Optimizing NVMe Server Performance on Linux: Sysctl, I/O Schedulers, and Interrupts

The Power of NVMe Multi-Queue (blk-mq)

Unlike legacy SATA disks limited to a single command queue with 32 commands, modern NVMe devices utilize the Linux Block Multi-Queue (blk-mq) subsystem, supporting up to 64,000 queues with 64,000 entries each.

This allows direct CPU core-to-queue mapping, eliminating global lock contention and unlocking the full potential of high-frequency AMD EPYC and Intel Xeon processors.

Selecting the Right I/O Scheduler: none vs mq-deadline

For high-throughput NVMe drives, traditional scheduling adds unnecessary CPU cycles. The recommended scheduler is none (bypassing scheduling altogether) or kyber for latency-sensitive workloads:

# Check current scheduler
cat /sys/block/nvme0n1/queue/scheduler

# Set scheduler to 'none' for maximum raw IOPS
echo "none" > /sys/block/nvme0n1/queue/scheduler

Sysctl Virtual Memory (VM) Parameters

Tuning dirty page writeback prevents sudden I/O stalls during intensive database bulk inserts:

vm.dirty_background_ratio = 5
vm.dirty_ratio = 10
vm.swappiness = 10
fs.file-max = 2097152