In HPC systems and hyper-scale data centers, the adoption of high-performance NVMe SSDs and high-speed networks has shifted storage bottlenecks to the network stack. Under high-concurrency workloads, frequent interrupt processing exhausts CPU resources while protocol-level control–data dependencies in the NVMe over TCP write path introduce additional serialization penalties. Existing optimizations either require specialized hardware, dedicate CPU cores to user-space polling, or apply semantically blind batching that delays time-sensitive control messages. We present SENS, a Semantic-aware NVMe over TCP Scheduler embedded within the NVMe over TCP driver of the Linux kernel. SENS combines two mechanisms: (1) PDU vectorization, which aggregates discrete Protocol Data Units into memory vectors before network transmission, amortizing per-I/O system call overhead and reducing soft-interrupt frequency; and (2) instruction-aware dispatch, which detects control PDUs such as R2T and triggers an early flush of the aggregation window, mitigating the serialization penalty on the write path. A prototype evaluation with physical NVMe SSDs and 100 GbE networks shows that SENS saturates the SSD throughput ceiling using 4–5 CPU cores, halving the host-side core budget compared to the native TCP driver. With a RAMDisk backend that removes storage-media constraints, SENS sustains up to 2.5× higher concurrent IOPS. These results show that exposing storage-protocol semantics to the batching layer improves the scalability of NVMe over TCP without additional hardware.

NVMe-over-Fabrics (NVMeoF) is expected to have high-performance and be highly scalable for disaggregating NVMe SSDs to High-Speed Network (HSN)-attached storage servers, thus the aggregated NVMe SSDs in storage servers can be elastically allocated to remote host servers for better utilization. However, due to the well-known connection scalability issue of RDMA NICs (RNICs), RDMA-enabled HSN can only provide a limited scale of performant Queue Pairs (QPs) for NVMeoF I/O queues to transfer capsule and data between the storage server and remote host servers. However, in current NVMeoF implementations, multiplexing multiple NVMeoF I/O queues onto a single RNIC QP is not supported yet. In this paper, we investigate how NVMeoF capsule and data transfers are performed efficiently over HSN with a limited number of RNIC QPs, and propose SPANoF, a Scalable and Performant Architecture for NVMe-over-Fabrics. SPANoF dissolves the intrinsic one-to-one mapping relationship between NVMeoF I/O queues and RNIC QPs, allocates a dedicated send-list for each NVMeoF I/O queue rather than for each RNIC QP, transfers NVMeoF capsules and data in send-lists with a QP-centric manner to remove lock-contention overhead, and polls for transfer completion notifications to remove interrupt-caused context switch overhead. We implemented SPANoF in the Linux kernel and evaluated it by the FIO benchmarks. Our experimental results demonstrate that SPANoF can avoid the performance collapses for commercial RNICs with a limited number of performant QPs and avoid the system crash for domain-specific RNICs with only limited-scale available QPs. Compared with the native NVMeoF implementation in Linux kernel, SPANoF can saturate an RNIC of the storage server with only three RNIC QPs of the remote host server. Compared with lock-based QP-sharing mechanisms, SPANoF improves bandwidth by up to 1.55× under 64 KB sequential write requests, improves throughput by up to 4.18× and reduces the average latency by 28.31% under 4 KB random read requests.