Remote Direct Memory Access (RDMA) technology provides a low-latency, high-bandwidth, and CPU-bypassed method for data transmission between servers. Recent works have proved that multipath transmission, especially packet spraying, can avoid network congestion, achieve load balancing, and improve overall performance in data center networks (DCNs). Multipath transmission can result in out-of-order (OOO) packet delivery. However, existing RDMA transport protocols, such as RDMA over Converged Ethernet version 2 (RoCEv2), are designed for handling sequential packets, limiting their ability to support multipath transmission. To address this issue, in this study, we propose ORNIC, a high-performance RDMA Network Interface Card (NIC) with out-of-order packet direct write method for multipath transmission. ORNIC supports both in-order and out-of-order packet reception. The payload of OOO packets is written directly to user memory without reordering. The write address is embedded in the packets only when necessary. A bitmap is used to check data integrity and detect packet loss. We redesign the bitmap structure into an array of bitmap blocks that support dynamic allocation. Once a bitmap block is full, it is marked and can be freed in advance. We implement ORNIC on a Xilinx U200 FPGA (Field-Programmable Gate Array), which consumes less than 15% of hardware resources. ORNIC can achieve 95 Gbps RDMA throughput, which is nearly 2.5 times that of MP-RDMA. When handling OOO packets, ORNIC’s performance is virtually unaffected, while the performance of Xilinx ERNIC and Mellanox CX-5 drops below 1 Gbps. Moreover, compared with MELO and LEFT, our bitmap has higher performance and lower bitmap block usage.

Remote direct memory access (RDMA) is widely used within and across data centers due to its low latency, high throughput, and low CPU overhead. To further enhance the transmission performance of RDMA, techniques such as multi-path RDMA have been proposed. However, while these techniques increase throughput, they also introduce significant out-of-order (OoO) packet issues that standard RDMA network interface cards (RNICs) struggle to handle effectively. To address the OoO challenges in RDMA network and ensure the integrity of data, we propose an FPGA-based bitmap which is capable of maintaining high throughput and low latency under OoO conditions. Our design segments the bitmap and maintains status information, achieving the low-latency processing of OoO packets with excellent scalability, thus making it suitable for various network environments. We implement this design on Xilinx AU200 FPGA and test it in a simulated 100 Gbps data center network. The results show that the performance under OoO transmission conditions is comparable to that under in-order conditions, demonstrating the solution’s effectiveness in handling RDMA OoO packets efficiently and ensuring high-performance data transfer in RDMA networks.

Remote Direct Memory Access (RDMA) enables low-latency datacenter networks but suffers from inefficient loss recovery using Go-Back-N (GBN). GBN retransmits entire packet windows, degrading Flow Completion Time (FCT) under congestion. We introduce RoSR, a novel selective retransmission architecture for Field-Programmable Gate Array (FPGA)-based RDMA NICs that supports hardware-accelerated direct writes of out-of-order (OoO) packets. RoSR supports efficient OoO packet reception and enables fine-grained retransmission using a dynamic shared bitmap for packet tracking. By extending the RDMA over Converged Ethernet version 2 (RoCEv2) packet format, RoSR facilitates selective retransmission. It triggers retransmissions via timeouts using bitmap blocks and introduces new Nack-bitmap and rd-req-bitmap messages for loss reporting. Under 1% packet loss, RoSR achieves up to 13.5× (RDMA Write) and 15.6× (RDMA Read) higher throughput than Xilinx ERNIC. In NS-3 simulations using the HPCC RDMA stack, RoSR reduces FCT slowdown by 3× to 6× compared to GBN across various packet loss rates, congestion control algorithms (DCQCN, HPCC, Timely), and traffic patterns, while maintaining robustness under high round-trip time (RTT) conditions.

Field-programmable gate arrays (FPGAs) are extensively utilized to accelerate virtualized network functions (VNFs) within cloud networks. Imposing rate limits on different flows can enhance the overall bandwidth utilization of the network. Existing hardware token bucket approaches fundamentally trade off resource efficiency against configuration granularity when supporting massive queues (>512). This paper proposes a novel rate-limiting method based on the token bucket algorithm and achieves efficient resource utilization through head packet scheduling and token-to-time conversion. The experimental results show that our method achieves 1.16% lookup-table (LUT) and 2.62% flip flop (FF) resource usage compared to state-of-the-art methods, while supporting 512 queues with <0.4% rate deviation across a 100 Kbps–10 Gbps range (5-decade dynamic range).