As the key module of programmable switches or the SmartNIC card, the packet processing pipeline undertakes the task of packet forwarding and processing. However, the current pipeline for the FPGA-based SmartNIC is inflexible, and the related reconfigurable commercial device designs are closed-source. To solve this problem, this paper proposes a high-performance reconfigurable pipeline design, which has fully reconfigurable match-action units, supporting various network functions by its flexible reconfiguration. The fields of the match key and the size of the match table can be reconfigured without recompiling the HDL code or modifying the hardware. The processing rules and action instructions for the pipeline can be dynamically installed by the configuration module at runtime. We implement our design on the Xilinx Alveo U200 board with a Virtex UltraScale+ XCU200-2FSGD2104E FPGA and show that the designed pipeline supports fast reconfiguration to implement new network functions and that the throughput of the designed pipeline reaches 100 Gbps with low latency.

Nowadays, driven by the exponential growth of parameters and training data of AI applications and Large Language Models, a single GPU is no longer sufficient in terms of computing power and storage capacity. Building high-performance multi-GPU systems or a GPU cluster via vertical scaling (scale-up) has thus become an effective approach to break the bottleneck and has further emerged as a key research focus. Given that traditional inter-GPU communication technologies fail to meet the requirement of GPU interconnection in vertical scaling, a variety of high-performance inter-GPU communication protocols tailored for the scale-up domain have been proposed recently. Notably, due to the emerging nature of these demands and technologies, academic research in this field remains scarce, with limited deep participation from the academic community. Inspired by this trend, this article identifies the challenges and requirements of a scale-up network, analyzes the bottlenecks of traditional technologies like PCIe in a scale-up network, and surveys the emerging scale-up targeted technologies, including NVLink, OISA, UALink, SUE, and other X-Links. Then, an in-depth comparison and discussion is conducted, and we express our insights in protocol design and related technologies. We also highlight that existing emerging protocols and technologies still face limitations, with certain technical mechanisms requiring further exploration. Finally, this article presents future research directions and opportunities. As the first review article fully focusing on intra-node GPU interconnection in a scale-up network, this article aims to provide valuable insights and guidance for future research in this emerging field, and we hope to establish a foundation that will inspire and direct subsequent studies.

Longest prefix matching (LPM) is often used in network forwarding and it is usually implemented by ternary content addressable memory (TCAM) on field programmable gate array (FPGA). The traditional SRAM based TCAM only has one clock cycle in search delay, but it has a non-negligible update latency. Different from the general TCAM words, the ternary bits in LPM words are arranged continuously from low bits to high, and the prefix lengths of words have some distribution characteristics on actual network traffics. In this brief, a design is proposed to reduce the update latency of LPM by fast update algorithm with reorder mechanism. The key idea is reorder LPM prefix and partition don’t care bits into each sub-prefix evenly in SRAM based LPM, so that the update latency in each sub-table would be minimized and the whole update latency would be reduced. The design of this brief is implemented on Xilinx Kintex-7 field-programmable gate array. Compared to the prior methods, the update latency of LPM is reduced by at least half or more in most cases with our method. As far as we know, this is the fastest update mechanism in SRAM based LPM, which consumes the least possible clock cycles.

Match table is the key part to perform packet processing and forwarding for programmable switches in a software-defined network (SDN). However, the match table in current field-programmable gate array (FPGA)-based switches is inflexible or undisclosed. When the network function changes, the match table on FPGA needs to be redesigned or reset size parameters, and after recompilation and reimplementation, it could work again; this time-consuming and labor-intensive operation seriously reduces the flexibility and configurability of the switch. To address this issue, this article presents a design of reconfigurable match table (RMT) for FPGA-based programmable switches. A three-layer table structure is introduced to realize the reconfiguration and hardware-plane mapping of user-defined tables, and the logical tables in packet processing pipeline are interconnected with the physical tables in memory pool by the designed resource-efficient segment crossbar. To the best of our knowledge, this article is the first to publicly present the entire FPGA-based RMT design scheme and implementation details. The proposed design implements reconfigurable ternary content addressable memory (TCAM) based and static random access memory (SRAM) based match tables on Xilinx FPGA and verifies them with a packet filter system. In the proposed RMT system, a user could reconfigure the number, depth, and width of user-defined match tables (UMTs) in pipeline via control plane without modifying hardware, which enhances the flexibility of the data plane of FPGA-based switch greatly.

Edge deployment of low-batch large language models (LLMs) faces critical memory bandwidth bottlenecks when executing memory-intensive general matrix-vector multiplications (GEMV) operations. While digital processing-in-memory (PIM) architectures promise to accelerate GEMV operations, existing PIM-equipped edge devices still suffer from three key limitations: limited bandwidth improvement, component under-utilization in mixed workloads, and low compute capacity of computing units (CUs). In this paper, we propose CD-PIM to address these challenges through three key innovations. First, we introduce a high-bandwidth compute-efficient mode (HBCEM) that enhances bandwidth by dividing each bank into four pseudo-banks through segmented global bitlines. Second, we propose a low-batch interleaving mode (LBIM) to improve component utilization by overlapping GEMV operations with GEMM operations. Third, we design a compute-efficient CU that performs enhanced GEMV operations in a pipelined manner by serially feeding weight data into the computing core. Forth, we adopt a column-wise mapping for the key-cache matrix and row-wise mapping for the value-cache matrix, which fully utilizes CU resources. Our evaluation shows that compared to a GPU-only baseline and state-of-the-art PIM designs, our CD-PIM achieves 11.42x and 4.25x speedup on average within a single batch in HBCEM mode, respectively. Moreover, for low-batch sizes, the CD-PIM achieves an average speedup of 1.12x in LBIM compared to HBCEM.

Weight Fair Queuing is an ideal scheduling algorithm to guarantee the bandwidth of different queues according to their configured Weights when the switching nodes of the network are congested. Many of the switching nodes based on FPGA in the current network support four physical ports or hundreds of virtual ports. Massive logic and storage resources would be consumed if each port implemented a WFQ scheduler. This paper proposes a Queue-Group-Based WFQ Scheduler (QGWFQS), which can support WFQ scheduling across multiple ports through the reuse of tag calculation and encoding circuits. We also propose a novel finish tag calculation algorithm to accommodate the variation in the link rate of each port. The remainder of integer division is also taken into account, which makes the bandwidth allocation fairer. Experimental results show that the proposed scheduler supports up to 512 ports, with 32 queues allocated on each individual port. The scheduler has the capability to operate at 200 MHz and the total scheduling capacity reaches 200 Mpps.