26 May 2023 in IEEE Transactions on Circuits and Systems II: Express Briefs
Graph attention networks (GATs) are a mainstream graph neural network (GNN) model. They have better performance in some tasks compared to other GNN models. The challenge is that graph data structures are irregular and data dependency in GATs is complex. General-purpose hardware cannot provide enough performance or energy efficiency. Therefore, a specialized accelerator for GATs is needed. In this brief, we propose FTW-GAT to accelerate GAT inference. The key idea of our approach is to quantize the weights of GATs to ternary values, which can greatly simplify processing elements (PEs), eliminate the dependence on digital signal processors (DSPs) and reduce power consumption. Then, we use operation fusion, multi-level pipelining and graph partitioning to improve the parallelism. Finally, we implement the accelerator on a Xilinx VCU128 FPGA platform. The results show that FTW-GAT achieves performance speedup by $390\times $ , $17\times $ and $1.4\times $ , and energy efficiency by $4007\times $ , $261\times $ and $3.1\times $ compared to CPUs, GPUs and a prior GAT accelerator.
31 March 2025 in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
Transformers have demonstrated remarkable success across various deep learning tasks. However, their inference and fine-tuning require substantial computation and memory resources, posing challenges for existing hardware platforms, particularly resource-constrained edge devices. To address these limitations, we propose F3, an FPGA-based accelerator for transformer fine-tuning. To reduce computation and memory overhead, this paper proposes a flexible floating point (FFP) format which consumes fewer resources than traditional floating-point formats of the same bitwidth. We adapt low-rank adaptation to FFP format and propose a fine-tuning strategy named LR-FFP which reduces the number of trainable parameters without compromising fine-tuning accuracy. At the hardware level, we design specialized processing elements (PEs) for the FFP format. The PE maximizes the utilization of DSP resources, enabling a single DSP to perform two multiply-accumulate operations per cycle. The PEs are organized into a systolic array (SA) to efficiently handle general matrix multiplication during fine-tuning. Through theoretical analysis and experimental evaluation, we determine the optimal dataflow and SA parameters to balance performance and resource consumption. We implement the architecture on the Xilinx VCU128 FPGA platform and F3 achieves a performance of 8.2 TFlops at 250 MHz. Compared with CPU and GPU implementations, F3 achieves speedups of 15.22× and 3.44×, respectively, and energy efficiency improvements of 70.52× and 9.44×.