Graph attention networks (GATs) are a mainstream graph neural network (GNN) model. They have better performance in some tasks compared to other GNN models. The challenge is that graph data structures are irregular and data dependency in GATs is complex. General-purpose hardware cannot provide enough performance or energy efficiency. Therefore, a specialized accelerator for GATs is needed. In this brief, we propose FTW-GAT to accelerate GAT inference. The key idea of our approach is to quantize the weights of GATs to ternary values, which can greatly simplify processing elements (PEs), eliminate the dependence on digital signal processors (DSPs) and reduce power consumption. Then, we use operation fusion, multi-level pipelining and graph partitioning to improve the parallelism. Finally, we implement the accelerator on a Xilinx VCU128 FPGA platform. The results show that FTW-GAT achieves performance speedup by $390\times $ , $17\times $ and $1.4\times $ , and energy efficiency by $4007\times $ , $261\times $ and $3.1\times $ compared to CPUs, GPUs and a prior GAT accelerator.
15 June 2022 in IEEE Transactions on Circuits and Systems II: Express Briefs
Convolutional Neural Networks (CNNs) have been widely used in various image recognition applications due to their high precision and high versatility. In order to improve real-time performance, many efficient CNNs with low computational complexity have been proposed. However, compared with the reduction in the amount of calculation, the demand for memory access has not decreased simultaneously. How to balance the utilization of various resources has become a challenge. This article deeply deconstructs the CNN network and analyzes the relationship among computational resources, on-chip memory, off-chip bandwidth, and throughput. With a given bandwidth, we propose a cross-layer scheduling to minimize the on-chip memory occupancy, which can still fully utilize the computational resources. We implement a flexible FPGA-based hardware accelerator that can deploy the proposed design. According to the experiments on the MobileNetV2, our design reduces the on-chip memory usage by 91.7% and achieves state-of-the-art performance.