10 June 2022 in IEEE Transactions on Circuits and Systems I: Regular Papers
Multi-bit-width convolutional neural network (CNN) maintains the balance between network accuracy and hardware efficiency, thus enlightening a promising method for accurate yet energy-efficient edge computing. In this work, we develop state-of-the-art multi-bit-width accelerator for NAS Optimized deep learning neural networks. To efficiently process the multi-bit-width network inferencing, multi-level optimizations have been proposed. Firstly, differential Neural Architecture Search (NAS) method is adopted for the high accuracy multi-bit-width network generation. Secondly, hybrid Booth based multi-bit-width multiply-add-accumulation (MAC) unit is developed for data processing. Thirdly, vector systolic array is proposed for effectively accelerating the matrix multiplications. With vector-style systolic dataflow, both the processing time and logic resources consumption can be reduced when compared with the classical systolic array. Finally, The proposed multi-bit-width CNN acceleration scheme has been practically deployed on FPGA platform of Xilinx ZCU102. Average performance on accelerating the full NAS optimized VGG16 network is 784.2 GOPS, and peek performance of the convolutional layer can reach as high as 871.26 GOPS for INT8, 1676.96 GOPS for INT4, and 2863.29 GOPS for INT2 respectively, which is among the best results in previous CNN accelerator benchmarks.
22 December 2022 in IEEE Transactions on Circuits and Systems II: Express Briefs
Working on edging device, convolutional neural network (CNN) inference application demands low-power consumption and high-performance computation. Therefore, exploiting energy-efficient multiply-and-accumulate (MAC) unit and high-throughput sparse CNN accelerator is of great importance. In this paper, we develop a sparse CNN accelerator achieving a high MAC-unit utilization ratio and great power efficiency. The accelerator includes a radix-4 Booth multiplier for pre-encoding weights to reduce the number of partial products (PPs) and the encoder power consumption. The proposed accelerator has the following three features. Firstly, we reduce the bit number of PPs exploiting the features of radix-4 Booth algorithm and offline weight pre-processing. Secondly, we extract eight encoders from relevant multipliers and merge them into one pre-encoding module to reduce area. Finally, after encoding non-zero weights offline, we design an activation selector module to select the activations corresponding to non-zero weights for subsequent multiple-add operations. The proposed work is designed by Verilog HDL language and implemented in a 28nm process. The proposed accelerator achieves 7.0325 TOPS/W with 50% sparsity and scales with sparsity up to 14.3720 TOPS/W at 87.5%.
15 August 2023 in IEEE Transactions on Circuits and Systems I: Regular Papers
Deep neural networks (DNNs) in safety-critical applications demand high reliability even when running on edge-computing devices. Recent works on System-on-Chip (SoC) design with state-of-the-art (SOTA) hardware artificial intelligence (AI) accelerators and corresponding multi-bit-width (MBW) convolutional neural network (CNN) generation strategies show that MBW CNNs can effectively explore the trade-off between network accuracy and hardware efficiency. However, reliability has not been considered in such trade-off analysis, even though highly quantized CNNs may elevate the impact of bit flips in the hardware. Also, the reliability of the microcontroller and its interface operating with the AI accelerator are not studied. This work evaluates the reliability of DNN computation in an SoC that includes a processor, SOTA AI accelerator, and NN models highly optimized for computation efficiency using a neural architecture search (NAS) method. Focusing on neutron-induced soft error, which is the primary source of bit-flip errors in a terrestrial environment, we perform fault injection and neutron beam experiments. For these experiments, we prototype the SoC on a flash-based FPGA platform, in which the configuration memory is robust to neutron irradiation. Then, we analyze the experimental data and identify vulnerable components in the system. Furthermore, we evaluate how the SoC running different NAS-optimized MBW LeNet5 networks impact the performance, radiation sensitivity, failure rate of MBW accelerator, and crash rate of the system on the FPGAs. Our results show that instruction and data tightly coupled memory (I/DTCM) are the most vulnerable parts and the control status registers (CSRs) in our accelerator are the second most vulnerable component. Moreover, MBW networks have higher susceptibility to critical errors than single-precision networks, low-precision data are more likely to affect the classification results, and the high bits are more sensitive to faults.
15 November 2024 in IEEE Transactions on Circuits and Systems I: Regular Papers
Artificial intelligence (AI) edge devices demand high energy efficiency as well as inference accuracy. SRAM-based compute-in-memory (CIM) accelerators have great potential for power reduction but still need to exploit higher throughput and better linearity performance. To meet edge-AI computing demands by CIM works, it is crucial to optimize algorithms and parameters for specific circuit systems to achieve hardware acceleration. This work firstly employs neural network search (NAS) method to find out the layer-wise optimized precisions and sparsities for convolutional neural networks (CNNs). Then, a 144-Kb charge-domain signed mixed-precision (2/4/8-bit) CIM accelerator employing bootstrapped SRAM cells with 9-transistors and 1-capacitor (9T1C) structure is proposed that incorporates a bit-level sparsity-aware analog-to-digital converter (ADC). This work not only achieves highly linear parallel accumulation operations to meet AI computing demands but also implements a hardware and software co-optimization system tailored to specific data characteristics. The design is verified on NAS-optimized networks VGG-16 and ResNet-18 using Cifar-10 dataset, which could achieve an equivalent accuracy at 4-bit of 68.68% while maintaining a high energy efficiency at 2-bit of 135.19TOPS/W by measurements.
25 June 2025 in IEEE Transactions on Circuits and Systems II: Express Briefs
This brief presents a novel compute-SRAM bitcell design capable of performing 1-bit digital multiplication for digital compute-in-memory (DCIM) macros. Unlike the conventional approach, which requires 10 transistors (6T SRAM + 4T NOR), the proposed bitcell achieves the same functionality with only 8 transistors. This is accomplished by efficiently embedding a NOR gate into the latch pair of the traditional 6T bitcell through the addition of two transistors. The design enables efficient in-bitcell compute operations, achieving a 37.5% reduction in area and a 41.5% reduction in leakage compared to the 10T counterpart. Moreover, the proposed bitcell maintains robust stability in terms of write and hold static noise margins, thanks to the sizing strategy and gate overdrive. To validate the design, a 64× 64 DCIM macro was fabricated using a 22nm CMOS process. The macro demonstrates a peak energy efficiency of 126 TOPS/W at INT4 precision under a 0.55 V supply.
22 April 2025 in IEEE Journal of Solid-State Circuits
The increasing model size and computational load of deep neural networks (DNNs) present a significant challenge to deploy DNN models on constrained devices. To enhance performance without compromising accuracy, we introduce a neural architecture search (NAS) method to develop layer-wise mixed-precision and mixed-sparsity DNNs. However, the optimization cannot be directly applied to existing DNN accelerators due to the specific data requirements of layer-wise mixed precision and mixed sparsity. To address this issue, this research proposes separate mixed-precision and mixed-sparsity accelerators. Both accelerators demonstrate cutting-edge results. The mixed-precision accelerator utilizes a split-and-combination vector (SCV) to re-use variable precision units (1/2/4/8 bit) at each layer with a further vector systolic array (VSA) implementation. By optimizing NAS for mixed-precision VGG-16, the VSA achieves mixed energy efficiency reached 29.12 TOPS/W, which is equivalent to 2 bit and equivalent accuracy at 4 bit. The mixed-sparsity accelerator introduces a log-scale structured sparse encoding strategy, combined with MAC and group VSA (G-VSA) optimization, to enhance system performance. It achieves an average energy efficiency of up to 21.7 TOPS/W at 0.7 V and 400 MHz using 28-nm CMOS. The measured results show that the mixed-precision chip exhibits better energy efficiency and accuracy than the mixed-sparsity chip.