SRAM based computing-in-memory (SRAM-CIM) techniques have been widely studied for neural networks (NNs) to solve the “Von Neumann bottleneck”. However, as the scale of the NN model increasingly expands, the weight cannot be fully stored on-chip owing to the big device size (limited capacity) of SRAM. In this case, the NN weight data have to be frequently loaded from external memories, such as DRAM and Flash memory, which results in high energy consumption and low efficiency. In this paper, we propose a hybrid-device computing-in-memory (HD-CIM) architecture based on SRAM and MRAM (magnetic random-access memory). In our HD-CIM, the NN weight data are stored in on-chip MRAM and are loaded into SRAM-CIM core, significantly reducing energy and latency. Besides, in order to improve the data transfer efficiency between MRAM and SRAM, a high-speed pipelined MRAM readout structure is proposed to reduce the BL charging time. Our results show that the NN weight data loading energy in our design is only 0.242 pJ/bit, which is 289 $\times $ less in comparison with that from off-chip DRAM. Moreover, the energy breakdown and efficiency are analyzed based on different NN models, such as VGG19, ResNet18 and MobileNetV1. Our design can improve $\mathbf {58\times \,\,to\,\,124\times }$ energy efficiency.
28 December 2023 in IEEE Transactions on Circuits and Systems I: Regular Papers
Recently, Adder Neural Networks (AdderNets) have gained widespread attention as an alternative to traditional Convolutional Neural Networks (CNNs) for deep learning tasks. AdderNets use lightweight addition operations to replace multiplication and accumulation (MAC) operations, but can keep almost the same accuracy compared to other CNNs. Nevertheless, challenges still exist with regards to hardware resources, power consumption, and communication bandwidth, primarily due to the ‘Von-Neumann bottlenecks’. However, computing-in-memory (CIM) architecture based on magnetic random-access memory (MRAM) has great potential for edge DNN implementation. In this paper, we propose a novel CIM paradigm using a novel Toggle-Spin-Torques (TST) driven MRAM for energy-efficient AdderNets (called CiTST_AdderNets). In CiTST_AdderNets, MRAM is driven by the interplay of the field-free spin orbit torque (SOT) effect and the spin transfer torque (STT) effect, which offers a fascinating prospect for energy efficiency and speed. Furthermore, a novel CIM paradigm is proposed to implement the dominating subtraction and sum operations in AdderNets, reducing data transfer and the related energy. Meanwhile, a highly parallel array structure integrating computation and storage is designed to support CiTST_AdderNets. In addition, a mapping strategy is proposed to efficiently map the convolution layer on the array. Fully connected layers can also be efficiently computed. The CiTST-AdderNets macro is designed by using a 65-nm CMOS process. Results show that our CiTST-AdderNets consumes about 1.65 mJ, 9.29 mJ, and 42.46 mJ for running VGG8, ResNet-50, and ResNet-18 respectively at 8-bit fixed-point precision. Compared to state-of-the-art platforms, our macro achieves an energy efficiency improvement of 1.45 x to 66.78 x.
9 February 2023 in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
Convolutional neural networks (CNNs) have been widely utilized in modern artificial intelligent (AI) systems. In particular, GoogLeNet, one of the most popular CNNs, consisting of a number of inception layers and max-pooling layers, has been intensively studied for mobile and embedded scenarios. However, the energy efficiency of GoogLeNet in hardware is still limited as the huge data movement between the processor and the memory. Therefore, designing a dataflow and the corresponding hardware architecture to achieve parallel processing with minimal data movement is rather critical to achieve high energy efficiency and throughput. In this paper, we propose a novel column stationary (CS) dataflow that maximally exploits the local data reuse of both the filter weights and feature maps. Moreover, a reconfigurable spatial architecture was proposed to map multiple convolution kernels (with different types and dimensions) in parallel to the processing engines (PEs) array. In this case, multiple convolution kernels can share the same input feature maps (activations) in computing process. In our hardware design, we utilize three typical convolution kernels (i.e., 5×5, 3×3, 1×1, corresponding to the inception layers of GoogLeNet) as an example to test the efficiency of our proposed dataflow and hardware architecture. The accelerator was implemented for one inception layer of the GoogLeNet in a 55-nm foundry’s CMOS process. The test results show that our CS dataflow can reduce ~85% energy consumption for memory access and save area of 13% and power of 12% for computing. In summary, our CS dataflow is 1.2× to 2.5× more energy-efficient compared to state-of-the-art dataflows.
19 April 2022 in IEEE Transactions on Electron Devices
Nonvolatile memories (NVMs) have been widely studied in the post-Moore era. However, noise/defect in NVMs has brought a challenging reliability problem in write/read access operations. A linear error correction code (ECC) has been widely used in memories to correct errors. In general, a codec hardware is required to encode or decode the information, resulting in area and energy overhead. In this article, we propose a new approach to implement the ECC codec by exploiting the existing write and read peripheral circuitry of NVMs based on an in-memory computing (IMC) architecture. In our approach, the desired code words can be generated through using write-like logic operations, given the original information and generation matrix during the encoding (write) process, while syndrome vectors (or error correction) can also be achieved through using read-like logic operations, given the parity-check matrix during the decoding (read) process, respectively. Therefore, we do not need a specific codec hardware but exploit the write-like and read-like logic operations. With an emerging toggle spin torque magnetic random access memory (TST-MRAM) and Hamming code as an example, we validated the feasibility of our approach and evaluated the performance of the design in the 40-nm technology node. The results show that our approach can significantly reduce energy consumption compared with state-of-the-art designs.
9 October 2024 in IEEE Transactions on Emerging Topics in Computing
In the post-Moore's era, compute-in-memory (CIM) techniques are promising to break the memory wall. In particular, SRAM-based CIMs (SRAM-CIMs) have attracted widespread attention owing to its good scalability with advanced process. At present, a rich variety of works focus on energy-efficiency improvement by either designing different bit-cell structures or optimizing circuit/chip architectures. However, owing to the CIM's primitive property to store one of the operands in the memory bit-cells, substantial computing resource is wasted by suspension during the operands loading procedure. In this paper, a high-throughput SRAM-CIM (HiT-CIM) architecture with simultaneous weight loading and computing capabilities is proposed by integrating on-chip nonvolatile MRAM (magnetic random-access memory). Meanwhile, both the mainstream current-domain and charge-domain SRAM bit-cell structures are optimized to support such an architecture. Furthermore, a reconfigurable fully-pipelined MRAM is designed to provide fast data loading in HiT-CIM, which can finetune weight loading strategy rapidly for different neural network models. Afterwards, an optimal evaluation and configuration strategy is proposed to improve the macro-level performance by considering the key components and parameters in terms of SRAM array, ADC, MRAM structure and frequency. Finally, the HiT-CIM's feasibility is verified under a 40-nm foundry's process. The results show that a multiple-fold speed improvement can be obtained on VGG19, ResNet18 and MobileNetV1, respectively. In specific, the area efficiency of HiT-CIM on VGG19 achieves 1124 GOPS/mm2 and 1880.12 GOPS/mm2 for the current-domain and chargedomain SRAM-CIMs, respectively. Up to 5.3× improvement is realized compared with prior works
In-memory computing (IMC) has emerged as a promising approach for accelerating deep neural network (DNN) inference by relocating computations to memory arrays. However, the efficacy of analog IMC diminishes when higher computational precision is required due to inherent device non-idealities. In this paper, we present a reconfigurable heterogeneous architecture that integrates a digital computing unit (DCU) with an analog IMC unit (AIMCU). The computational data is partitioned into most significant bits (MSBs) and least significant bits (LSBs); the sparse MSBs are processed by the DCU with lossless precision, and the dense LSBs are computed by the AIMCU for high energy efficiency, thereby enhancing inference accuracy and optimizing area efficiency. The architecture also features multiple modes that support variable-precision input splitting and weight splitting computation. Additionally, by leveraging hardware characteristics, we have developed several optimization strategies for neural network deployment, including parameter splitting, shifting algorithms, and sparse weight mapping. The experimental results show that the perceptual evaluation of speech quality (PESQ) of the deep complex convolution recurrent network (DCCRN) improved by 28.98%, while the peak signal-to-noise ratio (PSNR) of the super-resolution network (SRN) increased by 17.27%. Compared to previous state-of-the-art (SOTA) work, the reconfigurable heterogeneous-IMC-based system on a chip (SoC) demonstrates a significant improvement in energy efficiency while achieving accuracy close to that of pure digital computing.