Resistive random access memory (RRAM)-based compute-in-memory (CIM) has shown great potential for accelerating deep neural network (DNN) inference. However, device characteristics, such as low-resistance values, susceptibility to drift, and single-level cells, may limit the capabilities of RRAM-based CIM. In addition, prior works generally used the off-chip write-verify scheme to tighten RRAM resistance distributions and used off-chip analog-to-digital converter (ADC) references for fine-tuning partial sum quantization. Although off-chip techniques are viable for testing purposes, they may be unsuitable for practical applications. In this work, we present an RRAM-CIM macro to accelerate DNN inference. The chip features: 1) multi-level cell (MLC) RRAM for improving compute performance and density; 2) sparsity-aware input control to leverage the high activation sparsity in DNN models; 3) on-chip write-verify to speed up initial weight programming and periodically refresh cells to compensate for resistance drift under stress; and 4) on-chip ADC reference generation that provides column-wise tunability and stability with varying temperatures to guarantee the CIFAR-10 accuracy of 85.8% at 120 °C. The design is fabricated in TSMC 40-nm process with embedded RRAM technology and achieves a macro-level peak performance of 97.8 GOPS/mm² and 44.5 TOPS/W for multiply-and-accumulate (MAC) operations on VGG-8 network with ternary weights.
This article comprehensively investigates analog-to-digital converter (ADC) design for compute-in-memory array. The authors show that 6-bit ADC precision is sufficient to guarantee no loss of accuracy for large arrays, while achieving the best tradeoff between hardware performance and area overhead, compared to prior state-of-the-art designs.
21 February 2024 in IEEE Transactions on Circuits and Systems I: Regular Papers
Over the past decade, numerous compute-in-memory (CIM) platforms have been proposed in the literature. While emerging non-volatile memory based analog CIM (ACIM) has been widely studied, its silicon demonstrations are in the mature legacy node (22 nm or above). As an alternative, digital CIM (DCIM) based on static random access memory (SRAM) is recently drawing significant attention, as it enjoys the scaling benefits with the logic process to the leading-edge node (5 nm or below), and does not suffer from the accuracy loss due to process/voltage/temperature (PVT) variations. To assess the potential of DCIM in the future, we release NeuroSim V1.4, a CIM benchmark framework, which supports advanced technology nodes down to 1 nm node. We project the technology parameters (standard cell, transistor and interconnect) using TCAD device simulations, interconnect modeling, and the available industry/IRDS roadmaps. State-of-the-art technology trends such as fin-depopulation, buried power rail, stacked nanosheet, etc are captured in the updated parameters. Technology scaling down to 1 nm enables DCIM to achieve 1.4 $\sim$ 1.8 $\times$ and 44.1 $\sim$ 63.1 $\times$ higher system-level figure of merit than state-of-the-art 7 nm SRAM-based ACIM and 22 nm RRAM-based ACIM, respectively, for representative workloads such as ResNet18 and ResNet34 inference.
8 August 2023 in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
After the success of the transformer networks on natural language processing (NLP), the application of transformers to computer vision (CV) has followed suit to deliver unprecedented performance gains on vision tasks, including image recognition and object detection. The multihead self-attention (MHSA) is the key component in transformers, allowing the models to learn the amount of attention paid to each input position. Despite its strong modeling capability, MHSA involves complex operations that make transformers prohibitively costly for hardware deployment. Existing acceleration efforts with conventional hardware platforms are challenged by the memory wall. To alleviate the memory wall problem, compute-in-memory (CIM) is a promising solution by storing all model parameters on-chip in compute-capable memory arrays. The footprint of 2-D CIM designs must, however, expand to accommodate the increasingly larger model sizes. In this work, we present a heterogeneous 3-D integrated (H3D) accelerator to target the MHSA workloads in vision transformers. H3D allows the proposed H3DAtten architecture to combine the merits of resistive random access memory (RRAM)-based analog CIM (ACIM) in 40 nm and static random access memory (SRAM)-based digital CIM (DCIM) in 16 nm. We perform comprehensive signaling and thermal analyses to examine the effects of 3-D stacking on the accelerator. Compared to iso-capacity 2-D baseline designs, the proposed 5-tier H3DAtten accelerator achieves 8.4 $\times$ compute density without experiencing accuracy loss on the ImageNet-1k dataset.
26 October 2021 in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
Compute-in-memory (CIM), where information can be processed and stored at the same locations, is emerging as a promising paradigm to address the memory wall bottleneck in traditional Von Neumann architectures. Static random-access memory (SRAM) has been demonstrated as a mature candidate for CIM accelerator for deep neural networks (DNNs) due to its availability in advanced technology nodes. However, as SRAM is volatile and could not hold weight after power down, the necessity for downloading models from the cloud to inference engine causes potential threats such as model leaking. Also, saving raw weights of the DNN model stationary in the memory cells will increase the vulnerabilities. This work aims at developing a secure inference engine with a lightweight yet effective countermeasure to protect the DNN models in SRAM-based CIM architecture. We propose a secure XOR-CIM engine with a modified reverse secure sketch protocol to enable on-chip authentication and key processing for XOR-based stream cipher encrypted models. In the XOR-CIM core, we modify the six-transistor SRAM bit cell with dual wordlines to implement XOR decryption without sacrificing the parallel computation’s efficiency. The evaluations at 28 nm show that the XOR-CIM could enhance security, achieving comparable energy efficiency and no throughput loss, with negligible area overhead compared with the normal-CIM design without encryption.
10 October 2022 in IEEE Transactions on Circuits and Systems I: Regular Papers
Compute-in-memory (CIM) is an attractive solution for machine learning hardware acceleration since it merges computation directly into memory arrays, performing parallel multiply-and-accumulate (MAC) operations. The primary challenge in the reported CIM designs is the analog-to-digital converters (ADCs) that digitize analog MAC values for further processing, causing accuracy loss, excessive power dissipation, latency penalty, and area overhead. In this work, we propose ENNA, a novel CIM architecture based on an ADC-free sub-array design, implementing inter-array data processing in an analog manner. A lightweight input encoding scheme based on pulse-width modulation (PWM) is proposed to improve the throughput. We taped-out a prototype macro and validated the proposed ADC-free RRAM array design in TSMC 40nm process. Based on the measured silicon data, we explore the system-level performance with a partition between analog and digital processing at a level higher than the sub-array. The evaluation results show that the proposed accelerator can achieve 73.6 $\sim$ 86.4 TOPS/W energy efficiency and 2.3 $\sim$ 7 TOPS throughput (normalized to binary operation) tested on various DNN models. Furthermore, we project the proposed design using a heterogeneous 3D integration (H3D) scheme, showing a 3 $\times $$\sim$ 37 $\times$ throughput improvement depending on different tasks and $\sim$ 50% reduced area overhead compared to 2D design.