Performing data-intensive tasks in the von Neumann architecture is challenging to achieve both high performance and energy efficiency due to the memory wall bottleneck. Compute-in-memory (CiM) is a promising mitigation approach by enabling parallel and in-situ multiply-accumulate (MAC) operations within the memory array. Thanks to the good matching of capacitors, SRAM-based charge-domain CiM (Q-CiM) has shown its potential for higher row-wise parallelism. However, the peripheral circuits of Q-CiM, such as the input drivers and analog-digital converters (ADCs), limit further improvement of throughput and area efficiency. This paper proposes a single-ADC multi-bit accumulation CiM macro architecture SAMBA, which can perform multi-bit MAC operation with ReLU of two vectors in one CiM cycle by only a single A/D conversion to mitigate the ADC overhead. In addition, post-correction methods are proposed to compensate the non-linearity of sensitive circuit modules in SAMBA to recover the accuracy drop due to the capacitor mismatch. A proof-of-concept macro is fabricated in a 65nm process and achieves 51.2GOPS throughput and 10.3TOPS/W energy efficiency, while showing 88.6% accuracy on CIFAR-10 and 64.8% accuracy on the CIFAR-100 with VGG-8 model.
7 November 2023 in IEEE Journal of Solid-State Circuits
Owing to the mature process and low access energy, static random-access memory (SRAM) has become a promising candidate for compute-in-memory (CiM) acceleration of multiply-accumulate (MAC) operations. However, SRAM-based CiM cells have rather low density and thus very limited total on-chip memory capacity. This fact, unfortunately, results in undesired weight data reload operations from the off-chip dynamic random-access memory (DRAM) in data-intensive scenarios and may even tarnish the energy efficiency of CiM at the task level. Therefore, exploration toward higher density CiM in CMOS is critical to ensure truly high energy efficiency in practice. Aligned with the goal of ultrahigh density, this article presents the first one-transistor (1T) multi-level-cell (MLC) read-only memory (ROM) CiM macro for multi-bit MAC. The highlights of the proposed ROM CiM techniques include: 1) multi-source-driven (MSD) 1T-MLC ROM; 2) charge-domain capacitor sharing (CDCS) for ultrahigh CiM memory density; and 3) ROM-based transfer-learning architectures to provide flexible support of different tasks with minor accuracy degradation. These techniques are demonstrated with a fabricated 2-Mb 1T-MLC ROM CiM macro for 8 b $\times$ 8 b MAC computing. This macro features a record-high cell density of 0.096- $\mu$ m $^{2}$ /bit and a macro weight density of 3984 kb/mm $^{2}$ in a 65-nm pure CMOS technology. It also achieves 3.8 $\times$ –55.3 $\times$ lower energy consumption per image inference than the state-of-the-art CiM macros when considering the possible DRAM access.
Computing-in-memory (CiM) is a promising technique to achieve high energy efficiency in data-intensive matrix-vector multiplication (MVM) by relieving the memory bottleneck. Unfortunately, due to the limited SRAM capacity, existing SRAM-based CiM needs to reload the weights from DRAM in large-scale networks. This undesired fact weakens the energy efficiency significantly. This work, for the first time, proposes the concept, design, and optimization of computing-in-ROM to achieve much higher on-chip memory capacity, and thus less DRAM access and lower energy consumption. Furthermore, to support different computing scenarios with varying weights, a weight fine-tune technique, namely Residual Branch (ReBranch), is also proposed. ReBranch combines ROM-CiM and assisting SRAM-CiM to achieve high versatility. YOLoC, a ReBranch-assisted ROM-CiM framework for object detection is presented and evaluated. With the same area in 28nm CMOS, YOLoC for several datasets has shown significant energy efficiency improvement by 14.8x for YOLO (DarkNet-19) and 4.8x for ResNet-18, with <8% latency overhead and almost no mean average precision (mAP) loss (−0.5% ~ +0.2%), compared with the fully SRAM-based CiM.
15 April 2025 in IEEE Journal of Solid-State Circuits
This article reports the first ultra-high-density hybrid static random access memory (SRAM)/read-only memory (ROM) compute-in-memory (CiM) architecture for multibit multiply-accumulate (MAC) operations of transformer models. It features several techniques that enhance the on-chip weight density, energy efficiency, and flexibility, namely: 1) the hybrid CiM structure of 6T SRAM and 1T multi-level cell (MLC) ROM for ultra-high memory density to improve the limited on-chip memory capacity and alleviate the frequent weight reload from dynamic random access memory (DRAM); 2) 2-/5-b adaptive-resolution analog-to-digital converters (ADCs) for energy reduction and low accuracy loss; and 3) post-fabrication 1-of-4 capacitor selection (PFCS) to reduce the capacitance of computing capacitors, which was originally limited by the capacitor variation. This leads to lower energy and higher accuracy. A record-high 22-Mb ROM CiM and 896-kb SRAM CiM macro has demonstrated the 8 × 8 b MAC operations of one transformer layer, featuring ultra-high weight density of 8928 kb/mm2 and significantly reduced DRAM access activities toward high inference energy efficiency.
1 November 2026 in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
The exponential growth of artificial intelligence (AI) applications has strained conventional von Neumann architectures, where frequent data transfers between compute units and memory create significant energy and latency bottlenecks. Compute-in-Memory (CIM) addresses this challenge by performing multiply-accumulate (MAC) operations directly in memory arrays, substantially reducing data movement. As transformers increasingly dominate AI workloads, extending CIM frameworks to support these architectures becomes critical. In this work, we present NeuroSim V1.5, an integrated framework for cooptimizing accuracy and hardware efficiency in CIM accelerator design. Key contributions include: (1) a hybrid ACIM/DCIM architecture enabling transformer acceleration, with a case study comparing vision transformer (ViT) and CNN performance; (2) an integrated co-optimization framework combining TensorRT-based quantization, flexible noise modeling, and circuit-level PPA estimation; (3) expanded device support including non-volatile capacitive memories; and (4) up to 6.5× faster runtime through GPU-accelerated simulation. Our ViT case study reveals that ViTs exhibit lower noise tolerance and area efficiency than CNNs of comparable size, suggesting CNNs remain well-suited for vision applications on CIM hardware. The hybrid architecture provides the foundation for future transformer workloads including large language models. All versions are available open-source at https://github.com/neurosim/NeuroSim.
Owing to mature fabrication support and high flexibility, static random-access memory (SRAM) has become a very promising candidate for compute-in-memory (CiM) in accelerating deep neural networks (DNNs). However, SRAM-based CiM has low memory density and thus very limited total on-chip capacity, resulting in frequent weights reloading and additional power consumption during end-to-end inference tasks. Analog ROM CiM increases memory density but suffers from low computing density caused by A/D converter (ADC) limitation. To address these challenges, for the first time, a fully digital compute-in-read-only-memory (DCiROM) design approach is proposed in this paper. DCiROM introduces a novel ROM-logic fusion CiM that successfully reduces CiM area by 51% while maintaining high memory density and computing performance. By reusing multiply-and-accumulation (MAC) resources, DCiROM further achieves flexibility with a minimal area cost. We have implemented a DCiROM chip loaded 3024Kb ResNet-56 parameters using 65nm CMOS technology. This macro achieves 10.2x-55.7x higher normalized FoM (memory density x computing density) than the state-of-the-art CiM works. It also reduces 2.9x-9.9x energy consumption per image inference than SRAM CiM works when considering off-chip access.