Performing data-intensive tasks in the von Neumann architecture is challenging to achieve both high performance and energy efficiency due to the memory wall bottleneck. Compute-in-memory (CiM) is a promising mitigation approach by enabling parallel and in-situ multiply-accumulate (MAC) operations within the memory array. Thanks to the good matching of capacitors, SRAM-based charge-domain CiM (Q-CiM) has shown its potential for higher row-wise parallelism. However, the peripheral circuits of Q-CiM, such as the input drivers and analog-digital converters (ADCs), limit further improvement of throughput and area efficiency. This paper proposes a single-ADC multi-bit accumulation CiM macro architecture SAMBA, which can perform multi-bit MAC operation with ReLU of two vectors in one CiM cycle by only a single A/D conversion to mitigate the ADC overhead. In addition, post-correction methods are proposed to compensate the non-linearity of sensitive circuit modules in SAMBA to recover the accuracy drop due to the capacitor mismatch. A proof-of-concept macro is fabricated in a 65nm process and achieves 51.2GOPS throughput and 10.3TOPS/W energy efficiency, while showing 88.6% accuracy on CIFAR-10 and 64.8% accuracy on the CIFAR-100 with VGG-8 model.
7 November 2023 in IEEE Journal of Solid-State Circuits
Owing to the mature process and low access energy, static random-access memory (SRAM) has become a promising candidate for compute-in-memory (CiM) acceleration of multiply-accumulate (MAC) operations. However, SRAM-based CiM cells have rather low density and thus very limited total on-chip memory capacity. This fact, unfortunately, results in undesired weight data reload operations from the off-chip dynamic random-access memory (DRAM) in data-intensive scenarios and may even tarnish the energy efficiency of CiM at the task level. Therefore, exploration toward higher density CiM in CMOS is critical to ensure truly high energy efficiency in practice. Aligned with the goal of ultrahigh density, this article presents the first one-transistor (1T) multi-level-cell (MLC) read-only memory (ROM) CiM macro for multi-bit MAC. The highlights of the proposed ROM CiM techniques include: 1) multi-source-driven (MSD) 1T-MLC ROM; 2) charge-domain capacitor sharing (CDCS) for ultrahigh CiM memory density; and 3) ROM-based transfer-learning architectures to provide flexible support of different tasks with minor accuracy degradation. These techniques are demonstrated with a fabricated 2-Mb 1T-MLC ROM CiM macro for 8 b $\times$ 8 b MAC computing. This macro features a record-high cell density of 0.096- $\mu$ m $^{2}$ /bit and a macro weight density of 3984 kb/mm $^{2}$ in a 65-nm pure CMOS technology. It also achieves 3.8 $\times$ –55.3 $\times$ lower energy consumption per image inference than the state-of-the-art CiM macros when considering the possible DRAM access.
15 April 2025 in IEEE Journal of Solid-State Circuits
This article reports the first ultra-high-density hybrid static random access memory (SRAM)/read-only memory (ROM) compute-in-memory (CiM) architecture for multibit multiply-accumulate (MAC) operations of transformer models. It features several techniques that enhance the on-chip weight density, energy efficiency, and flexibility, namely: 1) the hybrid CiM structure of 6T SRAM and 1T multi-level cell (MLC) ROM for ultra-high memory density to improve the limited on-chip memory capacity and alleviate the frequent weight reload from dynamic random access memory (DRAM); 2) 2-/5-b adaptive-resolution analog-to-digital converters (ADCs) for energy reduction and low accuracy loss; and 3) post-fabrication 1-of-4 capacitor selection (PFCS) to reduce the capacitance of computing capacitors, which was originally limited by the capacitor variation. This leads to lower energy and higher accuracy. A record-high 22-Mb ROM CiM and 896-kb SRAM CiM macro has demonstrated the 8 × 8 b MAC operations of one transformer layer, featuring ultra-high weight density of 8928 kb/mm2 and significantly reduced DRAM access activities toward high inference energy efficiency.
While large Transformer models have exhibited outstanding performance on multimodal tasks, the underlying massive parameters land up with memory-wall issues. To address this bottleneck, SRAM-based compute-in-memory (CiM) is a promising technique. However, frequent off-chip weight loading due to limited on-chip capacity could severely limit the systemlevel energy efficiency. Recently, a high-density CiM structure at 16.4Mb/mm2, YOLoC, has shown the potential of complete on-chip deployment of a large detection model using transistor-based read-only-memory (ROM). However, it is still challenging to deploy even larger Transformer models. With opportunities provided by LoRA for finetuning large pretrained models on ROM-CiM with very light SRAM-CiMs, this work achieves ultra-high density up to 165.6Mb/mm2 by eliminating the use of transistors for ROM-CiM with a proposed 3D-METRO and a 3D stacking array on the mature CMOS process. Unlike the usual belief that parasitics have negative impacts, this work observes that parasitics can be utilized for data storage. Furthermore, a local recovering unit (LRU) is proposed for addressing the interference due to the transistor-less structure. 3D-METRO achieves ultra-high density improvement over the previous YOLoC, which is hundreds of times higher than that of SRAM-CiM, enabling the opportunity for large language model (LLM) deployment on a single chip with 28x energy efficiency improvement.