8 August 2023 in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
After the success of the transformer networks on natural language processing (NLP), the application of transformers to computer vision (CV) has followed suit to deliver unprecedented performance gains on vision tasks, including image recognition and object detection. The multihead self-attention (MHSA) is the key component in transformers, allowing the models to learn the amount of attention paid to each input position. Despite its strong modeling capability, MHSA involves complex operations that make transformers prohibitively costly for hardware deployment. Existing acceleration efforts with conventional hardware platforms are challenged by the memory wall. To alleviate the memory wall problem, compute-in-memory (CIM) is a promising solution by storing all model parameters on-chip in compute-capable memory arrays. The footprint of 2-D CIM designs must, however, expand to accommodate the increasingly larger model sizes. In this work, we present a heterogeneous 3-D integrated (H3D) accelerator to target the MHSA workloads in vision transformers. H3D allows the proposed H3DAtten architecture to combine the merits of resistive random access memory (RRAM)-based analog CIM (ACIM) in 40 nm and static random access memory (SRAM)-based digital CIM (DCIM) in 16 nm. We perform comprehensive signaling and thermal analyses to examine the effects of 3-D stacking on the accelerator. Compared to iso-capacity 2-D baseline designs, the proposed 5-tier H3DAtten accelerator achieves 8.4 $\times$ compute density without experiencing accuracy loss on the ImageNet-1k dataset.
17 November 2022 in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
Compute-in-memory (CIM) employing non-volatile memories (NVMs) has been widely investigated as an attractive candidate to accelerate the heavy multiply-and-accumulate (MAC) workloads in deep neural networks (DNNs) inference. While DNNs exhibit error tolerance to a certain extent, the aggregated effects from device and circuit non-idealities may degrade network accuracy to unacceptable levels. In this work, we examine a scalable error-correcting code (ECC), called MAC-ECC, that can be applied to CIM outputs obtained from parallel row accesses. An extrapolative design methodology is proposed to select the suitable MAC-ECC scheme based on the target workload, desirable accuracy, and error level in the utilized CIM hardware. To demonstrate the techniques, we taped-out and validated a resistive random-access memory (RRAM)-based CIM macro with embedded MAC-ECC in TSMC 40 nm process. The design achieves macro-level peak performance of 42.7 TOPS/W energy efficiency and 97.8 GOPS/mm 2 compute efficiency at 25 °C. By using the proposed methodology to select the optimal MAC-ECC scheme, the design maintains around 2% accuracy degradation for the investigated workloads at 120 °C with low performance overhead.
1 November 2026 in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
The exponential growth of artificial intelligence (AI) applications has strained conventional von Neumann architectures, where frequent data transfers between compute units and memory create significant energy and latency bottlenecks. Compute-in-Memory (CIM) addresses this challenge by performing multiply-accumulate (MAC) operations directly in memory arrays, substantially reducing data movement. As transformers increasingly dominate AI workloads, extending CIM frameworks to support these architectures becomes critical. In this work, we present NeuroSim V1.5, an integrated framework for cooptimizing accuracy and hardware efficiency in CIM accelerator design. Key contributions include: (1) a hybrid ACIM/DCIM architecture enabling transformer acceleration, with a case study comparing vision transformer (ViT) and CNN performance; (2) an integrated co-optimization framework combining TensorRT-based quantization, flexible noise modeling, and circuit-level PPA estimation; (3) expanded device support including non-volatile capacitive memories; and (4) up to 6.5× faster runtime through GPU-accelerated simulation. Our ViT case study reveals that ViTs exhibit lower noise tolerance and area efficiency than CNNs of comparable size, suggesting CNNs remain well-suited for vision applications on CIM hardware. The hybrid architecture provides the foundation for future transformer workloads including large language models. All versions are available open-source at https://github.com/neurosim/NeuroSim.
21 March 2025 in IEEE Electron Device Letters
In this letter, a ferroelectric field effect transistor (FeFET)-based charge-domain ternary content addressable memory (TCAM) is experimentally demonstrated for in-memory search. Specifically, foundry based FeFETs are configured in non-volatile capacitor (nvCAP) mode for charge-domain computation. A single TCAM cell consisting of 2 nvCAPs (i.e. two FeFETs) is used to perform the Store: bit ‘0’, ‘1’, ‘X’ and Search: bit ‘0’, ‘1’, ‘X’ operations. Further, SPICE simulations are performed to evaluate the feasibility of a scaled nvCAP device in a large TCAM array. The low CON/COFF ratio of a scaled device can lead to false-positive matchings, which can be countered by introducing a dummy column and input dependent charge cancellation. Finally, the charge-domain nvCAP TCAM is benchmarked against other proposed candidates, showing a ~1.5× improvement in energy efficiency over FeFET based current domain TCAM, and a ~3.3× improvement compared to resistive random access memory (RRAM) based TCAM.
27 January 2025 in IEEE Journal on Exploratory Solid-State Computational Devices and Circuits
3D heterogeneous integration (3D HI) offers promising solutions for incorporating substantial embedded memory into cutting-edge analog compute-in-memory (CIM) AI accelerators, addressing the need for on-chip acceleration of large AI models. However, this approach faces challenges with power supply noise (PSN) margins due to VDD scaling and increased power delivery network (PDN) impedance. This study demonstrates the necessity and benefits of 3D-HI for large-scale CIM accelerators, where 2D implementations would exceed manufacturing reticle limits. Our 3D designs achieve 39% higher energy efficiency, 8× higher operation density, and improved throughput through shorter vertical interconnects. We quantify steady-state IR-drop impacts in 3D-HI CIM architectures using a framework that combines PDN modeling, 3D-HI power, performance, area estimation, and behavioral modeling. We demonstrate that a drop in supply voltage to CIM arrays increases sensitivity to process, voltage, and temperature (PVT) noise. Using our framework, we model IR-drop and simulate its impact on the accuracy of ResNet-50 and ResNet-152 when classifying images from the ImageNet 1k dataset in the presence of injected PVT noise. We analyze the impact of through-silicon via (TSV) design and placement to optimize the IR-drop and classification accuracy. For ResNet architectures in 3D integration, we demonstrate that peripheral TSV placement provides an optimal balance between interconnect complexity and performance, achieving IR-drop below 10% of VDD while maintaining high classification accuracy.