29 May 2024 in IEEE Journal of Solid-State Circuits
Simultaneous localization and mapping (SLAM), a pivotal technology in robotics, autonomous vehicles, and surveillance, has gained prominence with the emergence of edge intelligence. Developing energy-efficient, low-latency SLAM systems is essential due to resource constraints and real-time demands. Compute-in-memory (CIM) architectures have been proven to be efficient for matrix multiplications. However, applications for SLAM raise new challenges in memory access and computation aspects: the linear system solving (LS) requires row transformation and causes frequent CIM updates, while the backend optimization causes redundant memory access; back-end optimization dominates SLAM’s computation and requires high precision and high dynamic range. Thus, we propose SLAM-CIM, a visual SLAM backend processor for edge robotics. A dynamic-range-driven-skipping CIM macro is designed to realize energy-efficient floating point (FP)-multiply-and-accumulate (MAC) operations. A preconditional-conjugate-gradient-based in-memory linear solver (PILARS) is designed to achieve LS without additional row transformations. This reduces memory access by $2.08{\times }$ and linear-system-solving latency by $3.84{\times }$ . SLAM-CIM further minimizes CIM weight updates through incremental bundle adjustment (BA), increasing average CIM utilization by $2.8{\times }$ . A silicon prototype is fabricated using 28-nm CMOS technology. The measurements show that SLAM-CIM achieves accurate and efficient SLAM operations with an average energy efficiency of 31.53 TFLOPS/W.
4 July 2018 in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
This paper presents OCEAN: an artificial neural network processor designed for accelerating gated-recurrent-unit (GRU) inference and on-chip incremental learning for sequential modeling. Implemented in 65-nm CMOS with silicon area of 2.9 × 3.5 mm 2 , the OCEAN processor features a 32-bit reduced instruction set computing core, 64-KB on-chip SRAM, and eight 16-bit four-cell GRU accelerators for inference and gradient computation. Each GRU accelerator is optimized and enhanced for efficient gradient computation. The processor is measured to consume 155 mW at the peak clock rate of 400 MHz and the supply of 1.2 V or 6.6 mW at 20 MHz/0.8 V. Both inference and on-chip incremental learning are accomplished on well-known AI tasks such as handwritten digit recognition, semantic natural language processing, and biomedical waveform-based seizure detection.
17 June 2025 in IEEE Journal of Solid-State Circuits
The resistive random access memory (RRAM)-based computing-in-memory features high density and high energy efficiency on edge. However, fine-tuning RRAM-based SoCs remains challenging due to the inherent limitations of non-volatile memory (NVM) characteristics. This work proposes an NVM-endurance/latency-aware collaborative RRAM/static random access memory (SRAM) compute-in-memory (CIM) accelerator that addresses the difficulties associated with RRAM write operations during fine-tuning tasks for both CNN and transformer models. The major contributions are: 1) RRAM-most significant bit (MSB)–SRAM-least significant bit (LSB) (RMSL)-based collaborative CIM macros, mitigating RRAM cell flipping times and alleviating the endurance concern; 2) an RRAM-sparse-SRAM-dense (RSSD) weight updating engine, minimizing the long reading and writing latency associated with RRAM access; and 3) a row-wise pipeline weight gradient (WG) computing data flow with low-hardware overhead. With a maximum bit update of ten times per fine-tuning (20 epochs, from start to convergence), the system achieves an energy efficiency of 76.25 TOPS/W for the CIM macro and 22.07 TOPS/W for the overall system. Thanks to the bit-level collaborative CIM, the RRAM CIM macro achieves the same hardware utilization during fine-tuning and inference processes to support RRAM’s energy-efficient computing. The proposed CIM accelerator, fabricated using 28-nm CMOS technology, achieves up to $143{\times }$ RRAM endurance improvement, $117{\times }$ and $144{\times }$ reduction in RRAM write power and latency.
7 August 2020 in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
This paper presents a communication-aware processing-in-memory deep neural network accelerator, which implements an in-memory entry-counting scheme for low bitwidth quantized multiplication-and-accumulations (MACs). To maintain good accuracy on ImageNet, the proposed design adopts a full-stack co-design methodology, from algorithms, circuits to architectures. In the algorithm level, an entry-counting based MAC is proposed to fit the learned step-sized quantization scheme, and exploit the sparsity of both activations and weights intrinsically. In the circuit level, content addressable memory cells and multiplexed arrays are developed in the processing-inmemory macro. In the architecture level, the proposed design is compatible with different stationary dataflow mappings, further reducing the memory access. An in-memory entry-counting silicon prototype and its entire peripheral circuits are fabricated in 65nm LP CMOS technology with an active area of 0.76×0.66 mm2. The 7.36-Kb processing-in-memory macro with 128 search entries can reduce the multiplication number by 12.8×. The peak throughput is 3.58 GOPS, achieved at a clock rate of 143MHz and a power supply of 1.23V. The peak energy efficiency of the processing-in-memory macro is 11.6 TOPS/W, achieved at a clock rate of 40MHz and a power supply of 1.01V. Note that the physical design of the entry-counting memory is completed in a standard digital placement and routing flow by augmenting the library with two dedicated memory cells. A 3-bit quantized ResNet-18 on the ImageNet dataset is performed, where the top-1 accuracy is 64.4%.
28 May 2025 in IEEE Journal of Solid-State Circuits
Large language models (LLMs) have demonstrated exceptional performance in complex artificial intelligence (AI) tasks. However, their rapidly increasing parameter sizes lead to significant communication and computational overhead, posing challenges to the energy efficiency (EEF) and memory footprint of AI processors. Compute-in-memory (CIM) architecture has emerged as a promising solution to alleviate bandwidth constraints and improve EEF. Nonetheless, both integer (INT) and floating-point (FP) CIM implementations struggle with the trade-off between accuracy and memory requirement when applied to LLMs. Outlier-aware quantization (OAQ), which employs low-precision formats for normal values and retains FP formats for high-magnitude outliers, has proven effective in matching the accuracy of full-FP baselines and has become a mainstream approach for efficient LLM deployment. Therefore, this work presents OA-CIM, an SRAM-based digital CIM macro that facilitates element-wise hybrid processing of BF16 outliers and INT4 normal values. The major contributions are: 1) an LUT-based multiply-and-accumulate (MAC) circuit design, which supports efficient FP/INT-compatible (FIC) MAC operations; 2) an xor-sharing non-maximum exponent gating scheme that reduces latency and area by bypassing unnecessary exponent comparisons in FP dataflow; and 3) a sparsity-aware readout circuit with distribution-offset weight encoding (DOWE) to mitigate the power-intensive charging/discharging process on the bitline. A 22-nm 512-kB 8T SRAM OA-CIM prototype is fabricated, which achieves an EEF of 346.6 TOPS/W in INT4 mode and 249.5 TFLOPS/W in outlier mode, representing a $2.7{\times }$ to $3.1{\times }$ improvement over state-of-the-art mixed-precision CIMs.