20 December 2023 in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
Self-attention-based transformers have outperformed recurrent and convolutional neural networks (RNN/ CNNs) in many applications. Despite the effectiveness, calculating self-attention is prohibitively costly due to quadratic computation and memory requirements. To solve this challenge, this article proposes a hybrid analog-ReRAM and digital-SRAM in-memory computing accelerator (HARDSEA), a computing-in-memory (CIM) accelerator supporting self-attention in transformer applications. To trade off between energy efficiency and algorithm accuracy, HARDSEA features an algorithm-architecture-circuit codesign. A product-quantization-based scheme dynamically facilitates self-attention sparsity by predicting lightweight token relevance. A hybrid in-memory computing architecture employs both high-efficiency analog ReRAM-CIM and high-precision digital SRAM-CIM to implement the proposed new scheme. The ReRAM-CIM, whose precision is sensitive to circuit nonidealities, takes charge of token relevance prediction where only computing monotonicity is demanded. The SRAM-CIM, utilized for exact sparse attention computing, is reorganized as an on-memory-boundary computing scheme, thus adapting to irregular sparsity patterns. In addition, we propose a time-domain winner-take-all (WTA) circuit to replace the expensive ADCs in ReRAM-CIM macros. Experimental results show that HARDSEA prunes BERT and GPT-2 models to 12%–33% sparsity without accuracy loss, achieving $13.5\times $ – $28.5\times $ speedup and $291.6\times $ – $1894.3\times $ energy efficiency over GPU. Compared to state-of-the-art transformer accelerators, HARDSEA has $1.2\times $ – $14.9\times $ better energy efficiency at the same level of throughput.
3 April 2026 in IEEE Transactions on Circuits and Systems I: Regular Papers
This paper presents an energy-efficient spike-sorting system-on-chip (SoC) designed for closed-loop brain-computer interfaces of massive probing channels. The design first incorporates a sparsity/similarity-aware spike detection scratchpad, leveraging a bit-wise differential encoder and zero-friendly read-out circuits, reducing the dynamic power consumption of spike detection by 77.7%. To mitigate static power dissipation, it also introduces an ultra-low-leakage dual-voltage 5T-SRAM array with level-shifter embedded sense amplifiers, achieving an 82.2% leakage power reduction of neural signal buffering by applying half $V_{DD}$ on SRAM cells. Additionally, a memory hierarchy architecture combining on-chip SRAM and off-chip FeRAM, along with a firing-rate-based Osort for cluster template management, minimizes off-chip memory access to only 9.7% with a latency of $11.7\mu $ s for 1024-channel spike sorting. A silicon prototype is fabricated in 28-nm CMOS technology, which achieves a power consumption of 583nW/channel and an area consumption of 0.0012mm2/channel. The chip supports real-time spike sorting with up to 16K templates, $21.3\times $ greater than the state-of-the-art spike-sorting processor.
3 February 2026 in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
Transformer-based large language models (LLMs) have achieved unprecedented advances across diverse AI tasks. However, their execution remains power-hungry, primarily due to the rapidly growing model size and the quadratic complexity of the self-attention. In-memory computing (IMC) has emerged as a promising solution for accelerating LLMs, delivering high storage density, wide memory bandwidth, and intrinsic compute parallelism. To further improve energy efficiency, this paper addresses LLM inference by leveraging irregular sparse computing in digital SRAM-based IMCs. The proposed butterfly-network-based feed-forward IMC engine enables irregular zero-weight skipping while sustaining high compute utilization. In parallel, a local-attention-reusable IMC engine supports variable sparse attention spans and eliminates redundant self-attention operations. Finally, we implement a four-chip system to scale the parallel performance of the IMC chip. Extensive experiments are deployed on the system, including BERT, GPT to Llama models on 11 AI tasks. The results show that the fabricated accelerator in 28nm CMOS achieves a peak throughput of 186.9 tokens/s and an energy cost of 0.21 J/token, yielding 4.2× speedup and 27.6× energy efficiency over the A100 GPU. Compared to the existing LLM-specific accelerators, the proposed accelerator further achieves 1.2× to 16.9× energy efficiency.