4 March 2026 in IEEE Transactions on Circuits and Systems I: Regular Papers
The autoregressive nature of Large Language Models (LLMs) has enabled remarkable performance in language generation, making them a cornerstone in natural language processing. As context windows lengthen, the per-token key–value (KV) cache grows linearly with sequence length and turns the generation stage into a memory-bound operation. However, existing works focus primarily on an isolated stage of KV cache reduction, and most efforts incur significant hardware overheads. This results in a lack of cross-stage co-optimization and leaves significant reduction potential untapped. To address these challenges, we propose CrossKV, a software-hardware co-design architecture that accelerates LLM inference to fully achieve the potential of KV caching optimization. Motivated by our observation, we identify the three stages in KV caching and present a cross-stage co-optimization algorithm, including: a derivative-enhanced dynamic progressive pruning method, a DCT-driven key-vector low-rank compressing, and a dynamic clustered hybrid run-length encoding to reduce the KV caching while maintaining high accuracy. Then, an efficient architecture featuring cross-stage dynamic co-optimization with negligible hardware overhead is proposed to fully harness the algorithm of CrossKV. Our evaluations of CrossKV over 9 popular LLMs models and various long-context tasks demonstrate an average of $6.03\times $ and $4.10\times $ improvement for energy efficiency and speedup compared to existing SoTA Accelerators architecture. Compared to the Nvidia A100 GPU, CrossKV achieves an average $23.43\times $ energy efficiency and $4.64\times $ speedup, respectively.
1 June 2026 in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
With the increasing demand for large-scale model deployment, cloud-based computing remains indispensable. However, energy and bandwidth overheads in edge-to-cloud data transmission pose significant challenges. This work presents a low-power, multimodal SoC that performs on-chip image and audio compression processing, enabling energy-efficient and low-latency cloud interaction for Internet of Things (IoT) applications. The SoC employs a two-level optimization strategy. At the microarchitecture level, a customized 16-bit fixed-length complex instruction set computing (CISC) microprocessor is introduced. Validated against a RISC-V minimal baseline RV32E equipped with equivalent peripheral engines, the proposed core achieves a $5.06\times $ reduction in core logic gate count and up to $2.42\times $ improvement in energy efficiency under iso-latency conditions. Additionally, a multipower domain strategy is adopted to further optimize energy efficiency, yielding $5.03\times $ and $4.13\times $ improvements for image and audio modes, respectively. Fabricated in a 65-nm CMOS process, our SoC integrates both image and audio processing, achieving an ultralow power consumption of 54.1– $387.5~\mu $ W. It outperforms state-of-the-art dual-modal processors, achieving a $2.84\times $ reduction in image-processing core power and a $12.78\times $ reduction in audio compression power. Furthermore, our SoC achieves an energy efficiency of 206 pJ/bit of audio task and 15.5 pJ/pixel of image task, which offers a scalable and energy-efficient solution for versatile IoT deployments.
1 January 2026 in IEEE Journal of Solid-State Circuits
1 October 2026 in IEEE Transactions on Very Large Scale Integration (VLSI) Systems