6 February 2025 in IEEE Circuits and Systems Magazine
The rapid development of large language models (LLMs) has significantly transformed the field of artificial intelligence, demonstrating remarkable capabilities in natural language processing and moving towards multi-modal functionality. These models are increasingly integrated into diverse applications, impacting both research and industry. However, their development and deployment present substantial challenges, including the need for extensive computational resources, high energy consumption, and complex software optimizations. Unlike traditional deep learning systems, LLMs require unique optimization strategies for training and inference, focusing on system-level efficiency. This paper surveys hardware and software co-design approaches specifically tailored to address the unique characteristics and constraints of large language models. This survey analyzes the challenges and impacts of LLMs on hardware and algorithm research, exploring algorithm optimization, hardware design, and system-level innovations. It aims to provide a comprehensive understanding of the trade-offs and considerations in LLM-centric computing systems, guiding future advancements in AI. Finally, we summarize the existing efforts in this space and outline future directions toward realizing production-grade co-design methodologies for the next generation of large language models and AI systems.
19 April 2022 in IEEE Transactions on Circuits and Systems II: Express Briefs
Due to the need for computing models that can process large quantities of data efficiently and with high throughput in many state-of-the-art machine learning algorithms, the processing-in-memory (PIM) paradigm is emerging as a potential replacement for standard digital architectures on these workloads. In this tutorial, we review the progress of PIM technology in recent years, at both the circuit and architecture level. We further present an analysis of when and how PIM technology surpasses the performance of conventional architectures. Finally, we outline our vision for the future of PIM technology.
16 March 2020 in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
4 September 2024 in IEEE Transactions on Circuits and Systems for Artificial Intelligence
Spiking neural networks (SNNs) have shown high efficiency in information processing. Time-to-first-spike (TTFS) coding, which encodes the information to the times of first spikes, has been introduced in SNNs due to its simple data representation. However, there is a lack of studies in synapse mapping and neuron circuits for compute-in-memory (CIM) neuromorphic design for low-power TTFS-based SNN inference. This work presents a CIM neuromorphic processing engine for TTFS-based SNNs, which comprises a 64x64 8T-SRAM array for synapse storage and current-based post-neuron circuits. We propose to map the positive and negative synaptic weights to adjacent columns in the SRAM array and accumulate the membrane potential in the dedicated post-neuron circuits. Two techniques are presented through hardware-software co-design: ”Multi-Level Firing Threshold Adjustment” to mitigate the impact of process variations, and “Timing Threshold Adjustment” to further speed up the computation. The proposed processing engine achieves the energy efficiency 249.8 TOPS/W under 8-bit inputs and signed 4-bit weights.
10 January 2023 in IEEE Transactions on Circuits and Systems II: Express Briefs
In-sensor-processing (ISP) paradigm has been exploited in state-of-the-art vision system designs to pave the way towards power-efficient sensing and processing. The redundant data transmission between sensors and processors is significantly minimized by local computation within each pixel. However, existing ISP designs suffer from limited frame rates and degraded fill factors. In this paper, we introduce a low-latency in-sensorintelligence neuromorphic vision system using neuromorphic spiking neurons, namely SpikeSen. SpikeSen directly operates on the photocurrents and executes the computation in the frequency domain, reducing the long exposure time and speeding up the computation. Experiments show that SpikeSen can achieve more than 6.1× computation speedup compared to existing ISP designs with competitive energy consumption per pixel.