Naigang Wang

Naigang Wang

Auto-generated profile, based on publicly available information

Naigang Wang

IBM T.J. Watson Research Center, Yorktown Heights, NY, USA

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 December 2020 in Proceedings of the IEEE
Verified
Journal Article
Closed access
SVSwagath Venkataramani
Swagath Venkataramani
XSXiao Sun
Xiao Sun
NWNaigang Wang
Naigang Wang
CCChia-Yu Chen
Chia-Yu Chen
JCJungwook Choi
Jungwook Choi
KGKailash Gopalakrishnan
Kailash Gopalakrishnan
Advances in deep neural networks (DNNs) and the availability of massive real-world data have enabled superhuman levels of accuracy on many AI tasks and ushered the explosive growth of AI workloads across the spectrum of computing devices. However, their superior accuracy comes at a high computational cost, which necessitates approaches beyond traditional computing paradigms to improve their operational efficiency. Leveraging the application-level insight of error resilience, we demonstrate how approximate computing (AxC) can significantly boost the efficiency of AI platforms and play a pivotal role in the broader adoption of AI-based applications and services. To this end, we present RaPiD, a multi-tera operations per second (TOPS) AI hardware accelerator core (fabricated at 14-nm technology) that we built from the ground-up using AxC techniques across the stack including algorithms, architecture, programmability, and hardware. We highlight the workload-guided systematic explorations of AxC techniques for AI, including custom number representations, quantization/pruning methodologies, mixed-precision architecture design, instruction sets, and compiler technologies with quality programmability, employed in the RaPiD accelerator.

13 November 2021 in IEEE Journal of Solid-State Circuits
Verified
Journal Article
Closed access
SLSae Kyu Lee
Sae Kyu Lee
AAAnkur Agrawal
Ankur Agrawal
JSJoel Silberman
Joel Silberman
MZMatthew Ziegler
Matthew Ziegler
MKMingu Kang
Mingu Kang
LCLeland Chang
Leland Chang
Reduced precision computation is a key enabling factor for energy-efficient acceleration of deep learning (DL) applications. This article presents a 7-nm four-core mixed-precision artificial intelligence (AI) chip that supports four compute precisions—FP16, Hybrid-FP8 (HFP8), INT4, and INT2—to support diverse application demands for training and inference. The chip leverages cutting-edge algorithmic advances to demonstrate leading-edge power efficiency for 8-bit floating-point (FP8) training and INT4 inference without model accuracy degradation. A new HFP8 format combined with separation of the floating- and fixed-point pipelines and aggressive circuit/architecture optimization enables performance improvements while maintaining high compute utilization. A high-bandwidth ring protocol enables efficient data communication, while power management using workload-aware clock throttling maximizes performance within a given power budget. The AI chip demonstrates 3.58-TFLOPS/W peak energy efficiency and 26.2-TFLOPS peak performance for HFP8 iso-accuracy training, and 16.9-TOPS/W peak energy efficiency and 104.9-TOPS peak performance for INT4 iso-accuracy inference.

6 June 2025 in IEEE Access
Verified
Journal Article
Open access
AZAozhong Zhang
Aozhong Zhang
ZYZi Yang
Zi Yang
NWNaigang Wang
Naigang Wang
YQYingyong Qi
Yingyong Qi
JXJack Xin
Jack Xin
PYPenghang Yin
Penghang Yin
Post-training quantization (PTQ) has emerged as a practical approach to compress large neural networks, making them highly efficient for deployment. However, effectively reducing these models to their low-bit counterparts without compromising the original accuracy remains a key challenge. In this paper, we propose an innovative PTQ algorithm termed COMQ, which sequentially conducts coordinate-wise minimization of the layer-wise reconstruction errors. We consider the widely used integer quantization, where every quantized weight can be decomposed into a shared floating-point scalar and an integer bit-code. Within a fixed layer, COMQ treats all the scaling factor(s) and bit-codes as the variables of the reconstruction error. Every iteration improves this error along a single coordinate while keeping all other variables constant. COMQ is easy to use and requires no hyper-parameter tuning. It instead involves only dot products and rounding operations. We update these variables in a carefully designed greedy order, significantly enhancing the accuracy. COMQ achieves remarkable results in quantizing 4-bit Vision Transformers, with a negligible loss of less than 1% in Top-1 accuracy. In 4-bit INT quantization of convolutional neural networks, COMQ maintains near-lossless accuracy with a minimal drop of merely 0.3% in Top-1 accuracy. The code is available at https://github.com/AozhongZhang/COMQ.

6 June 2025 in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
Verified
Journal Article
Open access
CZChuan Zhang
Chuan Zhang
YYYou You
You You
NWNaigang Wang
Naigang Wang
Jongsun ParkJongsun Park
Jongsun Park
LZLi Zhang
Li Zhang
Generative artificial intelligence (GenAI) has emerged as a pivotal focus in global innovation agendas, revealing transformative potential that extends beyond technological applications to reshape diverse societal domains. Given the fundamental dependency of GenAI deployment on circuits and systems (CAS), a co-evolutionary approach integrating both technological paradigms becomes imperative. This synergistic framework confronts three interrelated challenges: 1) developing deployment-ready GenAI algorithms, 2) engineering implementation-efficient CAS architectures, and 3) leveraging GenAI for autonomous CAS designs - each representing critical innovations vectors. Given the rapid advancement of GenAI-CAS technologies, a comprehensive synthesis has become an urgent priority across academia and industry. Consequently, this timely review systematically analyzes current advancements, provides integrative perspectives, and identifies emerging research trajectories. This review endeavors to serve both AI and CAS communities, thereby catalyzing an innovation feedback loop: GenAI-optimized CAS architectures in turn accelerate GenAI evolution through algorithm-hardware co-empowerment.

27 February 2025
Verified
Conference Paper
Open access
IEIsmail Erbas
Ismail Erbas
AAAporva Amarnath
Aporva Amarnath
VPVikas Pandey
Vikas Pandey
KSKarthik Swaminathan
Karthik Swaminathan
NWNaigang Wang
Naigang Wang
XIXavier Intes
Xavier Intes
Fluorescence lifetime imaging (FLI) is a widely used technique in the biomedical field for measuring the decay times of fluorescent molecules, providing insights into metabolic states, protein interactions, and ligand-receptor bindings [1, 5]. However, its broader application in fast biological processes, such as dynamic activity monitoring, and clinical use, such as in guided surgery, is limited by long data acquisition times and computationally demanding data processing. While deep learning has reduced post-processing times, time-resolved data acquisition remains a bottleneck for real-time applications. To address this,we propose a method to achieve near realtime FLI using an FPGA-based hardware accelerator. Specifically,we implemented a GRU-based sequence-to-sequence (Seq2Seq) model [3] on an FPGA board compatible with time-resolved cameras. The GRU model balances accurate processing with the resource constraints of FPGAs, which have limited DSP units and BRAM. The limited memory and computational resources on the FPGA require efficient scheduling of operations and memory allocation to deploy deep learning models for low-latency applications.We address these challenges by using STOMP, a queue-based discrete-event simulator that automates and optimizes task scheduling and memory management on hardware [4]. By integrating a GRU-based Seq2Seq model and its compressed version, called Seq2SeqLite [2], generated through knowledge distillation (KD), we were able to process multiple pixels in parallel, reducing latency compared to sequential processing. We explore various levels of parallelism to achieve an optimal balance between performance and resource utilization. Our results indicate that the proposed techniques achieved a 17.7x and 52.0x speedup over manual scheduling for the Seq2Seq model and the Seq2SeqLite model, respectively. Furthermore, the Seq2SeqLite model achieved a 98% reduction in parameters compared to the Seq2Seq model.

17 October 2025 in IEEE Computer Architecture Letters
Verified
Journal Article
Closed access
YFYunhua Fang
Yunhua Fang
RXRui Xie
Rui Xie
AHAsad Ul Haq
Asad Ul Haq
LMLinsen Ma
Linsen Ma
KMKaoutar El Maghraoui
Kaoutar El Maghraoui
TZTong Zhang
Tong Zhang
Large Language Model (LLM) inference is increasingly constrained by memory bandwidth, with frequent access to the key-value (KV) cache dominating data movement. While attention sparsity reduces some memory traffic, the relevance of past tokens varies over time, requiring the full KV cache to remain accessible and sustaining pressure on both bandwidth and capacity. With advances in interconnects such as NVLink and LPDDR5X, modern AI hardware now integrates high-bandwidth memory (HBM) with high-speed off-package DRAM, making heterogeneous memory systems a practical solution. This work investigates dynamic KV cache placement across such systems to maximize aggregated bandwidth utilization under capacity constraints. Rather than proposing a specific scheduling policy, we formulate the placement problem mathematically and derive a theoretical upper bound, revealing substantial headroom for runtime optimization. To our knowledge, this is the first formal treatment of dynamic KV cache scheduling in heterogeneous memory systems for LLM inference.