19 August 2023 in Journal of the Franklin Institute
12 January 2026 in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Quantization and sparsity are key techniques for CNN compression and acceleration. Independent exploration of value-based sparsity (VBS) or bit-level sparsity (BLS) in 8b weights yield limited gains. We analyze weight distributions of VGG16 and ResNet50 after 4b quantization and find over 80% zero-values with strong near-structured sparsity. On VGG16, 98.11% of 16-weight groups contain at least 8 zero-values. Based on this analysis, we propose a two-stage sparse compression (TSSC) method that integrates VBS and BLS to efficiently compress weights. In stage 1, we apply post-quantization pruning to enforce structured sparsity with at least 8 zero-values per 16 weights and utilize value-based structured sparsity (VBSS) to compress these 8 zero-values per group. In stage 2, among the remaining weights, we apply column-based BLS (CBBLS) to further compress remaining weights. Finally, a two-stage sparse accelerator, which integrates the TSSC method and corresponding hardware optimization (hardware-efficient activation selection unit and bit-serial multiply-accumulate), is implemented to accelerate the CNN models. Experimental results demonstrate that our proposed TSSC method can reduce the weights of VGG16 and ResNet50 to 0.99b and 0.95b while maintaining an acceptable accuracy drop. When deployed on FPGA, our accelerator achieves an 8.08× weight compression and 7.13× performance improvement compared with 8b non-sparse VGG16 model. Compared with state-of-the-art accelerators, we achieve 1.95×-7.97×, 2.00×-32.8× and 1.13×-6.53× improvement in LUT efficiency, equivalent LUT efficiency and energy efficiency, respectively.
17 April 2026 in IEEE Transactions on Circuits and Systems I: Regular Papers
Mixed-precision quantization and unstructured pruning have emerged as two effective compression techniques, demonstrating great potential in reducing model size and computational cost in the deployment of convolutional neural networks (CNNs). However, their joint deployment still faces two major challenges: 1) The former introduces heterogeneous bit-widths in the bit-level, while the latter results in irregular sparsity in the value-level; their fundamentally incompatible data representations and computation patterns require two distinct types of hardware overhead to process them separately, which severely limits hardware execution efficiency. 2) Jointly applying both techniques often leads to notable accuracy degradation. In this paper, we propose a novel compression perspective that reinterprets zero-values generated by unstructured pruning as multiple consecutive 0-bits. We further introduce column-based bit-level sparsity, which provides a unified representation for weights after mixed-precision quantization and unstructured pruning, requiring only a single type of hardware overhead. Based on these techniques, we develop a hybrid compression framework that jointly optimizes model size, accuracy, and hardware implementation. Our method achieves weight/activation precision of 2.13b/4.06b on VGG16, delivering 7.40 $\times $ compression and 2.74 $\times $ speedup with 0.92% accuracy loss compared to the 8b baseline. Compared to state-of-the-art accelerators, our design achieves 1.12 $\times $ -6.23 $\times $ and 1.31 $\times $ -6.60 $\times $ improvements in energy efficiency and LUT efficiency when deploying VGG16 and ResNet50.
1 January 2026 in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
1 January 2026 in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
23 February 2026 in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
N:M structured sparsity is key to convolutional neural network (CNN) compression and acceleration, but two challenges remain. From the algorithm perspective, prior works have mainly focused on 8-bit quantized models, where N:M sparsity yields limited hardware efficiency. From the hardware perspective, the cost differences across N:M sparsity have not been analyzed. To address these issues, we present a unified algorithm–hardware co-design framework for 4-bit CNN acceleration. We show that 4-bit quantization induces over 80% zero weights and strongly structured sparsity, with over 95% of weight groups satisfying 4:8, 8:16, or 16:32 patterns. We propose a pruning-after-quantization (PAQ) algorithm that enforces strict N:M sparsity with minimal accuracy loss. We also analyze the hardware overhead of activation fetch units (AFUs) under different N:M sparsity patterns (4:8, 8:16, 16:32), revealing that the 4:8 AFU reduces look-up table (LUT) cost by up to 66.7% compared to 16:32. Finally, we introduce a 4-bit LUT-based sign-magnitude multiplier (LBSMM) requiring only 11 LUT6 resources, outperforming existing multipliers. Integrated on a Xilinx VCU118 field-programmable gate array (FPGA), our accelerator achieves $2.51\times $ – $12.89\times $ improvements in equivalent LUT efficiency over SOTA designs. The implementations of the PAQ algorithm and the RTL of LBSMM are available at https://github.com/haden-01/PAQ-and-LBSMM.git