Recent quantized neural networks require various dynamic ranges and resolutions to enable better inference accuracy, necessitating mixed support of energy-efficient integer and floating point computation. This work presents an SRAM look-up table (LUT) based INT8/FP8 dual mode in-memory-computing (IMC) macro which supports configurable FP8 (E4M3 and E5M2) multiply-accumulate (MAC) operation and 2:8 structural sparsity pruning. Instead of suffering either excessive overheads of shifting and alignment logic or accuracy degradation for extreme energy efficiency, this macro exploits fine-grain exponent sharing and efficient LUT-based MAC circuits to balance the two aspects. The key features are: (1) a pipelined and SNR-oriented mantissa and exponent execution circuit for FP8 multiplication; (2) a LUT-based shifter and 1-stage adder to reduce digital logic overhead in FP MAC; (3) an in-macro decoder to support 2:8 structural sparsity pruning. The silicon prototype in 28nm CMOS achieves 81.69% accuracy for DeiT-base model for ImageNet dataset in dense FP8 mode, and 76.15% accuracy for ResNet-34 model in CIFAR100 dataset in sparse FP8 mode. The 2:8 pruning provides a 31.42 TFLOPS/W peak energy efficiency and 3.56 TFLOPS/mm2 peak area efficiency in sparse FP8 mode.
1 January 2026 in IEEE Journal of Solid-State Circuits
Transformer-based neural networks (NNs) have been dominant in modern AI due to their superior accuracy and generalization over traditional models. However, the quadratic complexity and dataflow challenges of self-attention hinder their efficient deployment on edge devices, especially for latency-sensitive, single-batch inference. Limited data parallelism necessitates both pipeline and tensor parallelism. Specifically, sequence parallelism offers strong scaling potential in domain-specific accelerators (DSAs), but is constrained by the temporal dependencies involved in the max-finding operations of softmax. This work presents SASDenSebLE, a compact, scalable vision Transformer (ViT) inference architecture co-designed with a max-finding-free softmax approximation across micro-architecture, architecture, and algorithm levels. A dense-sparse-hybrid datapath transforms large-value softmax operations into linear projections, enabling layer-fusion. A hierarchical system further improves sequence parallelism without excessive on-chip buffering or off-chip traffic. Experiments show up to $2.83\times $ , $28.02\times $ , and $1.40\times $ speedup over baseline DSAs and a state-of-the-art ViT accelerator design, respectively, with negligible accuracy loss. Demonstration on the camera encoder of a recent bird’s-eye-view model shows up to $10.82\times $ speedup over a typical edge GPU.