Large language models (LLMs) have sparked a new revolution in the field of natural language processing (NLP), and have garnered tremendous attention in both academic research and everyday life, thanks to their unprecedented performance in a wide range of applications. However, their deployment remains a significant challenge, primarily due to their intensive computational and memory requirements. Hardware acceleration and efficient quantization are promising solutions to address the two issues. In this paper, a quantization and hardware architecture co-design is presented for matrix-vector multiplications (MVMs) of LLMs. During quantization, we uniformly group weights and activations to ensure workload balance for hardware. To enhance the performance of quantization, we further propose two approaches called channel sorting and channel selection, which can be applied simultaneously. To support the proposed quantization scheme, we develop two precision-scalable MVM hardware architectures. They are specifically designed for high speed and high energy efficiency, respectively. Experimental results show that our proposed quantization scheme achieves state-of-the-art performance among all the reported post-training schemes that quantize both weights and activations into integers. Compared to MVM architecture of the state-of-the-art LLM accelerator OliVe, our design exhibits significant advantages in terms of area efficiency and energy efficiency.

Vision Transformers (ViTs) have achieved remarkable success in computer vision (CV) and are increasingly recognized as the new backbone for vision-language multi-modal tasks. Despite their success, the high computational cost associated with ViTs hinders their inference efficiency. In this paper, we introduce BSViT, a bit-serial Vision Transformer accelerator enhanced by algorithm-hardware co-design. BSViT can efficiently accelerate both plain and hierarchical Vision Transformer inference. At the algorithm level, we propose a post-training quantization scheme named dynamic patch and weight bit-group quantization. We first introduce a dynamic patch quantization (DPQ) scheme to dynamically allocate bit-width to different image patches based on their importance, thus reducing bit width and saving computation without significantly impacting accuracy. Second, we propose a weight bit-group quantization (BGQ) scheme to evenly distribute bits within groups and achieve workload balance across processing elements (PEs). At the hardware level, we propose a term-separate bit-serial accelerator to efficiently support DPQ and BGQ. We introduce dense and sparse bit-serial PEs to manipulate the dense least significant term (LST) and sparse most significant term (MST) workloads. A dense-sparse hybrid dataflow is devised to efficiently balance the two kinds of workloads. Our experiments show that BSViT can achieve up to $1.95\times$ speedup and $2.72\times$ energy efficiency compared to state-of-the-art (SOTA) bit-serial accelerators and achieve up to $3.69\times$ energy efficiency compared to SOTA Transformer accelerators.

Precision-scalable deep neural network (DNN) accelerator designs have attracted much research interest. Since the computation of most DNNs is dominated by multiply-accumulate (MAC) operations, designing efficient precision-scalable MAC (PSMAC) units is of central importance. This brief proposes two low-complexity PSMAC unit architectures based on the well-known one, Fusion Unit (FU), which is composed of a few basic units called Bit Bricks (BBs). We first simplify the architecture of BB through optimizing some redundant logic. Then a top-level architecture for PSMAC unit is devised by recursively employing BBs. Accordingly, two low-complexity PSMAC unit architectures are presented for two different kinds of quantization schemes. Moreover, we provide an insight into the decomposed multiplications and further reduce the bitwidths of the two architectures. Experimental results show that our proposed architectures can save up to 44.18% area cost and 45.45% power consumption when compared with the state-of-the-art design.

To meet the demand in a wide range of practical applications, precision-scalable deep neural network (DNN) accelerators are becoming an unavoidable trend. On the other hand, it has been demonstrated that a DNN accelerator may achieve better computation efficiency through exploiting the sparsity. Therefore, DNN accelerators with both precision scalability and sparsity exploitation are expected to have better performance. In this article, we propose an efficient precision-scalable DNN accelerator that can exploit the sparsity of activations. The precision scalability is obtained from the decomposable multiplier which is inspired by the well-known design, Bit Fusion. Besides, a zero-skipping scheme is adopted to leverage the inherent sparsity of activations. We first modify the architecture of the conventional fusion unit (FU) to make it amenable to the zero-skipping scheme. Then, a segmentation approach is devised to tackle the memory access conflict. Furthermore, a sparsity-aware mapping method is proposed to balance the workload of processing elements (PEs). Moreover, we present a bit-splitting strategy which can take advantage of the sparsity in the bit level. Compared with the state-of-the-art precision-scalable designs, our proposed accelerator can provide speedups of $4.12\times $ , $4.07\times $ , and $6.62\times $ in the precision modes $8b\times 8b$ , $4b\times 4b$ , and $2b\times 2b$ , respectively. Meanwhile, it also achieves $3.92\times $ peak area efficiency and competitive peak energy efficiency.

Generalized integrated interleaved (GII) codes have attracted much attention in distributed storage systems since they can reduce the overall redundancy by creating redundancy shared by the interleaves. The Berlekamp-Massey (BM) algorithm makes the GII decoding efficient by reusing the preceding results at each decoding stage. However, it also forms the speed bottleneck of GII decoders. The reformulated inversionless Berlekamp-Massey (riBM) algorithm is well known for its short critical path, but it is not generally suitable for GII decoding since its polynomials are difficult to be reused directly. In this paper, we address this issue by reinitializing the polynomials and propose a riBM-based GII decoding algorithm. In our simulations, slight performance degradation induced by the first-layer miscorrection is observed in the high signal-to-noise ratio (SNR) region. Hence, a subtle method is introduced to handle the miscorrected interleaves. Furthermore, the hardware architecture of the proposed decoding algorithm is developed and a high throughput decoder for the given example GII code is implemented. The synthesis results under TSMC 28-nm technology show that the decoder can achieve a throughput of up to 124 Gb/s. Moreover, the decoder has been implemented into a field-programmable gate array (FPGA) to validate its error-correction performance.

Generalized integrated interleaved Bose-Chaudhuri-Hocquenghem (GII-BCH) codes show great potentials in some storage applications, such as flash memory programming, since they need less redundancy than the simple interleaved BCH codes for a target reliability. The coefficient vectors that are precomputed from the conversion matrix play a very important role in the GII-BCH encoding and also characterize its hardware architecture. However, they require a large amount of computations, including matrix inversions and multiplications both modulo polynomials. In addition, the GII-BCH encoders also suffer from high complexity and latency. In this letter, we modify the original conversion matrix and directly obtain the coefficient vectors without sacrificing error-correction performance. The modified coefficient vectors are much simpler than the original ones due to their independence on the error-correction capabilities. Furthermore, encoder architectures for the original and the modified GII-BCH codes are developed and compared. It is shown that the encoders for the modified GII-BCH codes perform significantly better than those for the original ones in both hardware complexity and latency.