Chao Fang

Dr. Chao Fang

orcid

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
16 August 2022 in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
Verified
Journal Article
Closed access
CFChao Fang
Chao Fang
AZAojun Zhou
Aojun Zhou
ZWZhongfeng Wang
Zhongfeng Wang
The Transformer has been an indispensable staple in deep learning. However, for real-life applications, it is very challenging to deploy efficient Transformers due to the immense parameters and operations of models. To relieve this burden, exploiting sparsity is an effective approach to accelerate Transformers. Newly emerging Ampere graphics processing units (GPUs) leverage a 2:4 sparsity pattern to achieve model acceleration, while it can hardly meet the diverse algorithm and hardware constraints when deploying models. By contrast, we propose an algorithm–hardware co-optimized framework to flexibly and efficiently accelerate Transformers by utilizing general N:M sparsity patterns. First, from an algorithm perspective, we propose a sparsity inheritance mechanism along with inherited dynamic pruning (IDP) to obtain a series of N:M sparse candidate Transformers rapidly. A model compression scheme is further proposed to significantly reduce the storage requirement for deployment. Second, from a hardware perspective, we present a flexible and efficient hardware architecture, namely, STA, to achieve significant speedup when deploying N:M sparse Transformers. STA features not only a computing engine unifying both sparse–dense and dense–dense matrix multiplications with high computational efficiency but also a scalable softmax module eliminating the latency from intermediate off-chip data communication. Experimental results show that, compared to other methods, N:M sparse Transformers, generated using IDP, achieves an average of 6.7% improvement on accuracy with high training efficiency. Moreover, STA can achieve $14.47\times $ and $11.33\times $ speedups compared to Intel i9-9900X and NVIDIA RTX 2080 Ti, respectively, and perform $2.00 \,\,\sim 19.47 \times $ faster inference than the state-of-the-art field-programmable gate array (FPGA)-based accelerators for Transformers.

14 April 2020 in IEEE Transactions on Computers
Verified
Journal Article
Closed access
JLJinming Lu
Jinming Lu
CFChao Fang
Chao Fang
MXMingyang Xu
Mingyang Xu
JLJun Lin
Jun Lin
ZWZhongfeng Wang
Zhongfeng Wang
The training of Deep Neural Networks (DNNs) brings enormous memory requirements and computational complexity, which makes it a challenge to train DNN models on resource-constrained devices. Training DNNs with reduced-precision data representation is crucial to mitigate this problem. In this work, we conduct a thorough investigation on training DNNs with low-bit posit numbers, a Type-III universal number (Unum). Through a comprehensive analysis of quantization with various data formats, it is demonstrated that the posit format shows great potential to be employed in the training of DNNs. Moreover, a DNN training framework using 8-bit posit is proposed with a novel tensor-wise scaling scheme. The experiments show the same performance as the state-of-the-art (SOTA) across multiple datasets (MNIST, CIFAR-10, ImageNet, and Penn Treebank) and model architectures (LeNet-5, AlexNet, ResNet, MobileNet-V2, and LSTM). We further design an energy-efficient hardware prototype for our framework. Compared to the standard floating-point counterpart, our design achieves a reduction of 68%, 51%, and 75% in terms of area, power, and memory capacity, respectively.

20 September 2023 in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Verified
Journal Article
Closed access
CFChao Fang
Chao Fang
WSWei Sun
Wei Sun
AZAojun Zhou
Aojun Zhou
ZWZhongfeng Wang
Zhongfeng Wang
Sparse training is one of the promising techniques to reduce the computational cost of deep neural networks (DNNs) while retaining high accuracy. In particular, N:M fine-grained structured sparsity, where only ${N}$ out of consecutive ${M}$ elements can be nonzero, has attracted attention due to its hardware-friendly pattern and capability of achieving a high sparse ratio. However, the potential to accelerate N:M sparse DNN training has not been fully exploited, and there is a lack of efficient hardware supporting N:M sparse training. To tackle these challenges, this article presents a computation-efficient training scheme for N:M sparse DNNs using algorithm, architecture, and dataflow co-design. At the algorithm level, a bidirectional weight pruning method, dubbed BDWP, is proposed to leverage the N:M sparsity of weights during both forward and backward passes of DNN training, which can significantly reduce the computational cost while maintaining model accuracy. At the architecture level, a sparse accelerator for DNN training, namely, SAT, is developed to neatly support both the regular dense operations and the computation-efficient N:M sparse operations. At the dataflow level, multiple optimization methods ranging from interleave mapping, pregeneration of N:M sparse weights, and offline scheduling, are proposed to boost the computational efficiency of SAT. Finally, the effectiveness of our training scheme is evaluated on a Xilinx VCU1525 FPGA card using various DNN models (ResNet9, ViT, VGG19, ResNet18, and ResNet50) and datasets (CIFAR-10, CIFAR-100, Tiny ImageNet, and ImageNet). Experimental results show the SAT accelerator with the BDWP sparse training method under 2:8 sparse ratio achieves an average speedup of $1.75\times $ over that with the dense training, accompanied by a negligible accuracy loss of 0.56% on average. Furthermore, our proposed training scheme significantly improves the training throughput by $2.97\times $ – $25.22\times $ and the energy efficiency by $1.36\times $ – $3.58\times $ over prior FPGA-based accelerators.

20 June 2023 in ACS ES&T Water
Verified
Journal Article
Closed access
HTHaoyue Tan
Haoyue Tan
JJJinsha Jin
Jinsha Jin
CFChao Fang
Chao Fang
YZYing Zhang
Ying Zhang
BCBaodi Chang
Baodi Chang
WSWei Shi
Wei Shi

4 October 2024 in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
Verified
Journal Article
Closed access
CWChuanning Wang
Chuanning Wang
CFChao Fang
Chao Fang
XWXiao Wu
Xiao Wu
ZWZhongfeng Wang
Zhongfeng Wang
JLJun Lin
Jun Lin
Deploying deep neural networks (DNNs) on those resource-constrained edge platforms is hindered by their substantial computation and storage demands. Quantized multiprecision DNNs (MP-DNNs), denoted as MP-DNNs, offer a promising solution for these limitations but pose challenges for the existing RISC-V processors due to complex instructions, suboptimal parallel processing, and inefficient dataflow mapping. To tackle the challenges mentioned above, SPEED, a scalable RISC-V vector (RVV) processor, is proposed to enable efficient MP-DNN inference, incorporating innovations in customized instructions, hardware architecture, and dataflow mapping. First, some dedicated customized RISC-V instructions are introduced based on RVV extensions to reduce the instruction complexity, allowing SPEED to support processing precision ranging from 4-to 16-bit with minimized hardware overhead. Second, a parameterized multiprecision tensor unit (MPTU) is developed and integrated within the scalable module to enhance parallel processing capability by providing reconfigurable parallelism that matches the computation patterns of diverse MP-DNNs. Finally, a flexible mixed dataflow method is adopted to improve computational and energy efficiency according to the computing patterns of different DNN operators. The synthesis of SPEED is conducted on TSMC 28-nm technology. Experimental results show that SPEED achieves a peak throughput of 737.9 GOPS and an energy efficiency of 1383.4 GOPS/W for 4-bit operators. Furthermore, SPEED exhibits superior area efficiency compared with prior RVV processors, with the enhancements of 5.9 $\sim$ 26.9 $\times$ and 8.2 $\sim$ 18.5 $\times$ for 8-bit operator and best integer performance, respectively, which highlights SPEED’s significant potential for efficient MP-DNN inference.

11 October 2024 in Environmental Science & Technology
Verified
Journal Article
Closed access
JGJing Guo
Jing Guo
YLYiwen Luo
Yiwen Luo
CFChao Fang
Chao Fang
JJJinsha Jin
Jinsha Jin
PXPu Xia
Pu Xia
WSWei Shi
Wei Shi