Wei Niu

Wei Niu

Auto-generated profile, based on publicly available information

Wei Niu

University of Georgia, Athens, GA, USA

Research Focus & Metrics

Academic Career Length

16Years (2011-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
3 November 2022
Verified
Book Chapter
Closed access
PDF file
ZKZhenglun Kong
Zhenglun Kong
PDPeiyan Dong
Peiyan Dong
XMXiaolong Ma
Xiaolong Ma
XMXin Meng
Xin Meng
WNWei Niu
Wei Niu
YWYanzhi Wang
Yanzhi Wang

18 June 2021
Verified
Conference Paper
Open access
WNWei Niu
Wei Niu
JGJiexiong Guan
Jiexiong Guan
YWYanzhi Wang
Yanzhi Wang
GAGagan Agrawal
Gagan Agrawal
BRBin Ren
Bin Ren
Deep Neural Networks (DNNs) have emerged as the core enabler of many major applications on mobile devices. To achieve high accuracy, DNN models have become increasingly deep with hundreds or even thousands of operator layers, leading to high memory and computational requirements for inference. Operator fusion (or kernel/layer fusion) is key optimization in many state-of-the-art DNN execution frameworks, such as TensorFlow, TVM, and MNN, that aim to improve the efficiency of the DNN inference. However, these frameworks usually adopt fusion approaches based on certain patterns that are too restrictive to cover the diversity of operators and layer connections, especially those seen in many extremely deep models. Polyhedral-based loop fusion techniques, on the other hand, work on a low-level view of the computation without operator-level information, and can also miss potential fusion opportunities. To address this challenge, this paper proposes a novel and extensive loop fusion framework called DNNFusion. The basic idea of this work is to work at an operator view of DNNs, but expand fusion opportunities by developing a classification of both individual operators and their combinations. In addition, DNNFusion includes 1) a novel mathematical-property-based graph rewriting framework to reduce evaluation costs and facilitate subsequent operator fusion, 2) an integrated fusion plan generation that leverages the high-level analysis and accurate light-weight profiling, and 3) additional optimizations during fusion code generation. DNNFusion is extensively evaluated on 15 DNN models with varied types of tasks, model sizes, and layer counts. The evaluation results demonstrate that DNNFusion finds up to 8.8 × higher fusion opportunities, outperforms four state-of-the-art DNN execution frameworks with 9.3× speedup. The memory requirement reduction and speedups can enable the execution of many of the target models on mobile devices and even make them part of a real-time application.

1 April 2024 in Talanta
Verified
Journal Article
Closed access
PYPing Yang
Ping Yang
ATA-Ling Tang
A-Ling Tang
STShuai Tan
Shuai Tan
GWGuang-Ye Wang
Guang-Ye Wang
HHHou-Yun Huang
Hou-Yun Huang
SYSong Yang
Song Yang

9 November 2022
Verified
Book Chapter
Closed access
PDF file
YWYushu Wu
Yushu Wu
YGYifan Gong
Yifan Gong
PZPu Zhao
Pu Zhao
YLYanyu Li
Yanyu Li
ZZZheng Zhan
Zheng Zhan
YWYanzhi Wang
Yanzhi Wang

16 June 2021 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
WNWei Niu
Wei Niu
ZLZhengang Li
Zhengang Li
XMXiaolong Ma
Xiaolong Ma
PDPeiyan Dong
Peiyan Dong
GZGang Zhou
Gang Zhou
BRBin Ren
Bin Ren
It is appealing but challenging to achieve real-time deep neural network (DNN) inference on mobile devices, because even the powerful modern mobile devices are considered as “resource-constrained” when executing large-scale DNNs. It necessitates the sparse model inference via weight pruning, i.e., DNN weight sparsity, and it is desirable to design a new DNN weight sparsity scheme that can facilitate real-time inference on mobile devices while preserving a high sparse model accuracy. This paper designs a novel mobile inference acceleration framework GRIM that is General to both convolutional neural networks (CNNs) and recurrent neural networks (RNNs) and that achieves Real-time execution and high accuracy, leveraging fine-grained structured sparse model Inference and compiler optimizations for Mobiles. We start by proposing a new fine-grained structured sparsity scheme through the Block-based Column-Row (BCR) pruning. Based on this new fine-grained structured sparsity, our GRIM framework consists of two parts: (a) the compiler optimization and code generation for real-time mobile inference; and (b) the BCR pruning optimizations for determining pruning hyperparameters and performing weight pruning. We compare GRIM with Alibaba MNN, TVM, TensorFlow-Lite, a sparse implementation based on CSR, PatDNN, and ESE (a representative FPGA inference acceleration framework for RNNs), and achieve up to $14.08\times$ speedup.

27 April 2024
Verified
Conference Paper
Open access
WNWei Niu
Wei Niu
MSMusfiqur Rahman Sanim
Musfiqur Rahman Sanim
ZSZhihao Shu
Zhihao Shu
JGJiexiong Guan
Jiexiong Guan
XSXipeng Shen
Xipeng Shen
BRBin Ren
Bin Ren
This work is motivated by recent developments in Deep Neural Networks, particularly the Transformer architectures underlying applications such as ChatGPT, and the need for performing inference on mobile devices. Focusing on emerging transformers (specifically the ones with computationally efficient Swin-like architectures) and large models (e.g., Stable Diffusion and LLMs) based on transformers, we observe that layout transformations between the computational operators cause a significant slowdown in these applications. This paper presents SmartMem, a comprehensive framework for eliminating most layout transformations, with the idea that multiple operators can use the same tensor layout through careful choice of layout and implementation of operations. Our approach is based on classifying the operators into four groups, and considering combinations of producer-consumer edges between the operators. We develop a set of methods for searching such layouts. Another component of our work is developing efficient memory layouts for 2.5 dimensional memory commonly seen in mobile devices. Our experimental results show that SmartMem outperforms 5 state-of-the-art DNN execution frameworks on mobile devices across 18 varied neural networks, including CNNs, Transformers with both local and global attention, as well as LLMs. In particular, compared to DNNFusion, SmartMem achieves an average speedup of 2.8×, and outperforms TVM and MNN with speedups of 6.9× and 7.9×, respectively, on average.