Zhenglun Kong

Zhenglun Kong

Auto-generated profile, based on publicly available information

Zhenglun Kong

Department of Electrical and Computer Engineering, Northeastern University, Boston, USA

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
3 November 2022
Verified
Book Chapter
Closed access
PDF file
ZKZhenglun Kong
Zhenglun Kong
PDPeiyan Dong
Peiyan Dong
XMXiaolong Ma
Xiaolong Ma
XMXin Meng
Xin Meng
WNWei Niu
Wei Niu
YWYanzhi Wang
Yanzhi Wang

24 March 2024 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
XSXuan Shen
Xuan Shen
PDPeiyan Dong
Peiyan Dong
LLLei Lu
Lei Lu
ZKZhenglun Kong
Zhenglun Kong
ZLZhengang Li
Zhengang Li
YWYanzhi Wang
Yanzhi Wang
Large Language Models (LLMs) stand out for their impressive performance in intricate language modeling tasks. However, their demanding computational and memory needs pose obstacles for broad use on edge devices. Quantization is then introduced to boost LLMs' on-device efficiency. Recent works show that 8-bit or lower weight quantization is feasible with minimal impact on end-to-end task performance, while the activation is still not quantized. On the other hand, mainstream commodity edge devices still struggle to execute these sub-8-bit quantized networks effectively. In this paper, we propose Agile-Quant, an Activation-Guided quantization framework for faster Inference of popular Large Language Models (LLMs) on the Edge. Considering the hardware profiling and activation analysis, we first introduce a basic activation quantization strategy to balance the trade-off of task performance and real inference speed. Then we leverage the activation-aware token pruning technique to reduce the outliers and the adverse impact on attentivity. Ultimately, we utilize the SIMD-based 4-bit multiplier and our efficient TRIP matrix multiplication to implement the accelerator for LLMs on the edge. We apply our framework on different scales of LLMs including LLaMA, OPT, and BLOOM with 4-bit or 8-bit for the activation and 4-bit for the weight quantization. Experiments show that Agile-Quant achieves simultaneous quantization of model weights and activations while maintaining task performance comparable to existing weight-only quantization methods. Moreover, in the 8- and 4-bit scenario, Agile-Quant achieves an on-device speedup of up to 2.55x compared to its FP16 counterparts across multiple edge devices, marking a pioneering advancement in this domain.

26 June 2023 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
ZKZhenglun Kong
Zhenglun Kong
HMHaoyu Ma
Haoyu Ma
GYGeng Yuan
Geng Yuan
MSMengshu Sun
Mengshu Sun
YXYanyue Xie
Yanyue Xie
YWYanzhi Wang
Yanzhi Wang
Vision transformers (ViTs) have recently obtained success in many applications, but their intensive computation and heavy memory usage at both training and inference time limit their generalization. Previous compression algorithms usually start from the pre-trained dense models and only focus on efficient inference, while time-consuming training is still unavoidable. In contrast, this paper points out that the million-scale training data is redundant, which is the fundamental reason for the tedious training. To address the issue, this paper aims to introduce sparsity into data and proposes an end-to-end efficient training framework from three sparse perspectives, dubbed Tri-Level E-ViT. Specifically, we leverage a hierarchical data redundancy reduction scheme, by exploring the sparsity under three levels: number of training examples in the dataset, number of patches (tokens) in each example, and number of connections between tokens that lie in attention weights. With extensive experiments, we demonstrate that our proposed technique can noticeably accelerate training for various ViT architectures while maintaining accuracy. Remarkably, under certain ratios, we are able to improve the ViT accuracy rather than compromising it. For example, we can achieve 15.2% speedup with 72.6% (+0.4) Top-1 accuracy on Deit-T, and 15.7% speedup with 79.9% (+0.1) Top-1 accuracy on Deit-S. This proves the existence of data redundancy in ViT. Our code is released at https://github.com/ZLKong/Tri-Level-ViT

31 January 2019 in Journal of Healthcare Engineering
Verified
Journal Article
Open access
PDF file
ZKZhenglun Kong
Zhenglun Kong
Ting LiTing Li
Ting Li
JLJunyi Luo
Junyi Luo
SXShengpu Xu
Shengpu Xu
Image segmentation plays an important role in multimodality imaging, especially in fusion structural images offered by CT, MRI with functional images collected by optical technologies, or other novel imaging technologies. In addition, image segmentation also provides detailed structural description for quantitative visualization of treating light distribution in the human body when incorporated with 3D light transport simulation methods. Here, we first use some preprocessing methods such as wavelet denoising to extract the accurate contours of different tissues such as skull, cerebrospinal fluid (CSF), grey matter (GM), and white matter (WM) on 5 MRI head image datasets. We then realize automatic image segmentation with deep learning by using convolutional neural network. We also introduce parallel computing. Such approaches greatly reduced the processing time compared to manual and semiautomatic segmentation and are of great importance in improving the speed and accuracy as more and more samples are being learned. The segmented data of grey and white matter are counted by computer in volume, which indicates the potential of this segmentation technology in diagnosing cerebral atrophy quantitatively. We demonstrate the great potential of such image processing and deep learning-combined automatic tissue image segmentation in neurology medicine.

1 January 2024
Verified
Conference Paper
Open access
PDF file
PZPu Zhao
Pu Zhao
FSFei Sun
Fei Sun
XSXuan Shen
Xuan Shen
PYPinrui Yu
Pinrui Yu
ZKZhenglun Kong
Zhenglun Kong
XLXue Lin
Xue Lin
Pu Zhao, Fei Sun, Xuan Shen, Pinrui Yu, Zhenglun Kong, Yanzhi Wang, Xue Lin. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024.

1 August 2023
Verified
Conference Paper
Open access
PDF file
XSXuan Shen
Xuan Shen
ZKZhenglun Kong
Zhenglun Kong
MQMinghai Qin
Minghai Qin
PDPeiyan Dong
Peiyan Dong
GYGeng Yuan
Geng Yuan
YWYanzhi Wang
Yanzhi Wang
The conventional lottery ticket hypothesis (LTH) claims that there exists a sparse subnetwork within a dense neural network and a proper random initialization method, called the winning ticket, such that it can be trained from scratch to almost as good as the dense counterpart. Meanwhile, the research of LTH in vision transformers (ViTs) is scarcely evaluated. In this paper, we first show that the conventional winning ticket is hard to find at weight level of ViTs by existing methods. Then, we generalize the LTH for ViTs to input data consisting of image patches inspired by the input dependence of ViTs. That is, there exists a subset of input image patches such that a ViT can be trained from scratch by using only this subset of patches and achieve similar accuracy to the ViTs trained by using all image patches. We call this subset of input patches the winning tickets, which represent a significant amount of information in the input data. We use a ticket selector to generate the winning tickets based on the informativeness of patches for various types of ViT, including DeiT, LV-ViT, and Swin Transformers. The experiments show that there is a clear difference between the performance of models trained with winning tickets and randomly selected subsets, which verifies our proposed theory. We elaborate the analogical similarity between our proposed Data-LTH-ViTs and the conventional LTH for further verifying the integrity of our theory. The Source codes are available at https://github.com/shawnricecake/vit-lottery-ticket-input.