Deming Chen

Deming Chen

Auto-generated profile, based on publicly available information

Deming Chen

University of Illinois Urbana-Champaign, Urbana, IL, USA

Research Focus & Metrics

Academic Career Length

20Years (2007-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
21 September 2021 in Transactions of the Association for Computational Linguistics
Verified
Journal Article
Open access
PDF file
PGPrakhar Ganesh
Prakhar Ganesh
YCYao Chen
Yao Chen
XLXin Lou
Xin Lou
MKMohammad Ali Khan
Mohammad Ali Khan
YYYin Yang
Yin Yang
MWMarianne Winslett
Marianne Winslett
Pre-trained Transformer-based models have achieved state-of-the-art performance for various Natural Language Processing (NLP) tasks. However, these models often have billions of parameters, and thus are too resource- hungry and computation-intensive to suit low- capability devices or applications with strict latency requirements. One potential remedy for this is model compression, which has attracted considerable research attention. Here, we summarize the research in compressing Transformers, focusing on the especially popular BERT model. In particular, we survey the state of the art in compression for BERT, we clarify the current best practices for compressing large-scale Transformer models, and we provide insights into the workings of various methods. Our categorization and analysis also shed light on promising future research directions for achieving lightweight, accurate, and generic NLP models.

23 January 2019
Verified
Book Chapter
Open access
PDF file
MKMatej Kristan
Matej Kristan
ALAleš Leonardis
Aleš Leonardis
JMJiří Matas
Jiří Matas
MFMichael Felsberg
Michael Felsberg
RPRoman Pflugfelder
Roman Pflugfelder
ZHZhiqun He
Zhiqun He
The Visual Object Tracking challenge VOT2018 is the sixth annual tracker benchmarking activity organized by the VOT initiative. Results of over eighty trackers are presented; many are state-of-the-art trackers published at major computer vision conferences or in journals in the recent years. The evaluation included the standard VOT and other popular methodologies for short-term tracking analysis and a “real-time” experiment simulating a situation where a tracker processes images as if provided by a continuously running sensor. A long-term tracking subchallenge has been introduced to the set of standard VOT sub-challenges. The new subchallenge focuses on long-term tracking properties, namely coping with target disappearance and reappearance. A new dataset has been compiled and a performance evaluation methodology that focuses on long-term tracking capabilities has been adopted. The VOT toolkit has been updated to support both standard short-term and the new long-term tracking subchallenges. Performance of the tested trackers typically by far exceeds standard baselines. The source code for most of the trackers is publicly available from the VOT page. The dataset, the evaluation kit and the results are publicly available at the challenge website ( http://votchallenge.net ).

21 January 2014 in Bioinformatics
Verified
Journal Article
Open access
PDF file
YHYun Heo
Yun Heo
XWXiao-Long Wu
Xiao-Long Wu
DCDeming Chen
Deming Chen
JMJian Ma
Jian Ma
WHWen-Mei Hwu
Wen-Mei Hwu
Motivation: Rapid advances in next-generation sequencing (NGS) technology have led to exponential increase in the amount of genomic information. However, NGS reads contain far more errors than data from traditional sequencing methods, and downstream genomic analysis results can be improved by correcting the errors. Unfortunately, all the previous error correction methods required a large amount of memory, making it unsuitable to process reads from large genomes with commodity computers. Results: We present a novel algorithm that produces accurate correction results with much less memory compared with previous solutions. The algorithm, named BLoom-filter-based Error correction Solution for high-throughput Sequencing reads (BLESS), uses a single minimum-sized Bloom filter, and is also able to tolerate a higher false-positive rate, thus allowing us to correct errors with a 40× memory usage reduction on average compared with previous methods. Meanwhile, BLESS can extend reads like DNA assemblers to correct errors at the end of reads. Evaluations using real and simulated reads showed that BLESS could generate more accurate results than existing solutions. After errors were corrected using BLESS, 69% of initially unaligned reads could be aligned correctly. Additionally, de novo assembly results became 50% longer with 66% fewer assembly errors. Availability and implementation: Freely available at http://sourceforge.net/p/bless-ec Contact:dchen@illinois.edu Supplementary information: Supplementary data are available at Bioinformatics online.

18 May 2020 in IEEE Transactions on Computers
Verified
Journal Article
Closed access
CGCheng Gong
Cheng Gong
YCYao Chen
Yao Chen
YLYe Lu
Ye Lu
TLTao Li
Tao Li
CHCong Hao
Cong Hao
DCDeming Chen
Deming Chen
Quantization has been proven to be an effective method for reducing the computing and/or storage cost of DNNs. However, the trade-off of the quantization bitwidth and final accuracy is complex and non-convex, which makes it difficult to be optimized directly. Minimizing direct quantization loss (DQL) of the coefficient data is an effective local optimization method, but previous works often neglect the accurate control of the DQL, resulting in a higher loss of accuracy. In this paper, we propose a new method, VecQ, which uses vectorized quantization loss for DNN weights, aiming to achieve the minimal quantization loss and better model accuracy. In addition, in order to speed up the proposed quantization process during the model training, we accelerate the quantization process with a parameterized probability estimation method and template-based derivation calculation. We evaluate our algorithm on MNIST, CIFAR, ImageNet, IMDB movie review and THUCNews text data sets with numerical DNN models. The results demonstrate that our proposed quantization solution is more accurate and effective than the state-of-the-art approaches yet with more flexible bitwidth support. Moreover, the evaluation of our quantized models on Saliency Object Detection (SOD) tasks maintains comparable feature extraction quality with up to 16×weight size reduction.

12 February 2023
Verified
Conference Paper
Open access
JZJinming Zhuang
Jinming Zhuang
JLJason Lau
Jason Lau
HYHanchen Ye
Hanchen Ye
ZYZhuoping Yang
Zhuoping Yang
YDYubo Du
Yubo Du
PZPeipei Zhou
Peipei Zhou

31 March 2021 in IEEE Design & Test
Verified
Journal Article
Closed access
CHCong Hao
Cong Hao
JDJordan Dotzel
Jordan Dotzel
JXJinjun Xiong
Jinjun Xiong
LBLuca Benini
Luca Benini
ZZZhiru Zhang
Zhiru Zhang
DCDeming Chen
Deming Chen
This work is an introduction and a survey for the Special Issue on Machine Intelligence at the Edge. The authors argue that workloads that were formerly performed in the cloud are increasingly moving to resource-limited edge computing systems, which raises a new set of challenges for machine learning as well as new opportunities. The topic is introduced by means of building blocks ranging from edge fundamentals to edge AI enabling methodologies as well as future trends and challenges.