Qian Gu

Ms. Qian Gu

orcid

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
15 November 2024 in IEEE Access
Verified
Journal Article
Open access
BXBingjie Xiang
Bingjie Xiang
JWJiarui Wu
Jiarui Wu
XHXiaoying Han
Xiaoying Han
QGQian Gu
Qian Gu
FCFei Chao
Fei Chao
XFXin Fu
Xin Fu
To address the issue of the substantial computational resource consumption during the inference phase of large language models due to their vast number of parameters, model sparsification is an effective solution. However, current sparsification methods for large models are costly. We propose a comprehensive two-stage approach called ELO-Mask for the rapid sparsification of large language models using a small calibration dataset. The approach consists of two steps: 1) Mask Reordering Step, this step involves initializing the mask using predefined parameter importance metrics, followed by reordering the model masks in blocks using the Straight-Through Estimator method with a small sample dataset. 2) Mask Fine-Tuning Step, this step involves further fine-tuning the masks obtained from the first step in blocks, using the same small sample dataset. Our experiments demonstrate the effectiveness of this approach. When sparsifying the Llama-7B model, our method shows significant superiority over the standard sparsification plus LoRA fine-tuning approach. It achieves comparable performance in the final sparse model while consuming less computational power, using a smaller dataset, occupying less GPU memory, and not affecting the inference speed of the sparse model.