Fei Sun

Fei Sun

Auto-generated profile, based on publicly available information

Fei Sun

Meta

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2024
Verified
Conference Paper
Open access
PDF file
PZPu Zhao
Pu Zhao
FSFei Sun
Fei Sun
XSXuan Shen
Xuan Shen
PYPinrui Yu
Pinrui Yu
ZKZhenglun Kong
Zhenglun Kong
XLXue Lin
Xue Lin
Pu Zhao, Fei Sun, Xuan Shen, Pinrui Yu, Zhenglun Kong, Yanzhi Wang, Xue Lin. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024.

21 October 2024 in arXiv
Verified
Preprint
Open access
PDF file
PZPu Zhao
Pu Zhao
FSFei Sun
Fei Sun
XSXuan Shen
Xuan Shen
PYPinrui Yu
Pinrui Yu
ZKZhenglun Kong
Zhenglun Kong
XLXue Lin
Xue Lin
Despite the superior performance, it is challenging to deploy foundation models or large language models (LLMs) due to their massive parameters and computations. While pruning is a promising technique to reduce model size and accelerate the inference, the traditional pruning techniques can hardly be applied for LLMs as they need to finetune the model on the full dataset with multiple epochs consuming massive data and hardware resources. To deal with this problem, post-training pruning methods are proposed to prune LLMs in one-shot without retraining. However, their accuracy after pruning may suffer from certain performance degradation due to the lack of retraining with massive data. To address this issue, in this paper, we first formulate the post-training problem for layer-wise LLM compression to simultaneously prune multiple weights in LLMs. Next, we provide an optimal solution for this problem and design our post-training pruning algorithm for both unstructured and semi-structured sparsity. Our extensive experiments demonstrate the superior performance of the proposed methods in comparison to SOTA baselines across various LLM families including transformer-based LLMs and Mamba-based LLMs. Code link: https://github.com/piuzha/APT