Pu Zhao

Dr. Pu Zhao


Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
9 November 2022
Verified
Book Chapter
Closed access
PDF file
YWYushu Wu
Yushu Wu
YGYifan Gong
Yifan Gong
PZPu Zhao
Pu Zhao
YLYanyu Li
Yanyu Li
ZZZheng Zhan
Zheng Zhan
YWYanzhi Wang
Yanzhi Wang

21 November 2024
Verified
Book Chapter
Closed access
PDF file
ZMZichong Meng
Zichong Meng
JZJie Zhang
Jie Zhang
CYChangdi Yang
Changdi Yang
ZZZheng Zhan
Zheng Zhan
PZPu Zhao
Pu Zhao
YWYanzhi Wang
Yanzhi Wang

1 January 2024
Verified
Conference Paper
Open access
PDF file
PZPu Zhao
Pu Zhao
FSFei Sun
Fei Sun
XSXuan Shen
Xuan Shen
PYPinrui Yu
Pinrui Yu
ZKZhenglun Kong
Zhenglun Kong
XLXue Lin
Xue Lin
Pu Zhao, Fei Sun, Xuan Shen, Pinrui Yu, Zhenglun Kong, Yanzhi Wang, Xue Lin. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024.

11 April 2025 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
XSXuan Shen
Xuan Shen
ZSZhao Song
Zhao Song
YZYufa Zhou
Yufa Zhou
BCBo Chen
Bo Chen
YLYanyu Li
Yanyu Li
JGJiuxiang Gu
Jiuxiang Gu
Diffusion Transformers have emerged as the preeminent models for a wide array of generative tasks, demonstrating superior performance and efficacy across various applications. The promising results come at the cost of slow inference, as each denoising step requires running the whole transformer model with a large amount of parameters. In this paper, we show that performing the full computation of the model at each diffusion step is unnecessary, as some computations can be skipped by lazily reusing the results of previous steps. Furthermore, we show that the lower bound of similarity between outputs at consecutive steps is notably high, and this similarity can be linearly approximated using the inputs. To verify our demonstrations, we propose the **LazyDiT**, a lazy learning framework that efficiently leverages cached results from earlier steps to skip redundant computations. Specifically, we incorporate lazy learning layers into the model, effectively trained to maximize laziness, enabling dynamic skipping of redundant computations. Experimental results show that LazyDiT outperforms the DDIM sampler across multiple diffusion transformer models at various resolutions. Furthermore, we implement our method on mobile devices, achieving better performance than DDIM with similar latency.

11 April 2025 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
XSXuan Shen
Xuan Shen
ZSZhao Song
Zhao Song
BCBo Chen
Bo Chen
JLJing Liu
Jing Liu
RZRuiyi Zhang
Ruiyi Zhang
JGJiuxiang Gu
Jiuxiang Gu
Transformers have emerged as the leading architecture in deep learning, proving to be versatile and highly effective across diverse domains beyond language and image processing. However, their impressive performance often incurs high computational costs due to their substantial model size. This paper focuses on compressing decoder-only transformer-based autoregressive models through structural weight pruning to improve the model efficiency while preserving performance for both language and image generation tasks. Specifically, we propose a training-free pruning method that calculates a numerical score with Newton's method for the Attention and MLP modules, respectively. Besides, we further propose another compensation algorithm to recover the pruned model for better performance. To verify the effectiveness of our method, we provide both theoretical support and extensive experiments. Our experiments show that our method achieves state-of-the-art performance with reduced memory usage and faster generation speeds on GPUs.

1 January 2024
Verified
Conference Paper
Open access
PDF file
ZZZheng Zhan
Zheng Zhan
YWYushu Wu
Yushu Wu
ZKZhenglun Kong
Zhenglun Kong
CYChangdi Yang
Changdi Yang
YGYifan Gong
Yifan Gong
YWYanzhi Wang
Yanzhi Wang
Zheng Zhan, Yushu Wu, Zhenglun Kong, Changdi Yang, Yifan Gong, Xuan Shen, Xue Lin, Pu Zhao, Yanzhi Wang. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.