Xuebo Liu

Dr. Xuebo Liu


Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
5 October 2023 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
JRJun Rao
Jun Rao
XMXv Meng
Xv Meng
LDLiang Ding
Liang Ding
SQShuhan Qi
Shuhan Qi
XLXuebo Liu
Xuebo Liu
DTDacheng Tao
Dacheng Tao
Pre-trained models are frequently employed in multimodal learning. However, these models have too many parameters and need too much effort to fine-tune the downstream tasks. Knowledge distillation (KD) is a method to transfer knowledge using the soft label from this pre-trained teacher model to a smaller student, where the parameters of the teacher are fixed (or partially) during training. Recent studies show that this mode may cause difficulties in knowledge transfer due to the mismatched model capacities. To alleviate the mismatch problem, adjustment of temperature parameters, label smoothing and teacher-student joint training methods (online distillation) to smooth the soft label of a teacher network, have been proposed. But those methods rarely explain the effect of smoothed soft labels to enhance the KD performance. The main contributions of our work are the discovery, analysis, and validation of the effect of the smoothed soft label and a less time-consuming and adaptive transfer of the pre-trained teacher's knowledge method, namely PESF-KD by adaptive tuning soft labels of the teacher network. Technically, we first mathematically formulate the mismatch as the sharpness gap between teacher's and student's predictive distributions, where we show such a gap can be narrowed with the appropriate smoothness of the soft label. Then, we introduce an adapter module for the teacher and only update the adapter to obtain soft labels with appropriate smoothness. Experiments on various benchmarks including CV and NLP show that PESF-KD can significantly reduce the training cost while obtaining competitive results compared to advanced online distillation methods.

1 January 2020
Verified
Conference Paper
Open access
PDF file
ZWZilong Wang
Zilong Wang
MZMingjie Zhan
Mingjie Zhan
XLXuebo Liu
Xuebo Liu
DLDing Liang
Ding Liang
Zilong Wang, Mingjie Zhan, Xuebo Liu, Ding Liang. Findings of the Association for Computational Linguistics: EMNLP 2020. 2020.

1 January 2021
Verified
Conference Paper
Open access
PDF file
XLXuebo Liu
Xuebo Liu
LWLongyue Wang
Longyue Wang
DWDerek F. Wong
Derek F. Wong
LDLiang Ding
Liang Ding
LCLidia S. Chao
Lidia S. Chao
ZTZhaopeng Tu
Zhaopeng Tu
Xuebo Liu, Longyue Wang, Derek F. Wong, Liang Ding, Lidia S. Chao, Shuming Shi, Zhaopeng Tu. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 2021.

16 September 2019 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Closed access
XLXuebo Liu
Xuebo Liu
DWDerek F. Wong
Derek F. Wong
LCLidia S. Chao
Lidia S. Chao
YLYang Liu
Yang Liu
Neural machine translation (NMT) has achieved state-of-the-art performance in many translation tasks. However, because the computational cost increases with the size of the search space for predicting the target words, the translation quality of NMT is constrained by the limited vocabulary. To alleviate this problem, we propose a novel dynamic hierarchical decoder for NMT to utilize all of the target words in the training and decoding process. In the proposed model, a target word is represented by two latent attribute vectors rather than a word vector. The model is trained to dynamically put together those words that share similar linguistic attributes. The prediction of a target word is, therefore, turned into the prediction of attribute vectors, where the softmax functions are performed at the attribute level. This greatly reduces the model size and the decoding time. Our experimental results demonstrate that the proposed model significantly outperforms the NMT baselines in both Chinese-English and English-German translation tasks.

1 January 2024
Verified
Conference Paper
Open access
PDF file
JZJiaqi Zhao
Jiaqi Zhao
MZMiao Zhang
Miao Zhang
CZChao Zeng
Chao Zeng
MWMing Wang
Ming Wang
XLXuebo Liu
Xuebo Liu
LNLiqiang Nie
Liqiang Nie
Jiaqi Zhao, Miao Zhang, Chao Zeng, Ming Wang, Xuebo Liu, Liqiang Nie. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

1 January 2023
Verified
Conference Paper
Open access
PDF file
CCChi Cheang
Chi Cheang
HCHou Chan
Hou Chan
DWDerek Wong
Derek Wong
XLXuebo Liu
Xuebo Liu
ZLZhaocong Li
Zhaocong Li
LCLidia Chao
Lidia Chao
Chi Cheang, Hou Chan, Derek Wong, Xuebo Liu, Zhaocong Li, Yanming Sun, Shudong Liu, Lidia Chao. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.