Xingjian Li

Xingjian Li

Auto-generated profile, based on publicly available information

Xingjian Li

Ray and Stephanie Lane Computational Biology Department, Carnegie Mellon University, USA

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
14 September 2022 in Knowledge and Information Systems
Verified
Journal Article
Closed access
PDF file
XLXuhong Li
Xuhong Li
HXHaoyi Xiong
Haoyi Xiong
XLXingjian Li
Xingjian Li
XWXuanyu Wu
Xuanyu Wu
XZXiao Zhang
Xiao Zhang
DDDejing Dou
Dejing Dou

14 November 2022 in Artificial Intelligence
Verified
Journal Article
Closed access
XLXuhong Li
Xuhong Li
HXHaoyi Xiong
Haoyi Xiong
XLXingjian Li
Xingjian Li
XZXiao Zhang
Xiao Zhang
JLJi LIU
Ji LIU
DDDejing Dou
Dejing Dou

24 March 2024 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
XLXingjian Li
Xingjian Li
PYPengkun Yang
Pengkun Yang
YGYangcheng Gu
Yangcheng Gu
XZXueying Zhan
Xueying Zhan
TWTianyang Wang
Tianyang Wang
CXChengzhong Xu
Chengzhong Xu
Uncertainty estimation for unlabeled data is crucial to active learning. With a deep neural network employed as the backbone model, the data selection process is highly challenging due to the potential over-confidence of the model inference. Existing methods resort to special learning fashions (e.g. adversarial) or auxiliary models to address this challenge. This tends to result in complex and inefficient pipelines, which would render the methods impractical. In this work, we propose a novel algorithm that leverages noise stability to estimate data uncertainty. The key idea is to measure the output derivation from the original observation when the model parameters are randomly perturbed by noise. We provide theoretical analyses by leveraging the small Gaussian noise theory and demonstrate that our method favors a subset with large and diverse gradients. Our method is generally applicable in various tasks, including computer vision, natural language processing, and structural data analysis. It achieves competitive performance compared against state-of-the-art active learning baselines.

27 October 2025
Verified
Conference Paper
Open access
XXXi Xiao
Xi Xiao
YZYunbei Zhang
Yunbei Zhang
XLXingjian Li
Xingjian Li
TWTianyang Wang
Tianyang Wang
XWXiao Wang
Xiao Wang
MXMin Xu
Min Xu

30 November 2023 in IEEE Transactions on Neural Networks and Learning Systems
Verified
Journal Article
Closed access
HHHang Hua
Hang Hua
XLXingjian Li
Xingjian Li
DDDejing Dou
Dejing Dou
CXCheng-Zhong Xu
Cheng-Zhong Xu
JLJiebo Luo
Jiebo Luo
The advent of large-scale pretrained language models (PLMs) has contributed greatly to the progress in natural language processing (NLP). Despite its recent success and wide adoption, fine-tuning a PLM often suffers from overfitting, which leads to poor generalizability due to the extremely high complexity of the model and the limited training samples from downstream tasks. To address this problem, we propose a novel and effective fine-tuning framework, named l ayerwise n oise s tability r egularization ( LNSR ). Specifically, our method perturbs the input of neural networks with the standard Gaussian or in-manifold noise in the representation space and regularizes each layer’s output of the language model. We provide theoretical and experimental analyses to prove the effectiveness of our method. The empirical results show that our proposed method outperforms several state-of-the-art algorithms, such as $\text{L}^2$ norm and start point (L2-SP), Mixout, FreeLB, and smoothness inducing adversarial regularization and Bregman proximal point optimization (SMART). In addition to evaluating the proposed method on relatively simple text classification tasks, similar to the prior works, we further evaluate the effectiveness of our method on more challenging question-answering (QA) tasks. These tasks present a higher level of difficulty, and they provide a larger amount of training examples for tuning a well-generalized model. Furthermore, the empirical results indicate that our proposed method can improve the ability of language models to domain generalization.

8 September 2022 in ACM Transactions on Knowledge Discovery From Data
Verified
Journal Article
Open access
HXHaoyi Xiong
Haoyi Xiong
RWRuosi Wan
Ruosi Wan
Jian ZhaoJian Zhao
Jian Zhao
ZCZeyu Chen
Zeyu Chen
XLXingjian Li
Xingjian Li
JHJun Huan
Jun Huan
Regularization that incorporates the linear combination of empirical loss and explicit regularization terms as the loss function has been frequently used for many machine learning tasks. The explicit regularization term is designed in different types, depending on its applications. While regularized learning often boost the performance with higher accuracy and faster convergence, the regularization would sometimes hurt the empirical loss minimization and lead to poor performance. To deal with such issues in this work, we propose a novel strategy, namely Gr adients O rthogonal D ecomposition ( GrOD ), that improves the training procedure of regularized deep learning. Instead of linearly combining gradients of the two terms, GrOD re-estimates a new direction for iteration that does not hurt the empirical loss minimization while preserving the regularization affects, through orthogonal decomposition. We have performed extensive experiments to use GrOD improving the commonly used algorithms of transfer learning [ 2 ], knowledge distillation [ 3 ], and adversarial learning [ 4 ]. The experiment results based on large datasets, including Caltech 256 [ 5 ], MIT indoor 67 [ 6 ], CIFAR-10 [ 7 ], and ImageNet [ 8 ], show significant improvement made by GrOD for all three algorithms in all cases.