Changjian Shui

Changjian Shui

Auto-generated profile, based on publicly available information

Changjian Shui

Vector Institute, Toronto, ON, Canada

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
24 September 2022 in Knowledge-Based Systems
Verified
Journal Article
Closed access
CSChangjian Shui
Changjian Shui
QCQi Chen
Qi Chen
JWJun Wen
Jun Wen
FZFan Zhou
Fan Zhou
CGChristian Gagné
Christian Gagné
BWBoyu Wang
Boyu Wang

1 June 2021 in Knowledge-Based Systems
Verified
Journal Article
Closed access
FZFan Zhou
Fan Zhou
CSChangjian Shui
Changjian Shui
SYShichun Yang
Shichun Yang
BHBincheng Huang
Bincheng Huang
BWBoyu Wang
Boyu Wang
BCBrahim Chaib-Draa
Brahim Chaib-Draa

1 January 2022 in Machine Learning
Verified
Journal Article
Open access
PDF file
CSChangjian Shui
Changjian Shui
BWBoyu Wang
Boyu Wang
CGChristian Gagné
Christian Gagné
A crucial aspect of reliable machine learning is to design a deployable system for generalizing new related but unobserved environments. Domain generalization aims to alleviate such a prediction gap between the observed and unseen environments. Previous approaches commonly incorporated learning the invariant representation for achieving good empirical performance. In this paper, we reveal that merely learning the invariant representation is vulnerable to the related unseen environment. To this end, we derive a novel theoretical analysis to control the unseen test environment error in the representation learning, which highlights the importance of controlling the smoothness of representation. In practice, our analysis further inspires an efficient regularization method to improve the robustness in domain generalization. The proposed regularization is orthogonal to and can be straightforwardly adopted in existing domain generalization algorithms that ensure invariant representation learning. Empirical results show that our algorithm outperforms the base versions in various datasets and invariance criteria.

30 January 2025 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
GXGezheng Xu
Gezheng Xu
LYLi Yi
Li Yi
PXPengcheng Xu
Pengcheng Xu
JLJiaqi Li
Jiaqi Li
RPRuizhi Pu
Ruizhi Pu
CLCharles Ling
Charles Ling
Recent source-free domain adaptation (SFDA) methods have focused on learning meaningful cluster structures in feature space, successfully adapting the knowledge from the source domain to the unlabeled target domain without accessing the private source data. However, existing methods rely on pseudo-labels generated by source models that can be noisy due to domain shift, presenting a significant challenge to their efficacy. In this paper, we study SFDA from the perspective of learning with label noise (LLN) and prove that the label noise in SFDA, unlike in conventional LLN scenarios, follows a different distribution assumption. This discrepancy renders some existing LLN methods less effective in SFDA. To address this issue and comprehensively improve adaptation performance, we tackle label noise in SFDA from two perspectives. First, we demonstrate that the early-time training phenomenon (ETP), previously observed in LLN settings, still exists in SFDA. Hence, we introduce a simple yet effective approach to leveraging ETP to improve current SFDA algorithms. Second, we propose a noise and variance control module, mitigating the label noise discrepancy between SFDA and LLN and enhancing the effectiveness of LLN methods in SFDA. Extensive empirical evaluation and analysis of four benchmarks show that our methods substantially outperform existing baselines.

26 June 2024 in IEEE Transactions on Knowledge and Data Engineering
Verified
Journal Article
Closed access
JLJiaqi Li
Jiaqi Li
YLYuanhao Lai
Yuanhao Lai
RWRui Wang
Rui Wang
CSChangjian Shui
Changjian Shui
SSSabyasachi Sahoo
Sabyasachi Sahoo
FZFan Zhou
Fan Zhou
Continual learning aims to learn a series of tasks sequentially without forgetting the knowledge acquired from the previous ones. In this work, we propose the Hessian Aware Low-Rank Perturbation algorithm for continual learning. By modeling the parameter transitions along the sequential tasks with the weight matrix transformation, we propose to apply the low-rank approximation on the task-adaptive parameters in each layer of the neural networks. Specifically, we theoretically demonstrate the quantitative relationship between the Hessian and the proposed low-rank approximation. The approximation ranks are then globally determined according to the marginal change of the empirical loss estimated by the layer-specific gradient and low-rank approximation error. Furthermore, we control the model capacity by pruning less important parameters to diminish the parameter growth. We conduct extensive experiments on various benchmarks, including a dataset with large-scale tasks, and compare our method against some recent state-of-the-art methods to demonstrate the effectiveness and scalability of our proposed method. Empirical results show that our method performs better on different benchmarks, especially in achieving task order robustness and handling the forgetting issue.

13 April 2023 in IEEE Transactions on Knowledge and Data Engineering
Verified
Journal Article
Closed access
CSChangjian Shui
Changjian Shui
RPRuizhi Pu
Ruizhi Pu
GXGezheng Xu
Gezheng Xu
JWJun Wen
Jun Wen
FZFan Zhou
Fan Zhou
BWBoyu Wang
Boyu Wang
In this article, we present an analysis of unsupervised domain adaptation with a series of theoretical and algorithmic results. We derive a novel Rényi- $\alpha$ divergence-based generalization bound, which is tailored to domain adaptation algorithms with arbitrary loss functions in a stochastic setting. Moreover, our theoretical results provide new insights into the assumptions for successful domain adaptation: the closeness between the conditional distributions of the domains and the Lipschitzness on the source domain. With these assumptions, we reveal the following: if their conditional generation distributions are close, the Lipschitzness property of the target domain can be transferred from the Lipschitzness on the source domain, without knowing the exact target distribution. Motivated by our analysis and assumptions, we further derive practical principles for deep domain adaptation: 1) Rényi-2 adversarial training for marginal distributions matching and 2) Lipschitz regularization for the classifier. Our experimental results on both synthetic and real-world datasets support our theoretical findings and the practical efficiency of the proposed principles.