Yan Sun

Yan Sun

Auto-generated profile, based on publicly available information

Yan Sun

School of Computer Science, University of Sydney, Sydney, Australia

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
11 November 2024 in ACM Computing Surveys
Verified
Journal Article
Open access
LSLi Shen
Li Shen
YSYan Sun
Yan Sun
ZYZhiyuan Yu
Zhiyuan Yu
LDLiang Ding
Liang Ding
Xinmei TianXinmei Tian
Xinmei Tian
DTDacheng Tao
Dacheng Tao
The field of deep learning has witnessed significant progress in recent times, particularly in areas such as computer vision (CV), natural language processing (NLP), and speech. The use of large-scale models trained on vast amounts of data holds immense promise for practical applications, enhancing industrial productivity and facilitating social development. However, it suffers extremely from the unstable training process and stringent requirements of computational resources. With the increasing demands on the adaption of computational capacity, though numerous studies have explored the efficient training field to a certain extent, a comprehensive summarization/guideline on those general acceleration techniques of training large-scale deep learning models is still much anticipated. In this survey, we present a detailed review of the general techniques for training acceleration. We consider the fundamental update formulation and split its basic components into five main perspectives: (1) “data-centric,” including dataset regularization, data sampling, and data-centric curriculum learning techniques, which can significantly reduce the computational complexity of the data samples; (2) “model-centric,” including acceleration of basic modules, compression training, model initialization, and model-centric curriculum learning techniques, which focus on accelerating the training via reducing the calculations on parameters and providing better initialization; (3) “optimization-centric,” including the selection of learning rate, the employment of large batch size, the designs of efficient objectives, and model average techniques, which pay attention to the training policy and improving the generality for the large-scale models; (4) “budgeted training,” including some distinctive acceleration methods on source-constrained situations, e.g., for limitation on the total iterations; and (5) “system-centric,” including some efficient distributed frameworks and open source libraries that provide adequate hardware support for the implementation of the above-mentioned acceleration algorithms. By presenting this comprehensive taxonomy, our survey presents a comprehensive review to understand the general mechanisms within each component and their joint interaction. Meanwhile, we further provide a detailed analysis and discussion of future works on the development of general acceleration techniques, which could inspire us to re-think and design novel efficient paradigms. Overall, we hope that this survey will serve as a valuable guideline for general efficient training.

1 August 2023 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
YSYan Sun
Yan Sun
LSLi Shen
Li Shen
HSHao Sun
Hao Sun
LDLiang Ding
Liang Ding
DTDacheng Tao
Dacheng Tao
Adaptive optimization has achieved notable success for distributed learning while extending adaptive optimizer to federated Learning (FL) suffers from severe inefficiency, including (i) rugged convergence due to inaccurate gradient estimation in global adaptive optimizer; (ii) client drifts exacerbated by local over-fitting with the local adaptive optimizer. In this work, we propose a novel momentum-based algorithm via utilizing the global gradient descent and locally adaptive amended optimizer to tackle these difficulties. Specifically, we incorporate a locally amended technique to the adaptive optimizer, named Federated Local ADaptive Amended optimizer ( FedLADA ), which estimates the global average offset in the previous communication round and corrects the local offset through a momentum-like term to further improve the empirical training speed and mitigate the heterogeneous over-fitting. Theoretically, we establish the convergence rate of FedLADA with a linear speedup property on the non-convex case under the partial participation settings. Moreover, we conduct extensive experiments on the real-world dataset to demonstrate the efficacy of our proposed FedLADA , which could greatly reduce the communication rounds and achieves higher accuracy than several baselines.

24 December 2025 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
YSYan Sun
Yan Sun
LSLi Shen
Li Shen
DTDacheng Tao
Dacheng Tao
As two mainstream frameworks in federated learning (FL), both centralized and decentralized approaches have shown great application value in practical scenarios. However, existing studies do not provide sufficient evidence and clear guidance for analysis of which performs better in the FL community. Although decentralized methods have been proven to approach the comparable convergence of centralized with less communication, their test performance always falls short of expectations in empirical studies. To comprehensively and fairly compare their efficiency gaps in FL, in this paper, we explore their stability and generalization efficiency. Specifically, we prove that on the general smooth non-convex objectives, 1) centralized FL (CFL) always generalizes better than decentralized FL (DFL); 2) CFL achieves the best performance via adopting partial participation instead of full participation; and, 3) there is a necessary requirement for the topology in DFL to avoid performance collapse as the training scale increases. We also conduct extensive experiments on several common setups in FL to validate that our theoretical analysis is consistent with experimental phenomena and contextually valid in several general and practical scenarios.

24 March 2026 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
MZMiaoxi Zhu
Miaoxi Zhu
YSYan Sun
Yan Sun
LSLi Shen
Li Shen
BDBo Du
Bo Du
DTDacheng Tao
Dacheng Tao
Minimax optimization is gaining increasing attention in modern machine learning applications. Driven by large-scale models and massive volumes of data collected from edge devices, as well as the concern to preserve client privacy, distributed minimax optimization algorithms become popular, such as Local Stochastic Gradient Descent Ascent (Local-SGDA), and Local Decentralized SGDA (Local-DSGDA). While most existing research on distributed minimax algorithms focuses on convergence rates and communication efficiency, their generalization performance remains largely unexplored, whereas generalization ability is a pivotal indicator for evaluating the holistic performance of a model when fed with unknown data. In this paper, we propose the stability-based generalization analytical framework for Distributed-SGDA, which unifies two popular distributed minimax algorithms including Local-SGDA and Local-DSGDA, and conduct a comprehensive analysis of stability error, generalization gap, and population risk across different metrics under various settings, e.g., (S)C-(S)C, PL-SC, and NC-NC cases. Our theoretical results reveal the trade-off between the generalization gap and optimization error and suggest hyperparameters choice to obtain the optimal population risk. Numerical experiments for Local-SGDA and Local-DSGDA validate the theoretical results.

9 July 2026 in EGUsphere
Verified
Preprint
Open access
PDF file
XLXueying Li
Xueying Li
YSYan Sun
Yan Sun
Xihui GuXihui Gu
Xihui Gu
NWNiko Wanders
Niko Wanders
BSBridget R. Scanlon
Bridget R. Scanlon
LSLouise Slater
Louise Slater
Terrestrial water storage (TWS), comprising all surface and subsurface water components, is a key indicator of water availability. The Gravity Recovery and Climate Experiment (GRACE) satellite mission provides large-scale estimates of TWS anomalies (TWSA), but its coarse spatial resolution (3°, approximately 300 km) limits the analysis of hydrologic processes at sub-regional scales. Using a physically-guided deep learning framework, we downscale TWSA from the original 3° GRACE mascons to 0.1° (approximately 10 km) across China, generating a standard version (2002–2019) with comprehensive observations used for model constraints and independent evaluation and an extended version (2020–2023) to support more recent hydrologic analyses. The downscaled TWSA preserves large-scale GRACE signals at the 3° grid scale (median correlation coefficient (CC): 0.95; root-mean-square error (RMSE): 1.38 cm) and basin scale (median CC: 0.94; RMSE: 1.72 cm), with a low median uncertainty (0.88 cm) across China. Its reliability is supported by high consistency with physically informed TWSA spatial patterns at the 0.1° resolution (median CC: 0.91) and internally consistent water balance closure beyond the native GRACE resolution (median CC: 0.80; RMSE: 1.44 cm). Evaluation against independent observations demonstrates that the downscaled TWSA agrees well with groundwater variations in intensively irrigated regions (CC: 0.65 for irrigation intensity > 50 %) and annual glacier elevation change in cryospheric areas (CC: 0.97). The datasets improve fine-scale characterization of TWS variability and associated hydrologic processes in China, and can be used as a reference for evaluating performance of high-resolution hydrologic models. The two versions of the dataset are available at https://doi.org/10.5281/zenodo.19502906.

20 January 2026 in IEEE Transactions on Networking
Verified
Journal Article
Closed access
YSYifan Shi
Yifan Shi
YLYingqi Liu
Yingqi Liu
YSYan Sun
Yan Sun
ZLZihao Lin
Zihao Lin
XWXueqian Wang
Xueqian Wang
DTDacheng Tao
Dacheng Tao
Compared with general Federated Learning (FL), Decentralized FL (DFL) has diminished central communication burdens and lower risks of disruption. To be compatible with real-world scenarios, some existing works produce several local personalized models rather than a universal model for all edge devices. However, they still suffer from inferior performance due to the full model aggregation in heterogeneous datasets. Therefore, we propose a DFL framework DFedMDC through partial model personalization, which can adapt to resource-heterogeneous environments (e.g., the Internet of Things (IoT)). Specifically, it personalizes the “right” components of local models and trains the shared model ui and personal model vi alternately in each client i. To further accelerate the convergence process, we propose DFedSMDC with properly directed noise perturbation into the gradient update. Theoretically, we provide convergence analysis of both algorithms in the general non-convex setting. It can shed light on how vital factors affect the convergence rate, such as these alternate updates of partial models ui, vi, data heterogeneity δ2 as well as various communication topologies (characterized by the spectral gap 1 − λ). Empirically, we confirm the state-of-the-art (SOTA) superiority of the proposed methods on several real-world datasets with various data distributions, relative to both SOTA personalized FL (PFL) and DFL baselines.