Zhiyuan Wu

Zhiyuan Wu

Auto-generated profile, based on publicly available information

Zhiyuan Wu

Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
26 June 2023 in IEEE Transactions on Parallel and Distributed Systems
Verified
Journal Article
Closed access
ZWZhiyuan Wu
Zhiyuan Wu
SSSheng Sun
Sheng Sun
YWYuwei Wang
Yuwei Wang
MLMin Liu
Min Liu
QPQuyang Pan
Quyang Pan
Bo GaoBo Gao
Bo Gao
The growing interest in intelligent services and privacy protection for mobile devices has given rise to the widespread application of federated learning in Multi-access Edge Computing (MEC). Diverse user behaviors call for personalized services with heterogeneous Machine Learning (ML) models on different devices. Federated Multi-task Learning (FMTL) is proposed to train related but personalized ML models for different devices, whereas previous works suffer from excessive communication overhead during training and neglect the model heterogeneity among devices in MEC. Introducing knowledge distillation into FMTL can simultaneously enable efficient communication and model heterogeneity among clients, whereas existing methods rely on a public dataset, which is impractical in reality. To tackle this dilemma, Fed erated Mult I -task Distillation for Multi-access Edge C ompu T ing (FedICT) is proposed. FedICT direct local-global knowledge aloof during bi-directional distillation processes between clients and the server, aiming to enable multi-task clients while alleviating client drift derived from divergent optimization directions of client-side local models. Specifically, FedICT includes Federated Prior Knowledge Distillation (FPKD) and Local Knowledge Adjustment (LKA). FPKD is proposed to reinforce the clients' fitting of local data by introducing prior knowledge of local data distributions. Moreover, LKA is proposed to correct the distillation loss of the server, making the transferred local knowledge better match the generalized representation. Extensive experiments on three datasets demonstrate that FedICT significantly outperforms all compared benchmarks in various data heterogeneous and model architecture settings, achieving improved accuracy with less than 1.2% training communication overhead compared with FedAvg and no more than 75% training communication round compared with FedGKT in all considered scenarios.

5 February 2024 in IEEE Transactions on Mobile Computing
Verified
Journal Article
Closed access
ZWZhiyuan Wu
Zhiyuan Wu
SSSheng Sun
Sheng Sun
YWYuwei Wang
Yuwei Wang
MLMin Liu
Min Liu
KXKe Xu
Ke Xu
JLJinda Lu
Jinda Lu
Edge Intelligence (EI) allows Artificial Intelligence (AI) applications to run at the edge, where data analysis and decision-making can be performed in real-time and close to data sources. To protect data privacy and unify data silos distributed among end devices in EI, Federated Learning (FL) is proposed for collaborative training of shared AI models across multiple devices without compromising data privacy. However, the prevailing FL approaches cannot guarantee model generalization and adaptation on heterogeneous clients. Recently, Personalized Federated Learning (PFL) has drawn growing awareness in EI, as it enables a productive balance between local-specific training requirements inherent in devices and global-generalized optimization objectives for satisfactory performance. However, most existing PFL methods are based on the Parameters Interaction-based Architecture (PIA) represented by FedAvg, which suffers from unaffordable communication burdens due to large-scale parameters transmission between devices and the edge server. In contrast, Logits Interaction-based Architecture (LIA) allows to update model parameters with logits transfer and gains the advantages of communication lightweight and heterogeneous on-device model allowance compared to PIA. Nevertheless, previous LIA methods attempt to achieve satisfactory performance either relying on unrealistic public datasets or increasing communication overhead for additional information transmission other than logits. To tackle this dilemma, we propose a knowledge cache-driven PFL architecture, named FedCache, which reserves a knowledge cache on the server for fetching personalized knowledge from the samples with similar hashes to each given on-device sample. During the training phase, ensemble distillation is applied to on-device models for constructive optimization with personalized knowledge transferred from the server-side knowledge cache. Empirical experiments on four datasets demonstrate that FedCache achieves comparable performance with state-of-art PFL approaches, with more than two orders of magnitude improvements in communication efficiency. Our code and DEMO are available at https://github.com/wuzhiyuan2000/FedCache .

22 February 2024 in ACM Transactions on Intelligent Systems and Technology
Verified
Journal Article
Open access
ZWZhiyuan Wu
Zhiyuan Wu
SSSheng Sun
Sheng Sun
YWYuwei Wang
Yuwei Wang
MLMin Liu
Min Liu
QPQuyang Pan
Quyang Pan
QLQingxiang Liu
Qingxiang Liu
Federated learning (FL) is a privacy-preserving machine learning paradigm in which the server periodically aggregates local model parameters from cli ents without assembling their private data. Constrained communication and personalization requirements pose severe challenges to FL. Federated distillation (FD) is proposed to simultaneously address the above two problems, which exchanges knowledge between the server and clients, supporting heterogeneous local models while significantly reducing communication overhead. However, most existing FD methods require a proxy dataset, which is often unavailable in reality. A few recent proxy-data-free FD approaches can eliminate the need for additional public data, but suffer from remarkable discrepancy among local knowledge due to client-side model heterogeneity, leading to ambiguous representation on the server and inevitable accuracy degradation. To tackle this issue, we propose a proxy-data-free FD algorithm based on distributed knowledge congruence (FedDKC). FedDKC leverages well-designed refinement strategies to narrow local knowledge differences into an acceptable upper bound, so as to mitigate the negative effects of knowledge incongruence. Specifically, from perspectives of peak probability and Shannon entropy of local knowledge, we design kernel-based knowledge refinement (KKR) and searching-based knowledge refinement (SKR) respectively, and theoretically guarantee that the refined-local knowledge can satisfy an approximately-similar distribution and be regarded as congruent. Extensive experiments conducted on three common datasets demonstrate that our proposed FedDKC significantly outperforms the state-of-the-art on various heterogeneous settings while evidently improving the convergence speed.

18 June 2024 in IEEE Transactions on Mobile Computing
Verified
Journal Article
Closed access
SSSheng Sun
Sheng Sun
ZZZengqi Zhang
Zengqi Zhang
QPQuyang Pan
Quyang Pan
MLMin Liu
Min Liu
YWYuwei Wang
Yuwei Wang
ZWZhiyuan Wu
Zhiyuan Wu
Federated Learning (FL) is an emerging distributed learning paradigm with the privacy-preserving advantage of collaboratively training a shared model across multiple participants. Considering the prevailing device heterogeneity circumstance in practice, asynchronous interaction is introduced into FL to break the straggler barrier of synchronization, at the cost of significant accuracy degradation derived from model staleness. Although quite a few works attempt to partially mitigate the detrimental impact after occurring staleness issue, they neglect to control the overall staleness degree of clients-side local models from the whole training perspective, resulting in highly-stale models for aggregation and slow convergence speed. To this end, we propose a Staleness-Controlled Asynchronous Federated Learning (SC-AFL) method, which enables to restrict staleness degree of local models within a certain bound via dynamically tuning the aggregated strategy of each round, aiming to strike a good balance between accuracy guarantee and convergence acceleration. Specifically, we leverage the Lyapunov optimization framework to decouple the troublesome round-coupling problem into the single-round sequential solving problem, and further develop a deterministic algorithm that selects the aggregated number of clients to minimize training time under the constraint of maintaining staleness queue stability. Besides, we derive the theoretical convergence analysis of SC-AFL and also present the upper bound of the performance gap with the optimum. Extensive experiments on three datasets demonstrate the superiority of SC-AFL in terms of time-to-accuracy speedup on both IID and Non-IID data distributions, achieving a good balance between model accuracy and convergence efficiency in AFL system.

21 October 2024
Verified
Conference Paper
Open access
XJXuefeng Jiang
Xuefeng Jiang
SSSheng Sun
Sheng Sun
JLJia Li
Jia Li
JXJingjing Xue
Jingjing Xue
RLRunhan Li
Runhan Li
MLMin Liu
Min Liu
Recently, federated learning (FL) has achieved wide successes for diverse privacy-sensitive applications without sacrificing the sensitive private information of clients. However, the data quality of client datasets can not be guaranteed since corresponding annotations of different clients often contain complex label noise of varying degrees, which inevitably causes the performance degradation. Intuitively, the performance degradation is dominated by clients with higher noise rates since their trained models contain more misinformation from data, thus it is necessary to devise an effective optimization scheme to mitigate the negative impacts of these noisy clients. In this work, we propose a two-stage framework FedELC to tackle this complicated label noise issue. The first stage aims to guide the detection of noisy clients with higher label noise, while the second stage aims to correct the labels of noisy clients' data via an end-to-end label correction framework which is achieved by learning possible ground-truth labels of noisy clients' datasets via back propagation. We implement sixteen related methods and evaluate five datasets with three types of complicated label noise scenarios for a comprehensive comparison. Extensive experimental results demonstrate our proposed framework achieves superior performance than its counterparts for different scenarios. Additionally, we effectively improve the data quality of detected noisy clients' local datasets with our label correction framework. The code is available at https://github.com/Sprinter1999/FedELC.

1 September 2025 in Journal of Computer Science and Technology
Verified
Journal Article
Closed access
PDF file
ZWZhi-Yuan Wu
Zhi-Yuan Wu
THTian-Liu He
Tian-Liu He
SSSheng Sun
Sheng Sun
YWYu-Wei Wang
Yu-Wei Wang
MLMin Liu
Min Liu
XJXue-Feng Jiang
Xue-Feng Jiang