Zhiwei Yao

Zhiwei Yao

Auto-generated profile, based on publicly available information

Zhiwei Yao

School of Computer Science and Technology, University of Science and Technology of China, Hefei, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
28 February 2024 in IEEE Transactions on Mobile Computing
Verified
Journal Article
Closed access
ZYZhiwei Yao
Zhiwei Yao
JLJianchun Liu
Jianchun Liu
HXHongli Xu
Hongli Xu
LWLun Wang
Lun Wang
CQChen Qian
Chen Qian
YLYunming Liao
Yunming Liao
Federated semi-supervised learning (FSSL) has been proposed to address the insufficient labeled data problem by training models with pseudo-labeling. In previous FSSL systems, a single global model is always trained without an equivalent generalization ability for the clients under the non-IID setting. Accordingly, model personalization methods have been proposed to overcome this problem. Intuitively, seeking labeling assistance from other clients with similar data distribution, i.e., model migration, can effectively improve the personalization on the clients with scarce labeled data. However, previous works require to migrate a pre-fixed number of models among the clients, causing unnecessary resource waste and accuracy degradation due to resource heterogeneity. Considering that the number of model migrations and the quality of pseudo-labels have a significant impact on the training performance (e.g., efficiency and accuracy), we propose a novel personalized FSSL system, called Ferrari, to boost the efficiency of pseudo-labeling and training accuracy through adaptive model migrations among the clients. Specifically, Ferrari first generates the similarity-based ranking using a Gaussian KD-Tree, considering the varied data distributions among the clients. Combined with the ranking and clients’ heterogeneous resource constraints, Ferrari then adaptively determines the proper model migration policy and confidence thresholds for high-quality pseudo-labeling and personalized training for clients. Extensive experiments on a physical platform show that Ferrari provides a 1.2 $\sim 5.5\times$ speedup without sacrificing model accuracy, compared to existing methods.

4 December 2024
Verified
Conference Paper
Open access
YLYunming Liao
Yunming Liao
YXYang Xu
Yang Xu
HXHongli Xu
Hongli Xu
ZYZhiwei Yao
Zhiwei Yao
LHLiusheng Huang
Liusheng Huang
CQChunming Qiao
Chunming Qiao
Mobile devices contribute more than half of the world's web traffic, providing massive and diverse data for powering various federated learning (FL) applications. In order to avoid the communication bottleneck on the parameter server (PS) and accelerate the training of large-scale models on resource-constraint workers in edge computing (EC) system, we propose a novel split federated learning (SFL) framework, termed ParallelSFL. Concretely, we split an entire model into a bottom submodel and a top submodel, and divide participating workers into multiple clusters, each of which collaboratively performs the SFL training procedure and exchanges entire models with the PS. However, considering the statistical and system heterogeneity in edge systems, it is challenging to arrange suitable workers to specific clusters for efficient model training. To address these challenges, we carefully develop an effective clustering strategy by optimizing a utility function related to training efficiency and model accuracy. Specifically, ParallelSFL partitions workers into different clusters under the heterogeneity restrictions, thereby promoting model accuracy as well as training efficiency. Meanwhile, ParallelSFL assigns diverse and appropriate local updating frequencies for each cluster to further address system heterogeneity. Extensive experiments are conducted on a physical platform with 80 NVIDIA Jetson devices, and the experimental results show that ParallelSFL can reduce the traffic consumption by at least 21%, speed up the model training by at least 1.36X, and improve model accuracy by at least 5% in heterogeneous scenarios, compared to the baselines.

6 November 2024 in IEEE Transactions on Mobile Computing
Verified
Journal Article
Closed access
JLJianchun Liu
Jianchun Liu
JLJun Liu
Jun Liu
HXHongli Xu
Hongli Xu
YLYunming Liao
Yunming Liao
ZYZhiwei Yao
Zhiwei Yao
CQChen Qian
Chen Qian
Federated learning (FL) is an efficient distributed learning method that facilitates collaborative model training among multiple edge devices (or clients). However, current research always assumes that clients have access to ground-truth data for training, which is unrealistic in practice because of a lack of expertise. Semi-supervised federated learning (SSFL) has been proposed in many existing works to address this problem, which always adopts a fixed model architecture for training, bringing two main problems with varying amounts of pseudo-labeled data. First, the shallow model cannot have the capability to fit the increasing pseudo-labeled data, leading to poor training performance. Second, the large model suffers from an overfitting problem when exploiting a few labeled data samples in SSFL, and also requires tremendous resource ( e.g. , computation and communication) costs. To tackle these problems, we propose a novel framework, called STAR , which adopts progressive training to enhance model training in SSFL. Specifically, STAR gradually increases the model depth through adding the sub-module ( e.g. , one or several layers) from a shallow model, and performs pseudo-labeling for unlabeled data with a specialized confidence threshold simultaneously. Then, we propose an efficient algorithm to determine the appropriate model depth for each client with varied resource budgets and the proper confidence threshold for pseudo-labeling in SSFL. Our proposed framework STAR innovatively applies progressive training to SSFL, which significantly contributes to the advancement of the FL field. STAR has been evaluated through extensive experiments, and the results demonstrate its high effectiveness. For instance, STAR can reduce the bandwidth consumption by about 40%, and achieve an average accuracy improvement of around 9.8% compared with the baselines, on CIFAR10. Besides, STAR achieves about 2.2× speedup compared to the baselines on ImageNet100.

19 March 2025 in IEEE Transactions on Networking
Verified
Journal Article
Closed access
ZYZhiwei Yao
Zhiwei Yao
JQJi Qi
Ji Qi
YXYang Xu
Yang Xu
YLYunming Liao
Yunming Liao
HXHongli Xu
Hongli Xu
LWLun Wang
Lun Wang
Federated learning (FL) has recently gained tremendous attention in edge computing and the Internet of Things, due to its capability of enabling clients to perform model training at the network edge or end devices (i.e., clients). However, these end devices are usually resource-constrained without the ability to train large-scale models. In order to accelerate the training of large-scale models on these devices, we incorporate Split Learning (SL) into Federated Learning (FL), and propose a novel FL framework, termed PairingFL. Specifically, we split a full model into a bottom model and a top model, and arrange participating clients into pairs, each of which collaboratively trains the two partial models as one client does in typical FL. Driven by the advantages of SL and FL, PairingFL is able to relax the computation burden on clients and protect model privacy. However, considering the features of system and statistical heterogeneity in edge networks, it is challenging to pair the clients by carefully developing the strategies of client partitioning and matching for efficient model training. To this end, we first theoretically analyze the convergence property of PairingFL, and obtain a convergence upper bound. Guided by this, we then design a greedy and efficient algorithm, which makes the joint decision of client partitioning and matching, so as to well balance the trade-off between convergence rate and model accuracy. The performance of PairingFL is evaluated through extensive simulation experiments. The experimental results demonstrate that PairingFL can speed up the training process by $4.6\times $ compared to baselines when achieving the corresponding convergence accuracy.

6 November 2025 in IEEE Transactions on Mobile Computing
Verified
Journal Article
Closed access
ZXZihuai Xu
Zihuai Xu
YXYang Xu
Yang Xu
HXHongli Xu
Hongli Xu
YLYunming Liao
Yunming Liao
ZYZhiwei Yao
Zhiwei Yao
ZXZuan Xie
Zuan Xie
Considering the hardware-friendly characteristics, structured pruning has emerged as an effective solution to reduce the resource requirements of large language models (LLMs) on resource-constrained devices. Since pruning a certain number of parameters will often reduce the model accuracy, fine-tuning is usually required to recover performance loss. However, fine-tuning relies on high device power and substantial data, making it unsuitable for on-device applications. Recent approaches propose post-training structured pruning techniques that do not require fine-tuning, following three granularities, i.e., neurons, attention heads, and layers. Unfortunately, the previous solutions still suffer from the high memory overhead of substructures’ importance evaluation at different granularities and the problem of only applying to models with specific structures, which limit their scope of applications. This paper proposes COMP, a lightweight and general post-training structured hybrid-granularity pruning method designed for on-device applications. COMP begins by assessing the importance of model layers and neurons individually using distributional distance and the matrix condition number. Subsequently, COMP performs fine-grained neuron pruning and simultaneously determines whether to apply coarse-grained layer pruning or not by comparing the outcomes of the two granularity strategies. Furthermore, COMP implements mask tuning to restore model accuracy without additional fine-tuning, minimizing dependency on external data resources and device power. Experimental results demonstrate that COMP performs well across models with various architectures. When pruning 20% of LLaMA-2-13B parameters, COMP requires less than 8 GB of memory, maintaining more than 90% model accuracy. Meanwhile, COMP improves performance by 3.54% compared to LLM-Pruner, while reducing memory overhead by 90%.

31 October 2025 in IEEE Transactions on Mobile Computing
Verified
Journal Article
Closed access
YZYing Zhu
Ying Zhu
YXYang Xu
Yang Xu
HXHongli Xu
Hongli Xu
YLYunming Liao
Yunming Liao
ZYZhiwei Yao
Zhiwei Yao
LHLiusheng Huang
Liusheng Huang
Recently, aided by edge computing (EC), federated meta-learning (FML) has been proposed to adequately utilize personalized data on different edge devices by incorporating meta-learning with federated learning (FL). Unlike conventional FL which conducts several stochastic gradient descent (SGD) steps to directly update the local model in each training round, FML first performs some SGD steps to obtain a reference model and then performs a meta-step to update the local model along the direction towards the reference model. In this way, FML can identify common patterns and structures across different clients (i.e., edge devices) and quickly adapt to new data distributions, which exhibits its advantage in dealing with statistical heterogeneity inherent in FL. However, existing FML approaches struggle to effectively cope with two key challenges: i) the disparity in dataset informativeness among clients with statistically heterogeneous local data; and ii) the disparity in computing capacities among clients with heterogeneous hardware configurations. Herein, we propose a heterogeneity-aware FML framework, termed HFML, which explores adjusting the execution frequencies of SGD step and meta-step in each training round to address the above statistical and system heterogeneity challenges simultaneously. We introduce usable information (UI) as a new metric in HFML to quantify dataset informativeness, and theoretically analyze the influence of the execution frequencies of SGD steps and meta-steps on model performance and convergence speed. Based on the theoretical analysis, we develop an efficient optimization algorithm to jointly adjust the execution frequencies for different clients to handle the heterogeneity issues, which contributes to accelerating model training and enhancing personalization performance. Experimental results demonstrate that HFML can speed up training by up to 4.02× and improve model accuracy by an average of 11.5% compared to the baselines under heterogeneous settings.