31 August 2022 in IEEE Internet of Things Journal
Federated learning is a popular framework designed to perform the distributed machine learning while protecting client privacy. However, the heterogeneous data distribution in real-world environments makes it difficult to converge when performing model training. In this article, we propose the federated gradient scheduling (FedGS), an improved historical gradient sampling utilization method for optimizers that utilize historical gradients in the federated learning to alleviate the instability problem of historical gradient information due to non-IID. FedGS consists of two main steps to improve federated learning performance. First, clustering uses clients’ label distributions, which relabel clients and their submitting gradients. Second, sampling gradient clusters to generate an IID gradient set, which feeds to optimizers to derive valid momentum information. Besides, we introduce differential privacy to collaborate with FedGS to enhance clients’ privacy protection strength. Compared to previous non-IID federated learning solutions, our method can achieve greater resistance to temporal non-IID. Moreover, experiments show that FedGS can achieve faster convergence and performance gain of up to 10% over existing state-to-art methods in some scenarios. FedGS can combine with existing methods easily to achieve better performance. We further verify that our method has robust performance gains in different non-IID scenarios, demonstrating the adaptability of FedGS for different scenarios.
19 January 2023 in IEEE Internet of Things Journal
Centralized learning now faces data mapping and security constraints that make it difficult to carry out. Federated learning with a distributed learning architecture has changed this situation. By restricting the training process to participants’ local, federated learning addresses the model training needs of multiple data sources while better protecting data privacy. However, in real-world application scenarios, federated learning faces the need to achieve fairness in addition to privacy protection. In practice, it could happen that some federated learning participants with specific motives may short join the training process to obtain the current global model with only limited contribution to the federated learning whole, resulting in unfairness to the participants who previously participated in federated learning. We propose the FedACC framework with a server-initiated global model accuracy control method to address this issue. Besides measuring the accumulative contributions of newly joined participants and providing participants with a model with an accuracy that matches their contributions, the FedACC still guarantees the validity of participant gradients based on the accuracy-decayed model. Under the FedACC framework, users do not have access to the full version of the current global model early in their training participation. However, they must produce a certain amount of contributions before seeing the full-accuracy model. We introduce an additional differential privacy mechanism to protect clients’ privacy further. Experiments demonstrate that the FedACC could obtain about 10% to 20% accuracy gain compared to the state-of-art methods while balancing the fairness, performance, and security of federated learning.
3 June 2024 in IEEE Transactions on Circuits and Systems for Video Technology
Federated learning is widely used and researched as an effective method for solving the privacy problems faced by centralized learning. To address the communication limitations and heterogeneity among clients, many existing methods based on the mixup algorithm share data mixed with the client’s local dataset to improve the model accuracy. However, due to the heterogeneity of federated learning, there may be some clients who join the mixup process with insufficient data, which will violate the privacy-preserving assumption of the mixup. Because of this weakness, many methods based on the mixup approach will face a serious privacy problem while trying to improve federated learning over other parts, e.g., accuracy or communication efficiency. Therefore, we propose the FedMDO framework to solve the privacy problem faced by mixup-based methods. In FedMDO, we introduce the auxiliary client to hold the auxiliary dataset that is related to the federated learning task and to generate the mixup templates for clients to increase the amount of data in the mixup process. By introducing the auxiliary client, the decrease in model accuracy can be suppressed as much as possible while taking advantage of the privacy-preserving gain from the increase in data volume. Furthermore, we introduce differential privacy into FedMDO with an elaborate redesign to enhance privacy protection. The corresponding analysis shows that under FedMDO, differential privacy can achieve the same protection with less negative impact. Experiments show that with at least an approximately 10%+ improvement in model accuracy and an average of 5 times greater communication savings compared to the FedAvg and non-IID SOTAs with weak privacy protection design, our method can yield significant improvement in the privacy of the shared data.
1 January 2026 in IEEE Transactions on Mobile Computing