Federated learning (FL) stands as a promising distributed machine learning approach today, allowing model training on local clients without data sharing. However, varied contributions among clients often impact the global model’s performance, while FL confronts challenges like communication overhead, data heterogeneity, and privacy concerns. In this paper, we introduce a novel federated learning server aggregation algorithm: the Federated Learning Algorithm with Optimized Weight Aggregation via Particle Swarm Optimization Algorithm (AdpFedPSO). This approach dynamically adjusts client model contribution weights based on their performance and stability, aiming to enhance global model accuracy, convergence speed, and make model aggregation smarter and more adaptable. Through experimental validation on real datasets, we find that AdpFedPSO enhances accuracy by about 15% on the 0.6-Dirichlet MNIST dataset, 7.3% on the FashionMNIST dataset, and 13.4% on the CIFAR-10 dataset compared to the traditional FL aggregation method, FedAvg. These results indicate that AdpFedPSO not only enhances the accuracy of the global model but also expedites the model’s convergence speed. Additionally, it demonstrates resilience across various levels of client numbers, participation rates, and client heterogeneity, providing valuable reference and guidance for the further development of FL technology. Moreover, the concept of employing the particle swarm optimization algorithm for FL model aggregation also offers new insights and directions for future research.
Federated learning, with its unique privacy protection mechanisms and distributed model training capabilities, provides an effective solution for data security by addressing the challenges associated with the inability to directly share private data due to privacy concerns. It exhibits broad application potential across various fields, particularly in scenarios such as autonomous vehicular networks, where collaborative learning is required from data sources distributed across different clients, thus optimizing and enhancing model performance. Nevertheless, in complex real-world environments, challenges such as data poisoning and labeling errors may cause some clients to introduce label noise that significantly exceeds ordinary levels, severely impacting model performance. The following conclusions are drawn from research on extreme label noise: highly polluted data severely affect the generalization capability of the global model and the stability of the training process, while the reweighting strategy can improve model performance. Based on these research conclusions, we propose a method named Enhanced Knowledge Distillation and Particle Swarm Optimization for Federated Learning (FedDPSO) to deal with extreme label noise. In FedDPSO, the server dynamically identifies extremely noisy clients based on uncertainty. It then uses the particle swarm optimization algorithm to determine client model weights for global model aggregation. In subsequent rounds, the identified extremely noisy clients construct an interpolation loss combining pseudo-label loss and knowledge distillation loss, effectively mitigating the negative impact of label noise overfitting on the local model. We carried out experiments on the CIFAR10/100 datasets to validate the effectiveness of FedDPSO. At the highest noise ratio under Beta = (0.1, 0.1), experiments show that FedDPSO improves the average accuracy on CIFAR10 by 15% compared to FedAvg and by 11% compared to the more powerful FOCUS. On CIFAR100, it outperforms FedAvg by 8% and FOCUS by 5%.
Federated learning (FL) is a distributed learning technique that ensures data privacy and has shown significant potential in cross-institutional image analysis. However, existing methods struggle with the inherent dynamic heterogeneity of real-world data, such as changes in cellular differentiation during disease progression or feature distribution shifts due to different imaging devices. This dynamic heterogeneity can cause catastrophic forgetting, leading to reduced performance in medical predictions across stages. Unlike previous federated learning studies that paid insufficient attention to dynamic heterogeneity, this paper proposes the FedDyH framework to address this challenge. Inspired by the adaptive regulation mechanisms of biological systems, this framework incorporates several core modules to tackle the issues arising from dynamic heterogeneity. First, the framework simulates intercellular information transfer through cross-client knowledge distillation, preserving local features while mitigating knowledge forgetting. Additionally, a dynamic regularization term is designed in which the strength can be adaptively adjusted based on real-world conditions. This mechanism resembles the role of regulatory T cells in the immune system, balancing global model convergence with local specificity adjustments to enhance the robustness of the global model while preventing interference from diverse client features. Finally, the framework introduces a genetic algorithm (GA) to simulate biological evolution, leveraging mechanisms such as gene selection, crossover, and mutation to optimize hyperparameter configurations. This enables the model to adaptively find the optimal hyperparameters in an ever-changing environment, thereby improving both adaptability and performance. Prior to this work, few studies have explored the use of optimization algorithms for hyperparameter tuning in federated learning. Experimental results demonstrate that the FedDyH framework improves accuracy compared to the SOTA baseline FedDecorr by 2.59%, 0.55%, and 5.79% on the MNIST, Fashion-MNIST, and CIFAR-10 benchmark datasets, respectively. This framework effectively addresses data heterogeneity issues in dynamic heterogeneous environments, providing an innovative solution for achieving more stable and accurate distributed federated learning.