Federated Learning, an approach to collaborative modeling, enables the training of a unified global model across multiple clients in a decentralized manner. However, the considerable impact of local data heterogeneity on algorithm performance has attracted significant attention. In this study, we introduce a novel Federated Learning algorithm called Federated Joint Server-Client Momentum (FedJSCM) to address data heterogeneity in real-world Federated Learning applications. FedJSCM efficiently utilizes global gradient information from previous communications and adjusts client gradient descent and server model fusion by transmitting gradient momentum information. This corrective mechanism effectively mitigates biases and improves the stability of Stochastic Gradient Descent (SGD). We offer theoretical analysis to highlight the advantages of FedJSCM and conduct extensive empirical studies, showcasing its superior performance across various tasks and its robustness in the face of varying degrees of data heterogeneity. Empirical studies demonstrate that FedJSCM outperforms existing algorithms, with a 1–3% accuracy increase.

In Visual Question Answering (VQA), the model’s ability to understand and reason across different modalities—language, visual, and multimodal—is crucial for accurate predictions. However, recent studies have identified a significant challenge of language bias in VQA models, where the model’s reasoning often hinges on incorrect linguistic associations rather than genuine multimodal understanding. To addressthis issue, We propose a novel Ensemble Bias Gradient Debiasing Approach (EBGDA) that combines bias detection with a dynamic forgetting mechanism. Our method utilizes a bias detector to identify and score biases across linguistic, visual, and multimodal data, enabling the model to focus on unbiased informationfor more accurate predictions. Additionally, inspired by human reasoning, we introduce the Forgotten Attention Algorithm (FAA), which iteratively “forgets” irrelevant visual content, progressively concentrating attention on the image regions most relevant to the question. This combination of bias mitigation andattention focusing enhances the model’s ability to make multimodal inferences, reducing bias and improving overall performance. Extensive experiments on the VQA-CP v2, VQA v2, VQA-VS, GQA-OOD, and VQA-CE datasets demonstrate the effectiveness of our approach, showing state-of-the-art performance inmitigating biases and excelling in complex multimodal scenarios. Our approach achieves a 21.32% improvement over UpDn on VQA-CP v2, establishing a new state-of-the-art among methods without data augmentation.