Federated Learning, an approach to collaborative modeling, enables the training of a unified global model across multiple clients in a decentralized manner. However, the considerable impact of local data heterogeneity on algorithm performance has attracted significant attention. In this study, we introduce a novel Federated Learning algorithm called Federated Joint Server-Client Momentum (FedJSCM) to address data heterogeneity in real-world Federated Learning applications. FedJSCM efficiently utilizes global gradient information from previous communications and adjusts client gradient descent and server model fusion by transmitting gradient momentum information. This corrective mechanism effectively mitigates biases and improves the stability of Stochastic Gradient Descent (SGD). We offer theoretical analysis to highlight the advantages of FedJSCM and conduct extensive empirical studies, showcasing its superior performance across various tasks and its robustness in the face of varying degrees of data heterogeneity. Empirical studies demonstrate that FedJSCM outperforms existing algorithms, with a 1–3% accuracy increase.