This paper presents a federated learning framework for multi-agent robotic systems, leveraging the ROS 2 framework to enable decentralized collaboration in both simulated and real-world environments. Traditional centralized machine learning approaches face challenges such as data privacy concerns, communication overhead, and limited scalability. To address these issues, we propose a federated reinforcement learning architecture where multiple robotic agents train local models and share their knowledge while preserving data privacy. The framework integrates deep reinforcement learning techniques, utilizing Unity for high-fidelity simulation. Experimental evaluations compare our federated approach against classical centralized learning, demonstrating that our proposal improves model generalization, stabilizes reward distribution, and reduces training variance. Additionally, results indicate that increasing the number of robots enhances task efficiency, reducing the number of steps required for successful navigation while maintaining consistent performance. This study highlights the potential of federated learning in robotics, offering a scalable and privacy-preserving approach to distributed multi-agent learning.