Deep reinforcement learning algorithms have demonstrated significant potential in showcasing robotic capabilities within virtual environments. However, applying DRL for practical robot development in realistic simulators like Webots remains challenging due to limitations in existing frameworks, such as complex dependencies and reliance on unrealistic control paradigms like a ‘supervisor’. This study introduces an open-source framework and a novel pattern-based method designed to facilitate the exploration of robot learning capabilities through reinforcement learning algorithms in specialized virtual testing environments built on Webots. Our approach simplifies setup by avoiding burdensome external package installations and, crucially, removes the dependency on an unrealistic ‘supervisor’ entity, offering a more practical and real-world-aligned solution. Designed to leverage Webots’ realistic simulation capabilities, the proposed method and system are validated through various examples, ranging from the classic inverted pendulum scenario to a production robot utilized in an actual assembly line. The developed code and examples are publicly accessible on GitHub for the deep reinforcement learning research community.

Reinforcement learning (RL) algorithms, particularly deep reinforcement learning (DRL), have shown transformative potential in robotics by enabling adaptive behaviour in virtual environments. However, a comprehensive framework for efficiently testing, training, and deploying robots in these environments remains underexplored. This study introduces a standardized, open-source framework designed specifically for the Webots simulation environment. Supported by a robust methodology, the framework integrates innovative design patterns and the digital twin (DT) concept with three distinct design patterns for structuring agent-environment interaction, notably including a novel pattern aimed at improving sim-toreal transferability, to enhance RL workflows. The proposed framework is validated through experimental studies on both a model the inverted pendulum and a production-grade Pioneer 3-AT robotic platform. The experiments highlight the framework’s ability to bridge the gap between virtual training and real-world implementation. All resources, including the framework, methodology, and experimental configurations, are openly accessible on GitHub.