We present a virtual reality (VR) framework for controlling dual-arm robotic manipulators through immersive interfaces, integrating both simulated and real-world platforms. The system combines the Webots robotics simulator with Unreal Engine 5.6.1 to provide real-time visualization and interaction, enabling users to manipulate each arm’s tool point via VR controllers with natural depth perception and motion tracking. The same control interface is seamlessly extended to a physical dual-arm robot, enabling teleoperation within the same VR environment. Our architecture supports real-time bidirectional communication between the VR layer and both the simulator and hardware, enabling responsive control and feedback. We describe the system design and performance evaluation in both domains, demonstrating the viability of immersive VR as a unified interface for simulation and physical robot control.
We present a virtual reality (VR) framework for controlling dual-arm robotic manipulators through immersive interfaces, integrating both simulated and real-world platforms. The system combines the Webots robotics simulator with Unreal Engine 5.6.1 to provide real-time visualization and interaction, enabling users to manipulate each arm’s tool point via VR controllers with natural depth perception and motion tracking. The same control interface is seamlessly extended to a physical dual-arm robot, allowing for teleoperation using the same VR environment. Our architecture supports real-time bidirectional communication between the VR layer and both the simulator and hardware, enabling responsive control and feedback. We describe the system design, and performance evaluation in both domains, demonstrating the viability of immersive VR as a unified interface for simulation and physical robot control.
We present V-DART (VR-based Dual-Arm Robot Teleoperation), an extended virtual reality framework for the teleoperation of mobile dual-arm robotic systems, building upon our previous work on stationary manipulator control. The enhanced system leverages Unreal Engine 5.6.1 (UE5) for real-time visualization and interaction, introducing critical advancements for complex environments: the control of the robot’s base movement via both joystick input and direct mapping of the user’s physical position and orientation within the tracked space. This allows users to navigate the robot intuitively by simply moving their body, while maintaining precise control over both arms using VR (Virtual Reality) controllers. Furthermore, we introduce a robust data logging layer that records expert movements during teleoperation tasks. By capturing timestamped streams of user pose, controller inputs, and robot states that can be synchronized downstream, this module facilitates the creation of high-quality datasets from expert demonstrations. These datasets can be leveraged for machine learning applications, such as behavior cloning, reducing the need for extensive manual annotation. The system maintains bidirectional communication between the VR interface and the physical robot, ensuring responsive control and real-time feedback. We describe the architectural enhancements, user interaction design, and performance evaluation, demonstrating the viability of this unified approach for both teleoperation and data-driven robotics development.