This paper proposes a MuJoCo-based sim-to-real reinforcement learning approach for transition control of a mini rotary double inverted pendulum (MRDIP). Transition control between multiple equilibrium points of the MRDIP is challenging because of its underactuated, nonlinear, and strongly coupled dynamics. To address this problem, a direct-drive MRDIP system is developed to enable torque-controlled actuation, and a MuJoCo model is constructed using CAD-based physical parameters and experimentally identified dynamic parameters. Based on the identified model, transition control policies for multiple equilibrium points are trained using the Truncated Quantile Critics (TQC) algorithm. The trained policies are directly implemented on the physical MRDIP and evaluated through successive equilibrium-point transitions without resetting the system between transitions. Experimental results show that the policies trained in MuJoCo successfully perform the intended transitions on the physical system. These results demonstrate the feasibility of the proposed MuJoCo-based framework for sim-to-real transition control of the MRDIP.
This study presents an experimentally validated MuJoCo-based sim-to-real reinforcement learning framework for transition control among all eight equilibrium points (EPs) of a physical triple inverted pendulum (TIP). A multibody model is constructed from the apparatus properties, and uncertain actuation, damping, and friction parameters are identified by matching measured and simulated trajectories using the covariance matrix adaptation evolution strategy. The resulting effective parameter sets are sampled during training using truncated quantile critics to reduce policy dependence on a single model. The trained policies are transferred to the physical TIP without additional tuning or calibration. In the EP0-to-EP7 experiment, the policy trained without parameter identification failed in all ten trials, whereas the policy trained with the identified parameter sets succeeded in all ten. In the sequential experiment, all eight transitions were completed in each of ten trials, yielding 80 successful transitions with no observed failures while satisfying the cart travel and tangential force limits. These results demonstrate sim-to-real transition control across all eight EPs without state resets or precomputed trajectories.