Doyoon Ju

Mr. Doyoon Ju

Inha University, Incheon, Republic of Korea
orcid

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
26 February 2025 in Machines
Verified
Journal Article
Open access
PDF file
TLTaegun Lee
Taegun Lee
DJDoyoon Ju
Doyoon Ju
YLYoung Sam Lee
Young Sam Lee
This study presents a sim2real reinforcement learning-based controller for transition control in a double-inverted pendulum system, addressing the limitations of traditional control methods that rely on precomputed trajectories and lack adaptability to strong external disturbances. By introducing the novel concept of ‘transition control’, this research expands the scope of inverted pendulum studies to tackle the challenging task of navigating between multiple equilibrium points. To overcome the reality gap—a persistent challenge in sim2real transfer—a hardware-centered approach was employed, aligning the physical system’s mechanical design with high-fidelity dynamic equations derived from the Euler–Lagrange equation. This design eliminates the need for software-based corrections, ensuring consistent and robust system performance across simulated and real-world environments. Experimental validation demonstrates the controller’s ability to reliably execute all 12 transition scenarios within the double-inverted pendulum system. Additionally, it exhibits recovery characteristics, enabling the system to stabilize and return to equilibrium point even under severe disturbances.

17 June 2025 in Mathematics
Verified
Journal Article
Open access
PDF file
DJDoyoon Ju
Doyoon Ju
JLJongbeom Lee
Jongbeom Lee
YLYoung Sam Lee
Young Sam Lee
This paper proposes a transition control strategy for a rotary double-inverted pendulum (RDIP) system using a sim-to-real reinforcement learning (RL) controller, built upon mathematical modeling and parameter estimation. High-resolution sensor data are used to estimate key physical parameters, ensuring model fidelity for simulation. The resulting mathematical model serves as the training environment in which the RL agent learns to perform transitions between various initial conditions and target equilibrium configurations. The training process adopts the Truncated Quantile Critics (TQC) algorithm, with a reward function specifically designed to reflect the nonlinear characteristics of the system. The learned policy is directly deployed on physical hardware without additional tuning or calibration, and the TQC-based controller successfully achieves all four equilibrium transitions. Furthermore, the controller exhibits robust recovery properties under external disturbances, demonstrating its effectiveness as a reliable sim-to-real control approach for high-dimensional nonlinear systems.

15 February 2025 in Mathematics
Verified
Journal Article
Open access
PDF file
DJDoyoon Ju
Doyoon Ju
TLTaegun Lee
Taegun Lee
YLYoung Sam Lee
Young Sam Lee
The rotary double inverted pendulum system is characterized by one stable equilibrium point and three unstable equilibrium points due to its kinematic properties. This paper defines the transition control problem between these equilibrium points to extend the conventional swing-up control problem and proposes an implementation method using a laboratory-developed rotary double inverted pendulum. To minimize energy consumption during the transition process while satisfying the boundary conditions of different equilibrium points, a two-point boundary value optimal control problem is formulated. The feedforward trajectory required for feedforward control is computed offline by solving this problem. The direct collocation method is employed to convert the constrained continuous optimal control problem into a nonlinear optimization problem. Furthermore, a time-varying linear–quadratic (LQ) controller is utilized as a feedback controller to accurately track the generated feedforward trajectory during real-time control, compensating for uncertainties in the feedforward control process. The proposed transition control strategy is experimentally implemented, and its effectiveness and practicality are validated through the successful tracking of 12 transition trajectories.

31 July 2023 in Journal of Institute of Control, Robotics and Systems
Verified
Journal Article
Closed access
TLTaegun Lee
Taegun Lee
DJDoyoon Ju
Doyoon Ju
YLYoung-Sam Lee
Young-Sam Lee
In recent years, reinforcement learning (RL)-based controller design methods have emerged as a powerful alternatives to traditional methods, providing a novel paradigm that overcomes limitations associated with the need for accurate model information. In this paper, we propose a development environment for RL-based controllers in real-world systems by integrating MATLAB/Simulink, Python, and the LW-RCP (Light-weight Rapid Control Prototyping) system developed by the authors’ laboratory. The proposed development environment utilizes LW-RCP’s library block in a Simulink-based RL controller model, enabling real-time experiments on real-world systems, and stores state information data in MATLAB’s workspace. Python obtains this data through the Python API after each episode and uses it to iteratively enhance the RL agent’s policy by using RL algorithms. Updated parameter values for the agent’s policy neural network are then sent back to MATLAB

12 September 2026 in Mathematics
Verified
Journal Article
Open access
PDF file
SLSungwon Lee
Sungwon Lee
DJDoyoon Ju
Doyoon Ju
YLYoung Sam Lee
Young Sam Lee
This paper proposes a MuJoCo-based sim-to-real reinforcement learning approach for transition control of a mini rotary double inverted pendulum (MRDIP). Transition control between multiple equilibrium points of the MRDIP is challenging because of its underactuated, nonlinear, and strongly coupled dynamics. To address this problem, a direct-drive MRDIP system is developed to enable torque-controlled actuation, and a MuJoCo model is constructed using CAD-based physical parameters and experimentally identified dynamic parameters. Based on the identified model, transition control policies for multiple equilibrium points are trained using the Truncated Quantile Critics (TQC) algorithm. The trained policies are directly implemented on the physical MRDIP and evaluated through successive equilibrium-point transitions without resetting the system between transitions. Experimental results show that the policies trained in MuJoCo successfully perform the intended transitions on the physical system. These results demonstrate the feasibility of the proposed MuJoCo-based framework for sim-to-real transition control of the MRDIP.

7 September 2026 in Electronics
Verified
Journal Article
Open access
PDF file
JKJihun Koo
Jihun Koo
SLSungwon Lee
Sungwon Lee
DJDoyoon Ju
Doyoon Ju
YLYoung Sam Lee
Young Sam Lee
This study presents an experimentally validated MuJoCo-based sim-to-real reinforcement learning framework for transition control among all eight equilibrium points (EPs) of a physical triple inverted pendulum (TIP). A multibody model is constructed from the apparatus properties, and uncertain actuation, damping, and friction parameters are identified by matching measured and simulated trajectories using the covariance matrix adaptation evolution strategy. The resulting effective parameter sets are sampled during training using truncated quantile critics to reduce policy dependence on a single model. The trained policies are transferred to the physical TIP without additional tuning or calibration. In the EP0-to-EP7 experiment, the policy trained without parameter identification failed in all ten trials, whereas the policy trained with the identified parameter sets succeeded in all ten. In the sequential experiment, all eight transitions were completed in each of ten trials, yielding 80 successful transitions with no observed failures while satisfying the cart travel and tangential force limits. These results demonstrate sim-to-real transition control across all eight EPs without state resets or precomputed trajectories.