In this study, we propose a novel method for enhancing the control performance of a degraded robot manipulator by leveraging digital twins and proximal policy optimization, a specific deep reinforcement learning algorithm. Recently, various robotic technologies with high levels of controllability, safety, and reliability that incorporate the fourth industrial technology have been developed. Nevertheless, repairs or replacements owing to the performance degradation of sophisticated robot hardware or control systems are still time-, cost-, and manpower-consuming. To address these challenges, we propose a new strategy:1) approximate the unstable dynamic characteristics of six-degree-of-freedom low-performance robot manipulators to a digital twin with parameter tuning of a physics engine, 2) improve the accuracy and stability of reaching target points under diverse conditions through deep reinforcement learning using the domain randomization method, and 3) deploy a trained policy on an actual robot manipulator with degraded capabilities to validate the control performance improvement. Our method reduced the position error of a real robot manipulator by 63.0% and 39.0% compared to the built-in control method and the proportional-integral-derivative control method, respectively. Randomizing parameters in the physics engine of the digital twin during training allowed the method to simulate the imprecise motions of an actual degraded robot manipulator, facilitating the development of a more robust policy. Notably, our method has the potential to be applicable to all types of articulated robots, and presents a promising solution for maintaining performance while reducing long-term costs.

With the rapid development of artificial intelligence (AI) technology and an increasing demand for redundant robotic systems, robot control systems are becoming increasingly complex. Although forward kinematics (FK) and inverse kinematics (IK) equations have been used as basic and perfect solutions for robot posture control, both equations have a significant drawback. When a robotic system is highly nonlinear, it is difficult or impossible to derive both the equations. In this paper, we propose a new method that can replace both the FK and IK equations of a seven-degrees-of-freedom (7-DOF) robot manipulator. This method is based on reinforcement learning (RL) and artificial neural networks (ANN) for supervised learning (SL). RL was used to acquire training datasets consisting of six posture data in Cartesian space and seven motor angle data in joint space. The ANN is used to make the discrete training data continuous, which implies that the trained ANN infers any new data. Qualitative and quantitative evaluations of the proposed method were performed through computer simulation. The results show that the proposed method is sufficient to control the robot manipulator as efficiently as the IK equation.

In this study, a new method was developed to extend the flight distance of unmanned aerial vehicles such as quadcopters that operate on a limited battery capacity. The suggested method consists of a high-altitude flight initiation system, which uses a carrier rocket, and an autonomous ramp flight algorithm. First, a quadcopter, called a mission quadcopter in this research, is loaded onto the carrier rocket by a folding mechanism, and the rocket flies to a designated height. Then, the carrier rocket releases the mission quadcopter at the high altitude and achieves flight deployment after an unfolding process. Subsequently, the airdropped mission quadcopter stabilizes its pose and flights toward the selected target point. During the flight, it adopts an autonomous ramp flying trajectory. Ramp flying is performed from the sky to the ground surface instead of conventional step shape flying to minimize the flight trajectory and energy consumed while flying. Experiments were performed in Nevada, USA during an international rocket-based airdrop competition, ARLISS (A Rocket Launch for International Student Satellite). The mission quadcopter's real-time latitude, longitude, and altitude were measured to confirm the position information regarding the quadcopter's flight trajectory. The mission quadcopter successfully separated from the carrier rocket, deployed for flight at a high altitude, turned to the target point, performed autonomous ramp flying, and landed.