4 January 2024 in IEEE Transactions on Industrial Electronics
In recent years, safe reinforcement learning (RL) with the actor-critic structure has gained significant interest for continuous control tasks. However, achieving near-optimal control policies with safety and convergence guarantees remains challenging. Moreover, few works have focused on designing RL algorithms that handle time-varying safety constraints. This article proposes a safe RL algorithm for optimal control of nonlinear systems with time-varying state and control constraints. The algorithm's novelty lies in two key aspects. Firstly, the approach introduces a unique barrier force-based control policy structure to ensure control safety during learning. Secondly, a multistep policy evaluation mechanism is employed, enabling the prediction of policy safety risks under time-varying constraints and guiding safe updates. Theoretical results on learning convergence, stability, and robustness are proven. The proposed algorithm outperforms several state-of-the-art RL algorithms in the simulated Safety Gym environment. It is also applied to the real-world problem of integrated path following and collision avoidance for two intelligent vehicles—a differential-drive vehicle and an Ackermann-drive one. The experimental results demonstrate the impressive sim-to-real transfer capability of our approach, while showcasing satisfactory online control performance.
1 March 2008 in Journal of Bionic Engineering
27 April 2023 in Robotica
Path-following control of wheeled mobile robots has been a crucial research topic in robotic control theory and applications. In path-following control with obstacles, the path-following control and collision avoidance goals might be conflicting, making it challenging to obtain near-optimal solutions for path-following control and obstacle avoidance with low tracking error and input energy consumption. To address this problem, we propose a potential field-based dual heuristic programming (P-DHP) algorithm with an actor–critic (AC) structure for path-following control of mobile robots with obstacle avoidance. In the proposed P-DHP, the path-following control and collision avoidance problems are decoupled into two ones to resolve the control conflict. Firstly, a neural network-based AC is constructed to approximate the near-optimal path-following control policy in a no-obstacle environment. Then, with the trained path-following control policy fixed, a potential field-based control policy structure is constructed by another AC network to generate opposite control forces as the robot moves toward the obstacle, which can guarantee the robot’s control safety and reduce the tracking error and input energy consumption in obstacle avoidance. The simulated and experimental results show that P-DHP can realize near-optimal path-following control with the satisfaction of safety constraints and outperforms state-of-the-art approaches in control performance.
27 January 2022 in Applied Intelligence