This article investigates the adaptive neural tracking control problem for a class of hyperbolic PDE with boundary actuator dynamics described by a set of nonlinear ordinary differential equations (ODEs). Particularly, the control input appears in the ODE subsystem with unknown nonlinearities requiring to be estimated and compensated, which makes the control task rather difficult. It is the first time to consider tracking control of such a class of systems, rendering our contributions essentially different from the existing literature that merely focus on the stabilization problem. By formulating a virtual exosystem to generate a reference trajectory, we propose a novel design of the adaptive geometric controller for the considered system where neural networks (NNs) are employed to approximately estimate nonlinearities, and finite and infinite-dimensional backstepping techniques are leveraged. Moreover, rigorously theoretical proofs based on the Lyapunov theory are provided to analyze the stability of the closed-loop system. Finally, we illustrate the results through two numerical simulations.
12 December 2022 in IEEE Transactions on Cybernetics
A reinforcement learning-based boundary optimal control algorithm for parabolic distributed parameter systems is developed in this article. First, a spatial Riccati-like equation and an integral optimal controller are derived in infinite-time horizon based on the principle of the variational method, which avoids the complex semigroups and operator theories. Using state data along the system trajectory, a value iteration algorithm via the Bellman optimality principle is proposed to obtain the solution of the spatial Riccati-like equation and the optimal control law. The convergence of the value iteration algorithm is proved. Subsequently, an approximation scheme based on weighted residuals is developed to implement the value iteration algorithm, where radial basis functions are chosen as the basic functions to approximate the solution of the spatial Riccati-like equation. Simulations on the diffusion–reaction process demonstrate the effectiveness of the developed method.
Reinforcement learning (RL) has achieved remarkable advancements in navigation tasks in recent years. However, tackling multi-goal navigation tasks with sparse rewards remains a complex and challenging problem due to the long-sequence decision-making involved. Such multi-goal navigation tasks inherently incorporate a hybrid action space, where the robot needs to select a navigation endpoint first before executing primitive actions. To address the problem of multi-goal navigation with sparse rewards, we introduce a novel hierarchical RL framework named Hierarchical RL with Multi-Goal (HRL-MG). The main idea of HRL-MG is to divide and conquer the hybrid action space, splitting long-sequence decisions into short-sequence decisions. The HRL-MG framework is composed of two main modules: a selector and an actuator. The selector employs a temporal abstraction hierarchical architecture designed to specify a desired end goal based on the discrete action space. Conversely, the actuator utilizes a continuous goal-oriented hierarchical architecture developed to enact continuous action sequences to reach the desired end goal specified by the selector. In addition, we incorporate a dynamic goal detection mechanism, grounded in hindsight experience replay, to mitigate the challenges posed by sparse reward landscapes. We validated the algorithm’s efficacy on both the discrete environment Maze_2D and the continuous robotic environment MuJoCo ‘Ant’. The results indicate that HRL-MG significantly outperforms other methods in multi-goal navigation tasks with sparse rewards.
8 August 2019 in International Journal of Robust and Nonlinear Control