27 July 2021 in IEEE Robotics and Automation Letters
This letter studies the adaptive optimal control problem for a wheel-legged robot in the absence of an accurate dynamic model. A crucial strategy is to exploit recent advances in reinforcement learning (RL) and adaptive dynamic programming (ADP) to derive a learning-based solution to adaptive optimal control. It is shown that suboptimal controllers can be learned directly from input-state data collected along the trajectories of the robot. Rigorous proofs for the convergence of the novel data-driven value iteration (VI) algorithm and the stability of the closed-loop robot system are provided. Experiments are conducted to demonstrate the efficiency of the novel adaptive suboptimal controller derived from the data-driven VI algorithm in balancing the wheel-legged robot to the equilibrium.
8 May 2023 in IEEE Transactions on Automatic Control
This paper studies the adaptive optimal control problem for a class of linear time-delay systems described by delay differential equations (DDEs). A crucial strategy is to take advantage of recent developments in reinforcement learning (RL) and adaptive dynamic programming (ADP) and develop novel methods to learn adaptive optimal controllers from finite samples of input and state data. In this paper, the data-driven policy iteration (PI) is proposed to solve the infinite-dimensional algebraic Riccati equation (ARE) iteratively in the absence of exact model knowledge. Interestingly, the proposed recursive PI algorithm is new in the present context of continuous-time time-delay systems, even when the model knowledge is assumed known. The efficacy of the proposed learning-based control methods is validated by means of practical applications arising from metal cutting and autonomous driving.
16 June 2023 in IEEE Transactions on Intelligent Vehicles
This paper presents a unified approach to the problem of learning-based optimal control of connected human-driven and autonomous vehicles in mixed-traffic environments including both the freeway and ring road settings. The stabilizability of a string of connected vehicles including multiple autonomous vehicles (AVs) and heterogeneous human-driven vehicles (HDVs) is studied by a model reduction technique and the Popov-Belevitch-Hautus (PBH) test. For this problem setup, a linear quadratic regulator (LQR) problem is formulated and a solution based on adaptive dynamic programming (ADP) techniques is proposed without a priori knowledge on model parameters. To start the learning process, an initial stabilizing control law is obtained using the small-gain theorem for the ring road case. It is shown that the obtained stabilizing control law can achieve general $\mathcal {L}_{p}$ string stability under appropriate conditions. Besides, to minimize the impact of external disturbance, a linear quadratic zero-sum game is introduced and solved by an iterative learning-based algorithm. Finally, the simulation results verify the theoretical analysis and the proposed methods achieve desirable performance for control of a mixed-vehicular network.
23 June 2021 in IEEE Control Systems Letters
This letter studies the trajectory tracking problem for an autonomous bicycle that is a non-minimum phase, strongly nonlinear system. As compared with most existing methods dealing only with approximate trajectory tracking, this letter proposes a constructive design to achieve asymptotic trajectory tracking. More specifically, under the assumption that the desired trajectory is generated by a virtual bicycle, a novel asymptotic trajectory tracking controller design scheme is presented. Firstly, the nonlinear dynamics of the autonomous bicycle is established. Secondly, it is decomposed into two interconnected subsystems: a tracking subsystem and a balancing subsystem. For the tracking subsystem, the popular backstepping approach is applied to determine the propulsive force of the bicycle. For the balancing subsystem, optimal control is applied to determine the steering angular velocity of the handlebar in order to balance the bicycle and align the bicycle with the desired yaw angle. Thirdly, to tackle the strong coupling between the tracking and the balancing systems, the small-gain technique is applied for the first time to prove the asymptotic stability of the closed-loop bicycle system. Finally, the efficacy of the proposed exact trajectory tracking control methodology is validated by numerical simulations.
30 April 2024 in Systems & Control Letters
8 January 2020 in IEEE Transactions on Systems, Man, and Cybernetics: Systems
This article proposes a novel image-based visual \nobreak servoing for a flexible aerial refueling boom with an eye-in-hand camera. The dynamic model of the flexible refueling boom is established by absolute nodal coordinate formulation (ANCF), after which the dynamic model is decomposed into a slow subsystem and a fast subsystem based on the singular perturbation approach. With respect to slow subsystem, the image feedback is used to control the flexible refueling boom so that the projection of the point marker on the back of the receiver converges to the desired position. With respect to fast subsystem, linear quadratic regulator (LQR) is applied to stabilize the vibration of the flexible refueling boom. The asymptotic convergence of the image error to zero is verified based on the Lyapunov theory. Simulation is used to demonstrate the effectiveness of the proposed method.