This article presents and practically validates an identifier-critic-based approximate dynamic programming (ADP) method to online address the optimal tracking control problem for nonlinear continuous-time unknown systems. The imposed assumption on precisely known system dynamics is obviated via a neural network (NN) identifier. A static control is first adopted to retain the steady-state tracking response, while an optimal control derived via the ADP method is proposed to regulate the tracking error by minimizing a cost function. A critic NN is then trained online to obtain the solution of the associated Hamilton-Jacobi-Bellman (HJB) equation. The learning of the identifier NN and critic NN is performed online simultaneously by tailoring a novel adaptation method, which can guarantee the convergence of the estimated NN weights. Consequently, the critic NN can be used to construct the optimal control policy directly, such that the actor NN used in the previous ADP schemes is avoided. Simulations are performed to verify the suggested control, and experiments on a helicopter plant are carried out to show its feasibility and improved control response.

The path tracking control problem for autonomous vehicle with uncertain dynamics requires simultaneous consideration of control optimality and safety-based performance constraints. In this paper, an adaptive optimal control method with prescribed performance is proposed to solve this problem, which contains two contributions: 1) by introducing a prescribed performance function (PPF) into adaptive dynamic programming (ADP), the controller can constrain the tracking error of the system within a specified performance boundary while optimizing the control cost; 2) the critic-only ADP is used for the controller design, which simplifies the commonly used actor-critic ADP scheme, and the convergence of the estimation error is guaranteed under FE conditions. On this basis, the neural network identification technique is introduced to deal with the unknown dynamic parameters of the vehicle system. The control scheme is able to strictly guarantee user-defined vehicle performance specifications with approximately optimal control performance. The stability of the closed-loop system is rigorously demonstrated by the Lyapunov method. In addition, the controller also embeds a radial basis function neural network (RBFNN) compensator to approximate the nonlinear external disturbances of the autonomous vehicle. Finally, the efficiency of the controller to achieve autonomous vehicle path tracking is verified by CarSim-Simulink simulation.

This paper develops a novel adaptive $H_{\infty }$ control scheme for adaptive cruise control (ACC) system via the data-driven learning. In the proposed technique, a continuous time ACC system is first constructed with unknown system dynamics. To estimate the unknown system dynamics, an adaptive estimator is then formulated utilizing the vectorization and Kronecker's products operations, enabling the reconstruction of the unknown system dynamics through the detectable input/output information. An adaptive law is utilized to ensure the convergence of the estimated parameters. Moreover, a data-driven learning technique is employed to resolve the constructed Riccati equation in an online manner. To accomplish this, the Riccati equation is reformulated via the Kronecker's products by using another adaptive law, whose convergence can be also effectively guaranteed. Unlike existing neural network-based approximate dynamic programming (ADP) algorithms, the data-driven learning scheme proposed in this paper does not involve neural network approximation errors, so the solution is relatively more accurate. Finally, simulation and experimental verification results are provided to verify the effectiveness of the presented $H_{\infty }$ control and data-driven learning algorithm.

Although robust control has been studied for decades, the output-feedback robust control design is still challenging in the control field. This article proposes a new approach to address the output-feedback robust control for continuous-time uncertain systems. First, we transform the robust control problem into an optimal control problem of the nominal linear system with a constructive cost function, which allows simplifying the control design. Then, a modified algebraic Riccati equation (MARE) is constructed by further investigating the corresponding relationship with the state-feedback optimal control. To solve the derived MARE online, the vectorization operation and Kronecker's product are applied to reformulate the output Lyapunov function, and then, a new online data-driven learning method is suggested to learn its solution. Consequently, only the measurable system input and output are used to derive the solution of the MARE. In this case, the output-feedback robust control gain can be obtained without using the unknown system states. The control system stability and convergence of the derived solution are rigorously proved. Two simulation examples are provided to demonstrate the efficacy of the suggested methods.

In this paper, a novel adaptive integral reinforcement learning (AIRL) is utilized to online handle the finite-horizon optimum control policies of the partially unknown multi-input nonlinear system. Firstly, the concept of Nash equilibrium is introduced to make the multiple cost functions reach the saddle point. Then, dual neural networks (NNs) are applied to approach the performance index functions based on the integral reinforcement signal. Simultaneously, two novel learning algorithms are proposed to update the NN weights, in which the convergence of weights is proved. Then, the optimal strategies can be obtained by using the obtained weights. The designed controllers based on the data-driven AIRL scheme can avoid the internal state of the system and the derivatives of NN activations in the weight learning process. Finally, the stability of the controlled system is analyzed. An F-16 aircraft model and another nonlinear system are utilized to prove the validity and rationality of the algorithm. Note to Practitioners —There exist many multi-input systems in practical engineering, which includes multi-engine driven F-16 aircraft, large radar servo system and large artillery systems, etc. The finite-horizon optimal control of these systems is crucial for the better performance of system state. However, an accurate engineering model is difficult to obtain, and the finite-horizon cannot generally be achieved. To address these issues, this paper proposes the finite-horizon optimal control for these systems based on a novel adaptive integral reinforcement learning (AIRL). The AIRL can realize an optimal performance for these multi-input systems in finite-time without internal system dynamics, which is a good development for the multi-input system in practical engineering.

Although state-feedback optimal control of the continuous-time (CT) systems has been extensively studied, resolving optimal control online via output-feedback is still challenging, especially only input-output information can be used. In this paper, we develop an innovative technique to online design the output-feedback optimal control (OFOC) of the CT systems. Firstly, to synthesis the OFOC, an output-feedback algebraic Riccati equation (OARE) is constructed, which can be solved using input-output information. Then, an online policy learning (PL) algorithm is developed to compute the solution of the OARE, where only the input-output information is required and the conventional offline learning procedure is avoided. Simulations based on an aircraft model are provided to test the developed control method and online learning algorithm.