This article presents and practically validates an identifier-critic-based approximate dynamic programming (ADP) method to online address the optimal tracking control problem for nonlinear continuous-time unknown systems. The imposed assumption on precisely known system dynamics is obviated via a neural network (NN) identifier. A static control is first adopted to retain the steady-state tracking response, while an optimal control derived via the ADP method is proposed to regulate the tracking error by minimizing a cost function. A critic NN is then trained online to obtain the solution of the associated Hamilton-Jacobi-Bellman (HJB) equation. The learning of the identifier NN and critic NN is performed online simultaneously by tailoring a novel adaptation method, which can guarantee the convergence of the estimated NN weights. Consequently, the critic NN can be used to construct the optimal control policy directly, such that the actor NN used in the previous ADP schemes is avoided. Simulations are performed to verify the suggested control, and experiments on a helicopter plant are carried out to show its feasibility and improved control response.

An online adaptive optimal control is proposed for continuous-time nonlinear systems with completely unknown dynamics, which is achieved by developing a novel identifier-critic-based approximate dynamic programming algorithm with a dual neural network (NN) approximation structure. First, an adaptive NN identifier is designed to obviate the requirement of complete knowledge of system dynamics, and a critic NN is employed to approximate the optimal value function. Then, the optimal control law is computed based on the information from the identifier NN and the critic NN, so that the actor NN is not needed. In particular, a novel adaptive law design method with the parameter estimation error is proposed to online update the weights of both identifier NN and critic NN simultaneously, which converge to small neighbourhoods around their ideal values. The closed-loop system stability and the convergence to small vicinity around the optimal solution are all proved by means of the Lyapunov theory. The proposed adaptation algorithm is also improved to achieve finite-time convergence of the NN weights. Finally, simulation results are provided to exemplify the efficacy of the proposed methods.

Inspired by Nash game theory, a multiplayer mixed-zero-sum (MZS) nonlinear game considering both two situations [zero-sum and nonzero-sum (NZS) Nash games] is proposed in this paper. A synchronous reinforcement learning (RL) scheme based on the identifier-critic structure is developed to learn the Nash equilibrium solution of the proposed MZS game. First, the MZS game formulation is presented, where the performance indexes for players 1 to N - 1 and N NZS Nash game are presented, and another performance index for players N and N + 1 zero-sum game is presented, such that player N cooperates with players 1 to N - 1, while competes with player N + 1, which leads to a Nash equilibrium of all players. A single-layer neural network (NN) is then used to approximate the unknown dynamics of the nonlinear game system. Finally, an RL scheme based on NNs is developed to learn the optimal performance indexes, which can be used to produce the optimal control policy of every player such that Nash equilibrium can be obtained. Thus, the widely used actor NN in RL literature is not needed. To this end, a recently proposed adaptive law is used to estimate the unknown identifier coefficient vectors, and an improved adaptive law with the error performance index is further developed to update the critic coefficient vectors. Both linear and nonlinear simulations are presented to demonstrate the existence of Nash equilibrium for MZS game and performance of the proposed algorithm.

This paper develops a novel adaptive $H_{\infty }$ control scheme for adaptive cruise control (ACC) system via the data-driven learning. In the proposed technique, a continuous time ACC system is first constructed with unknown system dynamics. To estimate the unknown system dynamics, an adaptive estimator is then formulated utilizing the vectorization and Kronecker's products operations, enabling the reconstruction of the unknown system dynamics through the detectable input/output information. An adaptive law is utilized to ensure the convergence of the estimated parameters. Moreover, a data-driven learning technique is employed to resolve the constructed Riccati equation in an online manner. To accomplish this, the Riccati equation is reformulated via the Kronecker's products by using another adaptive law, whose convergence can be also effectively guaranteed. Unlike existing neural network-based approximate dynamic programming (ADP) algorithms, the data-driven learning scheme proposed in this paper does not involve neural network approximation errors, so the solution is relatively more accurate. Finally, simulation and experimental verification results are provided to verify the effectiveness of the presented $H_{\infty }$ control and data-driven learning algorithm.