This article investigates an adaptive reinforcement learning (RL) optimal control design problem for a class of nonstrict-feedback discrete-time systems. Based on the neural network (NN) approximating ability and RL control design technique, an adaptive backstepping RL optimal controller and a minimal learning parameter (MLP) adaptive RL optimal controller are developed by establishing a novel strategic utility function and introducing external function terms. It is proved that the proposed adaptive RL optimal controllers can guarantee that all signals in the closed-loop systems are semiglobal uniformly ultimately bounded (SGUUB). The main feature is that the proposed schemes can solve the optimal control problem that the previous literature cannot deal with. Furthermore, the proposed MPL adaptive optimal control scheme can reduce the number of adaptive laws, and thus the computational complexity is decreased. Finally, the simulation results illustrate the validity of the proposed optimal control schemes.

In this paper, an adaptive neural network (NN) control problem is investigated for discrete-time nonlinear systems with input saturation. Radial-basis-function (RBF) NNs, including critic NNs and action NNs, are employed to approximate the utility functions and system uncertainties, respectively. In the previous works, a gradient descent scheme is applied to update weight vectors, which may lead to local optimal problem. To circumvent this problem, a multigradient recursive (MGR) reinforcement learning scheme is proposed, which utilizes both the current gradient and the past gradients. As a consequence, the MGR scheme not only eliminates the local optimal problem but also guarantees faster convergence rate than the gradient descent scheme. Moreover, the constraint of actuator input saturation is considered. The closed-loop system stability is developed by using the Lyapunov stability theory, and it is proved that all the signals in the closed-loop system are semiglobal uniformly ultimately bounded (SGUUB). Finally, the effectiveness of the proposed approach is further validated via some simulation results.

This article investigates the model-free fault-tolerant containment control problem for multiagent systems (MASs) with time-varying actuator faults. Depending on the relative state information of neighbors, a distributed containment control method based on reinforcement learning (RL) is adopted to achieve containment control objective without prior knowledge on the system dynamics. First, based on the information of agent itself and its neighbors, a containment error system is established. Then, the optimal containment control problem is transformed into an optimal regulation problem for the containment error system. Furthermore, the RL-based policy iteration method is employed to deal with the corresponding optimal regulation problem, and the nominal controller is proposed for the original fault-free system. Based on the nominal controller, a fault-tolerant controller is further developed to compensate for the influence of actuator faults on MAS. Meanwhile, the uniform boundedness of the containment errors can be guaranteed by using the presented control scheme. Finally, numerical simulations are given to show the effectiveness and advantages of the proposed method.

In this article, the tracking control problem of event-triggered multigradient recursive reinforcement learning is investigated for nonlinear multiagent systems (MASs). Attention is focused on the distributed reinforcement learning approach for MASs. The critic neural network (NN) is applied to estimate the long-term strategic utility function, and the actor NN is designed to approximate the uncertain dynamics in MASs. The multigradient recursive (MGR) strategy is tailored to learn the weight vector in NN, which eliminates the local optimal problem inherent in gradient descent method and decreases the dependence of initial value. Furthermore, reinforcement learning and event-triggered mechanism can improve the energy conservation of MASs by decreasing the amplitude of the controller signal and the controller update frequency, respectively. It is proved that all signals in MASs are semiglobal uniformly ultimately bounded (SGUUB) according to the Lyapunov theory. Simulation results are given to demonstrate the effectiveness of the proposed strategy.

In this article, the asynchronous fault detection (FD) strategy is investigated in frequency domain for nonlinear Markov jump systems under fading channels. In order to estimate the system dynamics and meet the fact that not all the running modes can be observed exactly, a set of asynchronous FD filters is proposed. By using statistical methods and the Lynapunov stability theory, the augmented system is shown to be stochastic stable with a prescribed $l_{2}$ gain even under fading transmissions. Then, a novel lemma is developed to capture the finite frequency performance. Some solvable conditions with less conservatism are subsequently deduced by exploiting novel decoupling techniques and additional slack variables. Besides, the FD filter gains could be calculated with the aid of the derived conditions. Finally, the effectiveness of the proposed method is shown by an illustrative example.