22 July 2020 in IEEE Transactions on Cybernetics
This article presents an event-sampled integral reinforcement learning algorithm for partially unknown nonlinear systems using a novel dynamic event-triggering strategy. This is a novel attempt to introduce the dynamic triggering into the adaptive learning process. The core of this algorithm is the policy iteration technique, which is implemented by two neural networks. A critic network is periodically tuned using the integral reinforcement signal, and an actor network adopts the event-based communication to update the control policy only at triggering instants. For overcoming the deficiency of static triggering, a dynamic triggering rule is proposed to determine the occurrence of events, in which an internal dynamic variable characterized by a first-order filter is defined. Theoretical results indicate that the impulsive system driven by events is asymptotically stable, the network weight is convergent, and the Zeno behavior is successfully avoided. Finally, three examples are provided to demonstrate that the proposed dynamic triggering algorithm can reduce samples and transmissions even more, with guaranteed learning performance.
1 July 2024 in IEEE Transactions on Automation Science and Engineering
23 February 2021 in IEEE Transactions on Neural Networks and Learning Systems
Static event-triggering-based control problems have been investigated when implementing adaptive dynamic programming algorithms. The related triggering rules are only current state-dependent without considering previous values. This motivates our improvements. This article aims to provide an explicit formulation for dynamic event-triggering that guarantees asymptotic stability of the event-sampled nonzero-sum differential game system and desirable approximation of critic neural networks. This article first deduces the static triggering rule by processing the coupling terms of Hamilton–Jacobi equations, and then, Zeno-free behavior is realized by devising an exponential term. Subsequently, a novel dynamic-triggering rule is devised into the adaptive learning stage by defining a dynamic variable, which is mathematically characterized by a first-order filter. Moreover, mathematical proofs illustrate the system stability and the weight convergence. Theoretical analysis reveals the characteristics of dynamic rule and its relations with the static rules. Finally, a numerical example is presented to substantiate the established claims. The comparative simulation results confirm that both static and dynamic strategies can reduce the communication that arises in the control loops, while the latter undertakes less communication burden due to fewer triggered events.
2 June 2020 in IEEE Transactions on Industrial Informatics
For the practical control system, the controller is normally implemented on a digital platform with a time-triggered scheme. This scheme maybe produces redundant control and resources wasting, and hence an event-triggered scheme is gradually favored. In this paper, the robust learning control scheme is proposed aiming at a class of disturbed control systems, in which the system information is processed by a novel dynamic event communication. First, the robust optimal control problem with external disturbances is re-described as a zero-sum differential game, and with integral reinforcement learning, a model-independent weight tuning law is devised for a critic neural network. Then, in order to further reduce the computational burden, an additional dynamic variable is put forward to incorporate the past triggering information. The application of a single-link joint arm system demonstrates that the proposed scheme can guarantee learning performance and robust control effect, along with larger triggering intervals. Finally, the load frequency control problem of single-area power system is studied. On one hand, the comparative results of five control schemes reveal that the dynamic event scheme can achieve the better frequency response at the lowest information transmission rate. On the other hand, the advantages of the proposed method are illustrated by comparing with other three event-triggered schemes.
26 September 2025 in IEEE Internet of Things Journal
This article presents a safety-critical control framework for navigation in complex environments with numerous obstacles. An online robust path planning scheme is developed by integrating reinforcement learning (RL) with control barrier functions (CBFs). First, a disturbance observer is designed to estimate the unknown disturbance along with a derived upper bound of the estimation error. Then, a nominal controller is designed using RL, where a critic neural network (NN) structure is established by using state-following (StaF) kernel function. Additionally, by employing a state extrapolation technique, the learning process leverages both real-time and simulated experience data. To ensure safety, obstacle avoidance is formulated as a forward invariance problem of safe sets defined by CBFs. Subsequently, the CBF-based safety-critical constraints are integrated into a quadratic programming (QP) framework to modify the nominal controller. Furthermore, these CBFs are incorporated into a composite CBF using smooth approximation, enabling efficient constraint consolidation. Then, an explicit safe control policy is proposed that guarantees collision-free path planning. Finally, the effectiveness of the proposed scheme is demonstrated through numerical simulations, and comparative results show the advantages over the existing methods in motion trajectory.
19 April 2022 in IEEE Transactions on Industrial Informatics
The optimal control of multi-input system can be described by a multiplayer nonzero-sum differential game. This article theoretically presents an event-based adaptive learning scheme to approximate the Nash equilibrium, and practically addresses the cruise control problem for Caltech vehicle systems. This design is deployed in two aspects. On one hand, the reinforcement learning is implemented through critic neural network architecture and recalling stored experience data. On the other hand, in view of that each player's preference is different, the decentralized triggering manner is considered to reduce communication. Based on the continuous state, the local sampled state is defined for each player, and a static triggering mechanism is formulated first. The decentralized dynamic triggering is then promoted by designing an auxiliary variable whose dynamics is constructed using static triggering information. The learning-based controller is tested on a single-vehicle system under different tracking commands, and then it is extended to multi-vehicle systems to realize cooperative optimization by introducing a novel game-in-game structure. Finally, the effectiveness of the proposed learning scheme is substantiated by comparison.