In this paper, a novel event-sampled robust optimal controller is proposed for a class of continuous-time constrained-input nonlinear systems with unknown dynamics. In order to solve the robust optimal control problem, an online data-driven identifier is established to construct the system dynamics, and an event-sampled critic-only adaptive dynamic programming method is developed to replace the conventional time-driven actor-critic structure. The designed online identification method runs during the solving process and is not applied as a priori part for the solutions, which simplifies the architecture and reduces computational load. The proposed robust optimal control algorithm tunes the parameters of critic-only neural network (NN) by event-triggering condition and runs in a plug-n-play framework without system functions, where fewer transmissions and less computation are required as all the measurements received simultaneously. Based on the novel design, the stability of system and the convergence of critic NN are demonstrated by Lyapunov theory, where the state is asymptotically stable and weight error is guaranteed to be uniformly ultimately bounded. Finally, the applications in a basic nonlinear system and the complex rotational-translational actuator problem demonstrate the effectiveness of the proposed method.

In this paper, a data-based control method based on adaptive dynamic programming (ADP) algorithm is proposed for a class of discrete-time (DT) systems in the case of multiple delays. Data-based ADP method is implemented by virtue of the measured input and output data. The condition of the existence of the corresponding equivalent multiple delays system is derived according to the characteristics of time-delay system. A novel data-based state equation is developed that only composed of input and output data, which is very meaningful in practical applications. By using the data-based ADP method, the output feedback control problem is solved only by measuring the input and output of the system with multiple delays. The convergence proofs of the designed policy iteration and value iteration algorithms are given, respectively. The simulation example is presented to demonstrate the validity of the proposed data-based ADP method.

This article is concerned with the convergence property and error bounds analysis of value iteration (VI) adaptive dynamic programming for continuous-time (CT) nonlinear systems. The size relationship between the total value function and the single integral step cost is described by assuming a contraction assumption. Then, the convergence property of VI is proved while the initial condition is an arbitrary positive semidefinite function. Moreover, the accumulated effects of approximation errors generated in each iteration are taken into consideration while using approximators to implement the algorithm. Based on the contraction assumption, the error bounds condition is proposed, which ensures the approximated iterative results converge to a neighborhood of the optimum, and the relation between the optimal solution and approximated iterative results is also derived. To make the contraction assumption more concrete, an estimation way is proposed to derive a conservative value of the assumption. Finally, three simulation cases are given to validate the theoretical results.

In this paper, a novel integral reinforcement learning method is proposed based on value iteration (VI) to design the $H_{\infty }$ controller for continuous-time nonlinear systems subject to input constraints. To confront the control constraints, a nonquadratic function is introduced to reconstruct the ${L_{2}}$ -gain condition for the $H_{\infty }$ control problem. Then, the VI method is proposed to solve the corresponding Hamilton–Jacobi–Isaacs equation initialized with an arbitrary positive semi-definite value function. Compared with most existing works developed based on policy iteration, the initial admissible control policy is no longer required which results in a more free initial condition. The iterative process of the proposed VI method is analyzed and the convergence to the saddle point solution is proved in a general way. For the implementation of the proposed method, only one neural network is introduced to approximate the iterative value function, which results in a simpler architecture with less computational load compared with utilizing three neural networks. To verify the effectiveness of the VI-based method, two nonlinear cases are presented, respectively.

In recent years, with the rapid development of network applications and the increasing demand for high-quality network service, quality-of-service (QoS) routing has emerged as a critical network technology. The application of machine learning techniques, particularly reinforcement learning and graph neural network, has garnered significant attention in addressing this problem. However, existing reinforcement learning methods lack research on the causal impact of agent actions on the interactive environment, and graph neural network fail to effectively represent link features, which are pivotal for routing optimization. Therefore, this study quantifies the causal influence between the intelligent agent and the interactive environment based on causal inference techniques, aiming to guide the intelligent agent in improving the efficiency of exploring the action space. Simultaneously, graph neural network is employed to embed node and link features, and a reward function is designed that comprehensively considers network performance metrics and causality relevance. A centralized reinforcement learning method is proposed to effectively achieve QoS-aware routing in Software-Defined Networking (SDN). Finally, experiments are conducted in a network simulation environment, and metrics such as packet loss, delay, and throughput all outperform the baseline.