This article investigates the optimal synchronization problem for unknown discrete-time nonlinear heterogeneous multiagent systems (MASs). It is very intractable to derive the analytical solutions of coupled Bellman’s equations, which are necessary to overcome this problem. We propose a data-based optimal synchronization control strategy based on a hierarchical and distributed optimal control framework composed of a model reference adaptive control (MRAC) layer and a distributed control layer. In the MRAC layer, the similar-offline MRAC algorithm is developed to make subsystems of MASs track their reference models, respectively. Then, the distributed optimal control problem of nonlinear heterogeneous MASs is transformed into that of homogeneous MASs composed of the reference models and the leader. In the distributed control layer, the distributed reference policy iteration algorithm is proposed to derive the solutions of coupled composite nonlinear Bellman’s equations, which ensure that the homogeneous MASs reach synchronization with optimum. The suboptimal synchronization control is achieved via optimization further. Convergence analysis of both algorithms is rigorously provided. The simulation results verify the effectiveness of the proposed strategy.

This article is concerned with the optimal synchronization problem for discrete-time nonlinear heterogeneous multiagent systems (MASs) with an active leader. To overcome the difficulty in the derivation of the optimal control protocols for these systems, we develop an observer-based adaptive synchronization control approach, including the designs of a distributed observer and a distributed model reference adaptive controller with no prior knowledge of all agents' dynamics. To begin with, for the purpose of estimating the state of a nonlinear active leader for each follower, an adaptive neural network distributed observer is designed. Such an observer serves as a reference model in the distributed model reference adaptive control (MRAC). Then, a reinforcement learning-based distributed MRAC algorithm is presented to make every follower track its corresponding reference model on behavior in real time. In this algorithm, a distributed actor-critic network is employed to approximate the optimal distributed control protocols and the cost function. Through convergence analysis, the overall observer estimation error, the model reference tracking error, and the weight estimation errors are proved to be uniformly ultimately bounded. The developed approach further achieves the synchronization by means of synthesizing these results. The effectiveness of the developed approach is verified through a numerical example.

Internet of Vehicles (IoV) is a typical application of Internet of Things (IoT) technology in the field of intelligent transportation systems. In the actual IoV, such as the autonomous vehicle fleet, there exists the problem of heterogeneous multivehicle coordination based on IoT communication. How to ensure the synchronization of multiple vehicles is a hot issue. In particular, when the system can only obtain a partial state of the vehicle, and does not know the dynamic model, including the vehicle itself and the companion model. To overcome these deficiencies, this paper deals with the model-free output consensus control problem for a class of partially observable heterogeneous multi-vehicle systems (MVSs). Using measurable input/output data without any system knowledge, this paper develops a Q-function-based adaptive dynamic programming (ADP). First, an adaptive distributed observer is designed to estimate the output of the leader. The augmented state representation is built using historical measurable input/output data to instead of the unmeasurable inner system state. Then, a Q-function-based ADP method using measurable input/output data was introduced. The method is used to solve this distributed tracking control problem without the requirement for the MVSs dynamics. The convergence analysis of the proposed method is also given. To facilitate the implementation of the proposed method, an actor-critic framework is adopted to approximate the optimal Q-functions and the optimal control policies. It shows that the approximated control policies achieve the distributed optimal tracking control. Finally, the simulation results verify the effectiveness of the developed method for solving multi-vehicle formation control problems.

The local navigation with collision avoidance is becoming increasingly important for the mobile robot in the crowd scenario. Previous work mainly concerns its point-to-point local navigation via deep reinforcement learning (DRL). However, applying DRL to the local path-following navigation poses extra challenges in generating smooth trajectory and enhancing safety. This article presents a danger-aware robot navigation algorithm by defining the pedestrians’ danger and introducing a virtual robot about the reference path. The main novelty of this algorithm is that the virtual robot is leveraged to derive the extra action and more sampling waypoints in pursuit of the robot motion smoothness and foresight. Moreover, a priority mechanism is established and incorporated into DRL navigation, so as to enhance safety of robot navigation. Experiments on the path-following social navigation demonstrate that our presented algorithm outperforms the state-of-the-art method in terms of the motion smoothness and the safety via evaluation metrics.