21 April 2021 in IEEE Transactions on Neural Networks and Learning Systems
In this article, we investigate the routing problem of packet networks through multiagent reinforcement learning (RL), which is a very challenging topic in distributed and autonomous networked systems. In specific, the routing problem is modeled as a networked multiagent partially observable Markov decision process (MDP). Since the MDP of a network node is not only affected by its neighboring nodes’ policies but also the network traffic demand, it becomes a multitask learning problem. Inspired by recent success of RL and metalearning, we propose two novel model-free multiagent RL algorithms, named multiagent proximal policy optimization (MAPPO) and multiagent metaproximal policy optimization (meta-MAPPO), to optimize the network performances under fixed and time-varying traffic demand, respectively. A practicable distributed implementation framework is designed based on the separability of exploration and exploitation in training MAPPO. Compared with the existing routing optimization policies, our simulation results demonstrate the excellent performances of the proposed algorithms.
13 February 2020 in IEEE Transactions on Industrial Electronics
Many cooperative robotic systems have not only modeling heterogeneity and uncertainty but also switched couplings, causing control difficulties. Here we develop a neural network adaptive control framework for cooperative robot manipulators with unknown Euler-Lagrange dynamics and Markovian switched couplings. Second-order Markovian switching networks are used for modeling such cooperative robotic systems, which also admit a hybrid neural network control with a desired tracking performance. The hybrid neural network control scheme contains a distributed adaptive controller and a hybrid adaptation law, enabling learning in the closed-loop system. The position and velocity tracking errors are shown to be practically uniformly exponentially stable in the mean-square sense, respectively, guaranteeing the second-order practical tracking. The results also suggest that the neural weight evolves with practical convergence to the ideal, showing the effect of network structure on the adaptation capacity.
6 October 2020 in IEEE Transactions on Circuits and Systems I: Regular Papers
28 April 2021 in IEEE Transactions on Cybernetics
This article surveys the interdisciplinary research of neuroscience, network science, and dynamic systems, with emphasis on the emergence of brain-inspired intelligence. To replicate brain intelligence, a practical way is to reconstruct cortical networks with dynamic activities that nourish the brain functions, instead of using only artificial computing networks. The survey provides a complex network and spatiotemporal dynamics (abbr. network dynamics) perspective for understanding the brain and cortical networks and, furthermore, develops integrated approaches of neuroscience and network dynamics toward building brain-inspired intelligence with learning and resilience functions. Presented are fundamental concepts and principles of complex networks, neuroscience, and hybrid dynamic systems, as well as relevant studies about the brain and intelligence. Other promising research directions, such as brain science, data science, quantum information science, and machine behavior are also briefly discussed toward future applications.
9 February 2022 in IEEE Transactions on Automatic Control
In this paper, the controllability is investigated for nonlinear impulsive and switching systems with input delay. Using the fixed point method, two controllability conditions are developed for such systems under different nonlinear constraints. First, a Lipschitz nonlinearity is considered. A control mapping is constructed based on the input matrix condition, and a nonlinear operator is introduced to convert the controllability into the existence of a fixed point. A sufficient controllability condition is obtained under the Banach's contraction mapping principle. Secondly, a linear growth condition is imposed on the nonlinearity. A switched control mapping is constructed using the inverse of controllability matrix, and a corresponding nonlinear operator is introduced. An alternative controllability condition is then developed by resorting to the Rothe's fixed point theorem. It is shown that the system controllability can be influenced by the impulsive and switching factors, beyond the delayed part and the nonlinearity. Two numerical examples are given to verify the two controllability conditions.