18 May 2022 in IEEE Transactions on Neural Networks and Learning Systems
Although value decomposition networks and the follow on value-based studies factorizes the joint reward function to individual reward functions for a kind of cooperative multiagent reinforcement problem, in which each agent has its local observation and shares a joint reward signal, most of the previous efforts, however, ignored the graphical information between agents. In this article, a new value decomposition with graph attention network (VGN) method is developed to solve the value functions by introducing the dynamical relationships between agents. It is pointed out that the decomposition factor of an agent in our approach can be influenced by the reward signals of all the related agents and two graphical neural network-based algorithms (VGN-Linear and VGN-Nonlinear) are designed to solve the value functions of each agent. It can be proved theoretically that the present methods satisfy the factorizable condition in the centralized training process. The performance of the present methods is evaluated on the StarCraft Multiagent Challenge (SMAC) benchmark. Experiment results show that our method outperforms the state-of-the-art value-based multiagent reinforcement algorithms, especially when the tasks are with very hard level and challenging for existing methods.
30 November 2022 in IEEE Transactions on Neural Networks and Learning Systems
The automated guided vehicle (AGV) dispatching problem is to develop a rule to assign transportation tasks to certain vehicles. This article proposes a new deep reinforcement learning approach with a self-attention mechanism to dynamically dispatch the tasks to AGV. The AGV dispatching system is modeled as a less complicated Markov decision process (MDP) using vehicle-initiated rules to dispatch a workcenter to an idle AGV. In order to deal with the highly dynamical environment, the self-attention mechanism is introduced to calculate the importance of different information. The invalid action masking technique is performed to alleviate false actions. A multimodal structure is employed to mix the features of various sources. Comparative experiments are performed to show the effectiveness of the proposed method. The properties of the learned policies are also investigated under different environment settings. It is discovered that the policies explore and learn the properties of different systems, and also smooth the traffic congestion. Under certain environment settings, the policy converges to a heuristic rule that assigns the idle AGV to the workcenter with the shortest queue length, which shows the adaptiveness of the proposed method.
18 March 2024 in IEEE Transactions on Cybernetics
In this article, the problem of inferring unknown isoperimetric constraints is considered given optimal state and control trajectories that solve the optimal control problem with isoperimetric constraints. By exploiting Pontryagin’s principle, the recovery equations for unknown isoperimetric constraints are established. Under verifiable dimensionality condition and matrix rank condition, the proposed method is guaranteed to infer the unknown isoperimetric constraints exactly. Furthermore, the proposed method is extended to multiple trajectory setting. Finally, the effectiveness of the proposed method is illustrated by two simulation examples with various settings.
12 January 2024 in Machine Intelligence Research
1 January 2014 in IFAC Proceedings Volumes