In this article, for the nonzero-sum (NZS) differential games problem of uncertain nonlinear systems with state constraints, an adaptive robust stabilization scheme based on the control barrier function (CBF) is presented under the influence of random disturbances and control input matrix uncertainty. To deal with the impact of uncertainty on the system, the nominal system of the original system is adopted and the cost functions associated with each player are appropriately chosen to convert the robust regulation problem of multiplayer differential games into an optimal regulation problem. Furthermore, the purpose of combining the cost function relevant to each player with the CBF is to make the system states evolve in the safe area. Different from the classical actor–critic dual neural network (NN), each player only needs a critic NN to approach the corresponding cost function without the restriction of the initial stabilizing control. Combined with the Lyapunov stability theory, under the combined influence of random disturbances and state constraints, the state and critic NN weights of the closed-loop system are guaranteed to be uniformly ultimately bounded (UUB). Finally, two simulation examples are used to verify the validity of the presented scheme.

Reinforcement learning (RL), as an emerging interdisciplinary field formed by the integration of artificial intelligence and control science, is currently demonstrating a cross-disciplinary development trend led by artificial intelligence and has become a research hotspot in the field of optimal control. This paper systematically reviews the development context of RL, focusing on the intrinsic connection between single-agent reinforcement learning (SARL) and multi-agent reinforcement learning (MARL). Firstly, starting from the formation and development of RL, it elaborates on the similarities and differences between RL and other learning paradigms in machine learning, and briefly introduces the main branches of current RL. Then, with the basic knowledge and core ideas of SARL as the basic framework, and expanding to multi-agent system (MAS) collaborative control, it explores the coherence characteristics of the two in theoretical frameworks and algorithm design. On this basis, this paper reconfigures SARL algorithms into dynamic programming, value function decomposition and policy gradient (PG) type, and abstracts MARL algorithms into four paradigms: behavior analysis, centralized learning, communication learning and collaborative learning, thus establishing an algorithm mapping relationship from single-agent to multi-agent scenarios. This innovative framework provides a new perspective for understanding the evolutionary correlation of the two methods, and also discusses the challenges and solution ideas of MARL in solving large-scale MAS problems. This paper aims to provide a reference for researchers in this field, and to promote the development of cooperative control and optimization methods for MAS as well as the advancement of related application research.

The surface defect detection (SDD) problem is one of the crucial techniques during production process, so it has become a key research area to control the quality of industrial products, which has been increasingly of greater interest to the researchers especially with the rapid development of artificial neural networks technology in recent years. Therefore, this paper proposes a novel deep convolutional neural network algorithm aiming at SDD. Firstly, a dense cross-stage partial Darknet backbone network is designed for feature extraction by optimizing cross-stage partial Darknet through the idea of dense connections, which can, not only enhance feature reuse but also greatly alleviate the overfitting issue. Secondly, a new cross-stage hierarchy module is presented combining the cross-stage feature fusion strategy and depthwise separable convolution technique for each node of the path aggregated feature pyramid network (PAN). Finally, an efficient channel attention (ECA) mechanism is introduced in PAN to construct a novel ECA PAN. The experimental results on three surface defect datasets show that the mean average precision of this network is 2.63, 5.48, and 1.16$\%$ which is higher than that of the baseline network, respectively. The proposed network outperforms not only the classical models but state-of-the-art models, which indicates the proposed algorithm can achieve higher accuracy and speed with fewer calculation parameters. And what is more, the proposed algorithm also has outstanding generalization ability.