In many existing multi-agent reinforcement learning tasks, each agent observes all the other agents from its own perspective. In addition, the training process is centralized, namely the critic of each agent can access the policies of all the agents. This scheme has certain limitations since every single agent can only obtain the information of its neighbor agents due to the communication range in practical applications. Therefore, in this paper, a multi-agent distributed deep deterministic policy gradient (MAD3PG) approach is presented with decentralized actors and distributed critics to realize multi-agent distributed tracking. The distinguishing feature of the proposed framework is that we adopted the multi-agent distributed training with decentralized execution, where each critic only takes the agent’s and the neighbor agents’ policies into account. Experiments were conducted in the distributed tracking tasks based on multi-agent particle environments where $N(N=3,N=5)$ agents track a target agent with partial observation. The results showed that the proposed method achieves a higher reward with a shorter training time compared to other methods, including MADDPG, DDPG, PPO, and DQN. The proposed novel method leads to a more efficient and effective multi-agent tracking.

In this paper, the stability problem of a class of multi-agent tracking systems with quasi-cyclic switching topologies is investigated. The existing results of systems with switching topologies are usually achieved based on the assumption that the piecewise constant communication topologies are connected and the switchings are cyclic. The communication topologies are possible to be unconnected and it is difficult to guarantee the topologies switch circularly. The piecewise unconnected topology makes the interactive multi-agent tracking system to be an unstable subsystem over this time interval. In order to relax the assumption constraint, a quasi-cyclic method is proposed, which allows the topologies of multi-agent systems to switch in a less conservative way. Moreover, the stability of the tracking system with the existence of unstable subsystems is analyzed based on switched control theory. It is obtained that the convergence rate is affected by the maximum dwell time of unstable subsystems. Finally, a numerical example is provided to demonstrate the effectiveness of the theoretical results.

High-speed trains are prone to severe lateral vibrations induced by track irregularity excitations under complex operating conditions, which deteriorate ride comfort, reduce running stability, and accelerate wheel–rail wear. Existing lateral suspension vibration control methods mainly rely on fixed parameters and expert experience. To further optimize and improve the control performance of semi-active lateral suspension systems for high-speed trains, this paper proposes the Physics-Constrained Dual-Attention Reinforcement Learning for Semi-Active Lateral Vibration Control of High-Speed Trains (SA-PRL). The proposed method is built upon a semi-active Twin Delayed Deep Deterministic Policy Gradient algorithm (SATD3). To ensure the physical realizability of control actions, the Logical Constraints of Skyhook Control (LCSC) are introduced to map the controller output into physically feasible damping commands. Furthermore, to enhance the perception of critical state features and improve value estimation capability, a Critic Network with Dual-Head Self-Attention (CDHA) is developed. Based on a railway vehicle lateral dynamic model, a series of simulation experiments are conducted under four excitation conditions, including single-peak sinusoidal excitation and multi-peak sinusoidal excitation, as well as the Chinese high-speed railway track irregularity (CHSRTI) and German low-interference track irregularity (GLITI) excitations. The results demonstrate the lateral vibration suppression capability of the proposed method over passive suspension, skyhook damping control, and existing reinforcement learning methods, with reductions of 47.5% and 47.7% in the RMS carbody lateral acceleration under the CHSRTI and GLITI spectra, respectively. The proposed method provides an effective solution for semi-active lateral vibration control of high-speed trains.

Railway intrusion detection is crucial for ensuring the safety and reliability of railway operations. Deep learning-based railway intrusion detection (RID) methods struggle with unknown foreign objects, while existing pixel-level out-of-distribution (OoD) detection methods lack spatial awareness for railway scenes. To address this limitation, we propose an uncertainty-driven spatial-aware energy learning (SAEL) method for railway intrusion detection. Our approach introduces a spatial-aware energy learning strategy that incorporates a distance transform and a Gaussian smoothing weighting matrix to prioritize OoD pixels near the boundaries of in-distribution pixels. Additionally, we design a texture representation module to enhance the model’s ability to capture edge and texture details of anomalous regions. We evaluate our method on both public datasets and a self-constructed dataset. Extensive experiments on both public datasets and our self-constructed OoD-RID test set demonstrate that our approach significantly outperforms state-of-the-art (SOTA) baseline methods. On the OoD-RID test set, the FPR95 is reduced by 10.68%, the AuPRC is improved by 5.40%, and the AuROC is increased by 1.73%. This work provides a robust and practical OoD segmentation solution for unseen intrusion detection in complex railway scenes, demonstrating a favorable balance between detection performance and computational efficiency for practical railway applications.