As an auxiliary facility, roadside units (RSUs) can well improve the shortcomings incurred by ad hoc networks and promote network performance in a vehicular ad hoc network (VANET). However, deploying a large number of RSUs will lead to high installation and maintenance costs. Therefore, trying to find the best locations is a key issue when deploying RSUs with the set delay and budget. In this paper, we study the delay-bounded and cost-limited RSU deployment (DBCL) problem in urban VANET. We prove it is non-deterministic polynomial-time hard (NP-hard), and a binary differential evolution scheme is proposed to maximize the number of roads covered by deploying RSUs. Opposite-based learning is introduced to initialize the first generation, and a binary differential mutation operator is designed to obtain binary coding. A random variable is added to the traditional crossover operator to increase population diversity. Also, a greedy-based individual reparation and promotion algorithm is adopted to repair infeasible solutions violating given constraints, and to gain optimal feasible solutions with the compromise of given limits. Moreover, after selection, a solution promotion algorithm is executed to promote the best solution found in generation. Simulation is performed on analog trajectories sets, and results show that our proposed algorithm has a higher road coverage ratio and lower packet loss compared with other schemes.

Multi-agent systems often face challenges such as elevated communication demands, intricate interactions, and difficulties in transferability. To address the issues of complex information interaction and model scalability, we propose an innovative hierarchical graph attention actor–critic reinforcement learning method. This method naturally models the interactions within a multi-agent system as a graph, employing hierarchical graph attention to capture the complex cooperative and competitive relationships among agents, thereby enhancing their adaptability to dynamic environments. Specifically, graph neural networks encode agent observations as single feature-embedding vectors, maintaining a constant dimensionality irrespective of the number of agents, which improves model scalability. Through the “inter-agent” and “inter-group” attention layers, the embedding vector of each agent is updated into an information-condensed and contextualized state representation, which extracts state-dependent relationships between agents and model interactions at both individual and group levels. We conducted experiments across several multi-agent tasks to assess our proposed method’s effectiveness, stability, and scalability. Furthermore, to enhance the applicability of our method in large-scale tasks, we tested and validated its performance within a curriculum learning training framework, thereby enhancing its transferability.

Deep reinforcement learning-based Multi-Agent Path Finding (MAPF) has gained significant attention due to its remarkable adaptability to environments. Existing methods primarily leverage multi-agent communication in a fullydecentralized framework to maintain scalability while enhancing information exchange among agents. However, as the number of agents and obstacles increases, the environment becomes more complex, making cooperation between agents becomes more difficult, and crowding occurs from time to time. To address these issues, we propose a decentralized planner C3PIL, which integrates a Controlled Communication mechanism for Crowd Perception and uses Imitation Learning to improve policy learning. C3PIL first introduces a crowd perception communication module that perceives environmental crowd information and incorporates it into the controlled communication. This effectively prevents and mitigates crowded situations. Furthermore, we employ generative adversarial imitation learning to learn a reward function from expert experiences. It reduces the possible misleading caused by the fixed reward function, improves the flexibility and diversity of agent behaviors, and ultimately enables agents to cooperate effectively. Finally, experimental results show that C3PIL not only outperforms previous learning-based MAPF methods, but also further enhances the cooperation of agents and significantly reduces crowding in complex environments. The code is available at https://github.com/JJingXie/C3PIL.

With the widespread application of robots in aerospace, medicine, automation, and other fields, their motion safety is essential for the well-being of humans and the accomplishment of vital socially beneficial programs. Conventional robot hardware and software designs mainly rely on experiential knowledge and manual testing to ensure safety, but this fails to cover all possible testing paths and adds risks. Alternatively, formal, mathematically rigorous verifications can provide predictable and reliable guarantees of robot motion safety. To demonstrate the feasibility of this approach, we formalize the mathematical coordinate transformation of a robot’s rigid-body kinematics using the Coq Proof Assistant to verify the correctness of its theoretical design. First, based on record-type matrix formalization, we define and verify a robot’s spatial geometry by constructing formal expressions of the matrix’ Frobenius norm, trace, and inner product. Second, we divide rotary motion into revolution and rotation construct and provide their formal definitions. Next, we formally verify the rotational matrices of angle conventions (e.g., roll–pitch–yaw and Euler), and we complete the formal verification of the Rodriguez formula to formally verify the correctness of the motion theory in specific rotating kinematics problems. The formal work of this paper has a variety of essential applications and provides a generalizable kinematics analysis framework for robot control system verification. Moreover, it paves the way for automatic programming capabilities.

Learning optimal policies in multi-agent cooperative settings with visual observations is significant and challenging. Agents must first perform state representation learning for their image observations and then learn policies in the abstracted state space. Aiming at this problem, we propose a novel model-based MARL method named Contrastive Latent World for Policy Optimization (CLWPO). In CLWPO, we first design a state representation model to facilitate learning in the latent state space. With the support of this model, we construct the latent world and introduce a contrastive variational bound (CVB) to optimize it. Subsequently, we develop a heuristic policy optimization (HPO) scheme, incorporating model-free learning with model-based planning to obtain robust policies that predict future behaviors. In particular, in the planning, we maintain a queue of teammate models and calculate an adaptive rollout length for each agent to support their self-imagination and reduce the model-based return discrepancy. Finally, we conducted extensive experiments in the PettingZoo benchmark, and results show that CLWPO significantly enhances learning efficiency and improves agent performance compared to state-of-the-art MARL methods.

Multi-agent reinforcement learning has been widely applied in solving various sequential decision-making problems in recent years. However, a key challenge is sparse rewards, where agents receive meaningful reward signals only upon task completion. This issue leads to inefficient exploration and slow learning progress. To address this problem, we propose an excellent-student learning method supported by a decentralized distributed learning paradigm, drawing inspiration from academically diverse classrooms. Specifically, we design a novel excellent-student learning model, which suggests that agents mimic the learning behaviors of excellent students. This model requires agents to share knowledge with other agents and engage in individual exploration driven by curiosity. Next, to foster collaborative team learning behaviors, similarity measurement techniques are integrated to enhance knowledge sharing among agents. An intrinsic reward function is designed, combining individual exploration with whole-class sharing, providing additional motivation for discovering new actions and states. This reward function is seamlessly incorporated into the policy learning process. Finally, experiments conducted in various multi-agent particle environments demonstrate significant improvements in training efficiency and stability.