One of the challenges in multi-agent systems comes from the environmental non-stationarity that policies of all agents are evolving individually over time. Many existing multi-agent reinforcement learning (MARL) methods have been proposed to address this problem. However, these methods rely on a large amount of training data and some of them require agents to intensely communicate, which is often impractical in real-world applications. To better tackle the non-stationarity problem, this article combines model-based reinforcement learning (MBRL) and meta-learning and proposes a method called Model-based Imagined Rollouts Augmentation (MIRA). Based on an environment dynamics model, distributed agents can independently perform multi-agent rollouts with opponent models during exploitation and learn to infer the environmental non-stationarity as a latent variable using the rollouts. Based on the world model and latent-variable inference module, we perform multi-agent soft actor-critic implementation for centralized training and decentralized decision making. Empirical results on the Multi-agent Particle Environment (MPE) have proved that the algorithm has a very considerable improvement in sample efficiency as well as better convergent rewards than state-of-the-art MARL methods, including COMA, MAAC, MADDPG, and VDN.

In this study, we propose an innovative approach to address a chronological planning problem involving the multiple agents required to complete tasks under precedence constraints. We model this problem as a stochastic game and solve it with multi-agent reinforcement learning algorithms. However, these algorithms necessitate relearning from scratch when confronted with changes in the chronological order of tasks, resulting in distinct stochastic games and consuming a substantial amount of time. To overcome this challenge, we present a novel framework that incorporates meta-learning into a multi-agent reinforcement learning algorithm. This approach enables the extraction of meta-parameters from past experiences, facilitating rapid adaptation to new tasks with altered chronological orders and circumventing the time-intensive nature of reinforcement learning. Then, the proposed framework is demonstrated through the implementation of a method named Reptile-MADDPG. The performance of the pre-trained model is evaluated using average rewards before and after fine-tuning. Our method, in two testing tasks, improves the average rewards from −44 to −37 through 10,000 steps of fine-tuning in two testing tasks, significantly surpassing the two baseline methods that only attained −51 and −44, respectively. The experimental results demonstrate the superior generalization capabilities of our method across various tasks, thus constituting a significant contribution towards the design of intelligent unmanned systems.

Crowdsensing has gained popularity across various domains such as urban transportation, environmental monitoring, and public safety. Unmanned aerial vehicle (UAV) crowdsensing is a novel approach that collects extensive data from targeted environments using UAVs equipped with built-in sensors. Unlike conventional methods that rely on fixed sensor networks or the mobility of humans, UAV crowdsensing offers high flexibility and scalability. With the rapid advancement of artificial intelligence techniques, UAV crowdsensing is becoming increasingly intelligent and autonomous. Previous studies on UAV crowdsensing have predominantly focused on algorithmic sensing strategies without considering the impact of different sensing environments. Thus, there is a research gap regarding the influence of environmental factors and sensing strategies in this field. To this end, we designed a $4\times 3$ empirical study, classifying sensing environments into four major categories: open, urban, natural, and indoor. We conducted experiments to understand how these environments influence three typical crowdsensing strategies: opportunistic, algorithmic, and collaborative. The statistical results reveal significant differences in both environments and sensing strategies. We found that an algorithmic strategy (machine-only) is suitable for open and natural environments, while a collaborative strategy (human and machine) is ideal for urban and indoor environments. This study has crucial implications for adopting appropriate sensing strategies for different environments of UAV crowdsensing tasks.