Wildfires destroy thousands of hectares every summer all over the globe. To provide an effective response and to mitigate wildfires impact, firefighters require a real-time monitoring of the fire front. This paper proposes a cooperative reinforcement learning (RL) framework that allows a team of autonomous unmanned aerial vehicles (UAVs) to learn how to monitor a fire front. In the literature, independent Q-learners were proposed to solve a wildfire monitoring task with two UAVs. Here we propose a framework that can be easily extended to a larger number of UAVs. Our framework builds on two methods: multiple single trained Q-learning agents (MSTA) and value decomposition networks (VDN). MSTA trains a single UAV controller, which is then "copied" to each of the UAVs in the team. In contrast, VDN trains agents to learn how to cooperate. We benchmarked in simulations our two considered methods – MSTA and VDN – against two state-of-the-art approaches: independent Q-learners and a joint Q-learner. Simulation results show that our considered methods outperform state-of-the-art approaches in a wildfire front monitoring task with up to 9 fixed-wing and multi-copter UAVs.

State-of-the-art multi-robot information gathering (MR-IG) algorithms often rely on a model that describes the structure of the information of interest to drive the robots motion. This causes MR-IG algorithms to fail when they are applied to new IG tasks, as existing models cannot describe the information of interest. Therefore, we propose in this paper a MR-IG algorithm that can be applied to new IG tasks with little algorithmic changes. To this end, we introduce DeepIG: a MR-IG algorithm that uses Deep Reinforcement Learning to allow robots to learn how to gather information. Nevertheless, there are IG tasks for which accurate models have been derived. Therefore, we extend DeepIG to exploit existing models for such IG tasks. This algorithm we term it model-based DeepIG (MB-DeepIG). First, we evaluate DeepIG in simulations, and in an indoor experiment with three quadcopters that autonomously map an unknown terrain profile built in our lab. Results demonstrate that DeepIG can be applied to different IG tasks without algorithmic changes, and that it is robust to measurement noise. Then, we benchmark MB-DeepIG against state-of-the-art information-driven Gaussian-processes-based IG algorithms. Results demonstrate that MB-DeepIG outperforms the considered benchmarks.

Information gathering (IG) algorithms aim to intelligently select a mobile sensor actions required to efficiently obtain an accurate reconstruction of a physical process, such as an occupancy map, or a magnetic field. Many recent works have proposed algorithms for IG that employ Gaussian processes (GPs) as underlying model of the process. However, most algorithms discretize the state space, which makes them computationally intractable for robotic systems with complex dynamics. Moreover, they are not suited for online information gathering tasks as they assume prior knowledge about GP parameters. This paper presents a novel approach that tackles the two aforementioned issues. Specifically, our approach includes two intertwined steps: (i) a Rapidly-Exploring Random Tree (RRT) search that allows a robot to identify unvisited locations, and to learn the GP parameters, and (ii) an RRT*-based informative path planning that guides the robot towards those locations by maximizing the information gathered while minimizing path cost. The combination of the two steps allows an online realization of the algorithm, while eliminating the need for discretization. We demonstrate that our proposed algorithm outperforms state-of-the-art both in simulations, and in a lab experiment in which a ground-based robot explores the magnetic field intensity within an indoor environment populated with obstacles.

Rapidly exploring random trees (RRTs) have been proven to be efficient for planning in environments populated with obstacles. These methods perform a uniform sampling of the state space, which is needed to guarantee the algorithm’s completeness but does not necessarily lead to the most efficient solution. In previous works it has been shown that the use of heuristics to modify the sampling strategy could incur an improvement in the algorithm performance. However, these heuristics only apply to solve the shortest path-planning problem. Here we propose a framework that allows us to incorporate arbitrary heuristics to modify the sampling strategy according to the user requirements. This framework is based on ‘learning from experience’. Specifically, we introduce a utility function that takes the contribution of the samples to the tree construction into account; sampling at locations of increased utility then becomes more frequent. The idea is realized by introducing an ant colony optimization concept in the RRT/RRT* algorithm and defining a novel utility function that permits trading off exploitation versus exploration of the state space. We also extend the algorithm to allow an anytime implementation. The scheme is validated with three scenarios: one populated with multiple rectangular obstacles, one consisting of a single narrow passage and a maze-like environment. We evaluate its performance in terms of the cost and time to find the first path, and in terms of the evolution of the path quality with the number of iterations. It is shown that the proposed algorithm greatly outperforms state-of-the-art RRT and RRT* algorithms.

Gas source localization tackles the problem of finding leakages of hazardous substances such as poisonous gases or radiation in the event of a disaster. In order to avoid threats for human operators, autonomous robots dispatched for localizing potential gas sources are preferable. This work investigates a Reinforcement Learning framework that allows a robotic agent to learn how to localize gas sources. We propose a solution that assists Reinforcement Learning with existing domain knowledge based on a model of the gas dispersion process. In particular, we incorporate a priori domain knowledge by designing appropriate rewards and observation inputs for the Reinforcement Learning algorithm. We show that a robot trained with our proposed method outperforms state-of-the-art gas source localization strategies, as well as robots that are trained without additional domain knowledge. Furthermore, the framework developed in this work can also be generalized to a large variety of information gathering tasks.

This paper presents a beehive-inspired multi-agent drone system for autonomous information collection to support the needs of first responders and emergency teams. The proposed system is designed to be simple, cost-efficient, yet robust and scalable at the same time. It includes several unmanned aerial vehicles (UAVs) that can be tasked with data collection, and a single control station that acts as a data accumulation and visualization unit. The system also provides a local communication access point for the UAVs to exchange information and coordinate the data collection routes. By avoiding peer-to-peer communication and using proactive collision avoidance and path-planning, the payload weight and per-drone costs can be significantly reduced; the whole concept can be implemented using inexpensive off-the-shelf components. Moreover, the proposed concept can be used with different sensors and types of UAVs. As such, it is suited for local-area operations, but also for large-scale information-gathering scenarios. The paper outlines the details of the system hardware and software design, and discusses experimental results for collecting image information with a set of 4 multirotor UAVs at a small experimental area. The obtained results validate the concept and demonstrate robustness and scalability of the system.