This paper summarizes in depth the state of the art of aerial swarms, covering both classical and new reinforcement-learning-based approaches for their management. Then, it proposes a hybrid AI system, integrating deep reinforcement learning in a multi-agent centralized swarm architecture. The proposed system is tailored to perform surveillance of a specific area, searching and tracking ground targets, for security and law enforcement applications. The swarm is governed by a central swarm controller responsible for distributing different search and tracking tasks among the cooperating UAVs. Each UAV agent is then controlled by a collection of cooperative sub-agents, whose behaviors have been trained using different deep reinforcement learning models, tailored for the different task types proposed by the swarm controller. More specifically, proximal policy optimization (PPO) algorithms were used to train the agents’ behavior. In addition, several metrics to assess the performance of the swarm in this application were defined. The results obtained through simulation show that our system searches the operation area effectively, acquires the targets in a reasonable time, and is capable of tracking them continuously and consistently.

Reinforcement learning (RL) has emerged as a powerful paradigm for enabling autonomous coordination in multi-UAV systems operating in complex and uncertain environments. However, the effectiveness of learned policies is strongly influenced by how actions are implemented at the control level, an aspect that has received limited attention in the literature. This paper presents a comparative study of three control methods (heading-based, waypoint-based, and deterministic) within a unified hybrid-AI architecture, in which the same RL policy structure is used across two of the three configurations. By isolating the control method as the sole variable, the study evaluates how different action abstractions affect learning efficiency, robustness, and operational performance in cooperative surveillance missions. A statistically rigorous Monte Carlo evaluation, supported by non-parametric hypothesis testing, demonstrates that heading-based control consistently achieves superior performance in terms of revisit period, target acquisition time, and tracking continuity. The analysis further reveals that these gains arise from improved reactivity and constraint handling rather than from differences in policy learning. The results highlight the critical role of control-level design in RL-based multi-agent systems and provide practical guidelines for selecting action abstractions in aerial swarm applications.

The increasing demand for autonomous aerial systems in surveillance, monitoring, and search-and-rescue operations has intensified the interest in unmanned aerial vehicle swarms as a means to achieve persistent coverage and robust performance in complex environments. Multirotor drones are particularly well suited for these missions due to their ability to hover, maneuver with precision, and operate in confined spaces. However, coordinating large-scale swarms presents significant challenges. Centralized planning methods often lack scalability, while fully distributed multi-agent reinforcement learning approaches face training difficulties, non-stationarity, and the curse of dimensionality. Moreover, models trained exclusively in simulation struggle to transfer seamlessly to real-world scenarios because of discrepancies in dynamics, sensing, and environmental conditions. This thesis addresses these challenges by introducing a hybrid methodology for multi-agent trajectory planning. The proposed framework combines reinforcement learning models, trained individually through curriculum learning for specific tasks such as search or tracking, with a deterministic centralized controller that ensures safe and coordinated behavior of the swarm. This integration leverages the adaptability of learning-based methods while maintaining the predictability and reliability of deterministic control. The design is modular, allowing different functionalities to be incorporated into the swarm without the need for retraining the complete system. The main objectives of the thesis are threefold. First, to develop a generalizable framework that integrates task-specific reinforcement learning models into a centralized control architecture. Second, to evaluate the performance of different trajectory planning strategies, including two reinforcement learning based methods and one deterministic method, under comparable conditions. Third, to validate the feasibility of transferring the methodology from simulation to real-world settings through a scaled physical mock-up. The findings demonstrate that the hybrid approach provides a balance between adaptability and robustness. Simulation results from Monte Carlo experiments show that the heading-based reinforcement learning strategy achieves superior performance in terms of revisit period, target detection rate, and tracking continuity, when compared to the waypoints-based reinforcement learning method and the deterministic baseline. The deterministic controller ensures safety and consistency, especially in situations where reinforcement learning models encounter conditions outside their training distribution. The real-world scaled mock-up further validates the approach by enabling hardware-in-the-loop experimentation, identifying transferability challenges, and confirming that the proposed architecture can be implemented with small multirotor drones in indoor environments. The thesis concludes that the integration of reinforcement learning models with deterministic control constitutes an effective methodology for UAV swarm coordination. The approach is scalable, modular, and generalizable to a variety of multi-agent tasks beyond surveillance. It demonstrates in qualitative terms that reinforcement learning, when combined with a structured control framework, can overcome the limitations of purely centralized or purely distributed methods. Future directions include scaling to larger and more complex scenarios, enhancing localization systems, integrating more realistic onboard sensors, and extending experiments to outdoor environments. This work provides a step forward toward bridging the gap between simulation and real-world deployments of drone swarms for persistent surveillance and related applications. RESUMEN La creciente demanda de sistemas aéreos autónomos en operaciones de vigilancia, monitoreo y búsqueda y rescate ha intensificado el interés en el uso de enjambres de vehículos aéreos no tripulados como medio para lograr una cobertura persistente y un desempeño robusto en entornos complejos. Los drones multirrotor son especialmente adecuados para este tipo de misiones debido a su capacidad de vuelo estacionario, su maniobrabilidad precisa y su operatividad en espacios confinados. No obstante, la coordinación de enjambres a gran escala presenta importantes desafíos. Los métodos de planificación centralizados suelen carecer de escalabilidad, mientras que los enfoques completamente distribuidos basados en aprendizaje por refuerzo multiagente enfrentan dificultades de entrenamiento, problemas de no estacionariedad y la denominada "maldición de la dimensionalidad". Además, los modelos entrenados exclusivamente en simulación presentan limitaciones al transferirse al mundo real debido a discrepancias en la dinámica, la sensorización y las condiciones ambientales. Esta tesis aborda dichos retos mediante la introducción de una metodología híbrida para la planificación de trayectorias en sistemas multiagente. El marco propuesto combina modelos de aprendizaje por refuerzo entrenados de forma individual mediante curriculum learning para tareas específicas como búsqueda o seguimiento, con un controlador centralizado determinista que garantiza un comportamiento seguro y coordinado del enjambre. Esta integración aprovecha la adaptabilidad de los métodos basados en aprendizaje, a la vez que mantiene la previsibilidad y fiabilidad de un control determinista. El diseño es modular, lo que permite incorporar distintas funcionalidades en el enjambre sin necesidad de reentrenar el sistema completo. Los objetivos principales de la tesis son tres. En primer lugar, desarrollar un marco generalizable que integre modelos de aprendizaje por refuerzo orientados a tareas dentro de una arquitectura de control centralizada. En segundo lugar, evaluar el rendimiento de...