We present a formalization of multi-agent planning problems in which agents are explicitly represented. In contrast with previous formalisations, we discuss the advantages of representing agents explicitly and show the implementation in the Unified Planning formalism and some practical examples. The proposed formalism is equivalent to other models, in particular to Multi Agent Planning Domain Definition Language (MA-PDDL), and can thus be compiled into it and solved by existing multi-agent planning solvers. Moreover, we present a further extension to define more complex multi-agent problems with explicit models of other agents.
The use of Unmanned Aerial Vehicle (UAVs) in deep reinforcement learning (DRL) applications is growing rapidly, as DRL is effective in handling complex environments. One major challenge is designing effective reward signals, as sparse rewards hinder the learning process of DRL. On the other hand, tabular reinforcement learning (RL) approaches, while faster, are typically limited to small, discrete environments and do not scale well to more complex settings. This work presents a hierarchical approach combining the strengths of both methods. We apply tabular RL to derive an optimal value function for a discrete abstract version of the original environment, and use it as a heuristic to generate a denser reward signal for the DRL algorithm, in the original environment. We empirically demonstrate that our approach significantly reduces the number of samples the DRL algorithm needs to achieve the same level of performance in a multi-UAV scenario, compared to traditional methods.