This paper presents LEVIOSA, a novel framework for text- and speech-based uncrewed aerial vehicle (UAV) trajectory generation. By leveraging multimodal large language models (LLMs) to interpret natural language commands, the system converts text and audio inputs into executable flight paths for UAV swarms. The approach aims to simplify the complex task of multi-UAV trajectory generation, which has significant applications in fields such as search and rescue, agriculture, infrastructure inspection, and entertainment. The framework involves two key innovations: a multi-critic consensus mechanism to evaluate trajectory quality and a hierarchical prompt structuring for improved task execution. The innovations ensure fidelity to user goals. The framework integrates several multimodal LLMs for high-level planning, converting natural language inputs into 3D waypoints that guide UAV movements and per-UAV low-level controllers to control each UAV in executing its assigned 3D waypoint path based on the high-level plan. The methodology was tested on various trajectory types with promising accuracy, synchronization, and collision avoidance results. The findings pave the way for more intuitive human–robot interactions and advanced multi-UAV coordination.

Intelligent systems are increasingly interacting with people, both in their daily lives and through their use in safety critical systems. Current research is focused on how to use intelligent systems in a collaborative way as a teammate, rather than a tool. This requires a better understanding of what behaviors enable effective human-agent teams. This paper reports an experiment where a human player collaborates with an agent to perform a maneuvering task while concurrently performing a memory task. The player must determine in which contexts the agent requires their input to achieve better combined game plus memory task accuracy scores. We hypothesized that high-performing teams would exhibit different patterns of control inputs when compared to low-performing teams and that these patterns of control would be made more evident with user interfaces that increased operator situation awareness. Preliminary results are inconclusive, but show different patterns of interaction between high- and low-performing teams.