The use of electrical vertical takeoff and landing (eVTOL) aircraft to provide efficient, high-speed, on-demand air transportation within a metropolitan area is a topic of increasing interest, which is expected to bring fundamental changes to the city infrastructures and daily commutes. NASA, Uber, and Airbus have been exploring this exciting concept of Urban Air Mobility (UAM), which has the potential to provide meaningful door-to-door trip time savings compared with automobiles. However, the ability to manage many of these eVTOL aircraft safely in a congested urban area presents a challenge unprecedented in air traffic management. In order to enable safe and efficient autonomous on-demand free flight operations in UAM, a computational guidance algorithm with collision avoidance capability is designed and analyzed. The approach proposed in this paper is to formulate this problem as a Markov Decision Process (MDP) and solve it using an online algorithm Monte Carlo Tree Search (MCTS). For illustration, a high-density free flight airspace simulator is created to test the proposed algorithm’s performance. Numerical experiment results show that this proposed algorithm has fewer conflicts and near mid-air collisions than Optimal Reciprocal Collision Avoidance (ORCA), a state-of-the-art collision avoidance strategy.
21 February 2022 in IEEE Transactions on Automation Science and Engineering
In this article, a scalable autonomous separation assurance framework is proposed for high-density en route airspace sectors with heterogeneous aircraft objectives. To handle the complex dynamic decision making under uncertainty, multi-agent reinforcement learning is used in a decentralized approach with each aircraft being represented as an agent. Based on this, each agent locally solves the separation assurance problem, allowing the framework to scale to a large number of aircraft. In addition, each agent has the ability to learn the intention of the intruder aircraft, which is essential in environments with heterogeneous agents. Numerical experiments are performed in a real-time air traffic simulator. The results demonstrate that the proposed framework is able to effectively ensure the safe separation of heterogeneous agents, while also optimizing the intrinsic agent objectives in high-density en route airspace sectors. In addition, the efficiency of the proposed framework is demonstrated and shown to provide real-time decision making for separation assurance. Note to Practitioners—In commercial aviation, the workload of human air traffic controllers increases with the growth of air traffic density. Robust decision making systems to augment human air traffic controllers allows for increased air traffic without increased workload, resulting in a safer airspace environment. In addition, advanced air mobility (AAM) is concerned with low-altitude airspace operations with both human and autonomous pilots. In this environment, autonomous real-time separation assurance systems are required. Most traditional separation assurance approaches fail to handle stochastic and high-density environments, rendering them inapplicable to future high-density traditional airspace and the envisioned low-altitude AAM operations. Therefore, it is important to study decentralized approaches that can place the separation assurance problem as an intrinsic objective of each aircraft to ensure cooperation in high-density airspace. With this consideration, a scalable, decentralized autonomous separation assurance framework capable of handling heterogeneous agents is proposed in this article. This framework is able to perceive the current air traffic environment and select speed advisories to ensure safe separation requirements, while balancing intrinsic objectives such as minimizing delay. While one limitation of multi-agent reinforcement learning is the long training time, this article demonstrates how the framework also can leverage modern computing clusters to significantly reduce training time without sacrificing performance.
Obstacle avoidance for small unmanned aircraft is vital for the safety of future urban air mobility (UAM) and Unmanned Aircraft System (UAS) Traffic Management (UTM). There are a variety of techniques for real-time robust drone guidance, but numerous of them solve in discretized airspace and control, which would require an additional path smoothing step to provide flexible commands for UAS. To deliver safe and computationally efficient guidance for UAS operations, we explore the use of a deep reinforcement learning algorithm based on Proximal Policy Optimization (PPO) to lead autonomous UAS to their destinations while bypassing obstacles through continuous control. The proposed scenario state representation and reward function can map the continuous state space to continuous control for both heading angle and speed. To verify the effectiveness of the proposed learning framework, we conducted numerical experiments with static and moving obstacles. Uncertainties associated with the environments and safety operation bounds are investigated in detail. Results show that the proposed model is able to provide accurate and robust guidance and resolve conflict with a success rate of over 99%.
13 April 2022 in IEEE Transactions on Intelligent Transportation Systems
The concept of Urban Air Mobility (UAM) proposes to use revolutionary new electrical vertical takeoff and landing (eVTOL) aircraft to provide efficient and on-demand air transportation service between places previously underserved by the current aviation market. A key challenge for the success of UAM is how to manage large-scale autonomous flight operations with safety guarantee in high-density, dynamic and uncertain airspace environments. In this paper, a safety assured decentralized online guidance algorithm with airborne self-separation capability is proposed and analyzed for multi-aircraft autonomous flight operations under uncertainties. The problem is formulated as a multi-agent Markov Decision Process with continuous action space and is solved by a customized decentralized online algorithm based on Monte Carlo Tree Search (MCTS). To guarantee the safety of real-time autonomous flight operations in uncertain environments, the formulation of loss of chance constrained separation is introduced and integrated with the proposed MCTS algorithm. In addition, Gaussian process regression along with Bayesian optimization is employed to discretize the continuous action space, which helps shorten the flight time. A comprehensive numerical study shows that the proposed algorithm can provide safe onboard guidance with guaranteed low near mid-air collision probability in uncertain and high-density airspace environments.
18 January 2024 in IEEE Transactions on Intelligent Transportation Systems
Urban air mobility (UAM) has the potential to revolutionize our daily transportation, offering rapid and efficient deliveries of passengers and cargo between dedicated locations within and around the urban environment. Before the commercialization and adoption of this emerging transportation mode, however, aviation safety must be guaranteed, i.e., all the aircraft have to be safely separated by strategic and tactical deconfliction. Reinforcement learning has demonstrated effectiveness in the tactical deconfliction of en route commercial air traffic in simulation. However, its performance is found to be dependent on the traffic density. In this project, we propose a novel framework that combines demand capacity balancing (DCB) for strategic conflict management and reinforcement learning for tactical separation. By using DCB to precondition traffic to proper density levels, we show that reinforcement learning can achieve much better performance for tactical safety separation. Our results also indicate that this DCB preconditioning can allow target levels of safety to be met that are otherwise impossible. In addition, combining strategic DCB with reinforcement learning for tactical separation can meet these safety levels while achieving greater operational efficiency than alternative solutions.