Safe flight in dynamic environments requires unmanned aerial vehicles (UAVs) to make effective decisions when navigating cluttered spaces with moving obstacles. Traditional approaches often decompose decision-making into hierarchical modules for prediction and planning. Although these handcrafted systems can perform well in specific settings, they might fail if environmental conditions change and often require careful parameter tuning. Additionally, their solutions could be suboptimal due to the use of inaccurate mathematical model assumptions and simplifications aimed at achieving computational efficiency. To overcome these limitations, this letter introduces the NavRL framework, a deep reinforcement learning-based navigation method built on the Proximal Policy Optimization (PPO) algorithm. NavRL utilizes our carefully designed state and action representations, allowing the learned policy to make safe decisions in the presence of both static and dynamic obstacles, with zero-shot transfer from simulation to real-world flight. Furthermore, the proposed method adopts a simple but effective safety shield for the trained policy, inspired by the concept of velocity obstacles, to mitigate potential failures associated with the black-box nature of neural networks. To accelerate the convergence, we implement the training pipeline using NVIDIA Isaac Sim, enabling parallel training with thousands of quadcopters. Simulation and physical experiments show that our method ensures safe navigation in dynamic environments and results in the fewest collisions compared to benchmarks.
20 November 2023 in IEEE Robotics and Automation Letters
Deploying autonomous robots in crowded indoor environments usually requires them to have accurate dynamic obstacle perception. Although plenty of previous works in the autonomous driving field have investigated the 3D object detection problem, the usage of dense point clouds from a heavy Light Detection and Ranging (LiDAR) sensor and their high computation cost for learning-based data processing make those methods not applicable to small robots, such as vision-based UAVs with small onboard computers. To address this issue, we propose a lightweight 3D dynamic obstacle detection and tracking (DODT) method based on an RGB-D camera, which is designed for low-power robots with limited computing power. Our method adopts a novel ensemble detection strategy, combining multiple computationally efficient but low-accuracy detectors to achieve real-time high-accuracy obstacle detection. Besides, we introduce a new feature-based data association and tracking method to prevent mismatches utilizing point clouds' statistical features. In addition, our system includes an optional and auxiliary learning-based module to enhance the obstacle detection range and dynamic obstacle identification. The proposed method is implemented in a small quadcopter, and the results show that our method can achieve the lowest position error (0.11 m) and a comparable velocity error (0.23 m/s) across the benchmarking algorithms running on the robot's onboard computer. The flight experiments prove that the tracking results from the proposed method can make the robot efficiently alter its trajectory for navigating dynamic environments. Our software is available on GitHub 1 as an open-source ROS package.
25 June 2003 in International Journal for Numerical Methods in Engineering
Aerial robots can enhance construction site productivity by autonomously handling inspection and mapping tasks. However, ensuring safe navigation near human workers remains challenging. While navigation in static environments has been well studied, navigating dynamic environments remains open due to challenges in perception and planning. Payload limitations restrict the robots to using cameras with limited fields of view, resulting in unreliable perception and tracking during collision avoidance. Moreover, the rapidly changing conditions of dynamic environments can quickly make the generated optimal trajectory outdated.To address these challenges, this paper presents a comprehensive navigation framework that integrates perception, intent prediction, and planning. Our perception module detects and tracks dynamic obstacles efficiently and handles tracking loss and occlusion during collision avoidance. The proposed intent prediction module employs a Markov Decision Process (MDP) to forecast potential actions of dynamic obstacles with the possible future trajectories. Finally, a novel intent-based planning algorithm, leveraging model predictive control (MPC), is applied to generate navigation trajectories. Simulation and physical experiments11Experiment video link: https://youtu.be/4xsEeMB9WPY demonstrate that our method improves the safety of navigation by achieving the fewest collisions compared to benchmarks.
5 June 2003 in International Journal for Numerical Methods in Engineering
Noninvasive ventilator support using bi-level positive airway pressure/continuous positive airway pressure (BiPAP/CPAP) is commonly utilized for chronic medical conditions like sleep apnea and neuromuscular disorders like amyotrophic lateral sclerosis (ALS) that lead to weakness of respiratory muscles. Generic masks come in standard sizes and are often perceived by patients as being uncomfortable, ill-fitting, and leaky. A significant number of patients are unable to tolerate the masks and eventually stop using their devices. The goal of this project is to develop custom-fit masks to increase comfort, decrease air leakage, and thereby improve patient compliance. A single-patient case study of a patient with variant ALS was performed to evaluate the custom-fit masks. His high nose bridge and overbite of lower jaw caused poor fit with generic masks, and he was noncompliant with his machine. Using desktop Stereolithography three-dimensional (3D) printing and magnetic resonance imaging (MRI) data, a generic mask was extended with a rigid interface such that it was complementary to the patient's unique facial contours. Patient or clinicians interactively select a desired mask shape using a newly developed computer program. Subsequently, a compliant silicone layer was applied to the rigid interface. Ten different custom-fit mask designs were made using computer-aided design software. Patient evaluated the comfort, extent of leakage, and satisfaction of each mask via a questionnaire. All custom-fit masks were rated higher than the standard mask except for two. Our results suggest that modifying generic masks with a 3D-printed custom-fit interface is a promising strategy to improve compliance with BiPAP/CPAP machines.