30 May 2024 in IEEE Robotics and Automation Letters
Chasing a dynamic target in a dense environment is one of the challenging applications of autonomous drones. The task requires multi-modal data, such as RGB and depth, to accomplish safe and robust maneuver. However, using different types of modalities can be difficult due to the limited capacity of drones in aspects of hardware complexity and sensor cost. Our framework resolves such restrictions in the target chasing task by using only a monocular camera instead of multiple sensor inputs. From an RGB input, the perception module can extract a cross-modal representation containing information from multiple data modalities. To learn cross-modal representations at training time, we employ variational autoencoder (VAE) structures and the joint objective function across heterogeneous data. Subsequently, using latent vectors acquired from the pre-trained perception module, the planning module generates a proper next-time-step waypoint by imitation learning of the expert, which performs a numerical optimization using the privileged RGB-D data. Furthermore, the planning module considers temporal information of the target to improve tracking performance through consecutive cross-modal representations. Ultimately, we demonstrate the effectiveness of our framework through the reconstruction results of the perception module, the target chasing performance of the planning module, and the zero-shot sim-to-real deployment of a drone.
25 August 2025 in IEEE Transactions on Intelligent Transportation Systems
In autonomous driving, a prior LiDAR map(PLM) is used as a powerful tool for correcting SLAM drift, but finding robust and accurate correspondences between cross modal sensors is a challenging problem. To address this cross-modality issue, this paper proposes a real-time plane-based stereo localization system with a PLM. In the proposed system, drift in visual pose estimation is eliminated through plane-based joint optimization and the registration module. Two types of planes are employed in this system: surfel, ensuring accuracy in a narrow domain, and global plane, providing robustness in a wide area. For accurate and robust matching between the visual map point and PLM, surfels and global planes in PLM are utilized collaboratively based on point-to-PLM distance. To reduce the computational cost of the registration module, a plane-to-plane drift estimation module is proposed. The performance of the proposed system is extensively validated across synthetic simulation and real-world indoor and outdoor datasets. We validate the effectiveness of each module through ablation studies and also assess the robustness against error that may exist in PLM and initial pose. In most of the validation, the proposed system shows more accurate and robust performance compared to the state-of-the-art methods.
1 October 2026 in IEEE Robotics and Automation Letters
1 January 2026 in IEEE Access
1 January 2026 in IEEE Robotics and Automation Letters
28 February 2022 in Korean Institute of Smart Media