10 November 2022 in IEEE Robotics and Automation Letters
Loop closing is a fundamental part of simultaneous localization and mapping (SLAM) for autonomous mobile systems. In the field of visual SLAM, bag of words (BoW) has achieved great success in loop closure. The BoW features for loop searching can also be used in the subsequent 6-DoF loop correction. However, for 3D LiDAR SLAM, the state-of-the-art methods may fail to effectively recognize the loop in real time, and usually cannot correct the full 6-DoF loop pose. To address this limitation, we present a novel B ag o f W ords for real-time loop closing in 3D LiDAR SLAM, called BoW3D. Our method not only efficiently recognizes the revisited loop places, but also corrects the full 6-DoF loop pose in real time. BoW3D builds the bag of words based on the 3D LiDAR feature LinK3D, which is efficient, pose-invariant and can be used for accurate point-to-point matching. We furthermore embed our proposed method into 3D LiDAR odometry system to evaluate loop closing performance. We test our method on public dataset, and compare it against other state-of-the-art algorithms. BoW3D shows better performance in terms of $F_{1}$ max and extended precision scores on most scenes. It is noticeable that BoW3D takes an average of 48 ms to recognize and correct the loops on KITTI 00 (includes 4K+ 64-ray LiDAR scans), when executed on a notebook with an Intel Core i7 @2.2 GHz processor.
16 January 2024 in IEEE Robotics and Automation Letters
Feature extraction and matching are the basic parts of many robotic vision tasks, such as 2D or 3D object detection, recognition, and registration. As is known, 2D feature extraction and matching have already achieved great success. Unfortunately, in the field of 3D, the current methods may fail to support the extensive application of 3D LiDAR sensors in robotic vision tasks due to their poor descriptiveness and inefficiency. To address this limitation, we propose a novel 3D feature representation method: Lin ear K eypoints representation for 3D LiDAR point cloud, called LinK3D. The novelty of LinK3D lies in that it fully considers the characteristics (such as the sparsity and complexity) of LiDAR point clouds and represents the keypoint with its robust neighbor keypoints, which provide strong constraints in the description of the keypoint. The proposed LinK3D has been evaluated on three public datasets, and the experimental results show that our method achieves great matching performance. More importantly, LinK3D also shows excellent real-time performance, faster than the sensor frame rate at 10 Hz of a typical rotating LiDAR sensor. LinK3D only takes an average of $\bm {30}$ milliseconds to extract features from the point cloud collected by a 64-beam LiDAR and takes merely about $\bm {20}$ milliseconds to match two LiDAR scans when executed on a computer with an Intel Core i7 processor. Moreover, our method can be extended to LiDAR odometry task, and shows good scalability.
7 April 2025 in IEEE Transactions on Automation Science and Engineering
In the construction industry, the integration of artificial intelligence (AI) and robotics has led to significant advancements in automating various tasks. One critical aspect is the intelligent navigation of automatic mobile robot (AMR) within construction sites, where dynamic environments pose challenges such as inaccurate robot state estimation, irregular and textureless obstacle detection. To solve these issues, this paper designs an AI enabled obstacle-aware AMR intelligent navigation approach while ensuring robot safety and efficiency. Specifically, the robot is equipped with the complementary RGB camera, inertial measurement unit (IMU) and wheel encoder to estimate the states (i.e., positions, orientations and velocities), which have been tightly fused in the optimization framework. Furthermore, the RGB-Depth images are jointly fed into the AI model. It combines of Mask-RCNN and multi-scale attention network, to detect and segment the obstacles, and estimate the corresponding relative depth. It should be noted that the system incorporates obstacle awareness mechanisms to dynamically switch the robot’s velocities in response to obstacles in the environment, ensuring smooth and collision-free movement. Experimental results demonstrate the effectiveness and robustness of the proposed AI-enabled navigation system in real-world construction scenarios, showcasing its potential to enhance productivity and safety in construction automation.
19 December 2024 in Applied Intelligence