21 November 2022 in Transportation Safety and Environment
Computer vision algorithms have been utilized for 3-D road imaging and pothole detection for over two decades. Nonetheless, there is a lack of systematic survey articles on state-of-the-art (SoTA) computer vision techniques, especially deep learning models, developed to tackle these problems. This article first introduces the sensing systems employed for 2-D and 3-D road data acquisition, including camera(s), laser scanners and Microsoft Kinect. It then comprehensively reviews the SoTA computer vision algorithms, including (1) classical 2-D image processing, (2) 3-D point cloud modelling and segmentation and (3) machine/deep learning, developed for road pothole detection. The article also discusses the existing challenges and future development trends of computer vision-based road pothole detection approaches: classical 2-D image processing-based and 3-D point cloud modelling and segmentation-based approaches have already become history; and convolutional neural networks (CNNs) have demonstrated compelling road pothole detection results and are promising to break the bottleneck with future advances in self/un-supervised learning for multi-modal semantic segmentation. We believe that this survey can serve as practical guidance for developing the next-generation road condition assessment systems.
21 May 2021 in IET Cyber-Systems and Robotics
A fundamental task in robotics is to plan collision‐free motions among a set of obstacles. Recently, learning‐based motion‐planning methods have shown significant advantages in solving different planning problems in high‐dimensional spaces and complex environments. This article serves as a survey of various different learning‐based methods that have been applied to robot motion‐planning problems, including supervised, unsupervised learning, and reinforcement learning. These learning‐based methods either rely on a human‐crafted reward function for specific tasks or learn from successful planning experiences. The classical definition and learning‐related definition of motion‐planning problem are provided in this article. Different learning‐based motion‐planning algorithms are introduced, and the combination of classical motion‐planning and learning techniques is discussed in detail.
2 March 2021 in IEEE Transactions on Cognitive and Developmental Systems
Path planning plays an important role in autonomous robot systems. Effective understanding of the surrounding environment and efficient generation of an optimal collision-free path are both critical parts for solving path-planning problems. Although conventional sampling-based algorithms, such as the rapidly exploring random tree (RRT) and its improved optimal version (RRT*), have been widely used in path-planning problems because of their ability to find a feasible path in even complex environments, they fail to find an optimal path efficiently. To solve this problem and satisfy the two aforementioned requirements, we propose a novel learning-based path-planning algorithm which consists of a novel generative model based on the conditional generative adversarial networks (CGANs) and a modified RRT* algorithm (denoted by CGAN-RRT*). Given the map information, our CGAN model can generate an efficient probability distribution of feasible paths, which can be utilized by the CGAN-RRT* algorithm to find an optimal path with a nonuniform sampling strategy. The CGAN model is trained by learning from ground-truth maps, each of which is generated by putting all the results of executing the RRT algorithm 50 times on one raw map. We demonstrate the efficient performance of this CGAN model by testing it on two groups of maps and comparing the CGAN-RRT* algorithm with the Informed-RRT* algorithm and conventional RRT* algorithm.
2 November 2021 in IEEE Transactions on Automation Science and Engineering
Robot path planning in 3D space is a challenging problem for its complex configuration. Sampling-based algorithms have gained great success in solving path planning problems in 3D space, but the quality of the initial path is not guaranteed and the convergence to the optimal solution is slow. To address these problems, in this article, we present a novel sampling-based path planning framework enhanced by the deep neural network (DNN) with applications to 3D space. In the proposed framework, we first train the DNN with a number of successful path planning cases in 3D space. Then the DNN is utilized to predict the promising region where the feasible path probably exists for a given path planning problem. This predicted promising region serves as a nonuniform sampling heuristic to bias the sampling process of the path planner. In this way, the path planner can focus on the promising region in the exploration and exploitation process so that the path planning speed gets accelerated. We conduct numerical simulations to evaluate the performance of the proposed algorithm and the results show that it can perform much better than conventional path planning algorithms. Furthermore, we also investigate the performance of different DNN architectures for path planning in 3D space. Note to Practitioners—In this work, we aim to provide an efficient learning-based method to accelerate the robot path planning process in 3D space. Conventional path planning algorithms need to perceive the environment first, and then implement a series of calculations such as collision checking and data storing to generate a feasible path. When facing complex and high-dimensional environments, they do not perform well. But the proposed neural network method in this article can predict the promising region where the feasible path exists for any given environment. This prediction result is used to guide the path planning process so that the algorithm performance can get significantly improved. Apart from sampling-based algorithms, the proposed neural network model can also be extended to other types of path planning algorithms.
20 February 2025 in Knowledge-Based Systems
20 May 2024 in IEEE Transactions on Intelligent Transportation Systems
Over the past decade, automated methods have been developed to detect cracks more efficiently, accurately, and objectively, with the ultimate goal of replacing conventional manual visual inspection techniques. Among these methods, semantic segmentation algorithms have demonstrated promising results in pixel-wise crack detection tasks. However, training such networks requires a large amount of human-annotated datasets with pixel-level annotations, which is a highly labor-intensive and time-consuming process. Moreover, supervised learning-based methods often struggle with poor generalizability in unseen datasets. Therefore, we propose an unsupervised pixel-wise road crack detection network, known as UP-CrackNet. Our approach first generates multi-scale square masks and randomly selects them to corrupt undamaged road images by removing certain regions. Subsequently, a generative adversarial network is trained to restore the corrupted regions by leveraging the semantic context learned from surrounding uncorrupted regions. During the testing phase, an error map is generated by calculating the difference between the input and restored images, which allows for pixel-wise crack detection. Our comprehensive experimental results demonstrate that UP-CrackNet outperforms other general-purpose unsupervised anomaly detection algorithms, and exhibits satisfactory performance and superior generalizability when compared with state-of-the-art supervised crack segmentation algorithms. Our source code is publicly available at mias.group/UP-CrackNet.