Pavement crack detection from images is a challenging problem due to intensity inhomogeneity, topology complexity, low contrast, and noisy texture background. Traditional learning-based approaches have difficulties in obtaining representative training samples. We propose a new unsupervised multi-scale fusion crack detection (MFCD) algorithm that does not require training data. First, we develop a windowed minimal intensity path-based method to extract the candidate cracks in the image at each scale. Second, we find the crack correspondences across different scales. Finally, we develop a crack evaluation model based on a multivariate statistical hypothesis test. Our approach successfully combines strengths from both the large-scale detection (robust but poor in localization) and the small-scale detection (detail-preserving but sensitive to clutter). We analyze and experimentally test the computational complexity of our MFCD algorithm. We have implemented the algorithm and have it extensively tested on three public data sets, including two public pavement data sets and an airport runway data set. Compared with six existing methods, experimental results show that our method outperforms all counterparts. Specifically, it increases the precision, recall, and F1-measure over the state-of-the-art by 22%, 12%, and 19%, respectively, on one public data set.

Detection of subsurface defects is important for maintaining runway structural health and reliability. A potential solution is to employ a robot equipped with a Ground Penetrating Radar (GPR) to perform subsurface scanning. To automate the inspection process, we develop a subsurface defect detection algorithm which is a deep learning algorithm that fuses 2D planar features in each panel in GPR B-scans and 3D voxel-wise features in GPR C-scan to robustly detect regions with defects even in the presence of significant noises. Named as GPR-RCNN, we have tested our algorithm with real airport runway data collected from three international airports using our runway inspection robot. The experimental results show that our proposed GPR-RCNN achieves superior results when comparing to state-of-the-art techniques. Specifically, our method achieves F1-measures at 62%, 33%, 81%, and 87% for void, crack, subsidence and pipe, respectively.

We propose a novel subsurface pipeline mapping and 3D reconstruction method by fusing ground-penetrating radar (GPR) scans and camera images. To facilitate the simultaneous detection of multiple pipelines, we model the GPR sensing process and prove hyperbola response for general scanning with nonperpendicular angles. Furthermore, we fuse visual simultaneous localization and mapping outputs, encoder readings with GPR scans to classify hyperbolas into different pipeline groups. We extensively apply the J-linkage method and maximum likelihood estimation with error analysis to improve algorithm robustness and accuracy. As a result, we optimally estimate the radii and locations of all pipelines. We have implemented our method and tested it in physical experiments with representative pipeline configurations. Two different kinds of 3-m-long pipes are used, with radii being 4.62 and 3.02 cm, respectively. The results show that our method successfully reconstructs all subsurface pipes. Moreover, the average estimation errors for two orientation angles of pipelines are 1.73° and 0.73°, respectively. The average localization error is 4.47 cm.

In order to improve the accuracy and robustness of existing automated crack detection methods, a fully convolutional neural network for pixel-level detection based on densely connected and deeply supervised network is proposed. First, the densely connected layers are applied for enhancing the propagation and reuse of crack features. Then, the deeply supervised modules are designed to make network extract more significant features through multi-scale levels. Finally, the feature maps from different scales are fused to achieve complementarity at different levels. In addition, a class-balanced cross-entropy loss function is designed to balance backgrounds and cracks by increasing the weight of crack pixel loss. The proposed method is tested on three public datasets, and the experiments show that our method is superior to state-of-the-art methods in accuracy, speed and robustness.

The detection and restoration of subsurface defects are essential for ensuring the structural reliability of airport runways. Subsurface inspections can be performed with the aid of a robot equipped with a Ground Penetrating Radar (GPR). However, interpreting GPR data is extremely difficult, as GPR data usually contains severe clutter interference. In addition, many different types of subsurface defects present similar features in B-scan images, making them difficult to distinguish. Consequently, this makes later maintenance work harder as different subsurface defects require different restoration measures. Thus, to automate the inspection process and improve defect identification accuracy, a novel deep learning algorithm, MV-GPRNet, is proposed. Instead of traditionally using GPR B-scan images only, MV-GPRNet utilizes multi-view GPR data to robustly detect regions with defects despite significant interference. It originally fuses the 3D feature map in C-scan data and the 2D feature map in Top-scan data for defect classification and localization. With our runway inspection robot, a large number of real runway data sets from three international airports have been used to extensively test our method. Experimental results indicate that the proposed MV-GPRNet outperforms state-of-the-art (SOTA) approaches. In particular, MV-GPRNet achieves F1 measurements for voids, cracks, subsidences, and pipes at 91%, 69%, 90%, and 100%, respectively.

Detection of both surface and subsurface defects is a vital task for maintaining the structural health and reliability of airport runways. We report the automated data collection and analysis for airport runways based on our novel robotic system, which employs a camera and a GPR (Ground Penetrating Radar) to inspect the surface and subsurface conditions, respectively. To perform the automated data analysis, we propose a novel crack detection algorithm based on the images, and a subsurface defect detection method with GPR data. Additionally, to create a composite global view of a large airport runway span, a camera/GPR data sequence from the robot is aligned accurately to create a continuous mosaic for visualization. We combine these algorithms into a software to perform automated on-site analysis. We have put our robot and software into engineering practice over 20 airports in China, achieving the performance of 70% and 67% F1-measure for crack detection and subsurface defect detection, respectively. More importantly, the results of our algorithms can satisfy the requirement of applications.