The estimation of a fundamental matrix (F-matrix) from two-view images is a crucial problem in epipolar geometry, and a key point in visual simultaneous localization and mapping (VSLAM). Conventional robust methods proposed by the data calculation space, such as Random Sample Consensus (RANSAC), encounter computational inefficiency and low accuracy when the outliers exceed 50%. In this paper, a semantic filter-based on faster region-based convolutional neural network (faster R-CNN) is proposed to solve the outlier problem in RANSAC based F-matrix calculations. The semantic filter is trained using semantic patches tailored by inliers, providing different semantic labels in various image regions. First, the patches classified into the top three bad labels are filtered out during the pre-processing phase. Second, precise and robust correspondences are determined by the remaining high-level semantic contexts. Finally, the inliers are assessed using RANSAC to produce an accurate F-matrix. The proposed algorithm can improve the accuracy of F-matrix calculations, as low-quality feature correspondences are effectively decreased. Experiments on KITTI and ETH sequences illustrate that the 3D position error can be reduced by applying the semantic filter to the ORB-SLAM system. Further, indoor and real environment experiments demonstrate that an effective lower trajectory error is yielded with the proposed approach.

Epipolar geometry is a fundamental constraint used in computer vision systems to estimate parameters using correspondence. The most common way to describe epipolar geometry is by means of a 3x3 matrix called the fundamental matrix, and such matrices are used to store the precise geometric information relating a pair of stereo images. Its efficient estimation substantially improves initialization in visual simultaneous localization and mapping (vSLAM), which uses correspondence-based epipolar geometry to determine the trajectory of the camera and a three-dimensional scene. Conventional robust methods for epipolar geometry estimation can become computationally inefficient and inaccurate when there are low-quality correspondences. Because semantic information can be more stable than pixel intensities/descriptors, a novel Faster Region-based Convolutional Network (R-CNN) learning-based approach called the semantic filter is proposed in this paper to address these problems. The semantic filter is first trained on different semantic patches, which are described in terms of their different outlier distributions, providing different semantic labels for image contexts. Then, the patches with low-level semantic labels are filtered out. Finally, precise and robust correspondences can be determined by matches using the high-level semantic contexts, making the correspondence-based calculation more accurate. For dynamic outdoor scenes, the results of extensive experiments show that our semantic filter can help vSLAM localize accurately and robustly on a map from different viewpoints. In a completely static scenario, our semantic filter can remove the low-quality correspondences, enabling the mobile robot to operate well.

Cracks in road surfaces are among the most critical problems in the management of expressways. Their sizes, which not only have severe consequences in relation to expressway safety, decide maintenance timing, which if accurately estimated, can prevent the rapid development of crack disease. Currently, the main techniques used for crack size measurement makes use of either human inspection or sophisticated mobile equipment, which are respectively, very time consuming, labour-intensive, and difficult to implement in civil engineering. Thus, there is great interest in practical and highly efficacious automated crack identification systems coupled with low-cost data-processing sensors. A PTZ (pan/tilt/zoom) camera-based image processing idea is investigated in this paper for crack size measurement in expressways in China. The proposed method consists of four steps: 1) image acquisition, 2) corner-based line size calculation, 3) pixel size calculation, and 4) crack size estimation. A case study of roadway lane images and crack images, collected from China G4/Jingshi, is conducted to demonstrate the feasibility of the proposed method. The results demonstrate that PTZ camera-based image processing is capable of efficiently quantifying cracks, obtaining high accuracy crack size results at range of 0 m to 40 m. The method shows promise for crack size measurement in a cost-effective manner and can provide an objective and timely means for transportation agencies to conduct maintenance plans.

This study proposes a line segment detection that can efficiently and effectively handle non‐linear uniform intensity changes. The presented sketching algorithm applies the resistant to affine transformation and monotonic intensity change (RATMIC) descriptor to conduct binary translation in the image pre‐processing step, which can remove the unwanted smoothing of the Canny detector in most line detections. The Harris corner detector is applied to catch regions of line segments for the purpose of simulating the composition of sketching and achieving a sense of unity within the picture. Furthermore, the RATMIC descriptor is employed to obtain binary images of the regions of interest (ROIs). Finally, small eigenvalue analysis is implemented to detect straight lines in the ROIs. The experiments conducted on various images with image rotation, scaling, and translation validate the effectiveness of the proposed method. The experimental results also demonstrate that about 30% in the overall coverage of major lines and 20% in the coverage per major line are increased compared with the state‐of‐the‐art line detectors. Moreover, the performance of the proposed method produces a combined advantage of ∼17% in the coverage of line segments over the line segment detector with noisy images.

Cracks commonly develop in expressways owing to repeated vehicle loads and environmental impact. Therefore, timely and accurate detection of minor cracks can help prevent severe and expensive rehabilitation efforts as well as assist in inspections to avert significant transportation system failures. Consequently, crack detection and measurement in expressways has been extensively researched. Traditional two-dimensional approaches present challenges, as they typically require distance measurement data and large workload. A pan–tilt–zoom (PTZ) camera–based crack detection and measurement method based on deep learning to obtain the size measurement of detected cracks is proposed in this paper. A YOLOv5-based crack recognition model is first trained to detect and classify crack images captured by PTZ cameras. Our previously proposed crack size estimation method is then applied on these crack images to estimate length and width. The proposed approach successfully detects and measures cracks on highway pavements; the YOLOv5-based crack recognition results yield 87.7% mean average precision and more than 90% average precision for crack real size measurement on 12 cracks. Case studies conducted in the G4/Jingshi Highway demonstrate the applicability of the method and enable tuning of the measurement algorithm parameters, confirming the viability of our proposal. Cracks in highway pavement are among the most critical problems in expressway maintenance. Crack detection and measurement not only have severe consequences in relation to expressway safety and decide maintenance timing, but can also prevent the rapid development of crack disease. Owing to ITS development of an expressway network in China, PTZ camera–based crack detection and measurement using YOLOv5 networks is proposed. We train a crack recognition model using YOLOv5 to classify cracks before length and width estimation. Case studies conducted in the G4/Jingshi Highway in the Hebei province of China demonstrate that the proposed method was able to detect and quantify cracks in a cost-effective manner, providing objective and timely guidance for preventive maintenance. This method shows its promise in real-world civil engineering applications for network-level highway maintenance engineering.