This paper presents an accurate and stable method for object and defect detection and visualization on building and infrastructural facilities. This method uses drones and cameras to collect three-dimensional (3D) point clouds via photogrammetry, and uses orthographic or arbitrary views of the target objects to generate the feature images of points’ spectral, elevation, and normal features. U-Net is implemented in the pixelwise segmentation for object and defect detection using multiple feature images. This method was validated on four applications, including on-site path detection, pavement cracking detection, highway slope detection, and building facade window detection. The comparative experimental results confirmed that U-Net with multiple features has a better pixelwise segmentation performance than separately using each single feature. The developed method can implement object and defect detection with different shapes, including striped objects, thin objects, recurring and regularly shaped objects, and bulky objects, which will improve the accuracy and efficiency of inspection, assessment, and management of buildings and infrastructural facilities.

Three-dimensional (3D) surveying of a construction site using an image-based method may produce incorrect ground elevation results at vegetation-covered regions, because the light rays are reflected on the surface of vegetation in front of the “truth” ground. This paper presents a convolutional neural network (CNN) method to identify and locate static vegetation using drone-based high-resolution orthoimages. The developed CNN-based image classification models are supplemented with an overlapping disassembling algorithm to generate $8\times 8\text{-}\mathrm{pixel}$ , $16\times 16\text{-}\mathrm{pixel}$ , $32\times 32\text{-}\mathrm{pixel}$ , or $64\times 64\text{-}\mathrm{pixel}$ small-patches as model inputs. The training datasets are 10 pairs of $\mathrm{1,536}\times \mathrm{1,536}\text{-}\mathrm{pixel}$ orthoimage and label-image dataset. Experimental results show that cropping a high-resolution image into 9,025 overlapped $32\times 32\text{-}\mathrm{pixel}$ small-patches (with a site size of $17.28\times 17.28{\mathrm{cm}}^2$ ) for image classification, and assembling the small-patch label-image predictions to a patch-wise label-image prediction, has the average pixel accuracy of 92.6% in identifying objects on the experimental site. In addition, a vegetation-removing algorithm is designed to divide the label-image prediction into 36,864 nonoverlapping $8\times 8\text{-}\mathrm{pixel}$ patches and traverse them in 192 row-loops and 191 column-loops. The testing results show vegetation in label-images are modified with the “truth” ground elevation and verified with two datasets obtained on different dates. In addition, the measured elevation differentials are close to the measured vegetation heights on the experimental site. This research has advanced the drone-based orthoimaging method in construction site surveying, which can automatically identify the static obstacles and determine the ground elevations more accurately. Furthermore, an approach of using a CNN model to segment a construction site has been proven feasible.

This paper presents an automated as-is façade modeling method for existing and historic high-rise buildings, named Scan4Façade. To begin with, a camera drone with a spiral path is employed to capture building exterior images, and photogrammetry is used to conduct three-dimensional (3D) reconstruction and create mesh models for the scanned building façades. High-resolution façade orthoimages are then generated from mesh models and pixelwise segmented by an artificial intelligence (AI) model named U-net. A combined data augmentation strategy, including random flipping, rotation, resizing, perspective transformation, and color adjustment, is proposed for model training with a limited number of labels. As a result, the U-net achieves an average pixel accuracy of 0.9696 and a mean intersection over union of 0.9063 in testing. Then, the developed twoStagesClustering algorithm, with a two-round shape clustering and a two-round coordinates clustering, is used to precisely extract façade elements’ dimensions and coordinates from façade orthoimages and pixelwise label. In testing with the Michigan Central Station (office tower), a historic high-rise building, the developed algorithm achieves an accuracy of 99.77% in window extraction. In addition, the extracted façade geometric information and element types are transformed into AutoCAD command and script files to create CAD drawings without manual interaction. Experimental results also show that the proposed Scan4Façade method can provide clear and accurate information to assist BIM feature creation in Revit. Future research recommendations are also stated in this paper.

This paper presents a 3D reconstruction method for fast elevation determination on construction sites. The proposed method is intended to automatically and accurately determine construction site elevations using drone-based, low–high orthoimage pairs. This method requires fewer images than other methods for covering a large target area of a construction site. An up–forward–down path was designed to capture approximately $2:1$ -scale images at different altitudes over target stations. A pixel grid matching and elevation determination algorithm was developed to automatically match images in dense pixel grid-style via self-adaptive patch feature descriptors, and simultaneously determine elevations based on a virtual elevation model. The 3D reconstruction results were an elevation map and an orthoimage at each station. Then, the large-scale results of the entire site were easily stitched from adjacent results with narrow overlaps. Moreover, results alignment was automatically performed via the U-net detected ground control point. Experiments validated that in 10–20 and 20–40 orthoimage pairs, 92% of 2,500- and 4,761-pixels were matched in the strongest and strong levels, which was better than sparse reconstructions via structure from motion; moreover, the elevation measurements were as accurate as photogrammetry using multiscale overlapping images.

This paper presents the research results of using Google Earth imagery for visual condition surveying of highway pavement in the United States. A screenshot tool is developed to automatically track the highway for collecting end-to-end images and Global Position System (GPS). A highway segmentation tool based on a deep convolutional neural network (DCNN) is developed to segment the collected highway images into the predefined object categories, where the cracks are identified and labeled in each small patch of the overlapping assembled label-image prediction. Then, the longitudinal cracks and transverse cracks are detected using the x-gradient and y-gradient from the Sobel operator, and the developed pavement evaluation tool rates the longitudinal cracking in $0.3048\mathrm{m}/30.48\mathrm{m}\text{-}\mathrm{Station}$ (linear feet per 100 ft. station) and transverse cracking in number per $30.48\mathrm{m}$ -Station (100 ft. station), which can be visualized in ArcGIS Online. Experiments were conducted on Interstate 43 (I-43) in Milwaukee County with pavement in both defective and sound visual conditions. Experimental results showed the patch-wise highway segmentation in Google Earth imagery from the $16\times 16\text{-}\mathrm{pixel}$ DCNN model has as precise pixel accuracy as the U-net-based pixelwise crack/noncrack classifier. Compared to the manually crafted label image in the experimental area, the rated longitudinal cracking has an average error of overrating 20%, while transverse cracking has an average error of underrating 7%. This research project contributes to visual pavement condition surveying methodology with the free-to-access Google Earth imagery, which is a feasible, cost-effective option for accurately rating and geographically visualizing both project-level and network-level pavement.