Computer vision techniques for extracting dynamic structural displacements from videos are gaining increasing acceptance for the purposes of system identification and structural health monitoring. However, the application of video-based techniques for modal analysis of full-scale civil infrastructure has been limited, because obtaining measurements of all points on a large structure with a single video frame with sufficient resolution is seldom feasible. In this study, a new approach is presented to facilitate the extraction of frequencies and mode shapes of full-scale civil infrastructure from video obtained by an unmanned aerial vehicle (UAV). This approach addresses directly a number of difficulties associated with modal analysis of full-scale infrastructure using vision-based methods. The proposed approach is evaluated using a story-story shear-building model excited on a shaking table in a laboratory environment, and on a full-scale pedestrian suspension bridge. The results demonstrate the efficacy of the proposed approach.

This research investigates vision‐based automated bridge component recognition, which is critical for automating visual inspection of bridges during initial response after earthquakes. Semantic segmentation algorithms with up to 45 convolutional layers are applied to recognize bridge components from images of complex scenes. One of the challenges in such scenarios is to get the recognition results consistent with high‐level scene structure using limited amount of training data. To impose the high‐level scene consistency, this research combines 10‐class scene classification and 5‐class bridge component classification. Three approaches are investigated to combine scene classification results into bridge component classification: (a) naïve configuration, (b) parallel configuration, and (c) sequential configuration of classifiers. The proposed approaches, sequential configuration in particular, are demonstrated to be effective in recognizing bridge components in complex scenes, showing less than 1% of accuracy loss from the naïve/parallel configuration for bridge images, and less than 1% false positives for the nonbridge images.

Computer vision techniques have been employed to characterize dynamic properties of structures, as well as to capture structural motion for system identification purposes. All of these methods leverage image-processing techniques using a stationary camera. This requirement makes finding an effective location for camera installation difficult, because civil infrastructure (i.e., bridges, buildings, etc.) are often difficult to access, being constructed over rivers, roads, or other obstacles. This paper seeks to use video from Unmanned Aerial Vehicles (UAVs) to address this problem. As opposed to the traditional way of using stationary cameras, the use of UAVs brings the issue of the camera itself moving; thus, the displacements of the structure obtained by processing UAV video are relative to the UAV camera. Some efforts have been reported to compensate for the camera motion, but they require certain assumptions that may be difficult to satisfy. This paper proposes a new method for structural system identification using the UAV video directly. Several challenges are addressed, including: (1) estimation of an appropriate scale factor; and (2) compensation for the rolling shutter effect. Experimental validation is carried out to validate the proposed approach. The experimental results demonstrate the efficacy and significant potential of the proposed approach.

Manual visual inspection of civil infrastructure is high-risk, subjective, and time-consuming. The success of deep learning and the proliferation of low-cost consumer robots has spurred rapid growth in research and application of autonomous inspections. The major components of autonomous inspection include data acquisition, data processing, and decision making, which are usually studied independently. However, for robust real-world applicability, these three aspects of the overall process need to be addressed concurrently with end-to-end testing, incorporating scenarios such as variations in structure type, color, damage level, camera distance, view angle, lighting, etc. Developing real-world datasets that span all these scenarios is nearly impossible. In this paper, we propose a framework to create a virtual visual inspection testbed using 3D synthetic environments that can enable end-to-end testing of autonomous inspection strategies. To populate the 3D synthetic environment with virtual damaged buildings, we propose the use of a non-linear finite element model to inform the realistic and automated visual rendering of different damage types, the damage state, and the material textures of what are termed herein physics-based graphics models (PBGMs). To demonstrate the benefits of the autonomous inspection testbed, three experiments are conducted with models of earthquake damaged reinforced concrete buildings. First, we implement the proposed framework to generate a new large-scale annotated benchmark dataset for post-earthquake inspections of buildings termed QuakeCity. Second, we demonstrate the improved performance of deep learning models trained using the QuakeCity dataset for inference on real data. Finally, a comparison of deep learning-based damage state estimation for different data acquisition strategies is carried out. The results demonstrate the use of PBGMs as an effective testbed for the development and validation of strategies for autonomous vision-based inspections of civil infrastructure.