This article introduces the use of emerging augmented reality technology to enable the next generation of structural infrastructure inspection and awareness. This work is driven by the prevalence of visual structural inspection. It is known that current visual inspection techniques have multiple sources of variance that should be reduced in order to achieve less ambiguous visual inspections. Emerging augmented reality tools feature a variety of sensors, computation, and communication resources that can enable relevant structural inspection data to be collected at very high resolution in an unambiguous manner. This work shows how emerging augmented reality tools can be used to greatly enhance our ability to capture comprehensive, high-resolution, three-dimensional measurements of critical infrastructure. This work also provides detailed information on the software architecture for augmented reality structural inspection applications that helps meet the goals of the framework. The fact that the framework is designed to accommodate the considerations associated with high-consequence infrastructure implies that it is also comprehensive enough to be applied to less hazardous but still high-value infrastructure such as bridges, dams, and tunnels. Augmented reality has great potential to enable the next generation of smart infrastructure, and this work focuses on addressing how augmented reality can be leveraged to enable the next generation of structural awareness for high-consequence, long-lifespan structures.

Measurement of bridge displacements is important for ensuring the safe operation of railway bridges. Traditionally, contact sensors such as Linear Variable Displacement Transducers (LVDT) and accelerometers have been used to measure the displacement of the railway bridges. However, these sensors need significant effort in installation and maintenance. Therefore, railroad management agencies are interested in new means to measure bridge displacements. This research focuses on mounting Laser Doppler Vibrometer (LDV) on an Unmanned Aerial System (UAS) to enable contact-free transverse dynamic displacement of railroad bridges. Researchers conducted three field tests by flying the Unmanned Aerial Systems Laser Doppler Vibrometer (UAS-LDV) 1.5 m away from the ground and measured the displacement of a moving target at various distances. The accuracy of the UAS-LDV measurements was compared to the Linear Variable Differential Transducer (LVDT) measurements. The results of the three field tests showed that the proposed system could measure non-contact, reference-free dynamic displacement with an average peak and root mean square (RMS) error for the three experiments of 10% and 8% compared to LVDT, respectively. Such errors are acceptable for field measurements in railroads, as the interest prior to bridge monitoring implementation of a new approach is to demonstrate similar success for different flights, as reported in the three results. This study also identified barriers for industrial adoption of this technology and proposed operational development practices for both technical and cost-effective implementation.

Decaying infrastructure maintenance cost allocation depends heavily on accurate and safe inspection in the field. New tools to conduct inspections can assist in prioritizing investments in maintenance and repairs. The industrial revolution termed as “Industry 4.0” is based on the intelligence of machines working with humans in a collaborative workspace. Contrarily, infrastructure management has relied on the human for making day-to-day decisions. New emerging technologies can assist during infrastructure inspections, to quantify structural condition with more objective data. However, today’s owners agree in trusting the inspector’s decision in the field over data collected with sensors. If data collected in the field is accessible during the inspections, the inspector decisions can be improved with sensors. New research opportunities in the human–infrastructure interface would allow researchers to improve the human awareness of their surrounding environment during inspections. This article studies the role of Augmented Reality (AR) technology as a tool to increase human awareness of infrastructure in their inspection work. The domains of interest of this research include both infrastructure inspections (emphasis on the collection of data of structures to inform management decisions) and emergency management (focus on the data collection of the environment to inform human actions). This article describes the use of a head-mounted device to access real-time data and information during their field inspection. The authors leverage the use of low-cost smart sensors and QR code scanners integrated with Augmented Reality applications for augmented human interface with the physical environment. This article presents a novel interface architecture for developing Augmented Reality–enabled inspection to assist the inspector’s workflow in conducting infrastructure inspection works with two new applications and summarizes the results from various experiments. The main contributions of this work to computer-aided community are enabling inspectors to visualize data files from database and real-time data access using an Augmented Reality environment.

Traditional vibration measurement typically requires physically attached sensors, such as accelerometers and strain gauges. However, these discrete-point sensors provide only low spatial resolution vibration measurements, potentially foregoing valuable structural information such as localized damage. Noncontact optical measurement methods such as laser vibrometers can achieve high spatial resolution vibration measurements but only through time-consuming sequential measurements and are not cost-effective. As an alternative to traditional vibration measurement methods, digital video cameras are relatively low-cost, agile, and offer noncontact, simultaneous high spatial resolution measurements where every pixel on the structure becomes a measurement point. However, regular digital video cameras are frame-based where each pixel simultaneously performs temporally uniform (synchronous) measurements containing large amounts of redundant (background) data, which consumes considerable resources for video data measurement, management, and processing. To alleviate such a challenge, this work explores the use of event-based neuromorphic imagers, specifically silicon retinas, an efficient alternative to traditional frame-based video cameras, to perform full-field vibration measurements and operational modal analysis. By imitating biological vision, each silicon retina pixel independently and asynchronously records only intensity change events that contain structural motion information while excluding redundant (background) information. Such an asynchronous event-based data measurement mechanism allows for structural motion to be captured on the microsecond scale in an extremely data-efficient manner, which could benefit real-time vibration measurement and control applications. This study takes the first step toward these applications by formulating an existing video frame-based full-field operational modal analysis technique in the event-based, asynchronous silicon retina measurement framework. Specifically, local phase-based motion extraction and blind source separation are used to automatically and efficiently extract full-field vibration and dynamics parameters from silicon retina measurements. The developed method is validated by laboratory experiments on a bench-scale cantilever beam.

Wireless sensor networks (WSN) are used by engineers to record the behavior of structures. The sensors provide data to be used by engineers to make informed choices and prioritize decisions concerning maintenance procedures, required repairs, and potential infrastructure replacements. However, reliable data collection in the field remains a challenge. The information obtained by the sensors in the field frequently needs further processing, either at the decision-making headquarters or in the office. Although WSN allows data collection and analysis, there is often a gap between WSN data analysis results and the way decisions are made in industry. The industry depends on inspectors’ decisions, so it is of vital necessity to improve the inspectors’ access in the field to data collected from sensors. This paper presents the results of an experiment that shows the way Augmented Reality (AR) may improve the availability of WSN data to inspectors. AR is a tool which overlays the known attributes of an object with the corresponding position on the headset screen. In this way, it allows the integration of reality with a virtual representation provided by a computer in real time. These additional synthetic overlays supply data that may be unavailable otherwise, but it may also display additional contextual information. The experiment reported in this paper involves the application of a smart Strain Gauge Platform, which automatically measures strain for different applications, using a wireless sensor. In this experiment, an AR headset was used to improve actionable data visualization. The results of the reported experiment indicate that since the AR headset makes it possible to visualize information collected from the sensors in a graphic form in real time, it enables automatic, effective, reliable, and instant communication from a smart low-cost sensor strain gauge to a database. Moreover, it allows inspectors to observe augmented data and compare it across time and space, which then leads to appropriate prioritization of infrastructure management decisions based on accurate observations.

Video-based techniques for identification of structural dynamics have the advantage that they are very inexpensive to deploy compared to conventional accelerometer or strain gauge techniques. When structural dynamics from video is accomplished using full-field, high-resolution analysis techniques utilizing algorithms on the pixel time series such as principal components analysis and solutions to blind source separation the added benefit of high-resolution, full-field modal identification is achieved. An important property of video of vibrating structures is that it is particularly sparse. Typically video of vibrating structures has a dimensionality consisting of many thousands or even millions of pixels and hundreds to thousands of frames. However the motion of the vibrating structure can be described using only a few mode shapes and their associated time series. As a result, emerging techniques for sparse and random sampling such as compressive sensing should be applicable to performing modal identification on video. This work presents how full-field, high-resolution, structural dynamics identification frameworks can be coupled with compressive sampling. The techniques described in this work are demonstrated to be able to recover mode shapes from experimental video of vibrating structures when 70% to 90% of the frames from a video captured in the conventional manner are removed.