Most civil infrastructure worldwide is currently past its design life. This situation has precipitated the need for more systematic inspections and continual monitoring of infrastructure to assure structural integrity. With the advancement of sensing and computing technologies, rapid, remote, real-time, and robust structural health monitoring (SHM) techniques have been significantly developed. These smart SHM technologies often result in big data that require advanced data management, visualization, diagnostic, and prognostic techniques. Over the past few decades, researchers have developed numerous machine learning and artificial intelligence (AI)–based damage diagnostic and prognosis methods, which have been systematically reviewed in the recent state-of-the-art papers. In parallel, various data management and visualization techniques have been explored in SHM using building information modeling (BIM), virtual reality (VR), and augmented reality (AR). Both BIM and AR/VR present a unique opportunity to document, systematically interpret, and visualize SHM data in a three-dimensional (3D) environment and have shown significant promise in a wide range of infrastructure monitoring applications. Unlike diagnostics and prognostics methods of SHM, there has been a very limited systematic review of the latest visualization and data management techniques, which are both the objective and novelty of this review paper. Each subtopic reviews The recent data management and visualization technologies, their relevance to SHM, and their implementation challenges in broader structural engineering applications. Finally, this state-of-the-art review summarizes potential future research directions for BIM and VR/AR associated with SHM.

A significant percentage of bridges in the United States are serving beyond their 50-year design life, and many of them are in poor condition, making them vulnerable to fatigue cracks that can result in catastrophic failure. However, current fatigue crack inspection practice based on human vision is time-consuming, labor intensive, and prone to error. We present a novel human-centered bridge inspection methodology to enhance the efficiency and accuracy of fatigue crack detection by employing advanced technologies including computer vision and augmented reality (AR). In particular, a computer vision-based algorithm is developed to enable near-real-time fatigue crack detection by analyzing structural surface motion in a short video recorded by a moving camera of the AR headset. The approach monitors structural surfaces by tracking feature points and measuring variations in distances between feature point pairs to recognize the motion pattern associated with the crack opening and closing. Measuring distance changes between feature points, as opposed to their displacement changes before this improvement, eliminates the need of camera motion compensation and enables reliable and computationally efficient fatigue crack detection using the nonstationary AR headset. In addition, an AR environment is created and integrated with the computer vision algorithm. The crack detection results are transmitted to the AR headset worn by the bridge inspector, where they are converted into holograms and anchored on the bridge surface in the 3D real-world environment. The AR environment also provides virtual menus to support human-in-the-loop decision-making to determine optimal crack detection parameters. This human-centered approach with improved visualization and human–machine collaboration aids the inspector in making well-informed decisions in the field in a near-real-time fashion. The proposed crack detection method is comprehensively assessed using two laboratory test setups for both in-plane and out-of-plane fatigue cracks. Finally, using the integrated AR environment, a human-centered bridge inspection is conducted to demonstrate the efficacy and potential of the proposed methodology.