The data collected by unmanned aerial vehicles (UAV) provide unique opportunities for applications, such as construction progress tracking and facility monitoring. To increase the effectiveness of UAV-captured data, inspection mission plans should be designed prior to site visits. The data collection locations must be identified and adjusted based on the user’s objectives. The UAV flight mission must guarantee a visit to each identified location, and the inspection plans must satisfy the constraints imposed by safety requirements, no-fly zones, and UAV’s limited battery life. This problem becomes more complicated indoors due to complex and continuously changing building layouts. This work uses four-dimensional (4D) building information models (BIM) to automatically design optimal UAV mission plans in support of indoor UAV-enabled data acquisition. It automatically identifies the inspection targets based on users’ customizable and multicriteria description of objectives (e.g., columns behind schedule). The three-dimensional (3D) navigable space is automatically calculated by time stamping the 4D models based on the inspection date. The navigable space is further refined based on safety rules. An optimal UAV inspection mission plan is developed using swarm intelligence that ensures complete coverage of targets and minimizes battery use. The method is based on the industry foundation classes (IFC) schema, promoting OpenBIM and interoperability, which are core challenges in the construction information modeling domain.

The safety of human workers has been the main concern in human-robot close collaboration. Along with rapidly developed artificial intelligence techniques, deep learning models using two-dimensional images have become feasible solutions for human motion detection. These models serve as “sensors” in the closed-loop system that involve humans and robots. Most existing methods that detect human motion using images do not consider the uncertainty from the deep learning model itself. The mappings established by deep learning models should not be taken blindly, and thus uncertainty should be a natural part of this type of sensor. In particular, model uncertainty should be explicitly quantified and incorporated into robot motion control to guarantee safety. With this motivation, to rigorously quantify the uncertainty of these “sensors”, this letter proposes a probabilistic interpretation method and automatically provides a framework to benefit from a deep model's uncertainty. Experimental data from human-robot collaboration has been collected and used to validate the proposed method. A training strategy is proposed to efficiently train surrogate models that learn to refine the prediction of the main Bayesian models. The proposed framework is also compared with Ego hands benchmark showing a 4.7% increase in mIoU.

The existing machine learning (ML) models for vibration-based damage assessments often use highly compressed features or are restricted to preprocessed signals with fixed durations and sampling rates. Additionally, the learning capacities of ML models and computational resources are limited, which restricts using raw signals as direct input. This paper studied Mel filter banks (MFBs) for seismic signal processing, inspired by speech recognition technology. It is argued that the same filter designs in audio engineering may not be appropriate for seismic records, and therefore, a customized filter bank formulation was developed. Hybrid deep learning models for rapid assessments (HyDRA) were introduced as multibranch neural network architectures that enable end-to-end training for different types of processed vibration data structures. Moreover, the performance-based earthquake engineering (PBEE) equation was adjusted to integrate ML model uncertainties for probabilistic assessments. The proposed concepts were validated in a case study based on a data set of 32,400 nonlinear time-history analyses of a highway bridge in California. Insights and guidelines are provided for optimum filter design based on 5,184 experiments. Several HyDRA architectures were compared with benchmark models. The optimized MFB feature type outperformed features obtained from continuous wavelet transform and a stacked vector of conventional earthquake engineering indexes. A Bayesian variant of HyDRA was investigated to showcase its integration in the modified PBEE equation. Adopting custom filter banks with the HyDRA architecture enables effective feature extraction from raw vibration records by diversifying feature space and preserving information in the time and frequency domains.

Inspection planning is a primary element of computer vision- and unoccupied aerial vehicle (UAV)-enabled construction monitoring. Prior to the on-site deployment of camera-mounted UAVs, the inspection objectives need to be identified, and optimal inspection plans must be developed; Such plans should ensure complete data acquisition and minimize the use of UAV’s limited flight time. The image capture configuration must be taken into account since it directly affects the downstream applications of the captured data such as progress detection and as-built modeling. This paper proposes a framework and a novel technique which utilizes four-dimensional (4D) building information models (BIM) and swarm intelligence to automatically generate the UAV inspection mission plans. It computationally supports both static and dynamic site layouts. The inspection objectives, their geometry, and their semantics are automatically extracted from BIM, and the corresponding elements are identified. An optimal inspection plan is developed using artificial intelligence, ensuring complete coverage of inspection targets while minimizing flight duration. The method has been tested in UAV-enabled data acquisition scenarios. It is based on the industry foundation classes (IFC), facilitating OpenBIM and reducing the costs associated with the lack of interoperability, a core challenge in information modeling. Due to the target extraction at element and sub-element levels, it supports computer vision-based construction progress monitoring and automated as-built and as-is BIM development.

Bridges are the essential components in lifeline transportation systems, and their safe operation is of great importance. Information on structural damage could assist in timely repairs and reduce downtime. With the latest advancements in sensing technology, collecting vibration data from bridges has become more accessible. However, effective vibration processing is still a challenge, given the high dimensionality and massive size of vibration data. Existing studies have shown that machine/deep learning techniques can be valuable tools for this task. However, the learning and computational capacities of these models are challenged in the presence of large sensor arrays. We propose Trident as a novel deep learning framework that enables automatic damage feature extraction by simultaneously learning from temporal and three-dimensional (3D) spatial variations of 6D input data in instrumented bridges. Trident is equipped with 3 ConvLSTM3D branches to achieve this goal. A 3D steel truss bridge subject to dynamic traffic loads is monitored for its vibrations to evaluate Trident’s robustness in finding damaged elements. A damage dataset of 52,800 vehicle passing simulations is generated leveraging a database of 528 passenger vehicles in the United States, obtained from the National Highway and Traffic Safety Administration. Bayesian optimization is utilized to tune the model’s hyperparameters, achieving a test Node Average Geometric Mean Accuracy of 86%. This level of performance is promising given the high dimensionality and complexities of the output space in vibration-based monitoring. Trident’s concept can be extended to other vibration monitoring tasks with different time series data and damage labeling strategies.