Developing deep neural network (DNN) models for computer vision applications for construction is challenging due to the shortage of training data. To address this issue, we proposed a novel data augmentation method that integrates a conditional generative adversarial networks (GANs) framework with a target classifier. The integrated architecture enables adversarial attack and defense during end-to-end training, thereby making it possible to generate effective images for the target classifier’s training. We trained and tested two image classification DNNs with and without data augmentation, where we confirmed the effectiveness of the proposed method: with the data augmentation, the classification accuracy improved by 4.2 percentage points, from 71.24% to 75.46%, with qualitatively improved feature extraction more focused on the target object. Given that the application areas of our method are open-ended, the result is noteworthy. The proposed method can help construction researchers offset the data insufficiency, which will contribute to having more accurate and scalable DNN-powered vision models in construction applications.

Contact-driven accidents involving actuated excavators have led to a significant number of fatalities in the construction industry. The revolving mechanical arm of excavators poses a major risk of contact-driven accidents for workers in its proximity due to its articulated pose. Detecting the 3D pose of excavator arms is thus essential to prevent contact-driven accidents near excavators. Previous works have attempted to estimate 3D excavator poses using sensor-based or computer vision-based methods. However, existing methods require extensive preparation work, such as attaching physical sensors, calibrating stereo cameras, or collecting 3D training data. As a result, existing methods cannot be easily integrated into the current construction workflow and are seldom applied in real-world situations. The authors propose a projection-based 3D pose optimization method that utilizes excavator kinematic constraints to infer 3D excavator poses from monocular image sequences with no dependency on 3D training data. The proposed method first extracts the 2D excavator pose from images using a keypoint region-based convolution neural network. Then, the 2D pose is reconstructed into 3D by enforcing the rigid excavator kinematic constraints (e.g., arm length) and minimizing the 2D reprojection error of the excavator pose. Tests using a 1:14 miniature excavator model showed a 3D position error of 7.3 cm (or 1.03 m when scaled up to real-world dimensions) for keypoints on the excavator pose, demonstrating the capabilities of the proposed method in estimating 3D excavator poses from monocular images. The proximity measuring capacity of the estimated 3D pose was also evaluated, achieving a mean absolute distance error of 4.7 cm (or 0.66 m scaled). The proposed method offers a 3D excavator pose estimation method using only a monocular camera and without relying on 3D training data. The estimated 3D excavator pose enables safety managers to monitor potential contact-driven accidents near excavators and alert workers of unsafe situations and promotes safer working environments for construction workers near excavators. The authors present a monocular vision-based method for 3D excavator pose estimation, which can serve as the groundwork for monitoring contact-driven accidents near excavators. The method allows safety managers to monitor the 3D pose of an excavator using one single camera (e.g., smartphone, site surveillance camera, or action camera). Unlike previous methods, the proposed method does not require construction professionals to conduct challenging preparation work such as setting up multiple stereo cameras or collecting custom 3D excavator training datasets. For instance, for an excavator 21 m away, the proposed method can estimate the 3D excavator’s keypoint positions with an expected 3D position error of 1.03 m. Paired with other onsite information (e.g., utility line or worker location) collected from existing drawings or using other vision-based methods, the proposed method can measure the proximity between the excavator arm and surrounding objects with an expected error of 0.66 m, enabling proactive safety interventions (e.g., proximity alerts). Such safety interventions can make it safer for workers to conduct construction work near excavators. Though originally devised for excavators, the proposed method can be further adapted to monitor the 3D pose of other construction equipment such as backhoes or crawler cranes, enabling a wider range of safety applications for monitoring contact-driven accidents.

Off-site construction is well-known technology that facilitates parallel processes of manufacturing and construction processes. This method enhances productivity while reducing accident, cost, and environmental impact. Many studies have highlighted its benefits, prompting further encouragement of off-site construction. This study consolidates current research and charts future directions by reviewing the existing literature. However, reviewing papers is time-intensive and laborious. Consequently, generative AI models, particularly Large Language Models (LLMs), are increasingly employed for document summarization. Specifically, LangChain influences LLMs through chaining data, demonstrating notable potential for research paper reviews. This study aims to evaluate the well-documented advantages of off-site construction through LangChain integrated with an LLM. It follows a streamlined process from the collection of research papers to conducting network analysis, examining 47 papers to uncover that current research primarily demonstrates off-site construction’s superiority through cutting-edge technologies. Yet, a data deficiency remains a challenge. The findings demonstrate that LangChain can rapidly and effectively summarize research, making it a valuable tool for literature reviews. This study advocates the broader application of LangChain in reviewing research papers, emphasizing its potential to streamline the literature review process and provide clear insights into off-site construction’s evolving landscape.

Improving the rapidity of 3D reconstruction is vital for time-critical construction tasks such as progress monitoring and hazard detection, but the majority of construction studies in this area have focused on improving its quality. We applied a Direct Sparse Odometry with Loop Closure (LDSO)-based 3D reconstruction method, improving the existing algorithm and tuning its hyper-parameter settings, to achieve both near real-time operation and quality 3D point cloud simultaneously. When validated using a benchmark dataset, the proposed method showed notable improvement in 3D point cloud density, as well as loop closure robustness, compared to the original LDSO. In addition, we conducted a real field test to validate the tuned LDSO’s accuracy and speed at both object and site scales, where we demonstrated our method’s near real-time operation and capability to produce a quality 3D point cloud comparable to that of the existing method. The proposed method improves the accessibility of the 3D reconstruction technique, which in turn helps construction professionals monitor their jobsite safety and progress in a more efficient manner.

Digital management of excavators has seen limited progress due to challenges in artificial intelligence (AI) training. The AI required for digital twinning of excavators necessitates a large volume of diverse imagery data, currently scarce in the construction domain. Moreover, the absence of deployable robot agents hinders reinforcement learning, impeding task-oriented AI development. In response, we introduce an innovative approach utilizing a miniature-scale, radio-controlled excavator (RC-excavator). This presents a cost-effective method for automated data collection and labeling, as well as interactive reinforcement learning. The RC-excavator’s electric circuit was modified, its motion dynamics were modeled, and it was fully robotized for precise computer-directed motion control. Statistical validation of its motions achieved a Normalized Range Adjusted Accuracy (NRAA) of 99.14% for the bucket, 97.97% for the main arm, and 98.63% for the cabin. This confirms its adequacy for image labeling and task-oriented automation research.

Deploying construction robots alongside workers presents the risk of unwanted forcible contact—a critical safety concern. To address a semantic digital twin where such contact-driven hazards can be monitored accurately, the authors present a single-shot deep neural network (DNN) model that can perform proximity and relationship detections simultaneously. Given that workers and construction robots must sometimes collaborate in close proximity, their relationship must be considered, along with proximity, before concluding an event is a hazard. To address this issue, we leveraged a unique two-in-one DNN architecture called Pixel2Graph (i.e., object + relationship detections). The potential of this DNN architecture for relationship detection was confirmed by follow-up testing using real-site images, achieving 90.63% recall@5 when object bounding boxes and classes were given. When integrated with existing proximity monitoring methods, single-shot visual relationship detection will enable the accurate identification of contact-driven hazards in a digital twin platform, an essential step in realizing sustainable and safe collaboration between workers and robots.