In the realm of crack inspection for complex infrastructures, traditional methods have primarily relied on expensive structural health monitoring instruments and labor-intensive procedures. The emergence of Unmanned Aerial Vehicle (UAV) technology brings about effective and innovative solutions for bridge inspection. To advance the technology, this study presents a novel crack inspection system that employs LiDAR scanning to construct a 3D model of the target structure. A path planner is then developed to ensure complete coverage of all crack points on the structure being inspected. Through extensive testing, the proposed system demonstrates successful detection and localization of various types of cracks. Furthermore, our improved deep crack segmentation model, U-Net with Spectral block and Self-Attention module, surpasses the performance of the original U-Net model, exhibiting a 3.2% higher Dice coefficient and a 3.3% higher mean Intersection over Union ( mIoU ) evaluation metric on our self-established crack dataset. In the case of the Crack500 public dataset, our model outperforms the original U-Net model by 10% in Dice coefficient and 14% in mIoU . Moreover, our USSA-Net outperforms other latest state-of-the-art models on the DeepCrack500 dataset, surpassing the PAF-Net and PHCF-Net by approximately 5% in Dice coefficient and 2.7% in mIoU . For crack size estimations, our proposed system accurately estimates the horizontal and vertical dimensions of cracks, achieving a root-mean-square error of 9.9 mm and 6.2 mm, respectively. Overall, the system achieves mm-level crack size estimation accuracy. Moreover, our system is characterized by its low-cost nature and lightweight design. Experimental results showcase the system’s robustness and effectiveness in executing real-world crack inspection tasks, even within complex environments.
Traditional methods on crack inspection for large infrastructures require a number of structural health inspection devices and instruments. They usually use the signal changes caused by physical deformations from cracks to detect the cracks, which is time-consuming and cost-ineffective. In this work, we propose a novel real-time crack inspection system based on unmanned aerial vehicles for real-world applications. The proposed system successfully detects and classifies various types of cracks. It can accurately find the crack positions in the world coordinate system. Our detector is based on an improved YOLOv4 with an attention module, which produces 90.02% mean average precision (mAP) and outperforms the YOLOv4-original by 5.23% in terms of mAP. The proposed system is low-cost and lightweight. Moreover, it is not restricted by navigation trajectories. The experimental results demonstrate the robustness and effectiveness of our system in real-world crack inspection tasks.
The inspection of electrical and mechanical (E&M) devices using unmanned aerial vehicles (UAVs) has become an increasingly popular choice in the last decade due to their flexibility and mobility. UAVs have the potential to reduce human involvement in visual inspection tasks, which could increase efficiency and reduce risks. This paper presents a UAV system for autonomously performing E&M device inspection. The proposed system relies on learning-based detection for perception, multi-sensor fusion for localization, and path planning for fully autonomous inspection. The perception method utilizes semantic and spatial information generated by a 2-D object detector. The information is then fused with depth measurements for object state estimation. No prior knowledge about the location and category of the target device is needed. The system design is validated by flight experiments using a quadrotor platform. The result shows that the proposed UAV system enables the inspection mission autonomously and ensures a stable and collision-free flight.
14 October 2024 in IEEE Transactions on Instrumentation and Measurement
Mobile robots extensively employ multiple sensors, including RGBD cameras, LiDAR, and thermal sensors. Sensor fusion plays a vital role in localization and environment perception tasks. However, traditional manual target-based methods for achieving consistent alignment among multiple sensors, especially thermal cameras, are laborious and lack adaptability. This work introduces an autonomous targetless framework for calibrating LiDAR-RGB and LiDAR-thermal sensors in mobile robots. In our proposed framework, we examine the characteristics of thermal images, identify suitable calibration scenarios, and employ the thermal bridge and line-based PnL algorithm to enable autonomous and targetless calibration. Experimental results demonstrate the efficiency of our method, with overall translation errors of 2.77 cm and 3.86 cm, and overall rotation errors of 0.21 degrees and 0.46 degrees, respectively, in LiDAR-RGB and LiDAR-thermal calibrations. These results are comparable to state-of-the-art techniques and traditional target-based manual methods. The analysis of different thermal scenes highlights the importance of well-aligned and distinguishable edges across thermal-RGB-LiDAR modalities for optimal calibration results. Simulation tests using synthetic data and validation tests using real-world data showcase the robustness of our model in executing targetless extrinsic calibrations.
1. CONTEXTTeleoperated robotic systems are widely used in hazardous and complex domains, but their reliance on continuous human attention and control limits scalability, robustness, and long-term deployment. As robotic missions increase in duration and complexity, there is a growing need to transition such systems toward autonomy while preserving operational intent, safety assurances, and compatibility with existing hardware. Despite the need, this migration from teleoperation to autonomy remains largely ad hoc, with limited support for requirements formalisation, architectural structuring, and formal verification.2. OBJECTIVESThis paper provides a principled framework for transitioning remote-controlled robotic systems into autonomous systems, controlled by goal-driven agents.3. METHODSOperator-led behaviours are incrementally transformed into explicit, formally specified agent goals, enabling verification and deployment in realistic environments. To support robustness and reliability, the framework employs a layered verification strategy combining satisfiability checking, model checking, simulation-based testing and runtime monitoring. The approach is evaluated through a case study in underground robotic inspection, and we provide a high-fidelity Gazebo simulation of a curved tunnel environment for simulation-based testing.4. RESULTSExtensive tests are carried out in randomised environments, requiring the agent to satisfy all formalised requirements. The results demonstrate that, in this case study, the autonomous agent exhibits improved environmental awareness and executes safer actions with a higher task success rate than a teleoperated baseline.5. CONCLUSIONWe propose this transformation from remote-control to autonomy, through the use of verified goaldriven agents, as the basis of a general framework for moving to autonomous systems in a principled way.