This paper presents a method for mosaicking unmanned aerial vehicle (UAV) multispectral images. The main purpose of the proposed method is to reduce spatial distortion in the mosaicking process and increase robustness and the speed of the operation. Most UAV multispectral images have multiple bands, and in every band, ground targets have a variety of reflection characteristics that will result in diverse feature quality for feature matching. In this research, an information entropy-based evaluation method is used to select the optimal band for feature matching among the UAV images. To produce more robust matching results for the following alignment step, the evaluation method takes the contrast and spatial distribution of the feature points into consideration at the same time. In most common image mosaicking processes, the digital orthophoto map (DOM) is generated to achieve maximum spatial accuracy. During this process, the original image data will experience considerable irregular resampling, and the process is also unstable in some circumstances. The alignment step uses a simplified projection model that treats the ground as planar is provided, by which the alignment parameters are applied directly to the images instead of generating 3D points, to avoid irregular resampling and unstable 3D reconstruction. The proposed method is proved to be more efficient and accurate and has lower spectral distortion than state-of-the-art mosaicking software.

It is a challenging task to accurately segment damaged road markings from images, mainly due to their fragmented, dense, small-scale, and blurry nature. This study proposes a multi-scale spatial kernel selection net named M-SKSNet, a novel model that integrates a transformer and a multi-dilated large kernel convolutional neural network (MLKC) block to address these issues. Through integrating multiple scales of information, the model can extract high-quality and semantically rich features while generating damage-specific representations. This is achieved by leveraging both the local and global contexts, as well as self-attention mechanisms. The performance of M-SKSNet is evaluated both quantitatively and qualitatively, and the results show that M-SKSNet achieved the highest improvement in F1 by 3.77% and in IOU by 4.6%, when compared to existing models. Additionally, the effectiveness of M-SKSNet in accurately extracting damaged road markings from images in various complex scenarios (including city roads and highways) is demonstrated. Furthermore, M-SKSNet is found to outperform existing alternatives in terms of both robustness and accuracy.

To tackle the complexity and limited applicability of high-precision segmentation models for damaged road markings, this study proposes a Multi-level Adaptive Lightweight Network (MALNet) based on knowledge distillation. By incorporating multi-scale dilated convolution and adaptive spatial channel attention fusion modules, the MALNet model significantly enhances the precision, integrity, and robustness of its segmentation branch. Furthermore, it employs an intricate knowledge distillation strategy, channeling rich, layered insights from a teacher model to a student model, thus elevating the latter’s segmentation ability. Concurrently, it streamlines the student model by markedly reducing its parameter count and computational demands, culminating in a segmentation network that is both high-performing and pragmatic. Rigorous testing on three distinct data sets for damaged road marking detection—CDM_P (Collective Damaged road Marking—Public), CDM_H (Collective Damaged road Marking—Highways), and CDM_C (Collective Damaged road Marking—Cityroad)—underscores the MALNet model’s superior segmentation abilities across all damage types, outperforming competing models in accuracy and completeness. Notably, the MALNet model excels in parameter efficiency, computational economy, and throughput. After distillation, the student model’s parameters and computational load decrease to only 31.78% and 27.40% of the teacher model’s, respectively, while processing speeds increase to 1.9 times, demonstrating a significant improvement in lightweight design.

This paper presents an augmented reality-based method for geo-registering videos from low-cost multi-rotor Unmanned Aerial Vehicles (UAVs). The goal of the proposed method is to conduct an accurate geo-registration and target localization on a UAV video stream. The geo-registration of video stream requires accurate attitude data. However, the Inertial Measurement Unit (IMU) sensors on most low-cost UAVs are not capable of being directly used for geo-registering the video. The magnetic compasses on UAVs are more vulnerable to the interferences in the working environment than the accelerometers. Thus the camera yaw error is the main sources of the registration error. In this research, to enhance the low accuracy attitude data from the onboard IMU, an extended Kalman Filter (EKF) model is used to merge Real Time Kinematic Global Positioning System (RTK GPS) data with the IMU data. In the merge process, the high accuracy RTK GPS data can be used to promote the accuracy and stability of the 3-axis body attitude data. A method of target localization based on the geo-registration model is proposed to determine the coordinates of the ground targets in the video. The proposed method uses a modified extended Kalman Filter to combine the data from RTK GPS and the IMU to improve the accuracy of the geo-registration and the localization result of the ground targets. The localization results are compared to the reference point coordinates from satellite image. The comparison indicates that the proposed method can provide practical geo-registration and target localization results.

In emergency response and disaster rescue, unmanned aerial vehicles (UAVs) onboard thermal infrared (TIR) sensors are an essential means of acquiring ground information in the nighttime working environment. To enable field personnel to make better decisions based on TIR video streams returned from a UAV, the geographic information enhancement of TIR video streams is required. At present, it is difficult for low-cost UAVs to carry high-precision attitude sensors and thus obtain high-precision camera attitude information to meet the enhanced processing requirements of UAV TIR video streams. To this end, this paper proposes an improved Kalman filter algorithm to improve the geographic registration (geo-registration) accuracy by fusing the positioning and heading data from the dual-antenna real-time kinematic global positioning system (RTK-GPS) with onboard internal measurement unit (IMU) data. This method can yield high-precision position and attitude data in real time based on low-cost UAV hardware, based on which high-precision geo-registration results can be obtained. The computational complexity can be reduced compared with video stream feature tracking algorithms. Furthermore, the problem of unstable matching due to the low resolution and texture level of TIR video streams can be avoided. The experimental results prove that the proposed method can reduce the registration error by 66.15%, and significantly improve the geo-registration accuracy of UAV TIR video streams. Thus, it can strongly support the popularization and practicality of the application of augmented reality (AR) technology to low-cost UAV platforms.