Images captured from different viewpoints or devices have often exhibited significant geometric and photometric differences due to factors such as environmental variations, camera technology differences, and shooting conditions’ instability. To address this problem, homography estimation has attracted much attention as a method to describe the geometric projection relationship between images. Researchers have proposed numerous homography estimation methods for single-source and multimodal images in the past decades. However, the comprehensive review and analysis of homography estimation methods, from feature-based to deep learning-based, is still lacking. Therefore, we provide a comprehensive overview of research advances in homography estimation methods. First, we provide a detailed introduction to homography estimation’s core principles and matrix representations. Then, we review homography estimation methods for single-source and multimodal images, from feature-based to deep learning-based methods. Specifically, we analyze traditional and learning-based methods for feature-based homography estimation methods in detail. For deep learning-based homography estimation methods, we explore supervised, unsupervised, and other methods in-depth. Subsequently, we specifically review several metrics used to evaluate these methods. After that, we analyze the relevant applications of homography estimation and show the broad application prospects of this technique. Finally, we discuss current challenges and future research directions, providing a reference for computer vision researchers and engineers.

The Enhanced Flight Vision System (EFVS) plays a significant role in the Next-Generation low visibility aircraft landing technology, where the involvement of optical sensing systems increases the visual dimension for pilots. This paper focuses on deploying infrared and visible image fusion systems in civil flight, particularly generating integrated results to contend with registration deviation and adverse weather conditions. The existing enhancement methods push ahead with metrics-driven integration, while the dynamic distortion and the continuous visual scene are overlooked in the landing stage. Hence, the proposed visual enhancement scheme is divided into homography estimation and image fusion based on deep learning. A lightweight framework integrating hardware calibration and homography estimation is designed for image calibration before fusion and reduces the offset between image pairs. The transformer structure adopting the self-attention mechanism in distinguishing composite properties is incorporated into a concise autoencoder to construct the fusion strategy, and the improved weight allocation strategy enhances the feature combination. These things considered, a flight verification platform accessing the performances of different algorithms is built to capture image pairs in the landing stage. Experimental results confirm the equilibrium of the proposed scheme in perception-inspired and feature-based metrics compared to other approaches.

The homography estimation of infrared and visible images, a key technique for assisting perception, is an integral element within the 6G Space–Air–Ground Integrated Network (6G SAGIN) framework. It is widely applied in the registration of these two image types, leading to enhanced environmental perception and improved efficiency in perception computation. However, the traditional estimation methods are frequently challenged by insufficient feature points and the low similarity in features when dealing with these images, which results in poor performance. Deep-learning-based methods have attempted to address these issues by leveraging strong deep feature extraction capabilities but often overlook the importance of precisely guided feature matching in regression networks. Consequently, exactly acquiring feature correlations between multi-modal images remains a complex task. In this study, we propose a feature correlation transformer method, devised to offer explicit guidance for feature matching for the task of homography estimation between infrared and visible images. First, we propose a feature patch, which is used as a basic unit for correlation computation, thus effectively coping with modal differences in infrared and visible images. Additionally, we propose a novel cross-image attention mechanism to identify correlations between varied modal images, thus transforming the multi-source images homography estimation problem into a single-source images problem by achieving source-to-target image mapping in the feature dimension. Lastly, we propose a feature correlation loss (FCL) to induce the network into learning a distinctive target feature map, further enhancing source-to-target image mapping. To validate the effectiveness of the newly proposed components, we conducted extensive experiments to demonstrate the superiority of our method compared with existing methods in both quantitative and qualitative aspects.

Homography estimation for infrared and visible images is a critical and fundamental task in multimodal image processing. Recently, the coarse-to-fine strategy has been gradually applied to the homography estimation task and has proved to be effective. However, current coarse-to-fine homography estimation methods typically require the introduction of additional neural networks to acquire multi-scale feature maps and the design of complex homography matrix fusion strategies. In this paper, we propose a new unsupervised homography estimation method for infrared and visible images. First, we design a novel coarse-to-fine strategy. This strategy utilizes different stages in the regression network to obtain multi-scale feature maps, enabling the progressive refinement of the homography matrix. Second, we design a local correlation transformer (LCTrans), which aims to capture the intrinsic connections between local features more precisely, thus highlighting the features crucial for homography estimation. Finally, we design an average feature correlation loss (AFCL) to enhance the robustness of the model. Through extensive experiments, we validated the effectiveness of all the proposed components. Experimental results demonstrate that our method outperforms existing methods on synthetic benchmark datasets in both qualitative and quantitative comparisons.

In the current field of air traffic control speech, there is a lack of effective objective speech quality evaluation methods. This paper proposes a new network framework based on ResNet–BiLSTM to address this issue. Firstly, the mel-spectrogram of the speech signal is segmented using the sliding window technique. Next, a preceding feature extractor composed of convolutional and pooling layers is employed to extract shallow features from the mel-spectrogram segment. Then, ResNet is utilized to extract spatial features from the shallow features, while BiLSTM is used to extract temporal features, and these features are horizontally concatenated. Finally, based on the concatenated spatiotemporal features, the final speech quality score is computed using fully connected layers. We conduct experiments on the air traffic control speech database and compare the objective scoring results with the subjective scoring results. The experimental results demonstrate that the proposed method has a high correlation with the mean opinion score (MOS) of air traffic control speech.

Both single infrared and visible images have respective limitations. Fusion technology has been developed to conquer these restrictions. It is designed to generate a fused image with infrared information and texture details. Most traditional fusion methods use hand-designed fusion strategies, but some are too rough and have limited fusion performance. Recently, some researchers have proposed fusion methods based on deep learning, but some early fusion networks cannot adaptively fuse images due to unreasonable design. Therefore, we propose a mask and cross-dynamic fusion-based network called MCDFN. This network adaptively preserves the salient features of infrared images and the texture details of visible images through an end-to-end fusion process. Specifically, we designed a two-stage fusion network. In the first stage, we train the autoencoder network so that the encoder and decoder learn feature extraction and reconstruction capabilities. In the second stage, the autoencoder is fixed, and we employ a fusion strategy combining mask and cross-dynamic fusion to train the entire fusion network. This strategy is conducive to the adaptive fusion of image information between infrared images and visible images in multiple dimensions. On the public TNO dataset and the RoadScene dataset, we selected nine different fusion methods to compare with our proposed method. Experimental results show that our proposed fusion method achieves good results on both datasets.