Matching multimodal remote sensing images (RSIs) remains an ongoing challenge due to the significant nonlinear radiometric differences and geometric distortions, resulting in matches exhibiting one-to-many matches or mismatches. To tackle this challenge, we propose a novel approach for multimodal RSI matching called modality-independent consistency matching (MICM), which leverages the capabilities of deep convolutional neural networks and the transformer attention mechanism to improve the matching performance. The proposed MICM method consists of three key steps. First, a Unet-like feature extraction backbone network is employed to learn multiscale invariant features from multimodal RSIs, enabling the extraction of rich and evenly distributed feature keypoints. Second, a hybrid approach combining local learning features with the transformer attention mechanism is introduced to aggregate learning features, facilitating both detailed capture and long-range modeling to enhance the representation ability of the features. Third, a feature consistency correlation strategy is adopted to maximize the number of correct corresponding feature points, ensuring reliable matching performance. The performance of the proposed method has been extensively evaluated on both the same scene and different scene multimodal RSIs, which are captured from various imaging modes, wavebands, and platforms. The results show the superior matching performance of the proposed MICM method compared to commonly used and state-of-the-art handcrafted- and learning-based methods when evaluated on both the same scene and different scene datasets. The proposed method serves as a valuable reference for addressing common challenges in multimodal RSI matching.

Multimodal remote sensing image high-precision matching faces significant challenges due to the large nonlinear distortions in radiation and geometry. To address this problem, we propose a novel multimodal remote sensing image matching method, M2DT-Net, which takes advantage of deep convolutional neural networks (CNNs) and the Delaunay triangulation strategy. Firstly, a fine-tuned CNN model is applied to learn invariant features with high robustness across multimodal remote sensing images. Second, an adaptive feature descriptor distance constraint combined with the DEGENSAC refinement is employed to filter the obtained matches, and only the matches with high credibility are retained. Last, a Delaunay triangulation strategy is adopted to expand more underlying correct matches that are dropped inliers in the previous stage. The experimental results on seven types of multimodal remote sensing image pairs indicate that M2DT-Net is superior to seven state-of-the-art image matching algorithms in terms of model performance and efficiency, achieving a balance of robustness and time consumption. The image registration results further demonstrate the effectiveness of the proposed algorithm. Therefore, the proposed M2DT-Net method provides a reference for resolving common multimodal image matching problems.

In the study of human social development and ecological environment monitoring, building change detection (BCD) is essential. Rapid and accurate BCD in complicated scenes and multiview high-resolution (HR) remote sensing images have garnered a lot of attention with the continual increase of image resolution and diverse satellite imaging modes. However, conventional BCD methods almost exclusively focus on the performance under specific, standard datasets and do not consider the robustness under off-nadir imaging and complex scene conditions. To address these issues, we propose an adaptative knowledge-driven multilevel attention BCD framework, called intelligent-BCD, in which we innovatively deploy four attention mechanisms. The compatibility design of the channel and spatial attention mechanism enables the model to fully mine deep features and meet BCD tasks in multiple imaging modes and complex scenes. Meanwhile, we propose a weight pool concept, a knowledge accumulation way of modeling attention adaptive. The model parameters obtained from the training of different data are stored to form the weight pool, and the weight for transfer application is reused in the new BCD task. Furthermore, we propose a new loss function, the dynamic domain loss function, that successfully addresses the problem of sample imbalance while motivating the model to pay greater attention to challenging sample learning. The benchmark experimental results for the four datasets, namely LEVIR-CD, LEVIR-CD+, WHU Building Dataset, and S2Looking, show that intelligent-BCD is superior to the competing methods in quantitative and qualitative evaluation. The transfer experiment in MtS-WH dataset demonstrates that intelligent-BCD has an excellent generalization and robustness.

The rapid development of Earth observation technology has promoted the continuous accumulation of images in the field of remote sensing. However, a large number of remote sensing images still lack manual annotations of objects, which makes the strongly supervised deep learning object detection method not widely used, as it lacks generalization ability for unseen object categories. Considering the above problems, this study proposes a few-shot remote sensing image object detection method that integrates context dependencies and global features. The method can be used to fine-tune the model with a small number of sample annotations based on the model trained in the base class, as a way to enhance the detection capability of new object classes. The method proposed in this study consists of three main modules, namely, the meta-feature extractor (ME), reweighting module (RM), and feature fusion module (FFM). These three modules are respectively used to enhance the context dependencies of the query set features, improve the global features of the support set that contains annotations, and finally fuse the query set features and support set features. The baseline of the meta-feature extractor of the entire framework is based on the optimized YOLOv5 framework. The reweighting module of the support set feature extraction is based on a simple convolutional neural network (CNN) framework, and the foreground feature enhancement of the support sets was made in the preprocessing stage. This study achieved beneficial results in the two benchmark datasets NWPU VHR-10 and DIOR. Compared with the comparison methods, the proposed method achieved the best performance in the object detection of the base class and the novel class.

Robustness of aerial-ground multi-source image matching is closely related to the quality of the ground reference image. To explore the influence of reference images on the performance of air-ground multi-source image matching, we focused on the impact of the control point projection accuracy and tie point accuracy on bundle adjustment results for generating digital orthophoto images by using the Structure from Motion algorithm and Monte Carlo analysis. Additionally, we developed a method to learn local deep features in natural environments based on fine-tuning the pre-trained ResNet50 model and used the method to match multi-scale, multi-seasonal, and multi-viewpoint air-ground multi-source images. The results show that the proposed method could yield a relatively even distribution of feature corresponding points under different conditions, seasons, viewpoints, illuminations. Compared with state-of-the-art hand-crafted computer vision and deep learning matching methods, the proposed method demonstrated more efficient and robust matching performance that could be applied to a variety of unmanned aerial vehicle self- and target-positioning applications in GPS-denied areas.