Matching multimodal remote sensing images (RSIs) remains an ongoing challenge due to the significant nonlinear radiometric differences and geometric distortions, resulting in matches exhibiting one-to-many matches or mismatches. To tackle this challenge, we propose a novel approach for multimodal RSI matching called modality-independent consistency matching (MICM), which leverages the capabilities of deep convolutional neural networks and the transformer attention mechanism to improve the matching performance. The proposed MICM method consists of three key steps. First, a Unet-like feature extraction backbone network is employed to learn multiscale invariant features from multimodal RSIs, enabling the extraction of rich and evenly distributed feature keypoints. Second, a hybrid approach combining local learning features with the transformer attention mechanism is introduced to aggregate learning features, facilitating both detailed capture and long-range modeling to enhance the representation ability of the features. Third, a feature consistency correlation strategy is adopted to maximize the number of correct corresponding feature points, ensuring reliable matching performance. The performance of the proposed method has been extensively evaluated on both the same scene and different scene multimodal RSIs, which are captured from various imaging modes, wavebands, and platforms. The results show the superior matching performance of the proposed MICM method compared to commonly used and state-of-the-art handcrafted- and learning-based methods when evaluated on both the same scene and different scene datasets. The proposed method serves as a valuable reference for addressing common challenges in multimodal RSI matching.

Multimodal remote sensing image high-precision matching faces significant challenges due to the large nonlinear distortions in radiation and geometry. To address this problem, we propose a novel multimodal remote sensing image matching method, M2DT-Net, which takes advantage of deep convolutional neural networks (CNNs) and the Delaunay triangulation strategy. Firstly, a fine-tuned CNN model is applied to learn invariant features with high robustness across multimodal remote sensing images. Second, an adaptive feature descriptor distance constraint combined with the DEGENSAC refinement is employed to filter the obtained matches, and only the matches with high credibility are retained. Last, a Delaunay triangulation strategy is adopted to expand more underlying correct matches that are dropped inliers in the previous stage. The experimental results on seven types of multimodal remote sensing image pairs indicate that M2DT-Net is superior to seven state-of-the-art image matching algorithms in terms of model performance and efficiency, achieving a balance of robustness and time consumption. The image registration results further demonstrate the effectiveness of the proposed algorithm. Therefore, the proposed M2DT-Net method provides a reference for resolving common multimodal image matching problems.

In the study of human social development and ecological environment monitoring, building change detection (BCD) is essential. Rapid and accurate BCD in complicated scenes and multiview high-resolution (HR) remote sensing images have garnered a lot of attention with the continual increase of image resolution and diverse satellite imaging modes. However, conventional BCD methods almost exclusively focus on the performance under specific, standard datasets and do not consider the robustness under off-nadir imaging and complex scene conditions. To address these issues, we propose an adaptative knowledge-driven multilevel attention BCD framework, called intelligent-BCD, in which we innovatively deploy four attention mechanisms. The compatibility design of the channel and spatial attention mechanism enables the model to fully mine deep features and meet BCD tasks in multiple imaging modes and complex scenes. Meanwhile, we propose a weight pool concept, a knowledge accumulation way of modeling attention adaptive. The model parameters obtained from the training of different data are stored to form the weight pool, and the weight for transfer application is reused in the new BCD task. Furthermore, we propose a new loss function, the dynamic domain loss function, that successfully addresses the problem of sample imbalance while motivating the model to pay greater attention to challenging sample learning. The benchmark experimental results for the four datasets, namely LEVIR-CD, LEVIR-CD+, WHU Building Dataset, and S2Looking, show that intelligent-BCD is superior to the competing methods in quantitative and qualitative evaluation. The transfer experiment in MtS-WH dataset demonstrates that intelligent-BCD has an excellent generalization and robustness.

The rapid development of Earth observation technology has promoted the continuous accumulation of images in the field of remote sensing. However, a large number of remote sensing images still lack manual annotations of objects, which makes the strongly supervised deep learning object detection method not widely used, as it lacks generalization ability for unseen object categories. Considering the above problems, this study proposes a few-shot remote sensing image object detection method that integrates context dependencies and global features. The method can be used to fine-tune the model with a small number of sample annotations based on the model trained in the base class, as a way to enhance the detection capability of new object classes. The method proposed in this study consists of three main modules, namely, the meta-feature extractor (ME), reweighting module (RM), and feature fusion module (FFM). These three modules are respectively used to enhance the context dependencies of the query set features, improve the global features of the support set that contains annotations, and finally fuse the query set features and support set features. The baseline of the meta-feature extractor of the entire framework is based on the optimized YOLOv5 framework. The reweighting module of the support set feature extraction is based on a simple convolutional neural network (CNN) framework, and the foreground feature enhancement of the support sets was made in the preprocessing stage. This study achieved beneficial results in the two benchmark datasets NWPU VHR-10 and DIOR. Compared with the comparison methods, the proposed method achieved the best performance in the object detection of the base class and the novel class.

The rapid and accurate discovery of damage information of the affected buildings is of great significance for postdisaster emergency rescue. In some related studies, the models involved can detect damaged buildings relatively accurately, but their time cost is high. Models that can guarantee both detection accuracy and high efficiency are urgently needed. In this paper, we propose a new transfer-learning deep attention network (TDA-Net). It can achieve a balance of accuracy and efficiency. The benchmarking network for TDA-Net uses a pair of deep residual networks and is pretrained on a large-scale dataset of disaster-damaged buildings. The pretrained deep residual networks have strong sensing properties on the damage information, which ensures the effectiveness of the network in prefeature grasping. In order to make the network have a more robust perception of changing features, a set of deep attention bidirectional encoding and decoding modules is connected after the TDA-Net benchmark network. When performing a new task, only a small number of samples are needed to train the network, and the damage information of buildings in the whole area can be extracted. The bidirectional encoding and decoding structure of the network allows two images to be input into the model independently, which can effectively capture the features of a single image, thereby improving the detection accuracy. Our experiments on the xView2 dataset and three datasets of disaster regions achieve high detection accuracy, which demonstrates the feasibility of our method.