Feature representation is a crucial issue in multimodal image registration. The handcrafted features extracted by traditional methods are highly sensitive to nonlinear radiation differences, while supervised learning methods are limited by deficient labeled samples in the remote sensing field. Therefore, this paper proposes a consistency feature representation learning method for multimodal image registration, which involves mapping data into a common feature space to realize the accurate alignment of remote sensing images. First, a contrastive network with a spatial attention mechanism is driven to enhance the capability to highlight high-level features of images. Second, a positive sample augmentation strategy is implemented with contrastive loss, which helps the model learn the inherent features better, and imposes constraints on the sample similarity to optimize the feature projection. Finally, a multimodal image registration framework is proposed to enhance the stability of feature matching. The proposed framework achieves accurate feature extraction and consistency feature description for multimodal images, ensuring robustness against nonlinear radiometric differences. Experimental results demonstrate that the proposed method obtains more reliable registration results on the SEN1-2 dataset. Furthermore, the proposed algorithm achieves superior performance on data from other modalities, indicating strong generalization ability.
Hyperspectral image classification is essential for satellite Internet of Things (IoT) to build a large scale land-cover surveillance system. After acquiring real-time land-cover information, the edge of the network transmits all the hyperspectral images by satellites with low-latency and high-efficiency to the cloud computing center, which are provided by satellite IoT. A gigantic amount of remote sensing data bring challenges to the storage and processing capacity of traditional satellite systems. When hyperspectral images are used in annotation of land-cover application, data dimension reduction for classifier efficiency often leads to the decrease of classifier accuracy, especially the region to be annotated consists of natural landform and artificial structure. This paper proposes encoding spectral-spatial features for hyperspectral image classification in the satellite Internet of Things system to extract features effectively, namely attribute profile stacked autoencoder (AP-SAE). Firstly, extended morphology attribute profiles EMAP is used to obtain spatial features of different attribute scales. Secondly, AP-SAE is used to extract spectral features with similar spatial attributes. In this stage the program can learn feature mappings, on which the pixels from the same land-cover class are mapped as closely as possible and the pixels from different land-cover categories are separated by a large margin. Finally, the program trains an effective classifier by using the network of the AP-SAE. Experimental results on three widely-used hyperspectral image (HSI) datasets and comprehensive comparisons with existing methods demonstrate that our proposed method can be used effectively in hyperspectral image classification.
Multi-source image registration has often suffered from great radiation and geometric differences. Specifically, grayscale and texture from similar landforms in different source images often show significantly different visual features, and these differences disturb the corresponding point extraction in the following image registration process. Considering that edges between heterogeneous images can provide homogeneous information and more consistent features can be extracted based on image edges, an edge consistency radiation-change insensitive feature transform (EC-RIFT) method is proposed in this paper. Firstly, the noise and texture interference are reduced by preprocessing according to the image characteristics. Secondly, image edges are extracted based on phase congruency, and an orthogonal Log-Gabor filter is performed to replace the global algorithm. Finally, the descriptors are built with logarithmic partition of the feature point neighborhood, which improves the robustness of the descriptors. Comparative experiments on datasets containing multi-source remote sensing image pairs show that the proposed EC-RIFT method outperforms other registration methods in terms of precision and effectiveness.
30 January 2025 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Multi-source remote sensing image matching can provide more accurate data support for a variety of remote sensing tasks, but the disparities in imaging characteristics among different sensors bring significant challenges for effective image matching. In this paper, we propose an image matching method based on the contrastive network with similarity statistics weighting for multi-source remote sensing registration. First, we proposed a sampling strategy that selects hard negatives based on similarity statistics measures. These hard samples with more valuable information can promote the abundance of multimodal consistency information. Subsequently, the samples are encoded with a self-attention contrastive network for semantic feature embedding, enabling the learning of effective representations in the latent space. In addition, a similarity-weighted loss function is designed to guide the training process, aiming to improve the discrimination ability of the network. Experimental results demonstrate that our proposed method can derive accurate and robust results in multi-source remote sensing matching, exhibiting superior matching performance with respect to other methods.
21 May 2024 in IEEE Geoscience and Remote Sensing Letters
Common feature representation in optical and synthetic aperture radar (SAR) image registration is one of the most challenging tasks due to the significant geometric and radiometric differences. This letter proposed a Prominent structure-guided feature (PSGF) representation for SAR and optical image registration. Firstly, the prominent structure of the image is highlighted based on windowed inherent variations, which is conducive to identifying more accurate and reliable corresponding points. Secondly, the maximum response index filter banks are proposed to extract structure features with multi-orientation convolution results. Then the structure feature-guided representation generated from this filtering map is quantized in histograms. Finally, the descriptor with radiation invariance is employed for feature matching, enabling automatic image registration with high accuracy. Comparative analysis with state-of-the-art methods on diverse terrain data demonstrates the superiority of the proposed PSGF method for SAR and optical image registration.
Multi-source image registration often suffered from great radiation and geometric differences. Specifically, gray scale and texture from similar landforms in different source images often show significantly different visual features. And these differences disturb the corresponding point extraction in the following image registration process. Considering that edges between heterogeneous images can provide homogeneous information and more consistent features can be extracted based on image edges, an edge consistency radiation-change insensitive feature transform (EC-RIFT) method is proposed in this paper. Firstly, the noise and texture interference are reduced by preprocessing according to the image characteristics. Secondly, image edges are extracted based on phase congruency, and an orthogonal Log-Gabor filter is performed to replace the global algorithm. Finally, the descriptors are built with logarithmic partition of the feature point neighborhood, which improves the robustness of descriptors. Comparative experiments on datasets containing multi-source remote sensing image pairs show that the proposed EC-RIFT method outperforms other registration methods in terms of precision and effectiveness.