The fusion of image data from multiple sensors is crucial for many applications. However, there are significant nonlinear intensity deformations between images from different kinds of sensors, leading to matching failure. To address this need, this paper proposes an effective coarse-to-fine matching method for multimodal remote sensing images (3MRS). In the coarse matching stage, feature points are first detected on a maximum moment map calculated with a phase congruency model. Then, feature description is conducted using an index map constructed by finding the index of the maximum value in all orientations of convolved images obtained using a set of log-Gabor filters. At last, several matches are built through image matching and outlier removal, which can be used to estimate a reliable affine transformation model between the images. In the stage of fine matching, we develop a novel template matching method based on the log-Gabor convolution image sequence and match the template features with a 3D phase correlation matching strategy, given that the initial correspondences are achieved with the estimated transformation. Results show that compared with SIFT, and three state-of-the-art methods designed for multimodal image matching, PSO-SIFT, HAPCG, and RIFT, only 3MRS successfully matched all six types of multimodal remote sensing image pairs: optical–optical, optical–infrared, optical–depth, optical–map, optical–SAR, and day–night, with each including ten different image pairs. On average, the number of correct matches (NCM) of 3MRS was 164.47, 123.91, 4.88, and 4.33 times that of SIFT, PSO-SIFT, HAPCG, and RIFT for the successfully matched image pairs of each method. In terms of accuracy, the root-mean-square error of correct matches for 3MRS, SIFT, PSO-SIFT, HAPCG, and RIFT are 1.47, 1.98, 1.79, 2.83, and 2.45 pixels, respectively, revealing that 3MRS got the highest accuracy. Even though the total running time of 3MRS was the longest, the efficiency for obtaining one correct match is the highest considering the most significant number of matches. The source code of 3MRS and the experimental datasets and detailed results are publicly available.

Accurate geopositioning of optical satellite imagery is a fundamental step for many photogrammetric applications. Considering the imaging principle and data processing manner, SAR satellites can achieve high geopositioning accuracy. Therefore, SAR data can be a reliable source for providing control information in the orientation of optical satellite images. This paper proposes a practical solution for an accurate orientation of optical satellite images using SAR reference images to take advantage of the merits of SAR data. Firstly, we propose an accurate and robust multimodal image matching method to match the SAR and optical satellite images. This approach includes the development of a new structural-based multimodal applicable feature descriptor that employs angle-weighted oriented gradients (AWOGs) and the utilization of a three-dimensional phase correlation similarity measure. Secondly, we put forward a general optical satellite imagery orientation framework based on multiple SAR reference images, which uses the matches of the SAR and optical satellite images as virtual control points. A large number of experiments not only demonstrate the superiority of the proposed matching method compared to the state-of-the-art methods but also prove the effectiveness of the proposed orientation framework. In particular, the matching performance is improved by about 17% compared with the latest multimodal image matching method, namely, CFOG, and the geopositioning accuracy of optical satellite images is improved, from more than 200 to around 8 m.

The accurate matching of multisource, multi-temporal remote sensing images is challenging because of significant nonlinear intensity differences (NIDs) and severe geometric distortions. To address these problems, we developed a robust image matching method: oriented filter-based matching (OFM). OFM is insensitive to NIDs, while exhibiting scale and rotational invariance. First, salient feature points with multiscale attributes were detected in the Gaussian-scale space of the input images. Then, the images were convoluted using multi-oriented filters, and unified feature maps were constructed by the extraction of orientation indices using effective data pooling operations. The constructed feature maps were highly resistant to NIDs. Five filters were integrated into the OFM framework to investigate their applicabilities in different application scenarios. Next, a novel rotation-invariant feature descriptor was constructed, using a dominant direction determination approach and a descriptor-grouping strategy. The dominant direction determination approach enables accurate dominant direction estimation, whereas the descriptor-grouping strategy improves the stability of the method under different rotational angles. Finally, brute-force matching was implemented to obtain initial matches; an improved mismatch elimination method was used to identify reliable putative matches. To evaluate the performance of OFM, we created a large dataset comprising 4,427 pairs of multitemporal optical–optical, optical–synthetic aperture radar (SAR), optical–infrared, and optical–depth images. OFM outperformed state-of-the-art methods in terms of number of correct matches, recall, inlier ratio, root mean square error and success rate. Our implement is publicly available ¹ .

Multimodal image matching success rates are often low due to non-linear radiation differences. Furthermore, when geometric transformations such as rotation and resolution exist between images, the matching success rate between multimodal images decreases even further. (It is worth noting that experiments have shown the impact of scale to be relatively small; therefore, this discussion focuses only on the influence of resolution differences on multimodal image matching.) To tackle these challenges, we’ve enhanced the feature point extraction, description, and association processes in image matching. This has resulted in a robust multimodal image matching framework that is invariant to rotation and resolution, with a high success rate. Specifically, inspired by the concept of image pyramids, we designed a strategy for extracting feature points in multiple resolution dimensions. This enables assigning resolution dimension information to feature points and expanding the set of points to be matched. Building upon feature point extraction, we enhanced the Log-Gabor filter and designed a novel feature descriptor. This descriptor can work robustly in scenarios with modal differences and rotational variances ranging from 0 °to 360 °. Applying this descriptor to the matching framework helps to eliminate the influence of angle differences on feature point associations between images. Furthermore, to further improve the matching success rate, we adopted a resolution dimension traversal retrieval strategy for the association of feature points. Based on this strategy, the number of correct matching points can be increased under the condition of the same feature points, thereby increasing the inlier rate of the matching results and enhancing the success rate of the matching results. To evaluate the performance of this matching framework, we created a test dataset containing 42, 496 pairs of images using publicly available datasets. These images cover six categories including optical, SAR, DEM, infrared, map, and nighttime light, with three types of transformations between images: translation, rotation, and scaling. We conducted comparative experiments using the MIRRIFT method against five advanced multimodal feature matching methods with publicly available source code, namely RIFT, LNIFT, HAPCG, HOWP, and LGHD. The results demonstrate that the MIRRIFT method proposed in this paper exhibits robustness to rotational, resolution, and modal differences in images. Specifically, it achieved an average success rate improvement of 59%, an increase in average correct matching points by 12%, and an average matching accuracy of 1.97. The executable program and sample data will be made available at https://github.com/Geng-Zemin/MIRRIFT.

Feature matching underpins critical satellite photogrammetric workflows, yet high spatial resolution (HSR) optical satellite imagery (0.3–2.6m GSD) presents persistent challenges through complex radiometric-geometric variations across diverse acquisition conditions. Current research suffers from two fundamental gaps: the absence of authentic benchmarks capturing real-world satellite imaging complexities, and limited systematic evaluation of modern algorithms under operational constraints. This study introduces the High Spatial Resolution Optical Satellite Stereo Dataset (HSROSS), comprising imagery from five operational satellites under six challenging conditions including seasonal variations, wide baselines, radiometric inconsistencies, multi-scale discrepancies, spectral differences, and cross-sensor heterogeneity, establishing comprehensive real-world validation benchmark. We systematically evaluate 15 advanced algorithms across classical, detector-based, and detector-free paradigms using six comprehensive quantitative metrics, revealing critical performance-efficiency trade-offs. Our analysis yields a practical algorithm selection framework aligned with mission priorities and operational constraints, identifying optimal solutions for robustness-critical applications, geometric precision tasks, dense reconstruction workflows, and resource-constrained onboard processing. This work ultimately provides essential insights for algorithm selection and system design of robust feature matching capabilities in Earth observation applications, while bridging the gap between theoretical advances and practical deployment requirements.