Mismatching removal is at the core yet still a challenging problem in the photogrammetry and computer vision field. In this paper, we propose a coordinate embedding network (named CE-Net). We consider the mismatching problem as a graph node classification problem, and generate node descriptors by embedding point coordinates and aggregating geometric information from neighboring nodes based on self-attention and cross-attention mechanism. Finally, a binary classifier is used to separate node descriptors into two classes, namely matching inliers and outliers. Benefiting from the attention mechanism, firstly the node descriptors can get geometric information from “good neighbors” (i.e., matching inliers) and keep away from “bad neighbors” (i.e., matching outliers), improving the exactness of the descriptors; secondly the node descriptors can contain the information from both intra-graph and inter-graph, improving their distinctiveness. Experiments in testing datasets show that our proposed CE-Net achieves the state-of-the-art performance with a precision of 0.972, an outlier recall of 0.984, and an inlier recall of 0.963. Furthermore, CE-Net also outperforms the compared methods in real mismatching removal tasks in terms of positional accuracy, dispersion, and number of remaining point pairs, showing great potentials in practical applications. Our codes and data are available on https://github.com/csyhy1986 .

Image stitching based on a global alignment model is widely used in computer vision. However, the resulting stitched image may look blurry or ghosted due to parallax. To solve this problem, we propose a parallax-tolerant image stitching method based on nonrigid warping in this paper. Given a group of putative feature correspondences between overlapping images, we first use a semiparametric function fitting, which introduces a motion coherence constraint to remove outliers. Then, the input images are warped according to a nonrigid warp model based on Gaussian radial basis functions. The nonrigid warping is a kind of elastic deformation that is flexible and smooth enough to eliminate moderate parallax errors. This leads to high-precision alignment in the overlapped region. For the nonoverlapping region, we use a rigid similarity model to reduce distortion. Through effective transition, the nonrigid warping of the overlapped region and the rigid warping of the nonoverlapping region can be used jointly. Our method can obtain more accurate local alignment while maintaining the overall shape of the image. Experimental results on several challenging data sets for urban scene show that the proposed approach is better than state-of-the-art approaches in both qualitative and quantitative indicators.

Promoting the intensive utilization of arable land is a critical strategy for addressing the scarcity problem of arable land resources and thus ensuring food security. However, public emergencies pose significant challenges to the intensive utilization of arable land. Based on the pressure-state response (PSR) model and taking Sichuan Province, known as China’s “Heavenly Granary”, as an example, this study constructs a suitable evaluation system and analyzes the variation trend of the intensive utilization of arable land from the perspective of public emergencies. Key factors constraining the intensive utilization of arable land are further analyzed using the obstacle diagnostic model. The findings of this study are as follows: (1) Despite the shocks of public emergencies, the intensive utilization level of arable land in Sichuan Province in China shows an overall upward trend, indicating a high level of resilience and adaptability. (2) The pressure to utilize arable land intensively in Sichuan exhibits periodic fluctuations, yet the state remains generally stable. The whole system shows positive adaptive responses to external pressures and contemporary conditions during the mid-to-late stages of the research period. Nevertheless, coordination among subsystems within the PSR framework remains suboptimal, and a dynamic equilibrium across the subsystems has not yet been achieved. (3) Obstacle factors constraining the intensive arable land utilization in Sichuan exhibit notable temporal variations. Early-period constraints centered on multiple cropping indexes, grain yield per unit area, and irrigation index, reflecting limitations of traditional agricultural production modes. In the later stages, key obstacles shifted to factors including per capita cultivated land, population density, and pesticide/fertilizer input index, highlighting the impediment effects caused by evolving socio-demographic dynamics influenced by public emergencies. The findings of this study reveal critical pathways for local governments to achieve sustainable arable land management amidst global uncertainties.

Due to radiometric and geometric distortions between images, mismatches are inevitable. Thus, a mismatch removal process is required for improving matching accuracy. Although deep learning methods have been proved to outperform handcraft methods in specific scenarios, including image identification and point cloud classification, most learning methods are supervised and are susceptible to incorrect labeling, and labeling data is a time-consuming task. This paper takes advantage of deep reinforcement leaning (DRL) and proposes a framework named unsupervised learning for mismatch removal (ULMR). Resorting to DRL, ULMR firstly scores each state–action pair guided by the output of classification network; then, it calculates the policy gradient of the expected reward; finally, through maximizing the expected reward of state–action pairings, the optimal network can be obtained. Compared to supervised learning methods (e.g., NM-Net and LFGC), unsupervised learning methods (e.g., ULCM), and handcraft methods (e.g., RANSAC, GMS), ULMR can obtain higher precision, more remaining correct matches, and fewer remaining false matches in testing experiments. Moreover, ULMR shows greater stability, better accuracy, and higher quality in application experiments, demonstrating reduced sampling times and higher compatibility with other classification networks in ablation experiments, indicating its great potential for further use.

Due to the inherent limitations of matching algorithms and the complexities associated with image contents, mismatches are inevitable and can have detrimental effects on downstream tasks in computer vision and remote sensing. Researchers have published numerous reviews on mismatch removal, which may suffer from two primary deficiencies. Firstly, these reviews are often embedded within studies that primarily focus on image matching, thereby limiting the detailed and comprehensive analysis of mismatch removal methods. Secondly, reviews of deep learning (DL)-based methods, despite their numerous existence and interconnection, tend to be fragmentary and lack a systematic approach. To address these two shortcomings, this paper presents a comprehensive survey of DL-based mismatch removal principles and methods. We provide a summary of network architectures, techniques for extracting geometrical information, and various training modes. Specifically, we highlight the importance of permutation invariance in mining operations, enumerate a majority of existing mining methods, and provide an explanation of their permutation invariant properties. Furthermore, we present both the intuitive motivation and mathematical analysis of commonly used methods, elucidating their underlying principles and efficacy. In the conclusion, we predict upcoming trends based on the findings of our review, aiming to provide valuable insights into mismatch removal techniques and guide their practical applications.

Large radiometric and geometric distortion in multi-source images leads to fewer matching points with high matching blunder ratios, and global geometric relationship models between multi-sensor images are inexplicit. Thus, traditional matching blunder detection methods cannot work effectively. To address this problem, we propose two matching blunder detection methods based on graph theory. The proposed methods can build statistically significant clusters in the case of few matching points with high matching blunder ratios, and use local geometric similarity constraints to detect matching blunders when the global geometric relationship is not explicit. The first method (named the complete graph-based method) uses clusters constructed by matched triangles in complete graphs to encode the local geometric similarity of images, and it can detect matching blunders effectively without considering the global geometric relationship. The second method uses the triangular irregular network (TIN) graph to approximate a complete graph to reduce to computational complexity of the first method. We name this the TIN graph-based method. Experiments show that the two graph-based methods outperform the classical random sample consensus (RANSAC)-based method in recognition rate, false rate, number of remaining matching point pairs, dispersion, positional accuracy in simulated and real data (image pairs from Gaofen1, near infrared ray of Gaofen1, Gaofen2, panchromatic Landsat, Ziyuan3, Jilin1and unmanned aerial vehicle). Notably, in most cases, the mean false rates of RANSAC, the complete graph-based method and the TIN graph-based method in simulated data experiments are 0.50, 0.26 and 0.14, respectively. In addition, the mean positional accuracy (RMSE measured in units of pixels) of the three methods is 2.6, 1.4 and 1.5 in real data experiments, respectively. Furthermore, when matching blunder ratio is no higher than 50%, the computation time of the TIN graph-based method is nearly equal to that of the RANSAC-based method, and roughly 2 to 40 times less than that of the complete graph-based method.