Successful remote sensing image registration is an important step for many remote sensing applications. The scale-invariant feature transform (SIFT) is a well-known method for remote sensing image registration, with many variants of SIFT proposed. However, it only uses local low-level information, and loses much middle- or high-level information to register. Image features extracted by a convolutional neural network (CNN) have achieved the state-of-the-art performance for image classification and retrieval problems, and can provide much middle- and high-level information for remote sensing image registration. Hence, in this letter, we investigate how to calculate the CNN feature, and study the way to fuse SIFT and CNN features for remote sensing image registration. The experimental results demonstrate that the proposed method yields a better registration performance in terms of both the aligning accuracy and the number of correct correspondences.

Compared to fully supervised 3D large-scale point cloud segmentation methods, which necessitate extensive manual point-wise annotations, weakly supervised segmentation has emerged as a popular approach for significantly reducing labeling costs while maintaining effectiveness. However, the existing methods have exhibited inferior segmentation performance and unsatisfactory generalization capabilities in some scenarios with unique structures (e.g., building facades). In this paper, we propose an effective and generalized weakly supervised semantic segmentation framework, called multi-stage scene-level constraints (MSC), to solve the above problem. To address the issue regarding inadequate labeled data, we use pseudo-labels for unlabeled data and propose an uncertainty-guided adaptive reweighting strategy to reduce the negative impact of erroneous pseudo-labeled data on the model learning process. To address the class imbalance issue, we employ multi-stage scene-level constraints (i.e., encoder, decoder, and classifier stages) to treat each class equally and improve perception ability of the model for each class. Evaluations conducted on multiple large-scale point cloud datasets collected in different scenarios, including building facades, indoor scenes, outdoor scenes, and UAV scenes, show that our MSC achieves a large gain over the existing weakly supervised methods and even surpasses some fully supervised methods.

The cross-dimensional matching of 2D images and 3D point clouds is an effective method by which to establish the spatial relationship between 2D and 3D space, which has potential applications in remote sensing and artificial intelligence (AI). In this paper, we propose a novel multi-task network, 2D3D-DescNet, to learn 2D and 3D local feature descriptors jointly and perform cross-dimensional matching of 2D image patches and 3D point cloud volumes. The 2D3D-DescNet contains two branches with which to learn 2D and 3D feature descriptors, respectively, and utilizes a shared decoder to generate the feature maps of 2D image patches and 3D point cloud volumes. Specifically, the generative adversarial network (GAN) strategy is embedded to distinguish the source of the generated feature maps, thereby facilitating the use of the learned 2D and 3D local feature descriptors for cross-dimensional retrieval. Meanwhile, a metric network is embedded to compute the similarity between the learned 2D and 3D local feature descriptors. Finally, we construct a 2D-3D consistent loss function to optimize the 2D3D-DescNet. In this paper, the cross-dimensional matching of 2D images and 3D point clouds is explored with the small object of the 3Dmatch dataset. Experimental results demonstrate that the 2D and 3D local feature descriptors jointly learned by 2D3D-DescNet are similar. In addition, in terms of 2D and 3D cross-dimensional retrieval and matching between 2D image patches and 3D point cloud volumes, the proposed 2D3D-DescNet significantly outperforms the current state-of-the-art approaches based on jointly learning 2D and 3D feature descriptors; the cross-dimensional retrieval at TOP1 on the 3DMatch dataset is improved by over 12%.

Point cloud semantic segmentation is a fundamental task for 3D scene understanding. However, acquiring dense point-wise annotations is labor-intensive and expensive. To alleviate the annotation burden, weakly supervised methods using extremely sparse labels (e.g., 0.1%) have attracted increasing attention. While existing methods have demonstrated promising performance, they struggle to generate sufficiently reliable pseudo-labels from unlabeled points, which limits further improvement in segmentation performance under extremely sparse supervision. To address this challenge, we propose a Multi-stage Pseudo-label Guided (MPG) training framework, which mines high-quality pseudo-labels through a two-stage training strategy. In the first stage, we introduce a Prototype-guided Pseudo-label Filter (PPF) to capture latent semantic information by comparing unlabeled features with class prototypes and removes unreliable pseudo-labels using a predefined threshold. In the second stage, we employ a teacher-student architecture and introduce a Pseudo-Label Fusion (PLF) strategy. PLF applies consistency and confidence filtering on two data-augmentation branches of the frozen teacher model to produce refined pseudo-labels, which are then combined with PPF outputs to train the student network. This design allows the student model to inherit distilled knowledge while exploring additional semantic information. Extensive experiments on indoor and outdoor building scenes demonstrate the effectiveness of MPG. Under extremely sparse supervision, MPG achieves 69.5% mIoU on S3DIS and 59.5% on XIAMEN-Buildings, achieving state-of-the-art performance among weakly supervised methods and performance comparable to some fully supervised methods.