Conventional remote sensing image retrieval (RSIR) system usually performs single-label retrieval where each image is annotated by a single label representing the most significant semantic content of the image. In this scenario, however, the scene complexity of remote sensing images is ignored, where an image might have multiple classes (i.e., multiple labels), resulting in poor retrieval performance. We therefore propose a novel multilabel RSIR approach based on fully convolutional network (FCN). Specifically, FCN is first trained to predict segmentation map of each image in the considered image archive. We then obtain multilabel vector and extract region convolutional features of each image based on its segmentation map. The extracted region features are finally used to perform region-based multilabel retrieval. The experimental results show that our approach achieves state-of-the-art performance in contrast to handcrafted and convolutional neural network features.

Remote sensing image retrieval (RSIR) aims to search and retrieve the images of interest from a large remote sensing (RS) image archive, which has remained to be a hot topic over the past decade. Benefited from the advent and progress of deep learning, RSIR has been promoted by developing novel approaches, constructing new datasets, and exploring potential applications. To the best of our knowledge, there lacks a comprehensive review of RSIR achievements, including systematic and hierarchical categorization of RSIR methods and benchmark datasets in the past decade. This article therefore provides a systematic survey of the recently published RSIR methods and benchmarks by reviewing more than 200 papers. To be specific, in terms of image source, label, and modality, we first group the RSIR methods into some hierarchical categories, each of which is reviewed in detail. Following the categorization of the RSIR methods, we list the benchmark datasets publically available for performance evaluation, and present our newly collected RSIR dataset. Moreover, some of the existing RSIR methods are selected and evaluated on the representative benchmark datasets. The results demonstrate that deep learning-based methods are currently the dominant RSIR approaches and outperform handcrafted feature-based methods by a significant margin. Finally, we discuss the main challenges of RSIR, and point out some potential directions for the future RSIR research.

Leveraging multi-view remote sensing images in scene classification tasks significantly enhances the precision of such classifications. This approach, however, poses challenges due to the simultaneous use of multi-view images, which often leads to a misalignment between the visual content and semantic labels, thus complicating the classification process. In addition, as the number of image viewpoints increases, the quality problem for remote sensing images further limits the effectiveness of multi-view image classification. Traditional scene classification methods predominantly employ SoftMax deep learning techniques, which lack the capability to assess the quality of remote sensing images or to provide explicit explanations for the network’s predictive outcomes. To address these issues, this paper introduces a novel end-to-end multi-view decision fusion network specifically designed for remote sensing scene classification. The network integrates information from multi-view remote sensing images under the guidance of image credibility and uncertainty, and when the multi-view image fusion process encounters conflicts, it greatly alleviates the conflicts and provides more reasonable and credible predictions for the multi-view scene classification results. Initially, multi-scale features are extracted from the multi-view images using convolutional neural networks (CNNs). Following this, an asymptotic adaptive feature fusion module (AAFFM) is constructed to gradually integrate these multi-scale features. An adaptive spatial fusion method is then applied to assign different spatial weights to the multi-scale feature maps, thereby significantly enhancing the model’s feature discrimination capability. Finally, an evidence decision fusion module (EDFM), utilizing evidence theory and the Dirichlet distribution, is developed. This module quantitatively assesses the uncertainty in the multi-perspective image classification process. Through the fusing of multi-perspective remote sensing image information in this module, a rational explanation for the prediction results is provided. The efficacy of the proposed method was validated through experiments conducted on the AiRound and CV-BrCT datasets. The results show that our method not only improves single-view scene classification results but also advances multi-view remote sensing scene classification results by accurately characterizing the scene and mitigating the conflicting nature of the fusion process.

Scene-level change detection (SLCD) can provide semantic change information at image level, thus is of great significance for monitoring land use changes. Supervised SLCD approaches tend to outperform unsupervised ones. However, the existing supervised methods are relying on post-classification, which often result in unsatisfactory performance due to classification error accumulation. We therefore formulate SLCD as a similarity learning task, and propose a scene similarity learning network (SSLN) for land use SLCD. To be specific, SSLN is a two-branch network with ResNet as the backbone for feature extraction, where the global feature difference and the multi-scale local feature fusion modules are considered in order to better mine the temporal correlation between bi-temporal scenes. Then, the trained SSLN is further exploited to obtain the similarity of scene pairs for determining similarity threshold via threshold traversal algorithm. Finally, the land use scenes are categorized into changed or unchanged by comparing scene similarity with the threshold. Experimental results on the publicly available MtS-WH dataset and our newly released land use scene change detection (LUSCD) dataset show that the proposed approach achieves better performance than comparison methods, indicating that our approach is a simple yet effective solution to land use SLCD.

Traditional urban scene-classification approaches focus on images taken either by satellite or in aerial view. Although single-view images are able to achieve satisfactory results for scene classification in most situations, the complementary information provided by other image views is needed to further improve performance. Therefore, we present a complementary information-learning model (CILM) to perform multi-view scene classification of aerial and ground-level images. Specifically, the proposed CILM takes aerial and ground-level image pairs as input to learn view-specific features for later fusion to integrate the complementary information. To train CILM, a unified loss consisting of cross entropy and contrastive losses is exploited to force the network to be more robust. Once CILM is trained, the features of each view are extracted via the two proposed feature-extraction scenarios and then fused to train the support vector machine classifier for classification. The experimental results on two publicly available benchmark data sets demonstrate that CILM achieves remarkable performance, indicating that it is an effective model for learning complementary information and thus improving urban scene classification.