Remote sensing image scene classification, which aims at labeling remote sensing images with a set of semantic categories based on their contents, has broad applications in a range of fields. Propelled by the powerful feature learning capabilities of deep neural networks, remote sensing image scene classification driven by deep learning has drawn remarkable attention and achieved significant breakthroughs. However, to the best of our knowledge, a comprehensive review of recent achievements regarding deep learning for scene classification of remote sensing images is still lacking. Considering the rapid evolution of this field, this paper provides a systematic survey of deep learning methods for remote sensing image scene classification by covering more than 160 papers. To be specific, we discuss the main challenges of remote sensing image scene classification and survey (1) Autoencoder-based remote sensing image scene classification methods, (2) Convolutional Neural Network-based remote sensing image scene classification methods, and (3) Generative Adversarial Network-based remote sensing image scene classification methods. In addition, we introduce the benchmarks used for remote sensing image scene classification and summarize the performance of more than two dozen of representative algorithms on three commonly-used benchmark data sets. Finally, we discuss the promising opportunities for further research.

Object detection is a fundamental yet challenging task in computer vision. Despite the great strides made over recent years, modern detectors may still produce unsatisfactory performance due to certain factors, such as non-universal object features and single regression manner. In this paper, we draw on the idea of mutual-assistance (MA) learning and accordingly propose a robust one-stage detector, referred as MADet, to address these weaknesses. First, the spirit of MA is manifested in the head design of the detector. Decoupled classification and regression features are reintegrated to provide shared offsets, avoiding inconsistency between feature-prediction pairs induced by zero or erroneous offsets. Second, the spirit of MA is captured in the optimization paradigm of the detector. Both anchor-based and anchor-free regression fashions are utilized jointly to boost the capability to retrieve objects with various characteristics, especially for large aspect ratios, occlusion from similar-sized objects, etc. Furthermore, we meticulously devise a quality assessment mechanism to facilitate adaptive sample selection and loss term reweighting. Extensive experiments on standard benchmarks verify the effectiveness of our approach. On MS-COCO, MADet achieves 42.5% AP with vanilla ResNet50 backbone, dramatically surpassing multiple strong baselines and setting a new state of the art.

Oriented object detection in aerial images has made significant advancements propelled by well-developed detection frameworks and diverse representation approaches to oriented bounding boxes. However, within modern oriented object detectors, the insufficient consideration given to certain factors, like contextual priors in aerial images and the sensitivity of the angle regression, hinder further improvement of detection performance. In this article, we propose a dual-focused detector (DFDet), which simultaneously focuses on the exploration of contextual knowledge and the mitigation of angle sensitivity. Specifically, DFDet contains two novel designs: a contextual dependence mining network (CDMN) and a penalty-incentive allocation strategy (PIAS). CDMN constructs multiple features containing contexts across various ranges with low computational burden and aggregates them into a compact yet informative representation that empowers the model for robust inference. PIAS dynamically calibrates the angle regression loss with a scalable penalty term determined by the angle regression sensitivity, incentivizing model to boost regression capacity for large aspect ratio objects challenging to be localized accurately. Extensive experiments on four widely-used benchmarks demonstrate the effectiveness of our approach, and new state-of-the-arts for one-stage object detection in aerial images are established. Without bells and whistles, DFDet with ResNet50 achieves 74.71% mean of average precision (mAP) running at 23.4 frames per second (FPS) on the most widely-used DOTA-v1.0 dataset. The source code is available at https://github.com/DDGRCF/DFDet .

Fine-grained object detection (FGOD) in remote sensing images is an emerging and challenging task in the field of image intelligent interpretation. It aims to localize objects while classifying them into different fine-grained categories. Modern FGOD methods are mainly derived from well-developed detectors and have made compelling progress. Despite this, these methods struggle to perform well in classifying objects at the subordinate level due to the limitations of their representation manners. In this paper, we propose a network capable of learning discriminative representation (DR) for fine-grained object detection in remote sensing images, named DRNet. First, a fine-grained branch that works in parallel with other task branches is introduced, where objects’ features are re-encoded with dual refinement to generate discriminative representation, enabling accurate fine-grained classification. Second, we design a confusion-minimized loss that automatically scales loss contributions according to the separability of samples to train the fine-grained branch, further boosting discriminative ability of the representation and better addressing hard-to-distinguish objects. Moreover, we devise an interaction verification strategy that empowers the network to fully utilize the results of fine-grained classification and coarse classification for achieving robust inference. On large-scale FAIR1M-1.0 and FAIR1M-2.0 datasets, our DRNet with ResNet50 and 1× training schedule obtains 40.87% mAP and 47.04% mAP, respectively, establishing new state-of-the-arts for fine-grained object detection in remote sensing images. The source code is available at https://github.com//54wb//DRNet.

Object detection in remote sensing images has garnered significant attention due to its wide applications in real-world scenarios. However, most existing oriented object detectors still suffer from complex backgrounds and varying angles , limiting their performance to further improvement. In this paper, we propose a novel oriented detector with Hierarchical mask prompting and Robust integrated regression, termed HRDet. Specifically, to cope with the first issue, we construct a hierarchical mask prompting module consisting of a semantic mask prediction branch and hierarchical Softmax technique. The former aims to isolate object instances from cluttered interferences guided by coarse box-wise masks, while the latter propagates differentiated features for adjacent layers using hierarchical attentive weights. To deal with the second issue, we strive for robust integrated regression and formulate an efficient oriented IoU loss, explicitly measuring the discrepancies of three geometric factors in oriented regression, i.e ., the central point distance, side length, and angle. This innovative loss intends to overcome the problem that existing IoU-based losses are invariant during the regression of varying angles. We applied these two strategies to a simple one-stage detection pipeline, achieving a new level of trade-off between speed and accuracy. Extensive experiments on four large aerial imagery datasets, DOTA-v1.0, DOTA-v2.0, DIOR-R, and HRSC2016, demonstrate that our HRDet significantly improves the accuracy of the one-stage detector over refine-stage counter-parts while maintaining the efficiency advantage. The source code will be available at https://github.com/yanqingyao1994/HRDet.

Artificial intelligence solutions, especially those based on deep learning, have swept through the realm of remote sensing image understanding over the past decade. Despite remarkable research advancements, there remains a gap between state-of-the-art techniques and application requirements. Neural models trained under conventional paradigms typically demonstrate limited capabilities in rapid generalization to new tasks, accurate identification of unseen categories, effective engagement in continuous learning, and adequate adaptation to distributional differences. Therefore, developing generic and scalable imagery interpretation approaches tailored for open-world scenarios is crucial. This article provides the first systematic and comprehensive review of key visual tasks within the remote sensing domain, encompassing few/zero-shot learning, open-set recognition, incremental learning, and domain adaptation/generalization. The anticipated objectives, evaluation protocols, and representative works for each task are thoroughly discussed, particularly highlighting their unique contributions to open-world image understanding. The review spans nearly 300 publications, with a strong focus on those from the past three years (2022–2024). Furthermore, this article conducts an in-depth analysis of the challenges facing the field, both at the level of individual visual tasks and within the broader context of open-world topics. Potential coping strategies and promising research directions, such as foundation models, multimodality, and domain knowledge, are also outlined to shed new insights on future endeavors in the geoscience and remote sensing community.