Few-shot object detection (FSOD) has received numerous attention due to the difficulty and time-consuming of labeling objects. Recent researches achieve excellent performance in a natural scene by only using a few instances of novel classes to fine-tune the last prediction layer of the model well-trained on plentiful base data. However, compared with natural scene objects with a single direction and small size variety, the direction and size of the objects in remote sensing images (RSIs) vary greatly. The methods proposed for the natural scene cannot be directly applied to RSIs. In this article, we first propose a strong baseline for RSIs. It fine-tunes all detector components acting on high-level features and effectively improves the performance of novel classes. Further analyzing the results of the baseline, we find that the error for novel classes is mainly concentrated in classification. It misclassifies novel classes as confusable base classes or backgrounds due to the difficulty in extracting generalized information from limited instances. As is well-known, text-modal knowledge can highly summarize the generalized and unique characteristics of categories. Thus, we introduce text-modal descriptions for each category and propose an FSOD method guided by TExt-MOdal knowledge, called TEMO. Specifically, a text-modal knowledge extractor and a cross-modal assembly module are proposed to extract text features and fuse the text-modal features into visual-modal features. The fused features greatly reduce the classification confusion of novel classes. Furthermore, we introduce a mask strategy and a separation loss to avoid over-fitting and ambiguity of text-modal features. Experimental results on detection in optical remote sensing images (DIOR), Northwestern Polytechnical University (NWPU), and fine-grained object recognition in high-resolution remote sensing imagery (FAIR1M) illustrate that our TEMO achieves state-of-the-art performance in all settings.

Multi-object tracking (MOT) in satellite videos is an essential topic with many applications, such as traffic monitoring and disaster response. However, many multi-object trackers that perform well in natural scenes show weak generalization in satellite videos due to low object discrimination caused by low spatial resolution and the widespread indistinguishable background, such as clouds and reflections. In this paper, we design a novel multi-object tracking framework called CFTracker (Cross-Frame Tracker) for satellite videos from the point of both network structure and training method. On the one hand, in network structure design, a cross-frame feature update module (CFU) is proposed to enhance object recognition and reduce the response to background noises by using rich temporal semantic information. On the other hand, we reveal that the picture-pair training approach used by the mainstream MOT network is not entirely conducive to the network learning temporal semantic information. To better grasp the cross-frame feature connections and output time-consistent motion predictions, we train CFTracker by a novel cross-frame training flow (CT). Experiments demonstrate the effectiveness of our CFTracker and obtain state-of-the-art tracking accuracy and precision of 72.9% score on the AIR-MOT dataset and 57.1% score on the VISO dataset. The code will be available online.

Deep convolutional neural networks (CNNs) have brought a tremendous increase in detection accuracy, but too cumbersome model makes them hard to deploy on low computation edge devices, such as satellites and unmanned aerial vehicles. A promising method to tackle this problem is knowledge distillation (KD), which makes models lightweight with satisfactory accuracy. For remote sensing images, the objects are usually environment-related and located in a cluttered scene. The features that objects’ semantic information relies on are tangled. However, existing distillation methods only imitate feature distribution derived from regions, including objects resulting in poor performance. Furthermore, masses of instances generated by teachers are blindly inherited, even if some of them are outliers. In this article, we propose a general and effective KD framework called dynamic knowledge distillation (DKD). First, our framework leverages the dynamic global distillation (GD) module to discover valuable regions from the foreground and background for multiscale features imitation, avoiding ignoring the potential geographical spatial relationship. Second, we propose a dynamic instance selection distillation (ISD) module to give students the ability of self-judgment through the magnitude of detection loss. Third, toward more accurate handling of hard samples in regression, a training-status-aware loss is tailored to guide students mine knowledge about objects with large aspect ratio or small size. Extensive experiments are conducted to show the effectiveness of DKD framework. The detection results on DOTA and NWPU VHR-10 dataset illustrate that our method is suitable for single-stage, two-stage and even anchor-free detectors. It shows the state-of-the-art performance. The code will be publicly available.

Aircraft classification in synthetic aperture radar (SAR) images plays a considerable role in global region management and surveillance. Recently, deep learning has been applied to solve the classification problem and made significant progress. Due to the imaging variability at different angles and component scattering discreteness in SAR images, previous works have difficulty in achieving desirable classification results. To address these issues, we study the positional and semantic relationship between the scattering points and propose an innovative scattering topology network (ST-Net) in this article. First, considering the diversity of imaging results caused by different target attitude angles, we extract and transform the scattering cluster centers to update the information of various categories. It can guide the model to strengthen the discriminative features and mitigate the impact of imaging variability on classification performance. Second, a novel scattering topology module is introduced to model the spatial relationships and semantic information interaction of discrete scattering points. In this process, the topology relations and scattering characteristics are enhanced for further accurate classification. Third, the context attention excitation is designed to capture significant global and semantic information, which is conducive to suppressing the background interference and reducing category confusion. In conclusion, the ST-Net is presented with the SAR imaging mechanism and the topology geometric representation of aircrafts. We construct the SAR aircraft category dataset (SAR-ACD) and conduct extensive experiments on it to show the effectiveness of ST-Net, which illustrates that our method achieves superior classification performance.

Semantic segmentation of remote sensing images is vital in remote sensing technology. High-quality models for this task require a vast amount of images, manual annotation is a process that is time-consuming and labor-intensive. Consequently, this has catalyzed the emergence of semi-supervised semantic segmentation methods. However, the complexity of foreground categories in remote sensing images poses a challenge to maintaining prediction consistency. Moreover, inherent characteristics such as intra-class variations and inter-class similarities result in a certain degree of confusion among features of different classes in the feature space. This impacts the final classification results. In order to improve the model’s consistency and optimize the classification of categories based on features, this paper proposes a new semi-supervised semantic segmentation framework that combines consistency regularization and contrastive learning. In terms of consistency regularization, the proposed method incorporates dual teacher networks, introduces ClassMix for image augmentation, and utilizes confidence levels to integrate the predictions from these networks. By introducing perturbations at both the network and image levels, while simultaneously maintaining consistency, the predictive prowess and generalization ability of the model are enhanced. For contrastive learning, Postive-Unlabeled Learning (PU-Learning) is employed to improve the problem of mis-sampling when selecting features. At the same time, higher biased weights are allocated to more challenging negative samples, thereby elevating the complexity of feature learning and enhancing the discriminative capability of the final feature representation space. Our extensive experiments on the ISPRS Vaihingen dataset and the challenging iSAID dataset have served to underscore the superior performance of our approach.

In this article, we introduce the 2020 Gaofen Challenge and relevant scientific outcomes. The 2020 Gaofen Challenge is an international competition, which is organized by the China High-Resolution Earth Observation Conference Committee and the Aerospace Information Research Institute, Chinese Academy of Sciences and technically cosponsored by the IEEE Geoscience and Remote Sensing Society and the International Society for Photogrammetry and Remote Sensing. It aims at promoting the academic development of automated high-resolution earth observation image interpretation. Six independent tracks have been organized in this challenge, which cover the challenging problems in the field of object detection and semantic segmentation. With the development of convolutional neural networks, deep-learning-based methods have achieved good performance on image interpretation. In this article, we report the details and the best-performing methods presented so far in the scope of this challenge.