Semantic segmentation of remote sensing images is vital in remote sensing technology. High-quality models for this task require a vast amount of images, manual annotation is a process that is time-consuming and labor-intensive. Consequently, this has catalyzed the emergence of semi-supervised semantic segmentation methods. However, the complexity of foreground categories in remote sensing images poses a challenge to maintaining prediction consistency. Moreover, inherent characteristics such as intra-class variations and inter-class similarities result in a certain degree of confusion among features of different classes in the feature space. This impacts the final classification results. In order to improve the model’s consistency and optimize the classification of categories based on features, this paper proposes a new semi-supervised semantic segmentation framework that combines consistency regularization and contrastive learning. In terms of consistency regularization, the proposed method incorporates dual teacher networks, introduces ClassMix for image augmentation, and utilizes confidence levels to integrate the predictions from these networks. By introducing perturbations at both the network and image levels, while simultaneously maintaining consistency, the predictive prowess and generalization ability of the model are enhanced. For contrastive learning, Postive-Unlabeled Learning (PU-Learning) is employed to improve the problem of mis-sampling when selecting features. At the same time, higher biased weights are allocated to more challenging negative samples, thereby elevating the complexity of feature learning and enhancing the discriminative capability of the final feature representation space. Our extensive experiments on the ISPRS Vaihingen dataset and the challenging iSAID dataset have served to underscore the superior performance of our approach.

Abstract—Remote sensing images semantic segmentation is a fundamental yet challenging task, which has long relied heavily on sufficient pixel-wise annotations. Semi-supervised learning is proposed to address the problem of high dependence on labeled data by exploiting more learnable samples generated from the large amounts of accessible unlabeled data. However, affected by the complexity and diversity of remote sensing images, various misclassifications often occur and lead to errors accumulation during model training. Errors accumulation will destroy the consistency of model training and lead to degradation of final segmentation performance. In this paper, in order to further alleviate the damage caused by the errors to the consistency of model training and improve final segmentation accuracy, we propose a novel semi-supervised segmentation framework, paradigms integration and contrastive selection (PICS). Firstly, multiple proven semi-supervised paradigms are integrated to generate pseudo-labeled samples with less noise. Secondly, a loss-based contrastive selection method is explored to distinguish generated samples that contain different degrees of inevitable misclassification, thereby further maintaining the approximation of the generated samples and the ground truth in the sample space. By generating and selecting high-quality pseudo-labeled samples for selective self-training, we can better guarantee the consistency during model training and obtain better segmentation results. Extensive experiments over the ISPRS Vaihingen, Potsdam and the challenging iSAID benchmarks demonstrate that our method yields significant accuracy boosting on the segmentation results and achieves on-par performance with the state-of-the-arts.

In this paper, we briefly summarize the first challenge on moving object detection and tracking in satellite videos (SatVideoDT). This challenge has three tracks related to satellite video analysis, including moving object detection (Track 1), single object tracking (Track 2), and multiple-object tracking (Track 3). 123, 89, and 70 participants successfully registered, while 37, 42, and 29 teams submitted their final results on the test datasets for Tracks 1-3, respectively. The top-performing methods and their results in each track are described with details. This challenge establishes a new benchmark for satellite video analysis.

Height estimation has long been a pivotal topic within measurement and remote sensing disciplines, with monocular height estimation offering wide-ranging data sources and convenient deployment. This paper addresses the existing challenges in monocular height estimation methods, namely the difficulty in simultaneously achieving high-quality instance-level height and edge reconstruction, along with high computational complexity. This paper presents a comprehensive solution for monocular height estimation in remote sensing, termed HeightFormer, combining multilevel interactions and image-adaptive classification–regression. It features the Multilevel Interaction Backbone (MIB) and Image-adaptive Classification–regression Height Generator (ICG). MIB supplements the fixed sample grid in the CNN of the conventional backbone network with tokens of different interaction ranges. It is complemented by a pixel-, patch-, and feature map-level hierarchical interaction mechanism, designed to relay spatial geometry information across different scales and introducing a global receptive field to enhance the quality of instance-level height estimation. The ICG dynamically generates height partition for each image and reframes the traditional regression task, using a refinement from coarse to fine classification–regression that significantly mitigates the innate ill-posedness issue and drastically improves edge sharpness. Finally, the study conducts experimental validations on the Vaihingen and Potsdam datasets, with results demonstrating that our proposed method surpasses existing techniques.

In the field of remote sensing technology, the semantic segmentation of remote sensing images carries substantial importance. The creation of high-quality models for this task calls for an extensive collection of image data. However, the manual annotation of these images can be both time-consuming and labor-intensive. This has catalyzed the advent of semi-supervised semantic segmentation methodologies. Yet, the complexities inherent within the foreground categories of these remote sensing images present challenges in preserving prediction consistency. Moreover, remote sensing images possess more complex features, and different categories are confused within the feature space, making optimization based on the feature space challenging. To enhance model consistency and to optimize feature-based class categorization, this paper introduces a novel semi-supervised semantic segmentation framework based on Mean Teacher (MT). Unlike the conventional Mean Teacher that only introduces perturbations at the image level, we incorporate perturbations at the feature level. Simultaneously, to maintain consistency after feature perturbation, we employ contrastive learning for feature-level learning. In response to the complex feature space of remote sensing images, we utilize entropy threshold to assist contrastive learning, selecting feature key-values more precisely, thereby enhancing the accuracy of segmentation. Extensive experimental results on the ISPRS Potsdam dataset and the challenging iSAID dataset substantiate the superior performance of our proposed methodology.

Knowledge distillation has been one of the most effective methods for enhancing the performance of lightweight detectors, crucial for remote sensing edge intelligence models. However, many mainstream distillation methods that are centered around the paradigm of distilling positive sample show weak exploitation of the student’s potential. This arises due to these methods overlooking the core teacher-student difference in remote sensing scenarios with vast and object-similar background. In this article, from the point of distillation sample and knowledge hierarchy, we design a negative-core sample knowledge distillation (NSD) method for improving the performance of the lightweight object detection model. Specifically, a negative-core sample (NCS) is innovatively employed to transfer effective background discrimination knowledge for bridging the core difference. Knowledge distillation for NCS across four levels—pixel, logit, box, and angle are customized to fully leverage the teacher’s insights. Category direction estimation(CE) is incorporated into the angle knowledge distillation to more effectively convey NCS oriented knowledge in a discrete manner. Extensive experiments conducted on multiple remote sensing datasets achieve state-of-the-art performance, demonstrating the effectiveness of the proposed NSD. Codes are available at https://github.com/Changan00/NSD.