The trade-off between feature representation power and spatial localization accuracy is crucial for the dense classification/semantic segmentation of remote sensing images (RSIs). High-level features extracted from the late layers of a neural network are rich in semantic information, yet have blurred spatial details; low-level features extracted from the early layers of a network contain more pixel-level information but are isolated and noisy. It is therefore difficult to bridge the gap between high- and low-level features due to their difference in terms of physical information content and spatial distribution. In this article, we contribute to solve this problem by enhancing the feature representation in two ways. On the one hand, a patch attention module (PAM) is proposed to enhance the embedding of context information based on a patchwise calculation of local attention. On the other hand, an attention embedding module (AEM) is proposed to enrich the semantic information of low-level features by embedding local focus from high-level features. Both proposed modules are lightweight and can be applied to process the extracted features of convolutional neural networks (CNNs). Experiments show that, by integrating the proposed modules into a baseline fully convolutional network (FCN), the resulting local attention network (LANet) greatly improves the performance over the baseline and outperforms other attention-based methods on two RSI data sets.

Semantic change detection (SCD) extends the multiclass change detection (MCD) task to provide not only the change locations but also the detailed land-cover/land-use (LCLU) categories before and after the observation intervals. This fine-grained semantic change information is very useful in many applications. Recent studies indicate that the SCD can be modeled through a triple-branch convolutional neural network (CNN), which contains two temporal branches and a change branch. However, in this architecture, the communications between the temporal branches and the change branch are insufficient. To overcome the limitations in existing methods, we propose a novel CNN architecture for the SCD, where the semantic temporal features are merged in a deep CD unit. Furthermore, we elaborate on this architecture to reason the bi-temporal semantic correlations. The resulting bi-temporal semantic reasoning network (Bi-SRNet) contains two types of semantic reasoning blocks to reason both single-temporal and cross-temporal semantic correlations, as well as a novel loss function to improve the semantic consistency of change detection results. Experimental results on a benchmark dataset show that the proposed architecture obtains significant accuracy improvements over the existing approaches, while the added designs in the Bi-SRNet further improve the segmentation of both semantic categories and the changed areas. The codes in this article are accessible at https://github.com/ggsDing/Bi-SRNet .

Vision Foundation Models (VFMs) such as the Segment Anything Model (SAM) allow zero-shot or interactive segmentation of visual contents, thus they are quickly applied in a variety of visual scenes. However, their direct use in many Remote Sensing (RS) applications is often unsatisfactory due to the special imaging properties of RS images. In this work, we aim to utilize the strong visual recognition capabilities of VFMs to improve change detection (CD) in very high-resolution (VHR) remote sensing images (RSIs). We employ the visual encoder of FastSAM, a variant of the SAM, to extract visual representations in RS scenes. To adapt FastSAM to focus on some specific ground objects in RS scenes, we propose a convolutional adaptor to aggregate the task-oriented change information. Moreover, to utilize the semantic representations that are inherent to SAM features, we introduce a task-agnostic semantic learning branch to model the semantic latent in bi-temporal RSIs. The resulting method, SAM-CD, obtains superior accuracy compared to the SOTA fully-supervised CD methods and exhibits a sample-efficient learning ability that is comparable to semi-supervised CD methods. To the best of our knowledge, this is the first work that adapts VFMs to CD in VHR RS images.

In recent years, deep-learning-based hyperspectral image (HSI) classification methods have achieved significant development. The superior capability of feature extraction from these data-driven methods dramatically improves the classification performance. However, the previous methods usually require to retrain the network from scratch to obtain the capability of feature extraction adaptive for the target image when facing a new HSI to be classified, which is a time-consuming and redundant process. In this paper, we consider putting this process ahead and making the network have a robust capability of feature extraction with generalization through pre-training. Therefore, the network enables to directly extract features of the target HSI without re-training. For this purpose, we rethink the three-dimension (3D) HSI data from a perspective of spectral sequence, and we attempt to extract the spectral variation information as the spectrum motion feature. Then, we construct an unsupervised spectrum motion feature learning framework (SMF-UL), which can be pre-trained on mass unlabeled HSI data to learn the knowledge about perceiving spectral variation. Furthermore, to achieve the expansion of source data for pre-training, we develop an extendable training dataset construction method, which can integrate HSIs of different sizes, number of bands and sensors into a unified training set to utilize the rapidly growing mass unlabeled HSI data effectively. Finally, we use the trained network to directly extract the spectrum motion feature of the target HSI for classification, so the laborious re-training of the network can be avoided. Extensive experiments show that the proposed SMF-UL acquires the robust capability of feature extraction with generalization through unsupervised learning on mass unlabeled HSI data, and the classification performance of extracted spectrum motion feature is competitive to advanced in-domain and cross-domain methods, which shows its flexibility and superiority. The code of SMF-UL will be open at: https://github.com/sssssyf/SMF-UL.

Semantic Change Detection (SCD) refers to the task of simultaneously extracting the changed areas and the semantic categories (before and after the changes) in Remote Sensing Images (RSIs). This is more meaningful than Binary Change Detection (BCD) since it enables detailed change analysis in the observed areas. Previous works established triple-branch Convolutional Neural Network (CNN) architectures as the paradigm for SCD. However, it remains challenging to exploit semantic information with a limited amount of change samples. In this work, we investigate to jointly consider the spatio-temporal dependencies to improve the accuracy of SCD. First, we propose a Semantic Change Transformer (SCanFormer) to explicitly model the ’from-to’ semantic transitions between the bi-temporal RSIs. Then, we introduce a semantic learning scheme to leverage the spatio-temporal constraints, which are coherent to the SCD task, to guide the learning of semantic changes. The resulting network (SCanNet) significantly outperforms the baseline method in terms of both detection of critical semantic changes and semantic consistency in the obtained bi-temporal results. It achieves the SOTA accuracy on two benchmark datasets for the SCD.

Long-range contextual information is crucial for the semantic segmentation of high-resolution (HR) remote sensing images (RSIs). However, image cropping operations, commonly used for training neural networks, limit the perception of long-range contexts in large RSIs. To overcome this limitation, we propose a wide-context network (WiCoNet) for the semantic segmentation of HR RSIs. Apart from extracting local features with a conventional convolutional neural network (CNN), the WiCoNet has an extra context branch to aggregate information from a larger image area. Moreover, we introduce a context transformer to embed contextual information from the context branch and selectively project it onto the local features. The context transformer extends the vision transformer, an emerging kind of neural networks, to model the dual-branch semantic correlations. It overcomes the locality limitation of CNNs and enables the WiCoNet to see the bigger picture before segmenting the land-cover/land-use (LCLU) classes. Ablation studies and comparative experiments conducted on several benchmark datasets demonstrate the effectiveness of the proposed method. In addition, we present a new Beijing Land-Use (BLU) dataset. This is a large-scale HR satellite dataset with high-quality and fine-grained reference labels, which can facilitate future studies in this field.