Building Change Detection (BCD) holds significant value in the context of monitoring land use, while Building Damage Assessment (BDA) plays a crucial role in expediting humanitarian rescue efforts post-disasters. To address these needs, we propose the Bi-Temporal Attention Module (BAM) as an innovative cross-attention mechanism aimed at effectively capturing spatio-temporal semantic relations between a pair of bi-temporal remote sensing images. Within BAM, a shifted windowing scheme has been implemented to confine the scope of the cross-attention mechanism to a specific range, not only excluding remote and irrelevant information but also contributing to computational efficiency. Moreover, existing methods for BDA often overlook the inherent order of ordinal labels, treating the BDA task simplistically as a multi-class semantic segmentation problem. Recognizing the vital significance of ordinal relationships, we approach the BDA task as an ordinal regression problem. To address this, we introduce a rank-consistent ordinal regression loss function to train our proposed change detection network, Bi-temporal Attention Transformer (BAT). Our method achieves state-of-the-art accuracy on two BCD datasets (LEVIR-CD+ and S2Looking), as well as the largest BDA dataset (xBD).

Land cover classification is a multiclass segmentation task to classify each pixel into a certain natural or human-made category of the earth’s surface, such as water, soil, natural vegetation, crops, and human infrastructure. Limited by hardware computational resources and memory capacity, most existing studies preprocessed original remote sensing images by downsampling or cropping them into small patches less than 512 × 512 pixels before sending them to a deep neural network. However, downsampling incurs a spatial detail loss, renders small segments hard to discriminate, and reverses the spatial resolution progress obtained by decades of efforts. Cropping images into small patches causes a loss of long-range context information, and restoring the predicted results to their original size brings extra latency. In response to the above weaknesses, we present an efficient lightweight semantic segmentation network termed MKANet. Aimed at the characteristics of top view high-resolution remote sensing imagery, MKANet utilizes sharing kernels to simultaneously and equally handle ground segments of inconsistent scales, and also employs a parallel and shallow architecture to boost inference speed and friendly support image patches more than 10× larger. To enhance boundary and small segment discrimination, we also propose a method that captures category impurity areas, exploits boundary information, and exerts an extra penalty on boundaries and small segment misjudgments. Both visual interpretations and quantitative metrics of extensive experiments demonstrate that MKANet obtains a state-of-the-art accuracy on two land-cover classification datasets and infers 2× faster than other competitive lightweight networks. All these merits highlight the potential of MKANet in practical applications.

Semantic segmentation is crucial for enabling autonomous flight and landing of low-altitude unmanned aerial vehicles (UAVs) and is indispensable for various intelligent applications. However, real-time semantic segmentation is a computationally intensive task because it involves pixel-wise classification, which renders conventional semantic segmentation networks impractical for deployment on embedded systems of limited hardware resources. Moreover, variations in flight height and object appearance increase the likelihood of misjudgment in segmentation results. To address these challenges, we propose an efficient approach consisting of a convolutional neural network (CNN)–Transformer network and an auxiliary loss. The encoder of the network integrates a newly designed module, which equally handles objects with varying scales. The decoder is composed of the innovative query–value squeeze axial transformer attention (QVSATA), which reduces computational complexity from quadratic in terms of image size to $\mathcal {O}(2C(H^{2}+W^{2}))$ , linear in terms of image size. By incorporating Laplacian operator convolution, the novel network-agnostic loss effectively captures intricate patterns, boundaries, and small objects. This enables extra penalization of misjudgments in these areas and compels the network to focus on objects that are challenging to distinguish. Our approach attains impressive accuracy when processing 4K resolution images in real time (15 FPS) on a mobile GPU. It demonstrates over $2\times $ faster speed compared to representative lightweight networks, underscoring its suitability for onboard deployment.

The fast data acquisition rate due to the shorter revisit periods and wider observation coverage of satellites results in large amounts of remote sensing images every day. This brings the challenge of how to accurately search the images with similar visual content as the query image. Content-Based Image Retrieval (CBIR) is a solution to this challenge, its performance heavily depends on the effectiveness of the image representation features and similarity evaluation metrics. Ideal image feature representations have dispersed inter-class, compact intra-class distribution. However, the neural networks employed by many CBIR methods are trained with cross entropy loss, which does not directly optimize the metrics that evaluates inter-class variance over intra-class variance, hence their feature representations are suboptimal. Meanwhile, the traditional distance metrics used by many CBIR methods cannot index the similarity of feature representations well in high dimensional space. For better CBIR performance, we propose a discriminative feature learning approach with distinguishable distance metrics for remote sensing image classification and retrieval. By balancing the diagonal elements and non-diagonal elements of the within-class scatter matrix of Deep Linear Discriminant Analysis, our proposed loss function, Balanced Deep Linear Discriminant Analysis (BDLDA), can better optimize the Rayleigh-Ritz Quotient which measures inter-class variance over intra-class variance. In addition, the proposed distance metrics, Reciprocal Exponential Distance (RED), is more capable of maintaining distance contrast in high dimensionality, therefore, it can better index similarity for feature representations in high dimensionality. Both visual interpretations and quantitative metrics of extensive experiments demonstrated the effectiveness of our approach.

A near-real-time change detection network can consistently identify unauthorized construction activities over a wide area, empowering authorities to enforce regulations efficiently. Furthermore, it can promptly assess building damage, enabling expedited rescue efforts. The extensive adoption of deep learning in change detection has prompted a predominant emphasis on enhancing detection performance, primarily through the expansion of the depth and width of networks, overlooking considerations regarding inference time and computational cost. To accurately represent the spatio-temporal semantic correlations between pre-change and post-change images, we create an innovative transformer attention mechanism named Sliding-Window Dissimilarity Cross-Attention (SWDCA), which detects spatio-temporal semantic discrepancies by explicitly modeling the dissimilarity of bi-temporal tokens, departing from the mono-temporal similarity attention typically used in conventional transformers. In order to fulfill the near-real-time requirement, SWDCA employs a sliding-window scheme to limit the range of the cross-attention mechanism within a predetermined window/dilated window size. This approach not only excludes distant and irrelevant information but also reduces computational cost. Furthermore, we develop a lightweight Siamese backbone for extracting building and environmental features. Subsequently, we integrate an SWDCA module into this backbone, forming an efficient change detection network. Quantitative evaluations and visual analyses of thorough experiments verify that our method achieves top-tier accuracy on two building change detection datasets of remote sensing imagery, while also achieving a real-time inference speed of 33.2 FPS on a mobile GPU.

Onboard land cover classification provides ever-updating land cover information, supporting various intelligent satellite applications that demand timely autonomous decision-making based on current and continuous land cover data. However, due to space, weight, and power constraints, satellites possess limited computational resources, rendering them unable to execute conventional land cover classification networks. In response to this challenge, we have designed a lightweight network for land cover classification featuring two efficient Transformer attention mechanisms enhanced by Multi-Granularity Tokens. Diverging from traditional Transformer attention mechanisms that solely capture token-to-token correlations at a single granularity, our approach splits the tokens into four segments and utilizes Atrous Convolutions across various dilation rates to aggregate token segments from diverse receptive fields, forming token segments combinations that encompass not just point information but also information from patches of varying sizes. These Multi-Granularity Tokens are subsequently processed through the Windowed Squeeze Axial Transformer Attention (WSATA) and Multi-Granularity Bi-level Routing Attention (MGBRA) for feature enhancement. In another aspect, empirical observations reveal that prediction errors are more prone to manifest on land covers of small extent, however, conventional methods treat all pixels uniformly. This realization motivates us to propose a novel network-agnostic loss named Connected Component Loss, which specifically targets small-scale land covers and their boundaries. Quantitative metrics and visual interpretations from comprehensive experiments confirm that our method attains state-of-the-art accuracy on two land cover classification datasets while exhibits significantly faster inference speed than other lightweight networks, underscoring the practical potential of our method on embedded systems.