The existence of clouds is one of the main factors that contributes to missing information in optical remote sensing images, restricting their further applications for Earth observation, so how to reconstruct the missing information caused by clouds is of great concern. Inspired by the image-to-image translation work based on convolutional neural network model and the heterogeneous information fusion thought, we propose a novel cloud removal method in this paper. The approach can be roughly divided into two steps: in the first step, a specially designed convolutional neural network (CNN) translates the synthetic aperture radar (SAR) images into simulated optical images in an object-to-object manner; in the second step, the simulated optical image, together with the SAR image and the optical image corrupted by clouds, is fused to reconstruct the corrupted area by a generative adversarial network (GAN) with a particular loss function. Between the first step and the second step, the contrast and luminance of the simulated optical image are randomly altered to make the model more robust. Two simulation experiments and one real-data experiment are conducted to confirm the effectiveness of the proposed method on Sentinel 1/2, GF 2/3 and airborne SAR/optical data. The results demonstrate that the proposed method outperforms state-of-the-art algorithms that also employ SAR images as auxiliary data.

As a new earth observation tool, satellite video has been widely used in remote-sensing field for dynamic analysis. Video super-resolution (VSR) technique has thus attracted increasing attention due to its improvement to spatial resolution of satellite video. However, the difficulty of remote-sensing image alignment and the low efficiency of spatial–temporal information fusion make poor generalization of the conventional VSR methods applied to satellite videos. In this article, a novel fusion strategy of temporal grouping projection and an accurate alignment module are proposed for satellite VSR. First, we propose a deformable convolution alignment module with a multiscale residual block to alleviate the alignment difficulties caused by scarce motion and various scales of moving objects in remote-sensing images. Second, a temporal grouping projection fusion strategy is proposed, which can reduce the complexity of projection and make the spatial features of reference frames play a continuous guiding role in spatial–temporal information fusion. Finally, a temporal attention module is designed to adaptively learn the different contributions of temporal information extracted from each group. Extensive experiments on Jilin-1 satellite video demonstrate that our method is superior to current state-of-the-art VSR methods.

Change detection (CD) is a fundamental and important task for monitoring the land surface dynamics in the earth observation field. Existing deep learning-based CD methods typically extract bi-temporal image features using a weight-sharing Siamese encoder network and identify change regions using a decoder network. These CD methods, however, still perform far from satisfactorily as we observe that 1) deep encoder layers focus on irrelevant background regions and 2) the models’ confidence in the change regions is inconsistent at different decoder stages. The first problem is because deep encoder layers cannot effectively learn from imbalanced change categories using the sole output supervision, while the second problem is attributed to the lack of explicit semantic consistency preservation. To address these issues, we design a novel similarity-aware attention flow network (SAAN). SAAN incorporates a similarity-guided attention flow module with deeply supervised similarity optimization to achieve effective change detection. Specifically, we counter the first issue by explicitly guiding deep encoder layers to discover semantic relations from bi-temporal input images using deeply supervised similarity optimization. The extracted features are optimized to be semantically similar in the unchanged regions and dissimilar in the changing regions. The second drawback can be alleviated by the proposed similarity-guided attention flow module, which incorporates similarity-guided attention modules and attention flow mechanisms to guide the model to focus on discriminative channels and regions. We evaluated the effectiveness and generalization ability of the proposed method by conducting experiments on a wide range of CD tasks. The experimental results demonstrate that our method achieves excellent performance on several CD tasks, with discriminative features and semantic consistency preserved.

Different from object detection in natural image, optical remote sensing object detection is a challenging task, due to the diverse meteorological conditions, complex background, varied orientations, scale variations, etc. In this paper, to address this issue, we propose a novel object detection network (the global-local saliency constraint network, GLS-Net) that can make full use of the global semantic information and achieve more accurate oriented bounding boxes. More precisely, to improve the quality of the region proposals and bounding boxes, we first propose a saliency pyramid which combines a saliency algorithm with a feature pyramid network, to reduce the impact of complex background. Based on the saliency pyramid, we then propose a global attention module branch to enhance the semantic connection between the target and the global scenario. A fast feature fusion strategy is also used to combine the local object information based on the saliency pyramid with the global semantic information optimized by the attention mechanism. Finally, we use an angle-sensitive intersection over union (IoU) method to obtain a more accurate five-parameter representation of the oriented bounding boxes. Experiments with a publicly available object detection dataset for aerial images demonstrate that the proposed GLS-Net achieves a state-of-the-art detection performance.

Very high-resolution (VHR) earth observation systems provide an ideal data source for man-made structure detection such as building footprint extraction. Manually delineating building footprints from the remotely sensed VHR images, however, is laborious and time-intensive; thus, automation is needed in the building extraction process to increase productivity. Recently, many researchers have focused on developing building extraction algorithms based on the encoder-decoder architecture of convolutional neural networks. However, we observe that this widely adopted architecture cannot well preserve the precise boundaries and integrity of the extracted buildings. Moreover, features obtained by shallow convolutional layers contain irrelevant background noises that degrade building feature representations. This paper addresses these problems by presenting a feature decoupling network (FD-Net) that exploits two essential building information from the input image, including semantic information that concerns building integrity and edge information that improves building boundaries. The proposed FD-Net improves the existing encoder-decoder framework by decoupling image features into the edge subspace and the semantic subspace; the decoupled features are then integrated by a supervision-guided fusion process considering the heterogeneity between edge and semantic features. Furthermore, a lightweight and effective global context attention module is introduced to capture contextual building information and thus enhance feature representations. Comprehensive experimental results on three real-world datasets confirm the effectiveness of FD-Net in large-scale building mapping. We applied the proposed method to various encoder-decoder variants to verify the generalizability of the proposed framework. Experimental results show remarkable accuracy improvements with less computational cost.

Sudden natural disasters and man-made disasters pose a threat to human life and property safety, and real-time semantic segmentation of high-resolution remote sensing images is crucial for disaster damage assessment applications. In recent years, with the wide application of high spatial resolution (HSR) remote sensing images and semantic change detection methods based on deep learning (DL), the acquisition of information on damaged areas has become more and more convenient and accurate. However, due to the black box characteristics of existing methods, the lack of interpretability and prior knowledge embedding (such as building positioning information), as well as the low utilization of damage conditions around the building, lead to automatically learned feature representations that still need to be improved. To solve these problems, we proposed the position-constrained differential attention Siamese network (PCDASNet). The main idea is to merge building extraction and disaster damage assessment into a cascaded framework to improve building damage recognition results under the constraints of building positioning information. In particular, the proposed differential attention module (DAM) adaptively extracts change information corresponding to buildings and surrounding environments from dual-temporal images, with interpretability and theoretical guarantee, which enables the integration of prior positioning knowledge into the design of network architecture. The objective metrics of the method on the building damage dataset show that the method achieves a test F1 score of more than 73% compared with other baseline methods, and also outperforms several state-of-the-art methods in terms of visual results.