Optical remote sensing (RS) data suffer from the limitation of bad weather and cloud contamination, whereas synthetic aperture radar (SAR) can work under all weather conditions and overcome this disadvantage of optical RS data. However, due to the imaging mechanism of SAR and the speckle noise, untrained people are difficult to recognize the land cover types visually from SAR images. Inspired by the excellent image-to-image translation performance of Generative Adversarial Networks (GANs), a supervised Cycle-Consistent Adversarial Network (S-CycleGAN) was proposed to generate large optical images from the SAR images. When the optical RS data are unavailable or partly unavailable, the generated optical images can be alternative data that aid in land cover visual recognition for untrained people. The main steps of SAR-to-optical image translation were as follows. First, the large SAR image was split to small patches. Then S-CycleGAN was used to translate the SAR patches to optical image patches. Finally, the optical image patches were stitched to generate the large optical image. A paired SAR-optical image dataset which covered 32 Chinese cities was published to evaluate the proposed method. The dataset was generated from Sentinel-1 (SEN-1) SAR images and Sentinel-2 (SEN-2) multi-spectral images. S-CycleGAN was applied to two experiments, which were SAR-to-optical image translation and cloud removal, and the results showed that S-CycleGAN could keep both the land cover and structure information well, and its performance was superior to some famous image-to-image translation models.

Convolutional neural networks (CNNs) have demonstrated their ability object detection of very high resolution remote sensing images. However, CNNs have obvious limitations for modeling geometric variations in remote sensing targets. In this paper, we introduced a CNN structure, namely deformable ConvNet, to address geometric modeling in object recognition. By adding offsets to the convolution layers, feature mapping of CNN can be applied to unfixed locations, enhancing CNNs’ visual appearance understanding. In our work, a deformable region-based fully convolutional networks (R-FCN) was constructed by substituting the regular convolution layer with a deformable convolution layer. To efficiently use this deformable convolutional neural network (ConvNet), a training mechanism is developed in our work. We first set the pre-trained R-FCN natural image model as the default network parameters in deformable R-FCN. Then, this deformable ConvNet was fine-tuned on very high resolution (VHR) remote sensing images. To remedy the increase in lines like false region proposals, we developed aspect ratio constrained non maximum suppression (arcNMS). The precision of deformable ConvNet for detecting objects was then improved. An end-to-end approach was then developed by combining deformable R-FCN, a smart fine-tuning strategy and aspect ratio constrained NMS. The developed method was better than a state-of-the-art benchmark in object detection without data augmentation.

The launch of the Chinese Gaofen-3 (GF-3) satellite will provide enough synthetic aperture radar (SAR) images with different imaging modes for land cover classification and other potential usages in the next few years. This paper aims to propose an efficient and practical classification framework for a GF-3 polarimetric SAR (PolSAR) image. The proposed classification framework consists of four simple parts including polarimetric feature extraction and stacking, the initial classification via XGBoost, superpixels generation by statistical region merging (SRM) based on Pauli RGB image, and a post-processing step to determine the label of a superpixel by modified majority voting. Fast initial classification via XGBoost and the incorporation of spatial information via a post-processing step through superpixel-based modified majority voting would potentially make the method efficient in practical use. Preliminary experimental results on real GF-3 PolSAR images and the AIRSAR Flevoland data set validate the efficacy and efficiency of the proposed classification framework. The results demonstrate that the quality of GF-3 PolSAR data is adequate enough for classification purpose. The results also show that the incorporation of spatial information is important for overall performance improvement.

Convolutional neural networks (CNN) have achieved great success in the optical image processing field. Because of the excellent performance of CNN, more and more methods based on CNN are applied to polarimetric synthetic aperture radar (PolSAR) image classification. Most CNN-based PolSAR image classification methods can only classify one pixel each time. Because all the pixels of a PolSAR image are classified independently, the inherent interrelation of different land covers is ignored. We use a fixed-feature-size CNN (FFS-CNN) to classify all pixels in a patch simultaneously. The proposed method has several advantages. First, FFS-CNN can classify all the pixels in a small patch simultaneously. When classifying a whole PolSAR image, it is faster than common CNNs. Second, FFS-CNN is trained to learn the interrelation of different land covers in a patch, so it can use the interrelation of land covers to improve the classification results. The experiments of FFS-CNN are evaluated on a Chinese Gaofen-3 PolSAR image and other two real PolSAR images. Experiment results show that FFS-CNN is comparable with the state-of-the-art PolSAR image classification methods.

Limited labels and increasing multisource data promote domain adaptation (DA) problem as a challenging study for polarimetric synthetic aperture radar (PolSAR) interpretation. Existing DAs for optical images cannot generalize over PolSAR imagery due to its special side-imaging characteristics and complex distribution shifts. In this letter, a general feature paradigm (GFP) is proposed for unsupervised cross-domain PolSAR image classification. The GFP is based on a key observation that interclass aggregation is optimized after four-step feature transformations. This key observation leads to GFP that not only reduces the domain shifts but also compatible with typical DA methods. The GFPs are conducted on both source and target domain by unsupervised manner, including polarimetric basis extraction, the Wishart clustering, histogram statistics, and dimensionality reduction. After these transformations, the unlabeled target PolSAR image can be classified based on obtained GFP, DA, and limited labeled samples only from the source domain. Extensive unsupervised cross-domain experiments on 27 scenarios verified that GFP leads to at most 93.76% accuracy for full- and dual-polarized synthetic aperture radar (SAR) images’ classification. Moreover, the GFP shed light on extensive cross-domain PolSAR applications about built-up areas, vegetation, and bare land analysis.

The aim of weakly supervised object co-localization is to locate different objects of the same superclass in a dataset. Recent methods achieve impressive co-localization performance by multiple instance learning and self-supervised learning. However, these methods ignore the common part information shared by fine-grained objects and the influence of the complementary parts on the co-localization of the fine-grained objects. To solve these issues, we propose a complementary parts contrastive learning method for fine-grained weakly supervised object co-localization. The proposed method follows such an assumption that fine-grained object parts with the same/different semantic meaning should have similar/dissimilar feature representations in the feature space. The proposed method tackles two critical issues in this task: i ) how to spread the model’s attention and suppress the complex background noise, and ii ) how to leverage the cross-category common parts information to mitigate the context co-occurrence problem. To address i ), we attempt to integrate local and context cues via three types of attention including self-supervised attention, channel, and spatial attention to spread the model’s attention toward automatically identifying and localizing most discriminative parts of objects in the fine-grained images. To solve ii ), we propose a cross-category object complementarity part contrastive learning module to identify the extracted part regions with different semantic information by pulling the same part features closer and pushing different part features away, which can mitigate the confounding bias caused by the co-occurrence surroundings within specific classes. Extensive qualitative and quantitative evaluations demonstrate the effectiveness of the proposed method on four fine-grained co-localization datasets: CUB-200-2011, Stanford Cars, FGVC-Aircraft, and Stanford Dogs. Code and models are available at https://github.com/Zhao-fan/CPCL.