Convolutional neural networks have made a great breakthrough in recent remote sensing image super-resolution (SR) tasks. Most of these methods adopt upsampling layers at the end of the models to perform enlargement, which ignores feature extraction in the high-dimension space, and thus, limits SR performance. To address this problem, we propose a new SR framework for remote sensing images to enhance the high-dimensional feature representation after the upsampling layers. We name the proposed method as a transformer-based enhancement network (TransENet), where transformers are introduced to exploit features at different levels. The core of the TransENet is a transformer-based multistage enhancement structure, which can be combined with traditional SR frameworks to fuse multiscale high-/low-dimension features. Specifically, in this structure, the encoders aim to embed the multilevel features in the feature extraction part and the decoders are used to fuse these encoded embeddings. Experimental results demonstrate that our proposed TransENet can improve super-resolved results and obtain superior performance over several state-of-the-art methods.

Recently, deep convolutional neural networks (CNNs) have made great progress in remote sensing image super-resolution (SR). The CNN-based methods can learn powerful feature representation from plenty of low- and high-resolution counterparts. For remote sensing images, there are many similar ground targets recurred inside the image itself, both within the same scale and across different scales. In this article, we argue that this internal recurrence can be used for learning stronger feature representation, and we propose a new hybrid-scale self-similarity exploitation network (HSENet) for remote sensing image SR. Specifically, we introduce a single-scale self-similarity exploitation module (SSEM) to compute the feature correlation within the same scale image. Moreover, we design a cross-scale connection structure (CCS) to capture the recurrences across different scales. By combining SSEM and CCS, we further develop a hybrid-scale self-similarity exploitation module (HSEM) to construct the final HSENet, which simultaneously exploits single- and cross-scale similarities. Experimental results demonstrate that HSENet can obtain superior performance over several state-of-the-art methods. Besides, the effectiveness of our method is also verified by the assistance to the remote sensing scene classification task.

Generative adversarial network (GAN) has made great progress in recent natural image super-resolution tasks. The key to its success is the integration of a discriminator which is trained to classify whether the input is a real high-resolution (HR) image or a generated one. Arguably, learning a strong discriminative prior is essential for generating high-quality images. However, in remote sensing images, we discover, through extensive statistical analysis, that there are more low-frequency components than natural images, which may lead to a ``discrimination-ambiguity'' problem, i.e., the discriminator will become ``confused'' to tell whether its input is real or not when dealing with those low-frequency regions, and therefore, the quality of generated HR images may be deeply affected. To address this problem, we propose a novel GAN-based super-resolution algorithm named coupled-discriminated GANs (CDGANs) for remote sensing images. Different from the previous GAN-based super-resolution models in which their discriminator takes in a single image at one time, in our model, the discriminator is specifically designed to take in a pair of images: a generated image and its HR ground truth, to make better discrimination of the inputs. We further introduce a dual pathway network architecture, a random gate, and a coupled adversarial loss to learn better correspondence between the discriminative results and the paired inputs. Experimental results on two public data sets demonstrate that our model can obtain more accurate super-resolution results in terms of both visual appearance and local details compared with other state of the arts. Our code will be made publicly available.

Given a language expression, referring remote sensing image segmentation (RRSIS) aims to identify ground objects and assign pixel-wise labels within the imagery. The one of key challenges for this task is to capture discriminative multi-modal features via text-image alignment. However, the existing RRSIS methods use one vanilla and coarse alignment, where the language expression is directly extracted to be fused with the visual features. In this paper, we argue that a “fine-grained image-text alignment” can improve the extraction of multi-modal information. To this point, we propose a new referring remote sensing image segmentation method to fully exploit the visual and linguistic representations. Specifically, the original referring expression is regarded as context text, which is further decoupled into the ground object and spatial position texts. The proposed fine-grained image-text alignment module (FIAM) would simultaneously leverage the features of the input image and the corresponding texts, obtaining better discriminative multi-modal representation. Meanwhile, to handle the various scales of ground objects in remote sensing, we introduce a Text-aware Multi-scale Enhancement Module (TMEM) to adaptively perform cross-scale fusion and intersections. We evaluate the effectiveness of the proposed method on two public referring remote sensing datasets including RefSegRS and RRSIS-D, and our method obtains superior performance over several state-of-the-art methods. The code will be publicly available at https://github.com/Shaosifan/FIANet.

Despite its fruitful applications in remote sensing, image super-resolution is troublesome to train and deploy as it handles different resolution magnifications with separate models. Accordingly, we propose a highly-applicable super-resolution framework called FunSR, which settles different magnifications with a unified model by exploiting context interaction within implicit function space. FunSR composes a functional representor, a functional interactor, and a functional parser. Specifically, the representor transforms the low-resolution image from Euclidean space to multi-scale pixel-wise function maps; the interactor enables pixel-wise function expression with global dependencies; and the parser, which is parameterized by the interactor’s output, converts the discrete coordinates with additional attributes to RGB values. Extensive experimental results demonstrate that FunSR reports state-of-the-art performance on both fixed-magnification and continuous-magnification settings, meanwhile, it provides many friendly applications thanks to its unified nature. Our code is available at https://github.com/KyanChen/FunSR.

Spatial resolution and spectral resolution both play an important role in the recognition of objects in hyperspectral remote sensing. However, the imaging characteristics of hyperspectral images (HSIs) result in a mutually restrictive relationship between the spatial and spectral resolutions. Generative adversarial networks (GANs) have achieved significant success in image generation. The introduce of the discriminators plays a key role in improving the reality. In this article, we propose an RGB to multiband hyperspectral imagery (150 bands) generation method based on GAN (R2HGAN). The method solves the high ill-posed problem and introduces high spectral resolution into RGB images by learning from multiple scenes of HSI. In R2HGAN, we extend the adversarial learning from spatial to spectral dimensions and joint discrimination is designed to generate HSIs closer to the real ones, where two discriminators (the conditional D and the spectral D) are put forward to supervise the spectral similarity and the conditional reality of the HSI jointly. In detail, the conditional discriminator comprehensively judges the quality of each area in the reconstructed HSI. At the same time, to ensure that the generated spectra are close to the real ones, a spectral discriminator based on multilayer perceptron is designed. Through the experiments on GF-5 imagery, the method has significantly improved the quality of the generated images over other state-of-the-art methods.