Generative Adversarial Networks (GAN) have demonstrated the potential to recover realistic details for single image super-resolution (SISR). To further improve the visual quality of super-resolved results, PIRM2018-SR Challenge employed perceptual metrics to assess the perceptual quality, such as PI, NIQE, and Ma. However, existing methods cannot directly optimize these indifferentiable perceptual metrics, which are shown to be highly correlated with human ratings. To address the problem, we propose Super-Resolution Generative Adversarial Networks with Ranker (RankSRGAN) to optimize generator in the direction of different perceptual metrics. Specifically, we first train a Ranker which can learn the behaviour of perceptual metrics and then introduce a novel rank-content loss to optimize the perceptual quality. The most appealing part is that the proposed method can combine the strengths of different SR methods to generate better results. Furthermore, we extend our method to multiple Rankers to provide multi-dimension constraints for the generator. Extensive experiments show that RankSRGAN achieves visually pleasing results and reaches state-of-the-art performance in perceptual metrics and quality. Project page: https://wenlongzhang0517.github.io/Projects/RankSRGAN .
Satellite microwave radiometer data is affected by many degradation factors during the imaging process, such as the sampling interval, antenna pattern and scan mode, etc., leading to spatial resolution reduction. In this paper, a deep residual convolutional neural network (CNN) is proposed to solve these degradation problems by learning the end-to-end mapping between low-and high-resolution images. Unlike traditional methods that handle each degradation factor separately, our network jointly learns both the sampling interval limitation and the comprehensive degeneration factors, including the antenna pattern, receiver sensitivity and scan mode, during the training process. Moreover, due to the powerful mapping capability of the deep residual CNN, our method achieves better resolution enhancement results both quantitatively and qualitatively than the methods in literature. The microwave radiation imager (MWRI) data from the Fengyun-3C (FY-3C) satellite has been used to demonstrate the validity and the effectiveness of the method.
Microwave radiometer data is affected by many factors during the imaging process, including the antenna pattern, system noise, and the curvature of the Earth. Existing deconvolution methods such as Wiener filtering handle this degradation problem in the Fourier domain. However, under complex degradation conditions, the Wiener filtering results are not accurate. In this paper, a convolutional neural network (CNN) model is proposed to solve the degradation problem. The deconvolution procedure is defined as a regression problem in the spatial domain that can be solved with deep learning. For the real inverse process of microwave radiometer data, the CNN model has a more powerful reconstruction ability than Wiener filtering due to the multi-layer structure of the CNN, which enables the multiple feature transform of the data. Additionally, the complex degradation factor during the imaging process of a microwave radiometer can be solved with general framework-based learning. Experimental results demonstrated that the CNN model gains about 5 dB at the peak signal-to-noise ratio compared to the Wiener filtering deconvolution method, and can better distinguish the measured data.
Passive multi-frequency microwave remote sensing is often plagued with the problems of low- and non-uniform spatial resolution. In order to adaptively enhance and match the spatial resolution, an accommodative spatial resolution matching (ASRM) framework, composed of the flexible degradation model, the deep residual convolutional neural network (CNN), and the adaptive feature modification (AdaFM) layers, is proposed in this paper. More specifically, a flexible degradation model, based on the imaging process of the microwave radiometer, is firstly proposed to generate suitable datasets for various levels of matching tasks. Secondly, a deep residual CNN is introduced to jointly learn the complicated degradation factors of the data, so that the resolution can be matched up to fixed levels with state of the art quality. Finally, the AdaFM layers are added to the network in order to handle arbitrary and continuous resolution matching problems between a start and an end level. Both the simulated and the microwave radiation imager (MWRI) data from the Fengyun-3C (FY-3C) satellite have been used to demonstrate the validity and the effectiveness of the method.
Building a unified model for general low-level vision tasks holds significant research and practical value. Current methods encounter several critical issues. Multi-task restoration approaches can address multiple degradation-to-clean restoration tasks, while their applicability to tasks with different target domains (e.g., image stylization) is limited. Methods like PromptGIP can handle multiple input-target domains but rely on the Masked Autoencoder (MAE) paradigm. Consequently, they are tied to the ViT architecture, resulting in suboptimal image reconstruction quality. In addition, these methods are sensitive to prompt image content and often struggle with low-frequency information processing. In this paper, we propose a Visual task Prompt-based Image Processing (VPIP) framework to overcome these challenges. VPIP employs visual task prompts to manage tasks with different input-target domains and allows flexible selection of backbone network suitable for general tasks. Besides, a new prompt cross-attention is introduced to facilitate interaction between the input and prompt information. Based on the VPIP framework, we train a low-level vision generalist model, namely GenLV, on 30 diverse tasks. Experimental results show that GenLV can successfully address a variety of low-level tasks, significantly outperforming existing methods both quantitatively and qualitatively. Codes are available at https://github.com/chxy95/GenLV.