12 December 2023 in IEEE Transactions on Geoscience and Remote Sensing
Recently, convolutional networks have achieved remarkable development in remote sensing image (RSI) super-resolution (SR) by minimizing the regression objectives, e.g., MSE loss. However, despite achieving impressive performance, these methods often suffer from poor visual quality with oversmooth issues. Generative adversarial networks (GANs) have the potential to infer intricate details, but they are easy to collapse, resulting in undesirable artifacts. To mitigate these issues, in this article, we first introduce diffusion probabilistic model (DPM) for efficient RSI SR, dubbed efficient diffusion model for RSI SR (EDiffSR). EDiffSR is easy to train and maintains the merits of DPM in generating perceptual-pleasant images. Specifically, different from previous works using heavy UNet for noise prediction, we develop an efficient activation network (EANet) to achieve favorable noise prediction performance by simplified channel attention and simple gate operation, which dramatically reduces the computational budget. Moreover, to introduce more valuable prior knowledge into the proposed EDiffSR, a practical conditional prior enhancement module (CPEM) is developed to help extract an enriched condition. Unlike most DPM-based SR models that directly generate conditions by amplifying LR images, the proposed CPEM helps to retain more informative cues for accurate SR. Extensive experiments on four remote sensing datasets demonstrate that EDiffSR can restore visual-pleasant images on simulated and real-world RSIs, both quantitatively and qualitatively. The code of EDiffSR will be available at https://github.com/XY-boy/EDiffSR .
5 September 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Optical-flow-based and kernel-based approaches have been extensively explored for temporal compensation in satellite Video Super-Resolution (VSR). However, these techniques are less generalized in large-scale or complex scenarios, especially in satellite videos. In this paper, we propose to exploit the well-defined temporal difference for efficient and effective temporal compensation. To fully utilize the local and global temporal information within frames, we systematically modeled the short-term and long-term temporal discrepancies since we observe that these discrepancies offer distinct and mutually complementary properties. Specifically, we devise a Short-term Temporal Difference Module (S-TDM) to extract local motion representations from RGB difference maps between adjacent frames, which yields more clues for accurate texture representation. To explore the global dependency in the entire frame sequence, a Long-term Temporal Difference Module (L-TDM) is proposed, where the differences between forward and backward segments are incorporated and activated to guide the modulation of the temporal feature, leading to a holistic global compensation. Moreover, we further propose a Difference Compensation Unit (DCU) to enrich the interaction between the spatial distribution of the target frame and temporal compensated results, which helps maintain spatial consistency while refining the features to avoid misalignment. Rigorous objective and subjective evaluations conducted across five mainstream video satellites demonstrate that our method performs favorably against state-of-the-art approaches. Code will be available at https://github.com/XY-boy/LGTD.
24 June 2024 in ISPRS Journal of Photogrammetry and Remote Sensing
27 February 2023 in IEEE Geoscience and Remote Sensing Letters
Satellite video is a novel data source for earth observation, which can be applied in multiple fields for dynamic monitoring. It is always equipped with high temporal resolution at the cost of low spatial resolution of tiny moving objects. Video super-resolution (VSR) is utilized to improve the spatial resolution of satellite video and obtain high spatial-temporal resolution data. However, most existing VSR methods mainly focus on the local inter-frame information during feature alignment, which lack the ability to model long-distance correspondence. In this article, a novel two-branch alignment network with an efficient fusion module is proposed for satellite VSR. Both deformable convolution and transformer-like attention are employed to fully explore the local and global information between frames. Further, a fusion module is proposed to model the residuals between fusion features and compensate them for better fusion. Experiments on Jilin-1 satellite videos demonstrate that the proposed network can achieve comparable results to current state-of-the-art VSR methods with tiny parameters.
1 January 2024 in IEEE Transactions on Geoscience and Remote Sensing