Due to the limitations of current technology and budget, a single satellite sensor cannot obtain high spatiotemporal resolution remote sensing images. Therefore, remote sensing image spatio-temporal fusion technology is considered as an effective solution and has attracted extensive attention. In the field of deep learning, due to the fixed size of the perception field of a convolutional neural network, it is impossible to model the correlation of global features, and the features extracted only through convolution operation lack the ability to capture long-distance features. At the same time, complex fusion methods cannot better integrate temporal and spatial features. In order to solve these problems, we propose a multistage remote sensing image spatio-temporal fusion model based on texture transformer and convolutional neural network. The model combines the advantages of transformer and convolutional network, uses a lightweight convolution network to extract spatial features and temporal discrepancy features, uses transformer to learn global temporal correlation, and finally, fuses temporal features with spatial features. In order to make full use of the features obtained in different stages, we design a cross-stage adaptive fusion module CSAFM. The module adopts the self-attention mechanism to adaptively integrate the features of different scales while considering the temporal and spatial characteristics. To test the robustness of the model, the experiments are carried out on three datasets of CIA, LGC, and DX. Compared with five typical spatio-temporal fusion algorithms, we obtain excellent results, which prove the superiority of MSFusion model.

Due to the limitations of current technology and budget, as well as the influence of various factors, obtaining remote sensing images with high-temporal and high-spatial (HTHS) resolution simultaneously is a major challenge. In this paper, we propose the GAN spatiotemporal fusion model Based on multiscale and convolutional block attention module (CBAM) for remote sensing images (MCBAM-GAN) to produce high-quality HTHS fusion images. The model is divided into three stages: multi-level feature extraction, multi-feature fusion, and multi-scale reconstruction. First of all, we use the U-NET structure in the generator to deal with the significant differences in image resolution while avoiding the reduction in resolution due to the limitation of GPU memory. Second, a flexible CBAM module is added to adaptively re-scale the spatial and channel features without increasing the computational cost, to enhance the salient areas and extract more detailed features. Considering that features of different scales play an essential role in the fusion, the idea of multiscale is added to extract features of different scales in different scenes and finally use them in the multi loss reconstruction stage. Finally, to check the validity of MCBAM-GAN model, we test it on LGC and CIA datasets and compare it with the classical algorithm for spatiotemporal fusion. The results show that the model performs well in this paper.

Currently, within the realm of deep learning-based spatiotemporal fusion algorithms, those that employ solely convolutional operations are unable to efficiently extract the global image information. In addition, fusion networks that employ a combination of convolution and transformer neglect the 2-D structure of remote sensing images and the role of their channels during training, resulting in an increased computational cost. The current complex fusion methods introduce noise and disregard the correlation between low fractional rate image's time-varying features and high-resolution image's spatial features. To address these issues, we propose TFNet—a temporal feature extraction network that combines normal and deep convolutions to better extract temporal features while reducing computational costs. Second, we suggest utilizing a convolution-based attention module with a large kernel to replace the transformer (LAM), which facilitates adjustment in both spatial and channel dimensions while preserving the image structure. Furthermore, for improved image fusion, we recommend a two-stage fusion module to merge feature images of various scales. This module for fusion integrates features of varying scales and resolutions from various perspectives, thereby preventing noise inclusions and producing favourable fusion outcomes. In addition, we advocate for the utilization of spatiotemporal fusion techniques on other satellites by introducing a new dataset, SW, which is founded on satellite images from Gaofen-1 and moderate-resolution imaging spectroradiometer.

High spatial-temporal resolution plays a vital role in the application of geoscience dynamic observance and prediction. However, thanks to the constraints of technology and budget, it is troublesome for one satellite detector to get high spatial-temporal resolution remote sensing images. Individuals have developed spatiotemporal image fusion technology to resolve this downside, and deep remote sensing images with spatiotemporal resolution have become a possible and efficient answer. Due to the fixed size of the receptive field of convolutional neural networks, the features extracted by convolution operations cannot capture long-range features, so the correlation of global features cannot be modeled in the deep learning process. We propose a spatiotemporal fusion model of remote sensing images to solve these problems based on a dual branch feedback mechanism and texture transformer. The model separates the network from the coarse-fine images with similar structures through the idea of double branches and reduces the dependence of images on time series. It principally merges the benefits of transformer and convolution network and employs feedback mechanism and texture transformer to extract additional spatial and temporal distinction features. The primary function of the transformer module is to learn global temporal correlations and fuse temporal features with spatial features. To completely extract additional elaborated features in several stages, we have a tendency to design a feedback mechanism module. This module chiefly refines the low-level representation through high-level info and obtains additional elaborated features when considering the temporal and spacial characteristics. We have a tendency to receive good results by comparison with four typical spatiotemporal fusion algorithms, proving our model’s superiority and robustness.

The purpose of the panchromatic sharpening of remote sensing images is to generate high-resolution multispectral images through software technology without increasing economic expenditure. The specific method is to fuse the spatial information of a high-resolution panchromatic image and the spectral information of a low-resolution multispectral image. This work proposes a novel model for generating high-quality multispectral images. This model uses the feature domain of the convolution neural network to fuse multispectral and panchromatic images so that the fused images can generate new features so that the final fused features can restore clear images. Because of the unique feature extraction ability of convolution neural networks, we use the core idea of convolution neural networks to extract global features. To extract the complementary features of the input image at a deeper level, we first designed two subnetworks with the same structure but different weights, and then used single-channel attention to optimize the fused features to improve the final fusion performance. We select the public data set widely used in this field to verify the validity of the model. The experimental results on the GaoFen-2 and SPOT6 data sets show that this method has a better effect in fusing multi-spectral and panchromatic images. Compared with the classical and the latest methods in this field, our model fusion obtained panchromatic sharpened images from both quantitative and qualitative analysis has achieved better results. In addition, to verify the transferability and generalization of our proposed model, we directly apply it to multispectral image sharpening, such as hyperspectral image sharpening. Experiments and tests have been carried out on Pavia Center and Botswana public hyperspectral data sets, and the results show that the model has also achieved good performance in hyperspectral data sets.