28 December 2021 in IEEE Transactions on Image Processing
Pansharpening is used to fuse a panchromatic (PAN) image with a multispectral (MS) image to obtain a high-spatial-resolution multispectral (HRMS) image. Traditional pansharpening methods face difficulties in obtaining accurate details and have low computational efficiency. In this study, a unified pansharpening model based on the band-adaptive gradient and detail correction is proposed. First, a spectral fidelity constraint is designed by keeping each band of the HRMS image consistent with that of the MS image. Then, a band-adaptive gradient correction model is constructed by exploring the gradient relationship between a PAN image and each band of the MS image, so as to adaptively obtain an accurate spatial structure for the estimated HRMS image. To refine the spatial details, a detail correction constraint is defined based on the parameter transfer by designing a reduced-scale parameter acquisition model. Finally, a unified model is constructed based on the gradient and detail corrections, which is then solved by an alternating direction multiplier method. Both reduced-scale and full-scale experiments are conducted on several datasets. Compared with state-of-the-art pansharpening methods, the proposed method can achieve the best results in terms of fusion quality and has high efficiency. Specifically, our method improves the SAM and ERGAS metrics by 17.6% and 21.2% respectively compared to the traditional approach with the best average values, and improves these two metrics by 4.3% and 10.3% respectively compared to the learning-based approach with the best average values.
13 February 2025 in Pattern Recognition
1 February 2023 in IEEE Transactions on Geoscience and Remote Sensing
Deep learning (DL)-based pansharpening methods have shown great advantages in extracting spectral-spatial features from multispectral (MS) and panchromatic (PAN) images compared with traditional methods. However, most DL-based methods ignore the local inner connection between the source images and high-resolution MS (HRMS) image, which cannot fully extract spectral-spatial information, and attempt to improve the quality of fusion by increasing the complexity of the network. To solve these problems, a lightweight network based on adaptive weighted feature learning (AWFLN) is proposed for pansharpening. Specifically, a novel detail extraction model is first built by exploring the local relationship between HRMS and source images, thereby improving the accuracy of details and the interpretability of the network. Guided by this model, we then design a residual multiple receptive-field structure to fully extract spectral-spatial features of source images. In this structure, an adaptive feature learning block based on spectral-spatial interleaving attention is proposed to adaptively learn the weights of features and improve the accuracy of the extracted details. Finally, the pansharpened result is obtained by a detail injection model in AWFLN. Numerous experiments are carried out to validate the effectiveness of the proposed method. Compared to traditional and state-of-the-art methods, AWFLN performs the best both subjectively and objectively, with high efficiency. The code is available at https://github.com/yotick/AWFLN.
2 August 2021 in IEEE Transactions on Geoscience and Remote Sensing
Deep-learning-based pansharpening methods have achieved remarkable results due to their powerful feature representation ability. However, the existing deep-learning-based pansharpening methods not only lack information exchange and sharing between features of different resolutions but also cannot effectively use the residual information at different levels. These disadvantages may lead to the loss of spatial information and spectral information in the pansharpened image. To address the above problems, we propose a novel dual-stream convolutional neural network with residual information enhancement (DSCNN-RIE) for pansharpening. The proposed network is mainly composed of a set of dual-stream information complementation blocks (DSICBs), which can extract various spatial details at two different resolutions using convolutional filters of various sizes simultaneously, and can transfer complementary information effectively between two different resolutions. Furthermore, to improve the learning ability of the network and enhance the feature extraction, an RIE strategy is presented to stack different levels of residuals into the outputs of cascaded DSICBs. The final pansharpened image is obtained by integrating the extracted features using the shallow feature information of the source images. Experimental results on three datasets demonstrate that DSCNN-RIE outperforms ten other state-of-the-art pansharpening methods in both subjective and objective image-quality evaluations.
30 March 2020 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Pansharpening exploits the high-frequency component (HFC) of panchromatic (PAN) images to restore the spatial-resolution of the corresponding multispectral (MS) image. In this paper, a dual-injection model-based multiband remote sensing image pansharpening method is presented that focuses on how to correctly use the HFC to improve the MS image for obtaining a high-spatial resolution and multispectral (HRMS) image. The model is based on a two-step HFC injection algorithm with two different injection gains. In the first step, an HFC is reconstructed with sparse theory, and an injection gain based on the relationship between PAN and MS images is developed. Employing the previous injection gain and the reconstructed HFC on an upsampling MS image, an improved LRMS (ILRMS) image is then produced. In the second step, another injection gain based on the differences and similarities between the PAN and MS images is designed. With the help of this injection gain, the fusion image is achieved via the adaptive integration of the ILRMS image and the HFC from the PAN image. Experiments confirm that the proposed method is more effective than some popular widely used pansharpening methods.
21 October 2020 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Pansharpening is the process of fusing low spatial resolution multispectral (MS) images with high spatial resolution panchromatic (PAN) images, so as to obtain high spatial resolution multispectral (HRMS) images. In this study, a new pansharpening method based on a joint-guided detail extraction is proposed to maintain the spectral and spatial fidelity of a pansharpened image. First, to obtain details that are highly correlated with an MS image, a new PAN image is constructed and guided by the intensity component of the MS image through a variational model. The construction of the new PAN image improves the correlation between the PAN and MS images, and thus reduces the spectral distortion. The variational model is rapidly solved using the least-squares method. Second, to obtain accurate details from the new PAN image, the extraction of the details is guided by each band of the MS image through a regression model, which can further reduce the spatial distortion. The regression model is effectively solved using the gradient descend method. Finally, the details are injected into the up-sampled MS image to obtain a fused image. Numerous experiments on the proposed approach were conducted and the results were compared with previous state-of-the-art pansharpening methods. The experimental results verify that the proposed method can efficiently achieve high-quality HRMS images.