Hyperspectral (HS) pansharpening aims at fusing a low-resolution HS image with a high-resolution panchromatic (PAN) image to obtain a HS image with both higher spectral and spatial resolutions. However, existing HS pansharpening algorithms are mainly based on multispectral pansharpening approaches, which cannot perfectly restore much spectral information in the continuous spectral bands and much broader spectral range, leading to spectral distortion and spatial blur. In this paper, we develop a new hyperspectral pansharpening network architecture (called Hyper-DSNet) to fully preserve latent spatial details and spectral fidelity via a deep-shallow fusion structure with multi-detail extractor and spectral attention. First, to solve the problem of spatial ambiguity, five types of high-pass filter templates are used to fully extract the spatial details of the PAN image, constructing a so-called multi-detail extractor. Then, a multi-scale convolution module and a deep-shallow fusion structure, which reduces parameters by decreasing the number of output channels as the network goes deeper, is utilized sequentially. In final, a spectral attention module is conducted to preserve the spectrum for a wealth of spectral information of HS images. Visual and quantitative experiments on three commonly used simulated datasets and one full-resolution dataset demonstrate the effectiveness and robustness of the proposed Hyper-DSNet against the recent state-of-the-art hyperspectral pansharpening techniques. Ablation studies and discussions further verify our contributions, e.g., better spectral preservation and spatial detail recovery.

Data augmentation (DA) is an effective way to enrich the richness of data and improve a model’s generalization ability. It has been widely used in many advanced vision tasks (e.g., classification, recognition, etc.), while it can hardly be seen in hyperspectral image (HSI) tasks. In this paper, we analyze whether existing augmentation methods are suitable for the task of HSI denoising and find that the biggest challenge lies in neither losing the spatial information of the original image nor destroying the correlation between the various bands for HSI denoising. Based on this, a new data augmentation method named PatchMask is proposed, which makes the training samples as diverse as possible while preserving the spatial and spectral information. The training data augmented by this method are somewhere between clear and noisy, which can make the network learn more effectively and generalize. Experiments demonstrate that our method outperforms other data augmentation methods, such as the benchmark CutBlur, in enhancing HSI denoising. In addition, the given DA method was used on several popular denoising networks, such as QRNN3D, DnCNN, MPRnet, CBDNet, and HSID-CNN, to verify the effectiveness of the proposed method. The results show that the given DA could increase the value of the PSNR by 0.2∼0.5 dB in various examples.

Pansharpening is an important yet challenging remote sensing image processing task, which aims to reconstruct a high-resolution (HR) multispectral (MS) image by fusing a HR panchromatic (PAN) image and a low-resolution (LR) MS image. Though deep learning (DL)-based pansharpening methods have achieved encouraging performance, they are infeasible to fully utilize the deep semantic features and shallow contextual features in the process of feature fusion for a HR-PAN image and LR-MS image. In this paper, we propose an efficient full-depth feature fusion network (FDFNet) for remote sensing pansharpening. Specifically, we design three distinctive branches called PAN-branch, MS-branch, and fusion-branch, respectively. The features extracted from the PAN and MS branches will be progressively injected into the fusion branch at every different depth to make the information fusion more broad and comprehensive. With this structure, the low-level contextual features and high-level semantic features can be characterized and integrated adequately. Extensive experiments on reduced- and full-resolution datasets acquired from WorldView-3, QuickBird, and GaoFen-2 sensors demonstrate that the proposed FDFNet only with less than 100,000 parameters performs better than other detail injection-based proposals and several state-of-the-art approaches, both visually and quantitatively.

Pansharpening, which attempts to obtain a high-resolution multispectral (HR-MS) image by fusing a panchromatic (PAN) image with a low-resolution multispectral (LR-MS) image, is a critical yet difficult remote sensing image processing task. In this study, we present a novel convolution operation, self-guided spatial-channel adaptive convolution (SSCAConv), for pansharpening. Unlike the reported adaptive convolutions that only focus on spatial details, our SSCAConv also considers channel specificity by generating an individual convolution kernel for each channel patch according to its own content and supplements the inter-channel information by introducing a global bias. We further apply the designed SSCAConv to a simple residual network architecture to construct the image fusion network (SSCANet). Experimental results show that SSCANet outperforms state-of-the-art (SOTA) pansharpening algorithms and achieves better generalization ability with fewer parameters. Additionally, our network also yields the best results when extended to the hyperspectral image super-resolution (HISR) problem. The code is available at https://github.com/Pluto-wei/SSCAConv.

In this paper, we propose a novel dynamic fusion model (DFM) with joint variational and deep priors for the task of hyperspectral image super-resolution (HISR). The given model can benefit from both the advantages of traditional modeling and deep learning methods, thus achieving significant improvements based on existing deep pre-trained models. Specifically, the given model mainly contains two new designed terms, i.e., the weighted spatial fidelity (WSF) term and the deep fusion (DF) term. The WSF term focuses on the spatial recovery of the low-resolution hyperspectral image through the high-resolution multispectral image without the knowledge of the spectral response matrix, thus the proposed DFM can be viewed as a semi-blind model for HISR. Moreover, the DF term relied upon deep fusion with a designed adaptive weight matrix, which can effectively inject the deep priors into the traditional minimization model. Besides, the proposed DFM can be quickly and effectively solved using the alternating direction method of multipliers. Experimental results on widely used datasets demonstrate the superiority of our approach compared with state-of-the-art HISR methods.

Synthetic aperture radars (SARs) require a larger pulse repetition interval to achieve a greater range swath width, which is often addressed by designing non-uniform pulse transmission patterns. However, decoupling the pulse transmission pattern and SAR imaging algorithms in exiting works hinders the full utilization of the anti-aliasing capability of non-uniform transmission during the imaging procedure. To address this issue, this article proposes a joint optimization method of learnable pulse transmission pattern and SAR imaging algorithm, namely, JointSAR. Specifically, we optimize the pulse transmission intervals in a random parameter freezing way and develop a deep unfolding network based on approximate observation model. Simulation and measured data-based experiments verify that the proposed method achieves better SAR imaging quality and maintains wide-swath imaging capability in comparison with existing state-of-the-art methods.