Hyperspectral (HS) pansharpening seeks to integrate low spatial resolution HS (LRHS) images with connected panchromatic (PAN) images to produce high spatial resolution HS (HRHS) images. Traditional pansharpening convolutional neural networks (CNNs) directly map LRHS and PAN images into HRHS images under fixed network parameters, which imply static pansharpening rules. However, real-world HS data are often characterized by spatial variations, and intuitively, the pansharpening rules should be dynamic. To deal with the dilemma, in this article, we develop dynamic HS pansharpening CNNs. We first specify the concepts of dynamic pansharpening and static pansharpening. Then, we propose a learn-to-learn-oriented pansharpening CNN paradigm, which aims to learn a how-to-learn rule to produce spatially adaptive pansharpening rules and comprises three stages of preliminary fusion, scene-sensitive modulation, and spectral reconstruction. Finally, following the paradigm, we design two groups of dynamic pansharpening CNNs (DyPNNs), i.e., internal-connection-based and external-connection-based. They involve various spatial modulations, including spatial affine transform (AT), spatial dynamic convolution (DC), or improved spatial attention (SA), and, thus, consist of six specific DyPNNs: IC-AT-DyPNN, IC-DC-DyPNN, IC-SA-DyPNN, EC-AT-DyPNN, EC-DC-DyPNN, and EC-SA-DyPNN. Experimental results on several HS datasets verify the effectiveness of the proposed DyPNNs in terms of both the spatial reconstruction and spectral fidelity.

Pansharpening aims at fusing a low-resolution multiband optical (MBO) image, such as a multispectral or a hyperspectral image, with the associated high-resolution panchromatic (PAN) image to yield a high spatial resolution MBO image. Though having achieved superior performances to traditional methods, existing convolutional neural network (CNN)-based pansharpening approaches are still faced with two challenges: alleviating the phenomenon of spectral distortion and improving the interpretation abilities of pansharpening CNNs. In this work, we develop a novel spectral-aware pansharpening neural network (SA-PNN). On the one hand, SA-PNN employs a network structure composed of a detail branch and an approximation branch, which is consistent with the detail injection framework; on the other hand, SA-PNN strengthens processing along the spectral dimension by using a spectral-aware strategy, which involves spatial feature transforms (SFTs) coupling the approximation branch with the detail branch as well as 3D convolution operations in the approximation branch. Our method is evaluated with experiments on real-world multispectral and hyperspectral datasets, verifying its excellent pansharpening performance.

Hyperspectral (HS) pansharpening refers to fusing low spatial resolution HS (LRHS) images with the corresponding panchromatic (PAN) images to create high spatial resolution HS (HRHS) images. Most of existing HS pansharpening methods overlook the spatial and spectral imbalance of the ground objects of different types in the observed scenes. To address the dilemma, in this paper we develop a novel tree-structured neural network (Tree-SNet) to form an adaptive spatial-spectral processing for HS pansharpening. The Tree-SNet method maps a CNN onto a hierarchical tree structure, where routing nodes automatically tune the data distributed to tree paths, which is adaptive to the local characteristic of data, while spatial enhancement (SpatE) and spectral enhancement (SpecE) modules are dynamically performed in the tree paths to further strengthen the adaptive processing. The proposed Tree-SNet is evaluated on several datasets, and the experimental results verify its superiority.