Pan-sharpening is a significant task in remote sensing image processing, which merges a high-resolution panchromatic (PAN) image and a low-resolution multispectral (MS) image to create a high-resolution MS image. In this paper, we propose a novel deep learning-based MS image pan-sharpening method which combines a Shallow-Deep Convolutional Network (SDCN) and a Spectral Discrimination-based Detail Injection (SDDI) model. SDCN consists of a shallow network and a deep network, which can capture mid-level and high-level spatial features from PAN images. SDDI, inspired by the “Amélioration de la Résolution Spatial par Injection de Structures” (ARSIS) concept, is developed to merge the spatial details extracted by SDCN into MS images with minimal spectral distortion. SDCN and SDDI are collaboratively learned for achieving high spatial resolution MS image and preserving more spectral information. Both the visual assessment and the quantitative assessment results on IKONOS and QuickBird datasets confirmed that the proposed method outperforms several state-of-the-art pan-sharpening methods.

Pan sharpening is used to fuse a low-resolution multispectral (MS) image and a high-resolution panchromatic (PAN) image to obtain a high-resolution MS image. This article proposes PSMD-Net, an end-to-end pan-sharpening method based on a multi-scale dense network. A shallow feature extraction layer (SFEL) extracts the shallow features from the original images, and these are used as an input to a global dense feature fusion (GDFF) network to learn the global features for image reconstruction. A multiscale dense block (MDB) is designed to fully extract the spatial and spectral information from the shallow features in the GDFF network. In the proposed network, multiple MDBs are stacked to extract rich, multi-scale dense hierarchical features, and a global dense connection (GDC) is designed to allow direct connections from the state of the current MDB to all subsequent MDBs to extract more advanced features. The extracted hierarchical features are sent to the global feature fusion layer (GFFL) to adaptively learn the global features for image reconstruction. Finally, global residual learning (GRL) is adopted to force the network to pay more attention to the changing part of the image. We perform experiments on simulated and real data from WorldView-2 and WorldView-3 satellites. Visual and quantitative assessment results demonstrate that PSMD-Net yields higher-resolution fusion images than the state-of-the-art methods.

With the success of ViT (Vision Transformer), Transformer is being increasingly used for hyperspectral image (HSI) classification given its ability to extract global context dependencies. However, existing methods based on transformers tend to classify HSI in the traditional patch-wise manner. Thus, these methods cannot obtain true global features because the inputs of the model are local patches. To solve these problems, a hybrid convolution and ViT network (HCVN) is proposed for HSI classification. HCVN realizes the classification task from the perspective of semantic segmentation, and its input is the entire HSI, which makes it possible to obtain truly meaningful global features. By improving the original ViT, an HCV module is proposed, which enhances the ability of local structure characterization while extracting global features. The HCVN hybrid convolution layer and HCV module realize the extraction and fusion of local and global features. Finally, the dual branch network architecture is used to integrate the spatial and spectral features. Extensive experiments on two datasets verify the effectiveness of the proposed method.

Transfer learning is an effective way to alleviate the problem of insufficient samples in a hyperspectral image (HSI) classification. However, the present transfer learning-based methods usually transfer knowledge from a single source domain, such as the natural image domain. Therefore, these methods cannot simultaneously transfer spectral and spatial knowledge to the target domain in HSIs. Generally, the natural image has rich spatial structure and texture information, while the HSI has abundant spectral information. To better utilize the knowledge learned from natural image datasets and HSI datasets, we proposed a multimodal transfer feature fusion network (MTFFN) for HSI classification. In MTFFN, a dual-branch network structure is designed to transfer the two-modal knowledge from the natural image domain and the source HSI domain to the target domain in two branches, respectively. A multitask learning strategy is adopted to achieve feature fusion. The fused features are used to generate the final classification result. Moreover, a local attention mechanism is designed to extract more meaningful spectral features. Experiments on two public datasets show that the proposed method is effective ( https://github.com/HuaipYan/MTFFN ).