Convolutional neural networks (CNNs) have been attracting increasing attention in hyperspectral (HS) image classification due to their ability to capture spatial-spectral feature representations. Nevertheless, their ability in modeling relations between the samples remains limited. Beyond the limitations of grid sampling, graph convolutional networks (GCNs) have been recently proposed and successfully applied in irregular (or nongrid) data representation and analysis. In this article, we thoroughly investigate CNNs and GCNs (qualitatively and quantitatively) in terms of HS image classification. Due to the construction of the adjacency matrix on all the data, traditional GCNs usually suffer from a huge computational cost, particularly in large-scale remote sensing (RS) problems. To this end, we develop a new minibatch GCN (called miniGCN hereinafter), which allows to train large-scale GCNs in a minibatch fashion. More significantly, our miniGCN is capable of inferring out-of-sample data without retraining networks and improving classification performance. Furthermore, as CNNs and GCNs can extract different types of HS features, an intuitive solution to break the performance bottleneck of a single model is to fuse them. Since miniGCNs can perform batchwise network training (enabling the combination of CNNs and GCNs), we explore three fusion strategies: additive fusion, elementwise multiplicative fusion, and concatenation fusion to measure the obtained performance gain. Extensive experiments, conducted on three HS data sets, demonstrate the advantages of miniGCNs over GCNs and the superiority of the tested fusion strategies with regard to the single CNN or GCN models. The codes of this work will be available at https://github.com/danfenghong/IEEE_TGRS_GCN for the sake of reproducibility.

Hyperspectral (HS) images are characterized by approximately contiguous spectral information, enabling the fine identification of materials by capturing subtle spectral discrepancies. Due to their excellent locally contextual modeling ability, convolutional neural networks (CNNs) have been proven to be a powerful feature extractor in HS image classification. However, CNNs fail to mine and represent the sequence attributes of spectral signatures well due to the limitations of their inherent network backbone. To solve this issue, we rethink HS image classification from a sequential perspective with transformers and propose a novel backbone network called SpectralFormer . Beyond bandwise representations in classic transformers, SpectralFormer is capable of learning spectrally local sequence information from neighboring bands of HS images, yielding groupwise spectral embeddings. More significantly, to reduce the possibility of losing valuable information in the layerwise propagation process, we devise a cross-layer skip connection to convey memory-like components from shallow to deep layers by adaptively learning to fuse “soft” residuals across layers. It is worth noting that the proposed SpectralFormer is a highly flexible backbone network, which can be applicable to both pixelwise and patchwise inputs. We evaluate the classification performance of the proposed SpectralFormer on three HS datasets by conducting extensive experiments, showing the superiority over classic transformers and achieving a significant improvement in comparison with state-of-the-art backbone networks. The codes of this work will be available at https://github.com/danfenghong/IEEE_TGRS_SpectralFormer for the sake of reproducibility.

Classification and identification of the materials lying over or beneath the earth's surface have long been a fundamental but challenging research topic in geoscience and remote sensing (RS), and have garnered a growing concern owing to the recent advancements of deep learning techniques. Although deep networks have been successfully applied in single-modality-dominated classification tasks, yet their performance inevitably meets the bottleneck in complex scenes that need to be finely classified, due to the limitation of information diversity. In this work, we provide a baseline solution to the aforementioned difficulty by developing a general multimodal deep learning (MDL) framework. In particular, we also investigate a special case of multi-modality learning (MML)--cross-modality learning (CML) that exists widely in RS image classification applications. By focusing on ``what,'' ``where,'' and ``how'' to fuse, we show different fusion strategies as well as how to train deep networks and build the network architecture. Specifically, five fusion architectures are introduced and developed, further being unified in our MDL framework. More significantly, our framework is not only limited to pixel-wise classification tasks but also applicable to spatial information modeling with convolutional neural networks (CNNs). To validate the effectiveness and superiority of the MDL framework, extensive experiments related to the settings of MML and CML are conducted on two different multimodal RS data sets. Furthermore, the codes and data sets will be available at https://github.com/danfenghong/IEEE_TGRS_MDL-RS, contributing to the RS community.

As a list of remotely sensed data sources is available, how to efficiently exploit useful information from multisource data for better Earth observation becomes an interesting but challenging problem. In this paper, the classification fusion of hyperspectral imagery (HSI) and data from other multiple sensors, such as light detection and ranging (LiDAR) data, is investigated with the state-of-the-art deep learning, named the two-branch convolution neural network (CNN). More specific, a two-tunnel CNN framework is first developed to extract spectral-spatial features from HSI; besides, the CNN with cascade block is designed for feature extraction from LiDAR or high-resolution visual image. In the feature fusion stage, the spatial and spectral features of HSI are first integrated in a dual-tunnel branch, and then combined with other data features extracted from a cascade network. Experimental results based on several multisource data demonstrate the proposed two-branch CNN that can achieve more excellent classification performance than some existing methods.

Hyperspectral anomaly detection (HAD) aims to identify anomalous targets that deviate from the surrounding background in unlabeled hyperspectral images (HSIs). Most existing deep networks that exploit reconstruction errors to detect anomalies are prone to fit anomalous pixels, thus yielding small reconstruction errors for anomalies, which is not favorable for separating targets from HSIs. To achieve a superior background reconstruction network for HAD purposes, this article proposes a self-supervised blind-block network (termed BockNet) with a guard window. BockNet creates a blind-block (guard window) in the center of the network’s receptive field, rendering it unable to see the information inside the guard window when reconstructing the central pixel. This process seamlessly embeds a sliding dual-window model into our BockNet, in which the inner window is the guard window and the outer window is the receptive field outside the guard window. Naturally, BockNet utilizes only the outer window information to predict/reconstruct the central pixel of the perceptive field. During the reconstruction of pixels inside anomalous targets of varying sizes, the targets typically fall into the guard window, weakening the contribution of anomalies to the reconstruction results so that those reconstructed pixels converge to the background distribution of the outer window area. Accordingly, the reconstructed HSI can be deemed as a pure background HSI, and the reconstruction error of anomalous pixels will be further enlarged, thus improving the discrimination ability of the BockNet model for anomalies. Extensive experiments on four datasets illustrate the competitive and satisfactory performance of our BockNet compared to other state-of-the-art detectors.

Due to advances in remote sensing satellite imaging and image processing technologies and their wide applications, intelligent remote sensing satellites are facing an opportunity for rapid development. The key technologies, standards, and laws of intelligent remote sensing satellites are also experiencing a series of new challenges. Novel concepts and key technologies in the intelligent hyperspectral remote sensing satellite system have been proposed since 2011. The aim of these intelligent remote sensing satellites is to provide real-time, accurate, and personalized remote sensing information services. This article reviews the current developments in new-generation intelligent remote sensing satellite systems, with a focus on intelligent remote sensing satellite platforms, imaging payloads, onboard processing systems, and other key technological chains. The technological breakthroughs and current defects of intelligence-oriented designs are also analyzed. Intelligent remote sensing satellites collect personalized remote sensing data and information, with real-time data features and information interaction between remote sensing satellites or between satellites and the ground. Such developments will expand the use of remote sensing applications beyond government departments and industrial users to a massive number of individual users. However, this extension faces challenges regarding privacy protection, societal values, and laws regarding the sharing and distribution of data and information.