Remote sensing images (RSIs) containing various ground objects have been applied in many fields. To make semantic understanding of RSIs objective and interactive, the task remote sensing visual question answering (VQA) has appeared. Given an RSI, the goal of remote sensing VQA is to make an intelligent agent answer a question about the remote sensing scene. Existing remote sensing VQA methods utilized a nonspatial fusion strategy to fuse the image features and question features, which ignores the spatial information of images and word-level information of questions. A novel method is proposed to complete the task considering these two aspects. First, convolutional features of the image are included to represent spatial information, and the word vectors of questions are adopted to present semantic word information. Second, attention mechanism and bilinear technique are introduced to enhance the feature considering the alignments between spatial positions and words. Finally, a fully connected layer with softmax is utilized to output an answer from the perspective of the multiclass classification task. To benchmark this task, a RSIVQA dataset is introduced in this article. For each of more than 37 000 RSIs, the proposed dataset contains at least one or more questions, plus corresponding answers. Experimental results demonstrate that the proposed method can capture the alignments between images and questions. The code and dataset are available at https://github.com/spectralpublic/RSIVQA .

The land cover classification has been an important task in remote sensing. With the development of various sensors technologies, carrying out classification work with multisource remote sensing (MSRS) data has shown an advantage over using a single type of data. Hyperspectral images (HSIs) are able to represent the spectral properties of land cover, which is quite common for land cover understanding. Light detection and ranging (LiDAR) images contain altitude information of the ground, which is greatly helpful with urban scene analysis. Current HSI and LiDAR fusion methods perform feature extraction and feature fusion separately, which cannot well exploit the correlation of data sources. In order to make full use of the correlation of multisource data, an unsupervised feature extraction-fusion network for HSI and LiDAR, which utilizes feature fusion to guide the feature extraction procedure, is proposed in this article. More specifically, the network takes multisource data as input and directly output the unified fused feature. A multimodal graph is constructed for feature fusion, and graph-based loss functions including Laplacian loss and t-distributed stochastic neighbor embedding (t-SNE) loss are utilized to constrain the feature extraction network. Experimental results on several data sets demonstrate the proposed network can achieve more excellent classification performance than some state-of-the-art methods.

The rapid development of sensor technology has made multi-modal remote sensing data valuable for land cover classification due to its diverse and complementary information. Many feature extraction methods for multi-modal data, combining light detection and ranging (LiDAR) and hyperspectral imaging (HSI), have recognized the importance of incorporating multiple spatial scales. However, effectively capturing both long-range global correlations and short-range local features simultaneously on different scales remains a challenge, particularly in large-scale, complex ground scenes. To address this limitation, we propose a multi-scale graph encoder–decoder network (MGEN) for multi-modal data classification. The MGEN adopts a graph model that maintains global sample correlations to fuse multi-scale features, enabling simultaneous extraction of local and global information. The graph encoder maps multi-modal data from different scales to the graph space and completes feature extraction in the graph space. The graph decoder maps the features of multiple scales back to the original data space and completes multi-scale feature fusion and classification. Experimental results on three HSI-LiDAR datasets demonstrate that the proposed MGEN achieves considerable classification accuracies and outperforms state-of-the-art methods.

Land cover analysis has received significant attention in remote sensing-related fields. To take advantage of multi-modal data, Hyperspectral Images (HSI) and Light Detection and Ranging (LiDAR) are often combined. However, it is difficult to capture intricate local and global spectral-spatial associations between HSI and LiDAR. To exploit the complementary information of multi-modal data, a Spectral-Spatial Graph Network is proposed that integrates HSI and LiDAR data into intricate local and global spectral-spatial associations. Specifically, the network consists of a local module and a global module. The local module uses convolution techniques applied over image patches to preserve the local spatial relationships available in multi-modal data. The global module constructs a spectral-spatial multi-modal graph, which is used to preserve spectral-spatial proximity in multi-modal data. Both the local and global modules are utilized to their utmost capacity to generate the final multi-modal data representation. Experiments on multi-modal remote sensing data sets reveal that the proposed network has attained performance levels comparable to state-of-the-art methods.

Semi-supervised learning networks based on pseudo-label method have been widely applied to multimodal image classification tasks to improve model accuracy when labeled data are scarce. However, it remains a critical challenge for multimodal data to generate high-quality pseudo-labels and effectively leverage unreliable unlabeled samples. Facing the challenge, a semi-supervised multimodal contrastive regularization network (SMCRN) for hyperspectral and LiDAR classification is proposed to fully exploit unlabeled samples. Firstly, to generate rational pseudo-labels for all unlabeled samples, we design a multimodal label assignment strategy (MLAS) to establish a training sample set through cross-modal comparison of superpixel features. Then, substantial class-balance samples are fed into a three-stream semi-supervised network to execute the class prediction, in which the unlabeled samples are divided into unreliable and reliable subsets according to the predicted class probabilities. For the reliably predicted samples, consistency regularization enforces the prediction invariance in an identical sample across different streams for enhancing the model robustness, while the unreliable samples are utilized in contrastive learning by providing their negative features, thereby improving the discriminative ability. Finally, three loss functions are integrated to optimize the network. Extensive experiments on three multimodal remote sensing datasets demonstrate that the proposed SMCRN achieves superior classification accuracy compared to state-of-the-art methods.

Optical instrumentation with reliable performance is essential for the research of exoplanet atmosphere characteristics. However, due to long distances and weak signals, exoplanets are difficult to be imaged by traditional optical systems. To this end, a novel optical system based on transit spectroscopy is proposed in this paper. On the basis of the principle of the transit-spectroscopy method and the astronomical parameters of observed targets, the optional parameter ranges of a dedicated optical system are analyzed. The transit signal-to-noise ratio (SNR) is introduced for the determination of telescope aperture and throughput. Furthermore, an example of the optical system with a space telescope and spectrometer is proposed according to the above optical index, which is proven to meet the performance requirements. The optical system is required to cover the wavelength of 0.5–8 μm and the field of view (FOV) of 27.9${}^{″}$ within the diffraction limit. The collecting aperture should be greater than 2 m, and spectral resolutions of two spectrometer channels should approximately be 100 (2–4 μm) and 30 (4–8 μm). The point-spread function (PSF) of each channel at the minimal wavelength should cover 2 pixels. The telescope and dichroic system provide diffraction-limited input beams with the required aperture, FOV, and wavelength for the spectrometer slits. The simulation results of the optical system show that the spectral resolutions of the dual-channel spectrometer were 111–200 and 43–94. The image points of the spectrometer in each wavelength were smaller than the Airy spot within the slit FOV, and the full width at half-maximum (FWHM) of PSF at ${\lambda }_{min}$ provided 2 pixels of 18 μm sampling. The feasibility of the demonstrated optical parameters is proven by the design.