The rapid development of sensor technology has made multi-modal remote sensing data valuable for land cover classification due to its diverse and complementary information. Many feature extraction methods for multi-modal data, combining light detection and ranging (LiDAR) and hyperspectral imaging (HSI), have recognized the importance of incorporating multiple spatial scales. However, effectively capturing both long-range global correlations and short-range local features simultaneously on different scales remains a challenge, particularly in large-scale, complex ground scenes. To address this limitation, we propose a multi-scale graph encoder–decoder network (MGEN) for multi-modal data classification. The MGEN adopts a graph model that maintains global sample correlations to fuse multi-scale features, enabling simultaneous extraction of local and global information. The graph encoder maps multi-modal data from different scales to the graph space and completes feature extraction in the graph space. The graph decoder maps the features of multiple scales back to the original data space and completes multi-scale feature fusion and classification. Experimental results on three HSI-LiDAR datasets demonstrate that the proposed MGEN achieves considerable classification accuracies and outperforms state-of-the-art methods.

Semi-supervised learning networks based on pseudo-label method have been widely applied to multimodal image classification tasks to improve model accuracy when labeled data are scarce. However, it remains a critical challenge for multimodal data to generate high-quality pseudo-labels and effectively leverage unreliable unlabeled samples. Facing the challenge, a semi-supervised multimodal contrastive regularization network (SMCRN) for hyperspectral and LiDAR classification is proposed to fully exploit unlabeled samples. Firstly, to generate rational pseudo-labels for all unlabeled samples, we design a multimodal label assignment strategy (MLAS) to establish a training sample set through cross-modal comparison of superpixel features. Then, substantial class-balance samples are fed into a three-stream semi-supervised network to execute the class prediction, in which the unlabeled samples are divided into unreliable and reliable subsets according to the predicted class probabilities. For the reliably predicted samples, consistency regularization enforces the prediction invariance in an identical sample across different streams for enhancing the model robustness, while the unreliable samples are utilized in contrastive learning by providing their negative features, thereby improving the discriminative ability. Finally, three loss functions are integrated to optimize the network. Extensive experiments on three multimodal remote sensing datasets demonstrate that the proposed SMCRN achieves superior classification accuracy compared to state-of-the-art methods.

Multi-channel common-aperture optical systems, which excel at simultaneous multi-spectral information acquisition, are widely used for image fusion. However, complex systems for long-distance multi-band detection suffer from difficulties in assembly and adjustment and light vignetting. To resolve this, the paper proposes a modular design method that splits the optical path into independent modules: the common-aperture optical path adopts an off-axis reflective beam-shrinking structure to extend the focal length and ensure 100% light input, compared with coaxial multi-channel common-aperture systems. The relay optical path of each spectral channel uses a continuous zoom design for smooth detection–recognition switching. Based on the method, a three-channel common-aperture system is developed integrating visible light (VIS), short-wave infrared (SWIR), and mid-wave infrared (MWIR). The modulation transfer function (MTF) and wavefront distribution of the common-aperture optical path approach the diffraction limit. After integration with the relay optical paths, the system, without global optimization, can achieve the following performance: the root mean square (RMS) across the full field of view (FOV) at different focal lengths for each channel is smaller than the detector pixel size (3.45 μm for VIS, 15 μm for SWIR/MWIR); the MTF exceeds 0.2 at the cutoff frequency. Subsequently, the results of the tolerance analysis verify the feasibility of the design for each module and the advantage of the modular layout in the assembly and adjustment of the system. Finally, the paper discusses the influence of parallel plates on the wavefront distortion of the system and proposes optimization thinking using freeform surfaces. The design results of the study validate the feasibility of the modular layout in simplifying the design and assembly of multi-channel common-aperture optical systems.

Capturing the characteristics of exoplanetary atmospheres (CEA) through transit spectroscopy (TS) holds profound implications for our understanding of planetary formation and evolution. However, TS, the method employed for detecting CEA, indirectly extracts these characteristics from the subtle variations in stellar spectra during the transit process, necessitating a high level of observational stability in optical instrumentation. To mitigate observational errors in spectral energy within the optical system, this dissertation delves into the optimal design of a high-stability optical system tailored for atmospheric spectra in transit observations. Initially, a theoretical model of transit signal-to-noise ratios (S/Ns) catered to the EAC retrievals is formulated based on transit observation strategies. Subsequently, the optimal parameters and design approach for the optical system are explored through an analysis of the optical factors influencing S/N. Leveraging an observation simulator for optical instruments, the detection feasibility of the optimized optical system for capturing CEA is validated.

Deep learning has significantly advanced hyperspectral image (HSI) and synthetic aperture radar (SAR) joint classification, achieving promising land-cover mapping performance. However, most existing methods rely heavily on dense pixel-level annotations, which are costly and time-consuming to acquire in practical multimodal remote sensing applications. In this article, we propose a novel semi-supervised framework, called Dual-Branch Multimodal Scale-Interaction Transformer (DMSIT), for HSI-SAR joint classification with limited labeled samples. Specifically, the proposed DMSIT first employs dual-stream convolutional encoders coupled with a Cross-Modal Feature Separation (CMFS) module. The CMFS module explicitly disentangles modality-shared classification features from sample-specific private representations, thereby effectively reducing cross-modal feature interference. Subsequently, the separated shared features are transformed into token sequences and processed by a Multimodal Scale-Interaction Transformer (MSIT). By executing scale-aware attention, MSIT captures global cross-modal dependencies and adaptively integrates complementary multimodal features, compensating for the limited receptive fields of local convolutions. Furthermore, to exploit unlabeled data, we introduce a Graph-Guided Pseudo-Labeling (GGPL) strategy. By leveraging graph topological proximity rather than isolated prediction confidences, GGPL selects reliable pseudo-labels, effectively mitigating class imbalance and suppressing error accumulation. Extensive experiments on three datasets demonstrate the effectiveness and superiority of the proposed DMSIT in comparison with several state-of-the-art methods.

In this paper, we explore the spiking encoding methodology within spiking neural networks for affective state recognition, deriving inspiration from the principles of quantum entanglement. A pioneering encoding strategy is proposed based on the strategic utilization of the quantum mechanical phenomenon of entanglement. By integrating quantum mechanisms into the spike-encoding pipeline, we aim to match the accuracy of existing encoders on emotion-classification tasks while retaining the inherently low-power advantage of spiking neural networks. Notably, leveraging the superposition of quantum bits and their potential quantum entanglement of adjacent values in feature space during encoding calculations, this quantum-inspired encoding paradigm holds substantial promise for augmenting information processing capabilities in brain-like neural networks. Through quantum observation, we derive spike trains characterized by quantum states, thereby establishing a foundation for experimental validation and subsequent investigative pursuits. We conducted experiments on emotion recognition and validated the effectiveness of our method.