Although the collaborative use of hyperspectral images (HSIs) and LiDAR data in land cover classification tasks has demonstrated significant importance and potential, several challenges remain. Notably, the heterogeneity in cross-modal information integration presents a major obstacle. Furthermore, most existing research relies heavily on category names, neglecting the rich contextual information from language descriptions. Visual-language pretraining (VLP) has achieved notable success in image recognition within natural domains by using multimodal information to enhance training efficiency and effectiveness. VLP has also shown great potential for land cover classification in remote sensing. This paper introduces a dual-sensor multimodal semantic collaborative classification network (DSMSC2N). It uses large language models (LLMs) in an instruction-driven manner to generate land cover category descriptions enriched with domain-specific knowledge in remote sensing. This approach aims to guide the model to accurately focus on and extract key features. Simultaneously, we integrate and optimize the complementary relationship between HSI and LiDAR data, enhancing the separability of land cover categories and improving classification accuracy. We conduct comprehensive experiments on benchmark datasets like Houston 2013, Trento, and MUUFL Gulfport, validating DSMSC2N’s effectiveness compared to various baseline methods.
In view of the complexity and diversity of hyperspectral images (HSIs), the classification task has been a major challenge in the field of remote sensing image processing. Hyperspectral classification (HSIC) methods based on neural architecture search (NAS) is a current attractive frontier that not only automatically searches for neural network architectures best suited to the characteristics of HSI data, but also avoids the possible limitations of manual design of neural networks when dealing with new classification tasks. However, the existing NAS-based HSIC methods have the following limitations: (1) the search space lacks efficient convolution operators that can fully extract discriminative spatial–spectral features, and (2) NAS based on traditional differentiable architecture search (DARTS) has performance collapse caused by unfair competition. To overcome these limitations, we proposed a neural architecture search method with receptive field spatial–spectral attention (RFSS-NAS), which is specifically designed to automatically search the optimal architecture for HSIC. Considering the core needs of the model in extracting more discriminative spatial–spectral features, we designed a novel and efficient attention search space. The core component of this innovative space is the receptive field spatial–spectral attention convolution operator, which is capable of precisely focusing on the critical information in the image, thus greatly enhancing the quality of feature extraction. Meanwhile, for the purpose of solving the unfair competition issue in the traditional differentiable architecture search (DARTS) strategy, we skillfully introduce the Noisy-DARTS strategy. The strategy ensures the fairness and efficiency of the search process and effectively avoids the risk of performance crash. In addition, to further improve the robustness of the model and ability to recognize difficult-to-classify samples, we proposed a fusion loss function by combining the advantages of the label smoothing loss and the polynomial expansion perspective loss function, which not only smooths the label distribution and reduces the risk of overfitting, but also effectively handles those difficult-to-classify samples, thus improving the overall classification accuracy. Experiments on three public datasets fully validate the superior performance of RFSS-NAS.
31 October 2025 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Remote sensing multimodal data integrates information from various sensors, providing detailed characterization and reliable monitoring capabilities for Earth observation tasks. This data is crucial in environmental surveillance, disaster assessment, and related domains. However, current state-of-the-art methods predominantly rely on category names within prompts. These methods demonstrate reduced effectiveness when processing ambiguous or poorly defined category designations, thus compromising classification accuracy. To overcome these limitations, we propose the Composite Text Prompt Multimodal Synergy Network (CTPMSN). This architecture integrates a text encoder with a remote sensing visual joint encoder and implements dual-level composite text supervision at both categorical and content strata. Our approach enhances inter-class discriminability while effectively capturing intra-class variations. Additionally, we introduce the Text-Visual Prompt Alignment Module (TVPAM) to optimize the association between text prompts and visual features, thereby strengthening their correspondence. Extensive experiments on three benchmark datasets demonstrate the effectiveness of the proposed method, achieving overall accuracies of 93.99 % on MUUFL Gulfport, 93.74 % on Houston 2013, and 91.56 % on Augsburg.
Hyperspectral images(HSIs) have rich spatial and spectral information, which provide a strong basis for distinguishing different landcover objects. The HSI classification methods based on deep learning have achieved good performance. Recently, several Neural Architecture Search (NAS) algorithms have been proposed for HSI to further improve the accuracy to a new level. In this paper, the neural structure search method based onmultiple attention mechanism search space is proposed for hyperspectral images. In the search process, the network selects attention mechanism operation at an appropriate layer, which can help the network focus on useful features and discard useless features. In our experiments, we have verified the superiority of this method on two hyperspectral datasets.
Light Detection and Ranging (LiDAR) technology, as an efficient data acquisition method, is widely used in various practical scenarios. Among them, the classification of LiDAR-generated gridded digital surface models (LiDAR-DSM) constitutes a key aspect of LiDAR data post-processing. However, how to efficiently extract long-range dependency features and high-frequency details from LiDAR data, especially their decisive role in accurately distinguishing land cover classes, remains an urgent problem to be solved. Traditional attention mechanisms perform well in local feature extraction but are inadequate in utilizing global contextual information, especially when dealing with elevation data with wide variation ranges and complex structures. In view of the above issues, this paper introduces an innovative method named the Spectrum-LiDAR Network, which is based on the Transformer encoder architecture and integrates a frequency-domain self-attention mechanism at its core. By implementing the Fourier transform, the data is converted to the frequency domain, thereby revealing and learning the complex dependency structures in LiDAR data that are difficult to intuitively recognize in the time domain. The Spectrum-LiDAR Network not only deepens the deep learning of elevation information but also significantly enhances the model's ability to recognize subtle differences in various terrains and land cover types through global frequency domain analysis, thereby greatly improving the performance and accuracy of LiDAR data classification. This provides a novel and efficient solution for the recognition of complex land features. Experimental evaluations demonstrate that the Spectrum-LiDAR Network can provide comparable or even superior classification results compared to the current state-of-the-art methods, further validating the effectiveness of this approach.