In recent years, the fusion-based classification of hyperspectral image (HSI) and light detection and ranging (LiDAR) data has garnered increasing attention from researchers. However, significant image disparities exist between HSI and LiDAR data because of their distinct imaging mechanisms, which limits the fusion of HSI and LiDAR data. Therefore, establishing effective interaction and fusion between HSI and LiDAR data is crucial. Moreover, the classification of multimodal data with limited labeled instances is another challenging task. To address these issues, we propose a contrastive learning enhanced adaptive multimodal fusion network (CAMFNet) for the joint classification of HSI and LiDAR data. CAMFNet introduces a novel semantic similarity contrastive loss that fully utilizes a large amount of unlabeled data to learn potential complementary information between different modalities. In addition, a hierarchical fusion strategy is adopted to fuse multimodal features. At the shallow feature fusion stage, a cross-modal feature interaction fusion module is proposed to guide the extraction of multimodal features and enhance the complementary information through cross-modal interaction. At the same time, adaptive feature fusion is also used to dynamically assign corresponding weights to different modal features, achieving finer fusion. At the deep feature fusion stage, a channel split and integration block is constructed to further optimize the fused spectral-spatial and elevation features via a negative feedback mechanism. The effectiveness of CAMFNet is validated through experiments on three standard datasets, which demonstrate that CAMFNet outperforms state-of-the-art methods.

Road extraction techniques based on remote sensing image have significantly advanced. Currently, fully supervised road segmentation neural networks based on remote sensing images require a significant number of densely labeled road samples, limiting their applicability in large-scale scenarios. Consequently, semi-supervised methods that utilize fewer labeled data have gained increasing attention. However, the imbalance between a small quantity of labeled data and a large volume of unlabeled data leads to local detail errors and overall cognitive mistakes in semi-supervised road extraction. To address this challenge, this paper proposes a novel consistency self-training semi-supervised method (CSSnet), which effectively learns from a limited number of labeled data samples and a large amount of unlabeled data. This method integrates self-training semi-supervised segmentation with semi-supervised classification. The semi-supervised segmentation component relies on an enhanced generative adversarial network for semantic segmentation, which significantly reduces local detail errors. The semi-supervised classification component relies on an upgraded mean-teacher network to handle overall cognitive errors. Our method exhibits excellent performance with a modest amount of labeled data. This study was validated on three separate road datasets comprising high-resolution remote sensing satellite images and UAV photographs. Experimental findings showed that our method consistently outperformed state-of-the-art semi-supervised methods and several classic fully supervised methods.

Multi-source unsupervised domain adaptation (MUDA) has demonstrated its effectiveness in improving model performance for remote sensing (RS) scene classification, particularly in cases where the target domain lacks labeled data. However, most current methods based on convolutional neural networks (CNNs) or transformers fail to fully exploit the features within each source domain, which benefits classification accuracy. Additionally, these methods frequently overlook the varying levels of similarity between domains, limiting the potential of MUDA. To alleviate this limitation, we propose Hy-MSDA, a hybrid CNN-Transformer with consistency learning and dynamic weighting for multi-source unsupervised domain adaptation, which fully explores and utilizes valuable information from multiple sources at both the feature and decision levels. To achieve a better alignment of categories among domains, the consistency learning module aims to learn domain-invariant features, maintaining consistency in global high-level features and rich semantic information at the feature level. At the decision level, the dynamic weighting strategy balances the contribution of each source domain by considering their varying importance levels. These two components are closely interconnected and mutually reinforce each other, resulting in improved cross-domain collaboration of Hy-MSDA for domain adaptation. Experimental results on the RS classification datasets clearly demonstrate that Hy-MSDA outperforms the state-of-the-art methods, achieving a significant enhancement of 2%-7% in classification accuracy. Moreover, experiments on RS segmentation datasets reveal that Hy-MSDA performs comparably to supervised methods, further validating its effectiveness of Hy-MSDA in segmentation tasks. The source code is available at https://github. com/phaeton2017/Hy-MSDA.

Extracting roads from remote sensing images holds significant practical value across fields like urban planning, traffic management, and disaster monitoring. Current Convolutional Neural Network (CNN) methods, praised for their robust local feature learning enabled by inductive biases, deliver impressive results. However, they face challenges in capturing global context and accurately extracting the linear features of roads due to their localized receptive fields. To address these shortcomings of traditional methods, this paper proposes a novel parallel encoder architecture that integrates a CNN Encoder Module (CEM) with a Transformer Encoder Module (TEM). The integration combines the CEM’s strength in local feature extraction with the TEM’s ability to incorporate global context, achieving complementary advantages and overcoming limitations of both Transformers and CNNs. Furthermore, the architecture also includes a Linear Convolution Module (LCM), which uses linear convolutions tailored to the shape and distribution of roads. By capturing image features in four specific directions, the LCM significantly improves the model’s ability to detect and represent global and linear road features. Experimental results demonstrate that our proposed method achieves substantial improvements on the German-Street Dataset and the Massachusetts Roads Dataset, increasing the Intersection over Union (IoU) of road class by at least 3% and the overall F1 score by at least 2%.