Hyperspectral image (HSI) classification has been extensively studied in the context of Earth observation. However, its application in Mars exploration remains limited. Although convolutional neural networks (CNNs) have proven effective in HSI processing, their local receptive fields hinder their ability to capture long-range features. Transformers excel in global modeling and perform well in HSI classification, but they often neglect the effective representation of local spectral and spatial features and tend to be more complex. To address these challenges, we propose a mixed CNN-Transformer network for Mars HSI classification incorporating graph contrastive learning (MCTGCL) to enhance classification performance. Specifically, we introduce an information-enhanced attention module (IEAM) designed to aggregate attention features from multiple perspectives. Additionally, we develop a lightweight dual-branch CNN-Transformer (LDCT) network that efficiently extracts both local and global spectral-spatial features with lower complexity. To improve the discrimination of inter-class features, we apply graph contrastive learning to the topological structure of labeled samples. Furthermore, we annotated three Mars HSI datasets, referred to as HyMars, to validate the effectiveness of our proposed MCTGCL. Comprehensive experimental results across different amounts of labeled samples consistently demonstrate the superiority of the method. The source code is available at https://github.com/B-Xi/TGRS_2025_MCTGCL.
Planetary rover systems need to perform terrain segmentation to identify feasible driving areas and surround obstacles, which falls into the research area of semantic segmentation. Recently, deep learning (DL)-based methods were proposed and achieved great performance for semantic segmentation. However, due to the on-board processor platform’s strict comstraints on computational complexity and power consumption, existing DL approaches are almost impossible to be deployed on satellites under the burden of extensive computation and large model size. To fill this gap, this paper targeted studying effective and efficient Martian terrain segmentation solutions that are suitable for on-board satellites. In this article, we propose a lightweight ViT-based terrain segmentation method, namely, SegMarsViT. In the encoder part, the mobile vision transformer (MViT) block in the backbone extracts local–global spatial and captures multiscale contextual information concurrently. In the decoder part, the cross-scale feature fusion modules (CFF) further integrate hierarchical context information and the compact feature aggregation module (CFA) combines multi-level feature representation. Moreover, we evaluate the proposed method on three public datasets: AI4Mars, MSL-Seg, and S5Mars. Extensive experiments demonstrate that the proposed SegMarsViT was able to achieve 68.4%, 78.22%, and 67.28% mIoU on the AI4Mars-MSL, MSL-Seg, and S5Mars, respectively, under the speed of 69.52 FPS.
9 September 2024 in IEEE Transactions on Geoscience and Remote Sensing
Land use/land cover classification with multimodal data has attracted increasing attention. For hyperspectral images (HSI) and light detection and ranging (LiDAR) data, the combination of them can make the classification more accurate and robust. However, how to effectively utilize their respective strengths and integrate them with the classification task is still a challenging problem. In this paper, a multilevel attention dynamic-scale network (MADNet) is proposed. Firstly, in the feature extraction stage, the two modalities are divided into two branches with different scales, which are then fed into the convolutional neural networks (CNN) to learn shallow features. Then, considering the characteristics of the HSI, a spectral angle attention module (SAAM) with low-level attention is designed to highlight surrounding pixels that have similar spectra to the central pixel of the patch. After that, a dynamic-scale selection module (DSSM) is proposed to screen an appropriate scale for the patches by pixel similarity analysis. Next, combining the Transformer and the CNN, a global-local cross-attention module (GLCAM) is devised to investigate the fused deep-level multimodal features. Distinct from the vanilla Transformer, the GLCAM deploys a distance-weight operator to decrease the redundancies at long distances and effectively reduce misclassifications. Extensive experiments on three paired HSI and LiDAR datasets demonstrate that the proposed MADNet has certain advantages over the existing methods.
The limited resource of Geostationary Earth Orbit (GEO) is precious and most telecommunication, weather and navigational satellites are placed in this orbit. In order to guarantee the safety and health of active satellites, advanced surveillance and warning of unknown space targets such as space debris are crucial. However, space object detection still remains a very challenging problem because of the weak target characteristics and complex star background. To solve this problem, we conduct a deep-learning-based framework called PP-YOLOv2 for single-frame object detection and design a post-processing algorithm named CFS for further candidate filtration and supplement. First, we transform the label information and generate the according bounding boxes to train the PP-YOLOv2 detector to extract candidate coordinates for each frame. Then, the CFS technique is designed as an effective post-processing procedure to obtain the eventual prediction results. Experiments were conducted over a dataset from the Kelvins SpotGEO challenge, which demonstrate the effectiveness and the comparable detection performance of our proposed pipeline. Finally, the deployment results on NVIDIA Jetson Nano show that the proposed method has a competitive application prospect for a space target monitoring system.
The fusion of hyperspectral imagery (HSI) and light detection and ranging (LiDAR) data for classification has received widespread attention and has led to significant progress in research and remote sensing applications. However, existing common CNN architectures suffer from the significant drawback of not being able to model remote sensing images globally, while transformer architectures are not able to capture local features effectively. To address these bottlenecks, this paper proposes a classification framework for multisource remote sensing image fusion. First, a spatial and spectral feature projection network is constructed based on parallel feature extraction by combining HSI and LiDAR data, which is conducive to extracting joint spatial, spectral, and elevation features from different source data. Furthermore, in order to construct local–global nonlinear feature mapping more flexibly, a network architecture coupling together multiscale convolution and a multiscale vision transformer is proposed. Moreover, a plug-and-play nonlocal feature token aggregation module is designed to adaptively adjust the domain offsets between different features, while a class token is employed to reduce the complexity of high-dimensional feature fusion. On three open-source remote sensing datasets, the performance of the proposed multisource fusion classification framework improves about 1% to 3% over other state-of-the-art algorithms.