20 June 2024 in ISPRS Journal of Photogrammetry and Remote Sensing
7 August 2023 in IEEE Transactions on Geoscience and Remote Sensing
Semantic segmentation of Mars scenes has a crucial role in Mars rovers science missions. Current CNN-based composition of U-Net has powerful information extraction capabilities; however, convolutional localization suffers from the limited global context modeling capability. Although Transformer global modeling has performed well, it still encounters obstacles in the extraction and retention of low-level features. This issue is particularly relevant for Martian rocks with their varying shapes, textures, and sizes in Mars scenes. In this paper, we propose a novel Transformer semantic segmentation framework for Martian rock images, called MarsFormer, that consists of an encoder-decoder structure connected through a Feature Enhancement Module (FEM) and a Window Transformer Block (WTB). Specifically, multi-scale hierarchical features are generated by the Mix Transformer encoders, Upgraded-FFN decoder (UFD) fuses and filters features at different scales, preserving the rich local and global contextual information. FEM enhances the inter-multi-scale feature correlation from both spatial and channel perspectives. WTB captures the long-range contexts and preserves the local features. We built two datasets of synthetic and real Martian rocks. The synthetic dataset is SynMars, referencing data from the ZhuRong rover taken from its Virtual Terrain Engine. The other dataset is MarsData-V2, from real Mars scenes, and published recently in our previous study. Extensive experiments conducted on both datasets showed that MarsFormer achieves superiority in Martian rock segmentation, obtaining state-of-the-art performance with favorable computational simplicity. The data is available at: https://github.com/CVIR-Lab/SynMars.
9 January 2023 in IEEE Transactions on Geoscience and Remote Sensing
Martian rock segmentation aims to separate rock pixels from background, which plays a crucial role in downstream tasks, such as traversing and geologic analysis by Mars rovers. The U-Nets have achieved certain results in rock segmentation. However, due to the inherent locality of convolution operations, U-Nets are inadequate in modeling global context and long-range spatial dependencies. Although emerging Transformers can solve this, they suffer from difficulties in extracting and retaining sufficient low-level local information. These shortcomings limit the performance of existing networks for Martian rocks that are variable in shape, size, texture and color. Therefore, we propose RockFormer, the first U-shaped Transformer framework for Mars rock segmentation, consisting of a hierarchical encoder-decoder architecture with a Feature Refining Module (FRM) connected between them. Specifically, the encoder hierarchically generates multi-scale features using an improved vision Transformer (improved-ViT), where both abundant local information and long-range contexts are exploited. The FRM removes less representative features and captures global dependencies between multi-scale features, improving RockFormer’s robustness to Martian rocks with diverse appearances. The decoder is responsible for aggregating these features for pixel-wise rock prediction. For evaluation, we establish two Mars rock datasets, including both real and synthesized images. One is MarsData-V2, an extension of our previously published MarsData collected from real Mars rocks. The other is SynMars, a synthetic dataset sequentially photographed from a virtual terrain built referring to the TianWen-1 dataset. Extensive experiments on the two datasets show the superiority of RockFormer for Martian rock segmentation, achieving state-of-the-art performance with decent computational simplicity.
7 November 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
In future multiagent Mars detection schemes, the Mars helicopter can assist the scientific missions of Mars rovers by providing navigation information and scientific objects. However, Mars surface exhibits a complex topography with diverse objects and similar textures to the background, posing a great challenge for existing CNN-based object detection networks. In this article, we propose a novel deep learning-based object detection framework, AirFormer, for Mars helicopter. AirFormer embeds a new feature-fusion attention module, MAT, which injects various receptive field sizes into labels. This fusion module is capable of capturing the interrelations between objects with each other while simultaneously reducing computational complexity. In addition, we published a synthetic dataset from the viewpoint of the Mars helicopter: SynMars-Air, which refers to the data collected by the ZhuRong rover. Extensive experiments are conducted to validate the performance of AirFormer compared to SOTA methods. The results show that our method achieved the highest accuracy both on synthetic and real Mars landscapes.
6 October 2025 in IEEE Transactions on Geoscience and Remote Sensing
The Martian surface landforms are highly related to the safe landing and traversability of Mars rovers. Furthermore, landforms associated with the presence of water/ice, minerals and biosignatures can provide valuable insights for Mars exploration missions, particularly in relation to the selection of landing or sample collection sites. The small number of Martian landform datasets and the scarcity of labelable landform samples over Mars make the precise mapping of Martian landforms a challenging task. In this article, we propose a stepwise deep feature transfer (SDFT) model for the mapping of Martian landforms with a small number of labeled samples. The SDFT model comprises two transfer steps. In the first transfer step, a deep learning model trained on a large public source dataset from Earth is transferred to a medium sized public dataset from Mars. This transfer is conducted through a standard pre-training and fine-tuning procedure utilizing a linear classifier. In the second transfer step, the model is further transferred to a small number of target datasets on Mars through a pre-training and fine-tuning procedure with a cosine distance classifier. The stepwise training technique mitigates the challenges associated with varying datasets and small training samples. The proposed SDFT model has been validated on two self-built sample sets using images from the Mars Reconnaissance Orbiter’s Context Camera (CTX). It has also been employed for landform mapping in two local regions with small samples to evaluate its effectiveness in comparison with existing state-of-the-art methods.
6 September 2026 in International Journal of Applied Earth Observation and Geoinformation