Semantic segmentation of remote sensing images is a fundamental task in geoscience research. However, convolutional neural networks (CNNs) and Transformers have some significant shortcomings. The former are limited by insufficient long-range modeling capabilities, while the latter are hampered by computational complexity. Recently, a novel visual state space (VSS) model represented by Mamba has emerged, capable of modeling long-range relationships with linear computability. In this research, we propose a novel dual-branch network named Remote Sensing Image Semantic Segmentation Mamba (RS ³ Mamba) designed specifically for remote sensing tasks. RS ³ Mamba utilizes VSS blocks to construct an auxiliary branch, providing additional global information to a convolution-based main branch. Moreover, considering the distinct characteristics of the two branches, we introduce a collaborative completion module (CCM) to refine and fuse features from the dual-encoder using a novel adaptive mechanism. Through experiments on two widely used datasets, the proposed RS ³ Mamba was found to outperform the state-of-the-art methods in terms of mIoU with 0.66% on ISPRS Vaihingen and 1.70% on LoveDA Urban, demonstrating its effectiveness and potential. The source code is available at https://github.com/sstary/SSRS.

Accurate semantic segmentation of remote sensing data plays a crucial role in the success of geoscience research and applications. Recently, multimodal fusion-based segmentation models have attracted much attention due to their outstanding performance as compared to conventional single-modal techniques. However, most of these models perform their fusion operation using convolutional neural networks (CNN) or the vision transformer (Vit), resulting in insufficient local-global contextual modeling and representative capabilities. In this work, a multilevel multimodal fusion scheme called FTransUNet is proposed to provide a robust and effective multimodal fusion backbone for semantic segmentation by integrating both CNN and Vit into one unified fusion framework. Firstly, the shallow-level features are first extracted and fused through convolutional layers and shallow-level feature fusion (SFF) modules. After that, deep-level features characterizing semantic information and spatial relationships are extracted and fused by a well-designed Fusion Vit (FVit). It applies Adaptively Mutually Boosted Attention (Ada-MBA) layers and Self-Attention (SA) layers alternately in a three-stage scheme to learn cross-modality representations of high inter-class separability and low intra-class variations. Specifically, the proposed Ada-MBA computes SA and Cross-Attention (CA) in parallel to enhance intra- and cross-modality contextual information simultaneously while steering attention distribution towards semantic-aware regions. As a result, FTransUNet can fuse shallow-level and deep-level features in a multilevel manner, taking full advantage of CNN and transformer to accurately characterize local details and global semantics, respectively. Extensive experiments confirm the superior performance of the proposed FTransUNet compared with other multimodal fusion approaches on two fine-resolution remote sensing datasets, namely ISPRS Vaihingen and Potsdam. The source code in this work is available at https://github.com/sstary/SSRS.

Driven by the rapid development of Earth observation sensors, semantic segmentation using multimodal fusion of remote sensing data has drawn substantial research attention in recent years. However, existing multimodal fusion methods based on convolutional neural networks cannot capture long-range dependencies across multiscale feature maps of remote sensing data in different modalities. To circumvent this problem, this work proposes a crossmodal multiscale fusion network (CMFNet) by exploiting the transformer architecture. In contrast to the conventional early, late, or hybrid fusion networks, the proposed CMFNet fuses information of different modalities at multiple scales using the cross-attention mechanism. More specifically, the CMFNet utilizes a novel cross-modal attention architecture to fuse multiscale convolutional feature maps of optical remote sensing images and digital surface model data through a crossmodal multiscale transformer (CMTrans) and a multiscale context augmented transformer (MCATrans). The CMTrans can effectively model long-range dependencies across multiscale feature maps derived from multimodal data, while the MCATrans can learn discriminative integrated representations for semantic segmentation. Extensive experiments on two large-scale fine-resolution remote sensing datasets, namely ISPRS Vaihingen and Potsdam, confirm the excellent performance of the proposed CMFNet as compared to other multimodal fusion methods.

Semantic segmentation of remote sensing imagery plays a pivotal role in extracting precise information for diverse downstream applications. Recent development of the Segment Anything Model (SAM), an advanced general-purpose segmentation model, has revolutionized this field, presenting new avenues for accurate and efficient segmentation. However, SAM is limited to generating segmentation results without class information. Meanwhile, the segmentation map predicted by current methods generally exhibit excessive fragmentation and inaccuracy of boundary. This paper introduces a streamlined framework designed to leverage the raw output of SAM by exploiting two novel concepts called SAM-Generated Object (SGO) and SAM-Generated Boundary (SGB). More specifically, we propose a novel object consistency loss and further introduce a boundary preservation loss in this work. Considering the content characteristics of SGO, we introduce the concept of object consistency to leverage segmented regions lacking semantic information. By imposing constraints on the consistency of predicted values within objects, the object consistency loss aims to enhance semantic segmentation performance. Furthermore, the boundary preservation loss capitalizes on the distinctive features of SGB by directing the model’s attention to the boundary information of the object. Experimental results on two well-known datasets, ISPRS Vaihingen and LoveDA Urban, demonstrate the effectiveness and broad applicability of the proposed method. The source code for this work is accessible at https://github.com/sstary/SSRS.

Landslide inventory mapping (LIM) plays an important role in hazard assessment and hazard relief. Even though much research has taken place in past decades, there is space for improvements in accuracy and the usability of mapping systems. In this paper, a new landslide inventory mapping framework is proposed based on the integration of the majority voting method and the multiscale segmentation of a postevent images, making use of spatial feature of landslide. Compared with some similar state-of-the-art methods, the proposed framework has three advantages: 1) the generation of LIM is almost automatic; 2) the framework can achieve more accurate results because it takes into account the landslide spatial information in an irregular manner through multisegmentation of the postevent image and object-based majority voting (MV); and 3) it needs less parameter tuning. The proposed framework was applied to four landslide sites on Lantau Island, Hong Kong. Compared with existing methods, including region level set evolution (RLSE), edge level set evolution (ELSE) and change detection Markov random field (CDMRF) methods, quantitative evaluation shows the proposed framework is competitive in terms of Completeness. The framework outperformed RLSE, ELSE, and CDMRF for the four experiments by more than 9% in Correctness and by 8% in Quality. To the authors’ knowledge, this is the first-time that landslide spatial information has been utilized through the integration of multiscale segmentation of postevent image with the MV approach to obtain LIM using high spatial resolution remote sensing images. The approach is also of wide generality and applicable to other kinds of land cover change detection using remote sensing images.