17 July 2024 in IEEE Transactions on Geoscience and Remote Sensing
Semantic segmentation of Remote Sensing (RS) images refers to labeling each pixel with a class to identify objects or land cover types. Existing mainstream spatial-domain semantic segmentation methods are mainly categorized into Convolutional Neural Network (CNN)-based and Vision Transformer (ViT)-based approaches. The former excels at capturing local features, while the latter is adept at extracting global features. Several recent approaches consider combining CNN and ViT to efficiently capture local and global features. However, these approaches still struggle to capture complete features of the RS images, resulting in inaccurate segmentation. To address this issue, we introduce the Fast Fourier Transform (FFT), which transforms images into the frequency domain for feature extraction, acquiring the image-size receptive field that can complement spatial-domain methods. Based on this, we propose a Boundary-aware spatial and frequency Dual-domain Transformer, termed Dual-domain Transformer. Specifically, our Dual-domain Transformer incorporates a Dual-domain Mixer (DualM), where the spatial-domain branch combines depthwise convolution and the attention mechanism to extract local and global features effectively, while the frequency-domain branch uses FFT to extract image-size features. The two branches complement each other, enabling a more comprehensive feature extraction of RS images. Meanwhile, a boundary-guided training strategy utilizing a Boundary-Aware Module (BAM) is devised to constrain the model extract and predict boundary detail texture, which is an auxiliary task. Additionally, the decoder incorporates a Scale-feature Fusion Module (SFM) for adaptive information fusion between the encoder and decoder. Comprehensive experiments on the Zeebrugge and ISPRS datasets, including Vaihingen and Potsdam, showcase that the Dual-domain Transformer significantly outperforms state-of-the-art (SOTA) methods.
18 February 2026 in IEEE Transactions on Circuits and Systems for Video Technology
Videos captured under real-world adverse weather conditions typically suffer from uncertain hybrid weather efforts. However, existing algorithms can only remove one type of weather degradation at a time and deal with different weather conditions with separate models, thus may fail to handle real-world stochastic hybrid scenarios. Besides, the model training is also infeasible due to the lack of paired video data to characterize the coexistence of multiple weather. To ameliorate the aforementioned issue, we propose a novel unified model, dubbed UniWRV, to remove multiple heterogeneous video weather degradations in an all-in-one fashion. Specifically, to tackle degenerate spatial feature heterogeneity, we propose a tailored weather prior guided module that queries exclusive priors for different instances as prompts to steer spatial feature characterization. To tackle degenerate temporal feature heterogeneity, we propose a dynamic routing aggregation module that can automatically select optimal fusion paths for different instances to dynamically integrate temporal features. Furthermore, we propose a real-world adaptation training scheme that leverages CLIP priors to provide semantic supervision for unlabeled real-world weather-degraded videos, thereby enabling the model to better cope with the diverse and complex real-world weather conditions. Additionally, we managed to construct a new synthetic video dataset, termed HWVideo, for learning and benchmarking multiple hybrid adverse weather removal, which contains 15 hybrid weather conditions with a total of 1500 adverse-weather/clean paired video clips. Real-world hybrid weather videos are also collected to facilitate model generalizability. Comprehensive experiments demonstrate that our UniWRV exhibits robust and superior adaptation capability in multiple heterogeneous degradations learning scenarios, including various generic video restoration tasks beyond weather removal.
15 September 2025 in IEEE Transactions on Geoscience and Remote Sensing
Semantic segmentation in Remote Sensing (RS) involves classifying each pixel of an image into predefined categories. Despite existing deep learning-based methods having significantly advanced semantic segmentation performance, two key challenges remain: (1) these methods are task-specific and exhibit limited generalization across different RS scenario tasks, and (2) they require prior knowledge of the task types to select the appropriate model. To address the above challenges, we propose PromptSeg, a prompt learning method for universal RS semantic segmentation based on the Segment Anything Model (SAM). Specifically, (1) to achieve efficient adaptation to diverse RS tasks with minimal fine-tuning, PromptSeg leverages the unparalleled generalization capabilities of SAM. (2) we propose task-specific prompts that encode essential image features and key distinctions to reduce the reliance on prior knowledge. These task-specific prompts consist of two implicit prompts and one explicit prompt: the Task-related Implicit Prompt (TI-Prompt), which learns task scenario types; the Feature-aware Implicit Prompt (FI-Prompt), which captures essential image features; and the Explicit Prompt (E-Prompt), which facilitates accurate boundary identification and delineation. To effectively integrate these prompts, we introduce the Prompt Learning Module (PLM), which coordinates their fusion to enhance segmentation performance. Extensive experiments demonstrate the superior performance of PromptSeg across various scenarios, including road extraction, field extraction, cloud segmentation, and building segmentation.