The distribution of sea ice is one of the major safety hazards for sea navigation. As human activities in polar regions become more frequent, monitoring and forecasting of sea ice are of great significance. In this paper, we use SAR data from the C-band synthetic aperture radar (SAR) Gaofen-3 satellite in the dual-polarization (VV, VH) fine strip II (FSII) mode of operation to study the Arctic sea ice classification in winter. SAR data we use were taken in the western Arctic Ocean from January to February 2020. We classify the sea ice into four categories, namely new ice (NI), thin first-year ice (tI), thick first-year ice (TI), and old ice (OI), by referring to the ice maps provided by the Canadian Ice Service (CIS). Then, we use the deep learning model MobileNetV3 as the backbone network, input samples of different sizes, and combine the backbone network with multiscale feature fusion methods to build a deep learning model called Multiscale MobileNet (MSMN). Dual-polarization SAR data are used to synthesize pseudocolor images and produce samples of sizes 16 × 16 × 3, 32 × 32 × 3, and 64 × 64 × 3 as input. Ultimately, MSMN can reach over 95% classification accuracy on testing SAR sea ice images. The classification results using only VV polarization or VH polarization data are tested, and it is found that using dual-polarization data could improve the classification accuracy by 10.05% and 9.35%, respectively. When other classification models are trained using the training data from this paper for comparison, the accuracy of MSMN is 4.86% and 1.84% higher on average than that of the model built using convolutional neural networks (CNNs) and ResNet18 model, respectively.
30 December 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Stereo matching, a critical step of binocular 3D reconstruction, has fully shifted to deep learning due to its strong feature representation of remote sensing images. However, the ground truth for the stereo matching relies on expensive airborne LiDAR data, thus making it difficult to obtain enough samples for supervised learning. To improve the generalization ability of stereo matching networks on cross-domain data from different optical sensors and scenarios, in this paper, we are dedicated to studying the key training factors from three perspectives. (1) When selecting a training dataset, prioritize data with similar regional target distribution as the test set, rather than relying on data from the same sensor. (2) Regarding the training modes, unsupervised methods generalize better than supervised methods. (3) We devised an unsupervised early-stop strategy to help preserve the best model based on the pretrained weights. Extensive experiments are conducted to support the previous findings, on the basis of which we present an unsupervised stereo matching network with good generalization performance. We release the source code and the datasets at https://github.com/Elenairene/RKF RSSM to reproduce the results and encourage future work.
23 September 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
This paper investigates the safety risks associated with sea ice along the Arctic Route by using polar sea ice images obtained by Gaofen-3 (GF3) Synthetic Aperture Radar (SAR) satellites. We collected three SAR datasets, representative of GF3 satellites' operational modes, and constructed semantic segmentation datasets through meticulous annotation using concurrent optical satellite imagery obtained by Landsat satellites to ensure precision. We propose SI-CTFNet, an innovative sea ice semantic segmentation model that integrates Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) for enhanced multi-scale feature extraction. SI-CTFNet features a dual-pathway architecture designed to optimize feature extraction, complemented by the BiAFusion module, which effectively merges local and global features to improve decoding accuracy. Additionally, we introduce the MSDAM module to facilitate a comprehensive multi-scale contextual analysis, addressing the diverse distribution of ice types in the imagery. The model incorporates advanced decoding techniques, including a progressive upsampling (PUP) approach for the CNN-fusion branch and an efficient All-MLP module for the Transformer branch. Performance evaluations across three distinct datasets reveal that SI-CTFNet significantly outperforms existing methods in key metrics and maintains efficacy with supplementary Sentinel1A C-band satellite data. Furthermore, we present a streamlined variant of SI-CTFNet, which achieves a threefold increase in inference speed with minimal reduction in classification accuracy. The ultimate objective of this work is to advance a precise sea ice forecasting and navigation system for polar regions, aimed at automating sea ice classification within a smart polar shipping framework.
29 May 2023 in IEEE Transactions on Geoscience and Remote Sensing
Multispectral remote sensing images (RSI), including hyperspectral and multispectral images, contain adequate information of ground objects and areas and play important roles in environmental monitoring, weather forecasting, urban planning, etc. However, due to many inevitable external effects on the remote sensing pathway, remote sensing images are often degraded by blur. The fields of multispectral remote sensing image deblurring have witnessed great improvements in recent years, including both optimization-based and deep-learning-based methods. However, issues are to be addressed in the remote sensing image deblurring field, such as the incompatibility of general regularizations, lack of spectral correlations for multispectral RSIs, demands of blind deblurring for real world applications, and high costs of computation. To address these problems, we incorporate a novel prior exploiting gradient information similarity between different spectral bands, and we name it auxiliary band gradient information (ABGI) prior. We show that the ABGI prior is applicable to all gradient sparsity regularizations by a simple subtract-then-add step. Specifically, we apply ABGI prior to the patch-wise minimal pixel (PMP) prior based deblurring method, and we also prove that the PMP prior exhibits sparsity for clear natural RSIs. We estimate our method on remote sensing image datasets of different spectral types and geographic resolutions. Compared to other state-of-the-art deblurring methods, our method shows superior performance on both simulated and real world blur.
7 June 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
With the continuous development of edge computing, real-time processing of satellite images has gradually become a research hotspot. However, the current semantic segmentation methods for SAR images are suboptimal in terms of segmentation performance and inference speed. Therefore, we propose a fast fine semantic segmentation network for building regions in SAR images, called FFSSNet. The architecture of this network comprises a feature extractor, a Feature Reprocessing Module (FRM), a Multi-scale Feature Deep Fusion Module (MFDFM), and an auxiliary branch. Aiming at the problem of low signal-to-noise ratio of SAR images, the network extracts multi-scale features rich in context information through a feature extractor with a large receptive field. After that, through the FRM and MFDFM, the semantic information in each scale is deeply extracted and fused. This makes the feature maps contain semantic information of all scales. In addition, we design an auxiliary branch for reducing misclassification of pixels in non-target regions. We test the network using two completely different datasets. Compared with other state-of-the-art methods, our proposed method has the better performance in terms of segmentation accuracy, inference speed and generalisation ability.
There are limited studies on the semantic segmentation of high-resolution synthetic aperture radar (SAR) images in building areas due to speckle noise and geometric distortion. For this challenge, we propose the large receptive field feature fusion network (LRFFNet), which contains a feature extractor, a cascade feature pyramid module (CFP), a large receptive field channel attention module (LFCA), and an auxiliary branch. SAR images only contain single-channel information and have a low signal-to-noise ratio. Using only one level of features extracted by the feature extractor will result in poor segmentation results. Therefore, we design the CFP module; it can integrate different levels of features through multi-path connection. Due to the problem of geometric distortion in SAR images, the structural and semantic information is not obvious. In order to pick out feature channels that are useful for segmentation, we design the LFCA module, which can reassign the weight of channels through the channel attention mechanism with a large receptive field to help the network focus on more effective channels. SAR images do not include color information, and the identification of ground object categories is prone to errors, so we design the auxiliary branch. The branch uses the full convolution structure to optimize training results and reduces the phenomenon of recognizing objects outside the building area as buildings. Compared with state-of-the-art (SOTA) methods, our proposed network achieves higher scores in evaluation indicators and shows excellent competitiveness.