Semantic segmentation is a fundamental task in remote sensing image analysis (RSIA). Fully convolutional networks (FCNs) have achieved state-of-the-art performance in the task of semantic segmentation of natural scene images. However, due to distinctive differences between natural scene images and remotely-sensed (RS) images, FCN-based semantic segmentation methods from the field of computer vision cannot achieve promising performances on RS images without modifications. In previous work, we proposed an RS image semantic segmentation framework SDFCNv1, combined with a majority voting postprocessing method. Nevertheless, it still has some drawbacks, such as small receptive field and large number of parameters. In this paper, we propose an improved semantic segmentation framework SDFCNv2 based on SDFCNv1, to conduct optimal semantic segmentation on RS images. We first construct a novel FCN model with hybrid basic convolutional (HBC) blocks and spatial-channel-fusion squeeze-and-excitation (SCFSE) modules, which occupies a larger receptive field and fewer network model parameters. We also put forward a data augmentation method based on spectral-specific stochastic-gamma-transform-based (SSSGT-based) during the model training process to improve generalizability of our model. Besides, we design a mask-weighted voting decision fusion postprocessing algorithm for image segmentation on overlarge RS images. We conducted several comparative experiments on two public datasets and a real surveying and mapping dataset. Extensive experimental results demonstrate that compared with the SDFCNv1 framework, our SDFCNv2 framework can increase the mIoU metric by up to 5.22% while only using about half of parameters.
In recent years, remote sensing techniques such as satellite and drone-based imaging have been used to monitor Pine Wilt Disease (PWD), a widespread forest disease that causes the death of pine species. Researchers have explored the use of remote sensing imagery and deep learning algorithms to improve the accuracy of PWD detection at the single-tree level. This study introduces a novel framework for PWD detection that combines high-resolution RGB drone imagery with free-access Sentinel-2 satellite multi-spectral imagery. The proposed approach includes an PWD-infected tree detection model named YOLOv5-PWD and an effective data augmentation method. To evaluate the proposed framework, we collected data and created a dataset in Xianning City, China, consisting of object detection samples of infected trees at middle and late stages of PWD. Experimental results indicate that the YOLOv5-PWD detection model achieved 1.2% higher mAP compared to the original YOLOv5 model and a further improvement of 1.9% mAP was observed after applying our dataset augmentation method, which demonstrates the effectiveness and potential of the proposed framework for PWD detection.
19 July 2019 in IEEE Geoscience and Remote Sensing Letters
Auto-extraction of convective clouds is of great significance. Convective clouds often bring heavy rain, strong winds, and other disastrous weather. Early warning of convection can effectively reduce loss. Using remote sensing images, we can get large-scale cloud information, which provides many effective methods for convective clouds detection. In this letter, we proposed a novel method to extract convective clouds. We introduce a novel deep network using only 1 x1 convolution (3ONet) to extract the spectral characteristics. We then combine a 3ONet with the symmetrical dense-shortcut deep fully convolutional networks (SDFCNs) with a double-stream fully convolutional network to extract convective clouds. In the experiment, we used 12,000 Himawari-8 satellite image patches to verify the proposed framework. Experimental results with 0.5882 mean intersection over union (mIOU) pointed out the proposed method can extract convective clouds effectively.
28 November 2023 in IEEE Transactions on Geoscience and Remote Sensing
Stereo matching and semantic segmentation are two significant tasks in remote sensing. Recently, deep learning approaches have been applied to these tasks separately. However, the lack of semantic supervision makes the training of stereo matching model susceptible to data disturbance, resulting in inferior generalization ability; foreground objects are sometimes confused with background pixels in RGB images, limiting the classification accuracy. By exploring the relationship between these two tasks, semantic stereo solves these problems simultaneously with multitask learning. Previous methods took semantic stereo as two parallel processing tasks, so they did not take full advantages of the additional information from both tasks and only obtained slight improvement. In this work, we designed a multitask learning framework semantic stereo network (S2Net). The proposed network generates cost volumes with feature maps supervised by semantic information to estimate disparity maps and fuses RGB-D feature maps to predict classification maps, therefore gathering multitask learning information. To enhance the performance of trained model, we also considered the continuity of disparity values and the duality of stereo image pairs in data augmentation. When applied in datasets without training, S2Net obtained 2.937% D1-Error in the WHU dataset, lower than 4.297% of the previous best method, depicting the generalization ability improvement from semantic supervision. In terms of semantic segmentation, the introduction of disparity maps increases the mean intersection over union (mIoU) from 61.375% to 69.096% in the US3D datasets. The experiments on the KITTI semantics benchmark show that our proposed method obtains 60.76% mIoU, achieving state-of-the-art among multitask learning methods.
11 August 2022 in IEEE Geoscience and Remote Sensing Letters
Semantic segmentation is one of the fundamental tasks of pixel-level remote sensing image analysis. Currently, most high-performance semantic segmentation methods are trained in a supervised learning manner. These methods require a large number of image labels as support, but manual annotations are difficult to obtain. To address the problem, we propose an efficient unsupervised remote sensing image segmentation method based on superpixel segmentation and fully convolutional networks (FCNs) in this letter. Our method can achieve pixel-level images segmentation of various scales rapidly without any manual labels or prior knowledge. We use the superpixel segmentation results as synthetic ground truth to guide the gradient descent direction during FCN training. In experiments, our method achieved high performance compared with current unsupervised image segmentation methods on three public datasets. Specifically, our method achieves an adjusted mutual information (AMI) score of 0.2955 on the Gaofen Image Dataset (GID), while processing each image of size 7200 $\times6800$ pixels in just 30 s.
Net primary productivity (NPP) can indirectly reflect vegetation’s capacity for CO fixation, but its spatiotemporal dynamics are subject to alterations to some extent due to the influences of climate change and human activities. In this study, NPP is used as an indicator to investigate vegetarian carbon ability changes in the vital ecosystems of the Yangtze River Basin (YRB) in China. We also explored the NPP responses to climate change and human activities. We conducted a comprehensive analysis of the temporal dynamics and spatial variations in NPP within the YRB ecosystems from 2003 to 2020. Furthermore, we employed residual analysis to quantitatively assess the contributions of climate factors and human activities to NPP changes. The research findings are as follows: (1) Over the 18-year period, the average NPP within the basin amounted to 543.95 gC/m, displaying a noticeable fluctuating upward trend with a growth rate of approximately 3.1 gC/m; (2) The areas exhibiting an increasing trend in NPP account for 82.55% of the total study area. Regions with relatively high stability in the basin covered 62.36% of the total area, while areas with low stability accounted for 2.22%, mainly situated in the Hengduan Mountains of the western Sichuan Plateau; (3) NPP improvement was jointly driven by human activities and climate change, with human activities contributing more significantly to NPP growth. Specifically, the contributions were 65.39% in total, with human activities contributing 59.28% and climate change contributing 40.01%. This study provides an objective assessment of the contributions of human activities and climate change to vegetation productivity, offering crucial insights for future ecosystem development and environmental planning.