Deep learning has achieved great success in remote sensing image change detection (CD). However, most methods focus only on the changed regions of images and cannot accurately identify their detailed semantic categories. In addition, most CD methods using convolutional neural networks (CNN) have difficulty capturing sufficient global information from images. To address the above issues, we propose a novel symmetric multi-task network (SMNet) that integrates global and local information for semantic change detection (SCD) in this paper. Specifically, we employ a hybrid unit consisting of pre-activated residual blocks (PR) and transformation blocks (TB) to construct the (PRTB) backbone, which obtains more abundant semantic features with local and global information from bi-temporal images. To accurately capture fine-grained changes, the multi-content fusion module (MCFM) is introduced, which effectively enhances change features by distinguishing foreground and background information in complex scenes. In the meantime, the multi-task prediction branches are adopted, and the multi-task loss function is used to jointly supervise model training to improve the performance of the network. Extensive experimental results on the challenging SECOND and Landsat-SCD datasets, demonstrate that our SMNet obtains 71.95% and 85.65% at mean Intersection over Union (mIoU), respectively. In addition, the proposed SMNet achieves 20.29% and 51.14% at Separated Kappa coefficient (Sek) on the SECOND and Landsat-SCD datasets, respectively. All of the above proves the effectiveness and superiority of the proposed method.
13 October 2020 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Remote sensing image scene classification is one of the important means for remote sensing image understanding. Convolutional neural networks have been successfully applied to remote sensing image scene classifications and have demonstrated remarkable performance. However, with improvements in image resolution, remote sensing image categories are becoming increasingly diverse, and problems such as high intraclass diversity and high interclass similarity have arisen. The performance of ordinary convolutional neural networks at distinguishing increasingly complex remote sensing images is still limited. Therefore, we propose a feature fusion framework based on hierarchical attention and bilinear pooling called HABFNet for scene classification of remote sensing images. First, the deep convolutional neural network ResNet50 is used to extract the deep features from different layers of the image, and these features are fused to boost the robustness and effectiveness of the features. Second, we design an improved channel attention scheme to enhance the features from different layers. Finally, the enhanced features are cross-layer bilinearly pooled and fused, and the fused features are used for classification. Extensive experiments were conducted on three publicly available remote sensing image benchmarks. Comparisons with the state-of-the-art methods demonstrated that the proposed HABFNet achieved competitive classification performance.
30 October 2019 in IEEE Geoscience and Remote Sensing Letters
Point cloud classification of airborne light detection and ranging (LiDAR) data is essential to extract geoinformation. Although deep learning provides a new approach for classification, the time-consuming training process and data dependence prevent its widespread application to point clouds. To solve these problems and leverage the potential of high-performing neural networks, we propose an airborne LiDAR point cloud classification method based on transfer learning. A strategy to generate feature images considering the point cloud spatial distribution is first introduced for applying traditional convolutional neural networks to point clouds. Then, transfer learning is used to extract multiscale and multiview deep features. A simple neural network classifier is designed to reduce dimensionality, fuse and learn high-level features, and postprocessing considering contextual information further improves the classification accuracy. We verified the performance of the proposed method through experiments on two airborne LiDAR data sets with different characteristics and containing eight classes. The results demonstrate that the proposed method can achieve a satisfactory classification accuracy with relatively short training time and less training samples than if using conventional methods.
15 March 2022 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Characterized by complicated backgrounds, various types, large size variations, and arbitrary orientations, the detection and recognition of arbitrary-oriented objects in remote sensing images are challenging. To address the aforementioned problem, an anchor-free arbitrary-oriented object detector using box boundary-aware vectors is proposed. With the idea of CenterNet to detect objects as points, oriented object detection is achieved by predicting the center, the box boundary-aware vectors, the size, and the type of the bounding box. In the feature extraction stage of the designed architecture, Res2Net, a multiscale convolutional neural network, is used to extract feature maps of different scales and adaptively spatial feature fusion is adopted to improve the detector's adaptability to objects of different sizes. In the detector, a context enhancement module with a multibranch network is designed to enhance the contextual information of the objects and improve the detector's robustness to the complicated backgrounds. Experiments are carried on three challenging benchmarks (i.e., HRSC2016, UCAS-AOD, and DOTA) and our method achieves state-of-the-art performance with 90.30%, 89.70%, and 77.18% mAP, respectively.
Accurate ship trajectory prediction is crucial for real-time vessel position tracking and maritime safety management. However, existing methods for ship trajectory prediction encounter significant challenges. They struggle to effectively extract long-term and complex spatial–temporal features hidden within the data. Moreover, they often overlook correlations among multivariate dynamic features such as longitude (LON), latitude (LAT), speed over ground (SOG), and course over ground (COG), which are essential for precise trajectory forecasting. To address these pressing issues and fulfill the need for more accurate and comprehensive ship trajectory prediction, we propose a novel and integrated approach. Firstly, a Trajectory Point Correlation Attention (TPCA) mechanism is devised to establish spatial connections between trajectory points, thereby uncovering the local trends of trajectory point changes. Subsequently, a Temporal Pattern Attention (TPA) mechanism is introduced to handle the associations between multiple variables across different time steps and capture the dynamic feature correlations among trajectory attributes. Finally, a Great Circle Route Loss Function (GCRLoss) is constructed, leveraging the perception of the Earth’s curvature to deepen the understanding of spatial relationships and geographic information. Experimental results demonstrate that our proposed method outperforms existing ship trajectory prediction techniques, showing enhanced reliability in multi-step predictions.
Remote sensing for image object detection has numerous important applications. However, complex backgrounds and large object-scale differences pose considerable challenges in the detection task. To overcome these issues, we proposed a one-stage remote sensing image object detection model: a multi-feature information complementary detector (MFICDet). This detector contains a positive and negative feature guidance module (PNFG) and a global feature information complementary module (GFIC). Specifically, the PNFG is used to refine features that are beneficial for object detection and explore the noisy features in a complex background of abstract features. The proportion of beneficial features in the feature information stream is increased by suppressing noisy features. The GFIC uses pooling to compress the deep abstract features and improve the model’s ability to resist feature displacement and rotation. The pooling operation has the disadvantage of losing detailed feature information; thus, dilated convolution is introduced for feature complementation. Dilated convolution increases the receptive field of the model while maintaining an unchanged spatial resolution. This can improve the ability of the model to recognize long-distance dependent information and establish spatial location relationships between features. The detector proposed also improves the detection performance of objects at different scales in the same image using a dual multi-scale feature fusion strategy. Finally, classification and regression tasks are decoupled in space using a decoupled head. We experimented on the DIOR and NWPU VHR-10 datasets to demonstrate that the newly proposed MFICDet achieves competitive performance compared to current state-of-the-art detectors.