This page has only limited features, please log in for full access.
Automatic change detection from remotely sensed imagery is extremely important for many applications, including land use mapping. In recent years, a growing number of researchers have applied capable deep-learning methods to the research on change detection. The majority of deep learning-based change detection methods currently perform pixel-by-pixel classification at the original image scale, but they can hardly avoid the false changes caused by strong parallax effects and projected shadows, without considering the totality of changed objects/regions. In this study, we propose an object-level change detection framework to detect changed geographic entities (such as newly built buildings or changed artificial structures) by paying more attention to the overall characteristics and context association of changed object instances. The detected changed objects are represented as bounding boxes, which are simple, regular, and convenient to use in object feature extraction. In terms of data handling, a special data augmentation method for change detection called Alternative-Mosaic is proposed to effectively accelerate model training and improve model performance. For the model, we propose a one-stage change detection network called dual correlation attention-guided detector (DCA-Det) to detect the changed objects. In particular, we feed the dual-temporal images into a weight-shared backbone network to extract the change features of different scales. The change features on the same scale are further refined, and then the features between different scales are fused by the correlation attention-guided feature fusion neck. Finally, the change detection heads output the prediction results of the changed objects/regions of different scales. Experiments were conducted on public LEVIR building change detection and aerial imagery change detection (AICD) datasets. The quantitative evaluation and visualization results proved the superiority and robustness of our framework. Our DCA-Det can obtain state-of-the-art performance on object-level metrics (99.50% APIoU=.50 and 79.72% APIoU=.50:.05:.95) on the AICD-2012 dataset.
Lin Zhang; Xiangyun Hu; Mi Zhang; Zhen Shu; Hao Zhou. Object-level change detection with a dual correlation attention-guided detector. ISPRS Journal of Photogrammetry and Remote Sensing 2021, 177, 147 -160.
AMA StyleLin Zhang, Xiangyun Hu, Mi Zhang, Zhen Shu, Hao Zhou. Object-level change detection with a dual correlation attention-guided detector. ISPRS Journal of Photogrammetry and Remote Sensing. 2021; 177 ():147-160.
Chicago/Turabian StyleLin Zhang; Xiangyun Hu; Mi Zhang; Zhen Shu; Hao Zhou. 2021. "Object-level change detection with a dual correlation attention-guided detector." ISPRS Journal of Photogrammetry and Remote Sensing 177, no. : 147-160.
As one of the fundamental tasks in remote sensing (RS) image understanding, multi-label remote sensing image scene classification (MLRSSC) is attracting increasing research interest. Human beings can easily perform MLRSSC by examining the visual elements contained in the scene and the spatio-topological relationships of these visual elements. However, most of existing methods are limited by only perceiving visual elements but disregarding the spatio-topological relationships of visual elements. With this consideration, this paper proposes a novel deep learning-based MLRSSC framework by combining convolutional neural network (CNN) and graph neural network (GNN), which is termed the MLRSSC-CNN-GNN. Specifically, the CNN is employed to learn the perception ability of visual elements in the scene and generate the high-level appearance features. Based on the trained CNN, one scene graph for each scene is further constructed, where nodes of the graph are represented by superpixel regions of the scene. To fully mine the spatio-topological relationships of the scene graph, the multi-layer-integration graph attention network (GAT) model is proposed to address MLRSSC, where the GAT is one of the latest developments in GNN. Extensive experiments on two public MLRSSC datasets show that the proposed MLRSSC-CNN-GNN can obtain superior performance compared with the state-of-the-art methods.
Yansheng Li; Ruixian Chen; Yongjun Zhang; Mi Zhang; Ling Chen. Multi-Label Remote Sensing Image Scene Classification by Combining a Convolutional Neural Network and a Graph Neural Network. Remote Sensing 2020, 12, 4003 .
AMA StyleYansheng Li, Ruixian Chen, Yongjun Zhang, Mi Zhang, Ling Chen. Multi-Label Remote Sensing Image Scene Classification by Combining a Convolutional Neural Network and a Graph Neural Network. Remote Sensing. 2020; 12 (23):4003.
Chicago/Turabian StyleYansheng Li; Ruixian Chen; Yongjun Zhang; Mi Zhang; Ling Chen. 2020. "Multi-Label Remote Sensing Image Scene Classification by Combining a Convolutional Neural Network and a Graph Neural Network." Remote Sensing 12, no. 23: 4003.
Automatic extraction of region objects from high-resolution satellite imagery presents a great challenge, because there may be very large variations of the objects in terms of their size, texture, shape, and contextual complexity in the image. To handle these issues, we present a novel, deep-learning-based approach to interactively extract non-artificial region objects, such as water bodies, woodland, farmland, etc., from high-resolution satellite imagery. First, our algorithm transforms user-provided positive and negative clicks or scribbles into guidance maps, which consist of a relevance map modified from Euclidean distance maps, two geodesic distance maps (for positive and negative, respectively), and a sampling map. Then, feature maps are extracted by applying a VGG convolutional neural network pre-trained on the ImageNet dataset to the image X, and they are then upsampled to the resolution of X. Image X, guidance maps, and feature maps are integrated as the input tensor. We feed the proposed attention-guided, multi-scale segmentation neural network (AGMSSeg-Net) with the input tensor above to obtain the mask that assigns a binary label to each pixel. After a post-processing operation based on a fully connected Conditional Random Field (CRF), we extract the selected object boundary from the segmentation result. Experiments were conducted on two typical datasets with diverse region object types from complex scenes. The results demonstrate the effectiveness of the proposed method, and our approach outperforms existing methods for interactive image segmentation.
Kun Li; Xiangyun Hu; Huiwei Jiang; Zhen Shu; Mi Zhang. Attention-Guided Multi-Scale Segmentation Neural Network for Interactive Extraction of Region Objects from High-Resolution Satellite Imagery. Remote Sensing 2020, 12, 789 .
AMA StyleKun Li, Xiangyun Hu, Huiwei Jiang, Zhen Shu, Mi Zhang. Attention-Guided Multi-Scale Segmentation Neural Network for Interactive Extraction of Region Objects from High-Resolution Satellite Imagery. Remote Sensing. 2020; 12 (5):789.
Chicago/Turabian StyleKun Li; Xiangyun Hu; Huiwei Jiang; Zhen Shu; Mi Zhang. 2020. "Attention-Guided Multi-Scale Segmentation Neural Network for Interactive Extraction of Region Objects from High-Resolution Satellite Imagery." Remote Sensing 12, no. 5: 789.
In recent years, building change detection has made remarkable progress through using deep learning. The core problems of this technique are the need for additional data (e.g., Lidar or semantic labels) and the difficulty in extracting sufficient features. In this paper, we propose an end-to-end network, called the pyramid feature-based attention-guided Siamese network (PGA-SiamNet), to solve these problems. The network is trained to capture possible changes using a convolutional neural network in a pyramid. It emphasizes the importance of correlation among the input feature pairs by introducing a global co-attention mechanism. Furthermore, we effectively improved the long-range dependencies of the features by utilizing various attention mechanisms and then aggregating the features of the low-level and co-attention level; this helps to obtain richer object information. Finally, we evaluated our method with a publicly available dataset (WHU) building dataset and a new dataset (EV-CD) building dataset. The experiments demonstrate that the proposed method is effective for building change detection and outperforms the existing state-of-the-art methods on high-resolution remote sensing orthoimages in various metrics.
Huiwei Jiang; Xiangyun Hu; Kun Li; Jinming Zhang; Jinqi Gong; Mi Zhang. PGA-SiamNet: Pyramid Feature-Based Attention-Guided Siamese Network for Remote Sensing Orthoimagery Building Change Detection. Remote Sensing 2020, 12, 484 .
AMA StyleHuiwei Jiang, Xiangyun Hu, Kun Li, Jinming Zhang, Jinqi Gong, Mi Zhang. PGA-SiamNet: Pyramid Feature-Based Attention-Guided Siamese Network for Remote Sensing Orthoimagery Building Change Detection. Remote Sensing. 2020; 12 (3):484.
Chicago/Turabian StyleHuiwei Jiang; Xiangyun Hu; Kun Li; Jinming Zhang; Jinqi Gong; Mi Zhang. 2020. "PGA-SiamNet: Pyramid Feature-Based Attention-Guided Siamese Network for Remote Sensing Orthoimagery Building Change Detection." Remote Sensing 12, no. 3: 484.
Thanks to the recent development of laser scanner hardware and the technology of dense image matching (DIM), the acquisition of three-dimensional (3D) point cloud data has become increasingly convenient. However, how to effectively combine 3D point cloud data and images to realize accurate building change detection is still a hotspot in the field of photogrammetry and remote sensing. Therefore, with the bi-temporal aerial images and point cloud data obtained by airborne laser scanner (ALS) or DIM as the data source, a novel building change detection method combining co-segmentation and superpixel-based graph cuts is proposed in this paper. In this method, the bi-temporal point cloud data are firstly combined to achieve a co-segmentation to obtain bi-temporal superpixels with the simple linear iterative clustering (SLIC) algorithm. Secondly, for each period of aerial images, semantic segmentation based on a deep convolutional neural network is used to extract building areas, and this is the basis for subsequent superpixel feature extraction. Again, with the bi-temporal superpixel as the processing unit, a graph-cuts-based building change detection algorithm is proposed to extract the changed buildings. In this step, the building change detection problem is modeled as two binary classifications, and acquisition of each period’s changed buildings is a binary classification, in which the changed building is regarded as foreground and the other area as background. Then, the graph cuts algorithm is used to obtain the optimal solution. Next, by combining the bi-temporal changed buildings and digital surface models (DSMs), these changed buildings are further classified as “newly built,” “taller,” “demolished”, and “lower”. Finally, two typical datasets composed of bi-temporal aerial images and point cloud data obtained by ALS or DIM are used to validate the proposed method, and the experiments demonstrate the effectiveness and generality of the proposed algorithm.
Shiyan Pang; Xiangyun Hu; Mi Zhang; Zhongliang Cai; Fengzhu Liu. Co-Segmentation and Superpixel-Based Graph Cuts for Building Change Detection from Bi-Temporal Digital Surface Models and Aerial Images. Remote Sensing 2019, 11, 729 .
AMA StyleShiyan Pang, Xiangyun Hu, Mi Zhang, Zhongliang Cai, Fengzhu Liu. Co-Segmentation and Superpixel-Based Graph Cuts for Building Change Detection from Bi-Temporal Digital Surface Models and Aerial Images. Remote Sensing. 2019; 11 (6):729.
Chicago/Turabian StyleShiyan Pang; Xiangyun Hu; Mi Zhang; Zhongliang Cai; Fengzhu Liu. 2019. "Co-Segmentation and Superpixel-Based Graph Cuts for Building Change Detection from Bi-Temporal Digital Surface Models and Aerial Images." Remote Sensing 11, no. 6: 729.
In this work, a novel building change detection method from bi-temporal dense-matching point clouds and aerial images is proposed to address two major problems, namely, the robust acquisition of the changed objects above ground and the automatic classification of changed objects into buildings or non-buildings. For the acquisition of changed objects above ground, the change detection problem is converted into a binary classification, in which the changed area above ground is regarded as the foreground and the other area as the background. For the gridded points of each period, the graph cuts algorithm is adopted to classify the points into foreground and background, followed by the region-growing algorithm to form candidate changed building objects. A novel structural feature that was extracted from aerial images is constructed to classify the candidate changed building objects into buildings and non-buildings. The changed building objects are further classified as “newly built”, “taller”, “demolished”, and “lower” by combining the classification and the digital surface models of two periods. Finally, three typical areas from a large dataset are used to validate the proposed method. Numerous experiments demonstrate the effectiveness of the proposed algorithm.
Shiyan Pang; Xiangyun Hu; Zhongliang Cai; Jinqi Gong; Mi Zhang. Building Change Detection from Bi-Temporal Dense-Matching Point Clouds and Aerial Images. Sensors 2018, 18, 966 .
AMA StyleShiyan Pang, Xiangyun Hu, Zhongliang Cai, Jinqi Gong, Mi Zhang. Building Change Detection from Bi-Temporal Dense-Matching Point Clouds and Aerial Images. Sensors. 2018; 18 (4):966.
Chicago/Turabian StyleShiyan Pang; Xiangyun Hu; Zhongliang Cai; Jinqi Gong; Mi Zhang. 2018. "Building Change Detection from Bi-Temporal Dense-Matching Point Clouds and Aerial Images." Sensors 18, no. 4: 966.
The semi-global optimization algorithm, which approximates a global 2D smoothness constraint by combining several 1D constraints, has been widely used in the field of image dense matching for digital surface model (DSM) generation. However, due to occlusion, shadow and textureless area of the matching images, some inconsistency may exist in the overlapping areas of different DSMs. To address this problem, based on the DSMs generated by semi-global matching from multiple stereopairs, a novel semi-global merging algorithm is proposed to generate a reliable and consistent DSM in this paper. Two datasets, each covering 1 km2, are used to validate the proposed method. Experimental results show that the optimal DSM after merging can effectively eliminate the inconsistency and reduce redundancy in the overlapping areas.
S. Pang; X. Hu; M. Zhang; L. Ye. SEMI – GLOBAL MERGING OF DIGITAL SURFACE MODELS FROM MULTIPLE STEREOPAIRS. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences 2017, IV-2/W4, 267 -271.
AMA StyleS. Pang, X. Hu, M. Zhang, L. Ye. SEMI – GLOBAL MERGING OF DIGITAL SURFACE MODELS FROM MULTIPLE STEREOPAIRS. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2017; IV-2/W4 ():267-271.
Chicago/Turabian StyleS. Pang; X. Hu; M. Zhang; L. Ye. 2017. "SEMI – GLOBAL MERGING OF DIGITAL SURFACE MODELS FROM MULTIPLE STEREOPAIRS." ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences IV-2/W4, no. : 267-271.
Semantic image segmentation has recently witnessed considerable progress by training deep convolutional neural networks (CNNs). The core issue of this technique is the limited capacity of CNNs to depict visual objects. Existing approaches tend to utilize approximate inference in a discrete domain or additional aides and do not have a global optimum guarantee. We propose the use of the multi-label manifold ranking (MR) method in solving the linear objective energy function in a continuous domain to delineate visual objects and solve these problems. We present a novel embedded single stream optimization method based on the MR model to avoid approximations without sacrificing expressive power. In addition, we propose a novel network, which we refer to as dual multi-scale manifold ranking (DMSMR) network, that combines the dilated, multi-scale strategies with the single stream MR optimization method in the deep learning architecture to further improve the performance. Experiments on high resolution images, including close-range and remote sensing datasets, demonstrate that the proposed approach can achieve competitive accuracy without additional aides in an end-to-end manner.
Mi Zhang; Xiangyun Hu; Like Zhao; Ye Lv; Min Luo; Shiyan Pang. Learning Dual Multi-Scale Manifold Ranking for Semantic Segmentation of High-Resolution Images. Remote Sensing 2017, 9, 500 .
AMA StyleMi Zhang, Xiangyun Hu, Like Zhao, Ye Lv, Min Luo, Shiyan Pang. Learning Dual Multi-Scale Manifold Ranking for Semantic Segmentation of High-Resolution Images. Remote Sensing. 2017; 9 (5):500.
Chicago/Turabian StyleMi Zhang; Xiangyun Hu; Like Zhao; Ye Lv; Min Luo; Shiyan Pang. 2017. "Learning Dual Multi-Scale Manifold Ranking for Semantic Segmentation of High-Resolution Images." Remote Sensing 9, no. 5: 500.
Semantic image segmentation has recently witnessed considerable progress by training deep convolutional neural networks (CNNs). The core issue of this technique is the limited capacity of CNNs to depict visual objects. Existing approaches tend to utilize approximate inference in a discrete domain or additional aides and do not have a global optimum guarantee. We propose the use of the multi-label manifold ranking (MR) method in solving the linear objective energy function in a continuous domain to delineate visual objects and solve these problems. We present a novel embedded single stream optimization method based on the manifold ranking (MR) model to avoid approximations without sacrificing expressive power. In addition, we propose a novel network, which we refer to as dual multi-scale manifold ranking (\textbf{DMSMR}) network, that combines the dilated, multi-scale strategies with the single stream MR optimization method in the deep learning architecture to further improve the performance. Experiments on high resolution images, including close-range and remote sensing datasets, demonstrate that the proposed approach can achieve competitive accuracy without additional aides in an end-to-end manner.
Mi Zhang; Xiangyun Hu; Like Zhao; Ye Lv; Min Luo; Shiyan Pang. Learning Dual Multi-Scale Manifold Ranking for Semantic Segmentation of High-Resolution Images. 2017, 1 .
AMA StyleMi Zhang, Xiangyun Hu, Like Zhao, Ye Lv, Min Luo, Shiyan Pang. Learning Dual Multi-Scale Manifold Ranking for Semantic Segmentation of High-Resolution Images. . 2017; ():1.
Chicago/Turabian StyleMi Zhang; Xiangyun Hu; Like Zhao; Ye Lv; Min Luo; Shiyan Pang. 2017. "Learning Dual Multi-Scale Manifold Ranking for Semantic Segmentation of High-Resolution Images." , no. : 1.