Building change detection is important for urban area monitoring, disaster assessment and updating geo-database. 3D information derived from image dense matching or airborne light detection and ranging (LiDAR) is very effective for building change detection. However, combining 3D data from different sources is challenging, and so far few studies have focused on building change detection using both images and LiDAR data. This study proposes an automatic method to detect building changes in urban areas using aerial images and LiDAR data. First, dense image matching is carried out to obtain dense point clouds and then co-registered LiDAR point clouds using the iterative closest point (ICP) algorithm. The registered point clouds are further resampled to a raster DSM (Digital Surface Models). In a second step, height difference and grey-scale similarity are calculated as change indicators and the graph cuts method is employed to determine changes considering the contexture information. Finally, the detected results are refined by removing the non-building changes, in which a novel method based on variance of normal direction of LiDAR points is proposed to remove vegetated areas for positive building changes (newly building or taller) and nEGI (normalized Excessive Green Index) is used for negative building changes (demolish building or lower). To evaluate the proposed method, a test area covering approximately 2.1 km2 and consisting of many different types of buildings is used for the experiment. Results indicate 93% completeness with correctness of 90.2% for positive changes, while 94% completeness with correctness of 94.1% for negative changes, which demonstrate the promising performance of the proposed method.
27 October 2017 in Geo-spatial Information Science
Targeting at a reliable image matching of multiple remote sensing images for the generation of digital surface models, this paper presents a geometric-constrained multi-view image matching method, based on an energy minimization framework. By employing a geometrical constraint, the cost value of the energy function was calculated from multiple images, and the cost value was aggregated in an image space using a semi-global optimization approach. A homography transform parameter calculation method is proposed for fast calculation of projection pixel on each image when calculating cost values. It is based on the known interior orientation parameters, exterior orientation parameters, and a given elevation value. For an efficient and reliable processing of multiple remote sensing images, the proposed matching method was performed via a coarse-to-fine strategy through image pyramid. Three sets of airborne remote sensing images were used to evaluate the performance of the proposed method. Results reveal that the multi-view image matching can improve matching reliability. Moreover, the experimental results show that the proposed method performs better than traditional methods.
21 June 2022 in IEEE Geoscience and Remote Sensing Letters
The increase in self-supervised learning (SSL), especially contrastive learning, has enabled one to train deep neural network models with unlabeled data for remote sensing image (RSI) scene classification. Nevertheless, it still suffers from the following issues: 1) the performance of the contrastive learning method is significantly impacted by the hard negative sample (HNS) issue, since the RSI scenario is complex in semantics and rich in surface features; 2) the multiscale characteristic of RSI is missed in the existing contrastive learning methods; and 3) as the backbone of a deep learning model, especially in the case of limited annotation, a convolutional neural network (CNN) does not include the adequate receptive field of convolutional kernels to capture the broad contextual information of RSI. In this regard, we propose label-free self-distillation contrastive learning with a transformer architecture (LaST). We introduce the self-distillation contrastive learning mechanism to address the HNS issue. Specifically, the LaST architecture comprises two modules: scale alignment with a multicrop module and a long-range dependence capture backbone module. In the former, we present global–local crop and scale alignment to encourage local-to-global correspondence and acquire multiscale relations. Then, the distorted views are fed into a transformer as a backbone, which is good at capturing the long-range-dependent contextual information of the RSI while maintaining the spatial smoothness of the learned features. Experiments on public datasets show that in the downstream scene classification task, LaST improves the performance of the self-supervised trained model by a maximum of 2.18% compared to the HNS-impacted contrastive learning approaches, and only 1.5% of labeled data can achieve the performance of supervised training CNNs with 10% labeled data. Moreover, this letter supports the integration of a transformer architecture and self-supervised paradigms in RSI interpretation.
23 November 2023 in IEEE Transactions on Geoscience and Remote Sensing
Self-supervised contrastive learning (SSCL) has achieved significant milestones in remote sensing image (RSI) understanding. Its essence lies in designing an unsupervised instance discrimination pretext task to extract image features from a large number of unlabeled images that are beneficial for downstream tasks. However, existing instance discrimination based SSCL suffers from two limitations when applied to the RSI semantic segmentation task: 1) Positive sample confounding issue, SSCL treats different augmentations of the same RSI as positive samples, but the richness, complexity, and imbalance of RSI ground objects lead to the model actually pulling a variety of different ground objects closer while pulling positive samples closer, which confuse the feature of different ground objects. 2) Feature adaptation bias, SSCL treats RSI patches containing various ground objects as individual instances for discrimination and obtains instance-level features, which are not fully adapted to pixel-level or object-level semantic segmentation tasks. To address the above limitations, we consider constructing samples containing single ground objects to alleviate positive sample confounding issue, and make the model obtain object-level features from the contrastive between single ground objects. Meanwhile, we observed that the discrimination information can be mapped to specific regions in RSI through the gradient of unsupervised contrastive loss, these specific regions tend to contain single ground objects. Based on this, we propose contrastive learning with Gradient guided Sampling Strategy (GraSS) for RSI semantic segmentation. GraSS consists of two stages: 1) the instance discrimination warm-up stage to provide initial discrimination information to the contrastive loss gradients, 2) the gradient guided sampling contrastive training stage to adaptively construct samples containing more singular ground objects using the discrimination information. Experimental results on three open datasets demonstrate that GraSS effectively enhances the performance of SSCL in high-resolution RSI semantic segmentation. Compared to eight baseline methods from six different types of SSCL, GraSS achieves an average improvement of 1.57% and a maximum improvement of 3.58% in terms of mean intersection over the union. Additionally, we discovered that the unsupervised contrastive loss gradients contain rich feature information, which inspires us to utilize gradient information more extensively during model training to attain additional model capacity. The source code is available at https://github.com/GeoX-Lab/GraSS.
12 November 2022 in Innovation and Green Development
Owing to the limited field of view (FOV) and depth of field (DOF) of a conventional camera, it is quite difficult to employ a single conventional camera to simultaneously measure high-precision displacements at many points on a bridge of dozens or hundreds of meters. Researchers have attempted to obtain a large FOV and wide DOF by a multi-camera system; however, with the growth of the camera number, the cost, complexity and instability of multi-camera systems will increase exponentially. This study proposes a multi-point displacement measurement method for bridges based on a low-cost Scheimpflug camera. The Scheimpflug camera, which meets the Scheimpflug condition, can enlarge the depth of field of the camera without reducing the lens aperture and magnification; thus, when the measurement points are aligned in the depth direction, all points can be clearly observed in a single field of view with a high-power zoom lens. To reduce the impact of camera motions, a motion compensation method applied to the Scheimpflug camera is proposed according to the characteristic that the image plane is not perpendicular to the lens axis in the Scheimpflug camera. Several tests were conducted for performance verification under diverse settings. The results showed that the motion errors in x and y directions were reduced by at least 62% and 92%, respectively, using the proposed method, and the measurements of the camera were highly consistent with LiDAR-based measurements.