It is difficult to extract a digital elevation model (DEM) from an airborne laser scanning (ALS) point cloud in a forest area because of the irregular and uneven distribution of ground and vegetation points. Machine learning, especially deep learning methods, has shown powerful feature extraction in accomplishing point cloud classification. However, most of the existing deep learning frameworks, such as PointNet, dynamic graph convolutional neural network (DGCNN), and SparseConvNet, cannot consider the particularity of ALS point clouds. For large-scene laser point clouds, the current data preprocessing methods are mostly based on random sampling, which is not suitable for DEM extraction tasks. In this study, we propose a novel data sampling algorithm for the data preparation of patch-based training and classification named T-Sampling. T-Sampling uses the set of the lowest points in a certain area as basic points with other points added to supplement it, which can guarantee the integrity of the terrain in the sampling area. In the learning part, we propose a new convolution model based on terrain named Tin-EdgeConv that fully considers the spatial relationship between ground and non-ground points when constructing a directed graph. We design a new network based on Tin-EdgeConv to extract local features and use PointNet architecture to extract global context information. Finally, we combine this information effectively with a designed attention fusion module. These aspects are important in achieving high classification accuracy. We evaluate the proposed method by using large-scale data from forest areas. Results show that our method is more accurate than existing algorithms.

Ground filtering (GF) is a fundamental step for airborne laser scanning (ALS) data processing. The advent of deep learning techniques provides new solutions to this problem. Existing deep-learning-based methods utilize a segmentation or classification framework to extract ground/non-ground points, which suffers from a dilemma in keeping high spatial resolution while acquiring rich contextual information when dealing with large-scale ALS data due to the computing resource limits. To this end, we propose SeqGP, a novel deep-learning-based GF pipeline that explicitly converts the GF task into an iterative sequential ground prediction (SeqGP) problem using points-profiles. The proposed SeqGP utilizes deep reinforcement learning (DRL) to optimize the prediction sequence and retrieve the bare terrain gradually. The 3D sparse convolution is integrated with the SeqGP strategy to generate high-precision classification results with memory efficiency. Extensive experiments on two challenging test sets demonstrate the state-of-the-art filtering performance and universality of the proposed method in dealing with large-scale ALS data.

Semantic segmentation of airborne laser scanning (ALS) point clouds is a valuable yet challenging task in remote sensing. When processing large-scale ALS scenes, it is necessary to partition them into smaller blocks for ease of handling. However, this partitioning introduces a challenge in capturing the ample spatial context within each block to adequately recognize the objects with a significant spatial span. This limitation becomes particularly pronounced when relying solely on the 3-D representations as the input of neural networks. To incorporate sufficient contextual information in ALS data semantic segmentation, we propose a multimodal-based segmentation framework called projection-based context embedding (PCE) in this study. PCE effectively combines the advantages of 2-D image and 3-D point-voxel representations, which are the computational efficiency and the representation capability for fine-grained 3-D geometries. The 2-D projection is used to encode a large-scale semantic context, which is computationally expensive to be obtained using only pure 3-D representation. Simultaneously, the sparse-point-voxel convolution (SPVConv) is employed to focus on learning 3-D features from a small block of points centered on the large-scale context. Finally, to fully exploit the power of each modality, the embedding disentangling (ED) strategy is proposed additionally to combine the context embedding from the 2-D image with 3-D features for the final prediction. We demonstrate the state-of-the-art performance of PCE through extensive experiments on public large-scale ALS point cloud datasets.

Automatic extraction of vector polygons of buildings from remotely sensed images is an important but difficult task. Recent existing methods based on deep learning usually adopt a multi-stage solution of semantic segmentation, contour detection, and polygon simplification. Such a long processing chain may lead to unreliable results as the boundary regularization and optimization processes are ultimately completed by utilizing low-level features, which ignores the potential of deep features in polygon generation. In this paper, we present an algorithm for directly extracting simplified polygons of buildings in remotely sensed images. The key of this task is the encoding of the polygon structure. PolyMapper [1] utilizes a recurrent neural network (RNN) to produce vertices of a polygon sequentially. Due to the limitation of RNN, this approach is unstable and difficult to deal with objects with complex shapes. In this work, we encode the polygon into a tensor representation and utilize a non-recurrent manner to recover the polygon structure. In our algorithm, two types of points are utilized, i.e., the corner point and the connecting point. Corner points are utilized to delineate the building outlines and form the vertices of the final polygon. Meanwhile, connecting points are sampled from the edges of the buildings for the assistance of the connection of the corner points. Furthermore, we predict the forward and backward directions of each keypoint in a polygon and propose a bidirectional tracing strategy for the polygon structure recovery. Our approach is simple, effective and robust. Experiments on public datasets demonstrate the superiority of the proposed algorithm. The code is made publicly available at https://github.com/sz94/bldvec.

Accurate building extraction from remotely sensed images is essential for topographic mapping, cadastral surveying and many other applications. Fully automatic segmentation methods still remain a great challenge due to the poor generalization ability and the inaccurate segmentation results. In this work, we are committed to robust click-based interactive building extraction in remote sensing imagery. We argue that stability is vital to an interactive segmentation system, and we observe that the distance of the newly added click to the boundaries of the previous segmentation mask contains progress guidance information of the interactive segmentation process. To promote the robustness of the interactive segmentation, we exploit this information with the previous segmentation mask, positive and negative clicks to form a progress guidance map, and feed it to a convolutional neural network (CNN) with the original RGB image, we name the network as PGR-Net. In addition, an adaptive zoom-in strategy and an iterative training scheme are proposed to further promote the stability of PGR-Net. Compared with the latest methods FCA and f-BRS, the proposed PGR-Net basically requires 1–2 fewer clicks to achieve the same segmentation results. Comprehensive experiments have demonstrated that the PGR-Net outperforms related state-of-the-art methods on five natural image datasets and three building datasets of remote sensing images.