Three-dimensional (3-D) urban models are an integral part of numerous applications, such as urban planning and performance simulation, mapping and visualization, emergency response training and entertainment, among others. We consolidate various algorithms proposed for reconstructing 3-D models of urban objects from point clouds. Urban models addressed in this review include buildings, vegetation, utilities such as roads or power lines and free-form architectures such as curved buildings or statues, all of which are ubiquitous in a typical urban scenario. While urban modeling, building reconstruction, in particular, clearly demand specific traits in the models, such as regularity, symmetry, and repetition; most of the traditional and state-of-the-art 3-D reconstruction algorithms are designed to address very generic objects of arbitrary shapes and topology. The recent efforts in the urban reconstruction arena, however, strive to accommodate the various pressing needs of urban modeling. Strategically, urban modeling research nowadays focuses on the usage of specialized priors, such as global regularity, Manhattan-geometry or symmetry to aid the reconstruction, or efficient adaptation of existing reconstruction techniques to the urban modeling pipeline. Aimed at an in-depth exploration of further possibilities, we review the existing urban reconstruction algorithms, prevalent in computer graphics, computer vision and photogrammetry disciplines, evaluate their performance in the architectural modeling context, and discuss the adaptability of generic mesh reconstruction techniques to the urban modeling pipeline. In the end, we suggest a few directions of research that may be adopted to close in the technology gaps.

Geometric primitives that consist of a group of discrete points may be viewed as one kind of abstraction and representation of lidar data at the entity level. In recent years, many efforts from different scientific communities, such as photogrammetry, computer vision and computer graphics, have been made into geometric primitive detection, regularization, and in-depth applications. The most recent lidar-based surveys focus only on reconstruction, object segmentation, and recognition, as well as the data processing techniques based on a specific platform such as mobile LiDAR. However, in this paper, lidar point clouds are understood from a new perspective, i.e., geometric primitives embedded in versatile objects in physical world. We categorize geometric primitives into two classes: shape primitives, e.g., lines, surfaces, and volumetric shapes, as well as structure primitives, represented by skeletons and edges. Interpretations of geometric primitive from multiple disciplines try to convey the significance of geometric primitives, the latest processing techniques regarding geometric primitives, and their potential possibilities in the context of lidar point clouds. To this end, applications of these primitives are reviewed with an emphasis on object extraction and reconstruction to clearly show the significances of this paper. Next, we survey and compare methods for geometric primitive extraction and then survey primitive regularization methods that add real-world geometry constrains to detected primitives. Finally, we summarize the problems and challenges and describe possible future for primitive extraction methods that can be achieved globally optimal results efficiently, even with disorganized, uneven, noisy, incomplete, and large-scale lidar point clouds.

This paper presents a novel framework to extract metro tunnel cross sections (profiles) from Terrestrial Laser Scanning point clouds. The entire framework consists of two steps: tunnel central axis extraction and cross section determination. In tunnel central extraction, we propose a slice-based method to obtain an initial central axis, which is further divided into linear and nonlinear circular segments by an enhanced Random Sample Consensus (RANSAC) tunnel axis segmentation algorithm. This algorithm transforms the problem of hybrid linear and nonlinear segment extraction into a sole segmentation of linear elements defined at the tangent space rather than raw data space, significantly simplifying the tunnel axis segmentation. The extracted axis segments are then provided as input to the step of the cross section determination which generates the coarse cross-sectional points by intersecting a series of straight lines that rotate orthogonally around the tunnel axis with their local fitted quadric surface, i.e., cylindrical surface. These generated profile points are further refined and densified via solving a constrained nonlinear least squares problem. Our experiments on Nanjing metro tunnel show that the cross sectional fitting error is only 1.69 mm. Compared with the designed radius of the metro tunnel, the RMSE (Root Mean Square Error) of extracted cross sections’ radii only keeps 1.60 mm. We also test our algorithm on another metro tunnel in Shanghai, and the results show that the RMSE of radii only keeps 4.60 mm which is superior to a state-of-the-art method of 6.00 mm. Apart from the accurate geometry, our approach can maintain the correct topology among cross sections, thereby guaranteeing the production of geometric tunnel model without crack defects. Moreover, we prove that our algorithm is insensitive to the missing data and point density.

In this paper, we propose a hierarchical building detection framework based on deep learning model, which focuses on accurately detecting buildings from remote sensing images. To this end, we first construct the generation model of the multilevel training samples using the Gaussian pyramid technique to learn the features of building objects at different scales and spatial resolutions. Then, the building region proposal networks are put forward to quickly extract candidate building regions, thereby increasing the efficiency of the building object detection. Based on the candidate building regions, we establish the multilevel building detection model using the convolutional neural networks (CNNs), from which the generic image features of each building region proposal are calculated. Finally, the obtained features are provided as inputs for training CNNs model, and the learned model is further applied to test images for the detection of unknown buildings. Various experiments using the Datasets I and II (in Section V-A) show that the proposed framework increases the mean average precision values of building detection by 3.63%, 3.85%, and 3.77%, compared with the state-of-the-art methods, i.e., Method IV. Besides, the proposed method is robust to the buildings having different spatial textures and types.

Due to errors in sensors and positioning, there exist mismatches between different phases of mobile laser scanning point clouds, which impede the application of point cloud, such as changing detection and deformation monitoring. To rectify such mismatches, we designed a 3-D deep feature construction method for point cloud registration. The proposed method combines two 3-D convolutional neural networks into a uniform deep learning model to extract 3-D deep features. First, the corresponding points and noncorresponding points are set to train the deep learning model to minimize the distance between corresponding points' features and maximize the distance between features of noncorresponding points. Second, in the test phase, the 3-D deep feature for each keypoint was extracted by the trained deep learning model. This could be used to determine the corresponding points by the k-dimensional tree and random sample consensus (RANSAC) algorithm. Finally, a transformation matrix was calculated based on the corresponding points and was then applied to point cloud registration. The experimental results illustrated that the proposed method of using 3-D deep features is more efficient at a corresponding point search than representatives of three existing methods. It also improved registration accuracy.

Large-scale point clouds scanned by light detection and ranging (lidar) sensors provide detailed geometric characteristics of the indoor and outdoor scenes due to the provision of 3D structural data colored by intensity/reflectance information. The semantic segmentation of large-scale point clouds is a crucial step for an in-depth understanding of complex scenes. Of late, although a large number of point cloud semantic segmentation algorithms have been proposed, semantic segmentation methods are still far from being satisfactory in terms of precision and accuracy of large-scale point clouds. For machine learning (ML) and deep learning (DL) methodologies, the semantic segmentation is largely influenced by the quality of training sets and methods themselves. Therefore, we construct a new point cloud dataset, namely CSPC-Dataset (Complex Scene Point Cloud Dataset) for large-scale scene semantic segmentation. CSPC-Dataset point clouds are acquired by a wearable laser mobile mapping robot. It covers five complex urban and rural scenes and mainly includes six types of objects, i.e., ground, car, building, vegetation, bridge, and pole. It provides large-scale outdoor scenes, which has advantages such as the scene more complete, point density relatively uniform, diversity and complexity of objects and the high discrepancy between different scenes. Based on the CSPC-Dataset, we construct a new benchmark, which includes approximately 68 million points with explicit semantic labels. To extend the dataset into a wide range of applications, this paper provides the semantic segmentation results and comparative analysis of 7 baseline methods based on CSPC-Dataset. In the experiment part, three groups of experiments are conducted for benchmarking, which offers an effective way to make comparisons with different point-labeling algorithms. The labeling results have shown that the highest Intersection over Union (IoU) of pole, ground, building, car, vegetation, and bridge for all benchmarks is 36.0%, 97.8%, 93.7%, 65.6%, 92.0%, and 69.6%.