Three-dimensional (3-D) urban models are an integral part of numerous applications, such as urban planning and performance simulation, mapping and visualization, emergency response training and entertainment, among others. We consolidate various algorithms proposed for reconstructing 3-D models of urban objects from point clouds. Urban models addressed in this review include buildings, vegetation, utilities such as roads or power lines and free-form architectures such as curved buildings or statues, all of which are ubiquitous in a typical urban scenario. While urban modeling, building reconstruction, in particular, clearly demand specific traits in the models, such as regularity, symmetry, and repetition; most of the traditional and state-of-the-art 3-D reconstruction algorithms are designed to address very generic objects of arbitrary shapes and topology. The recent efforts in the urban reconstruction arena, however, strive to accommodate the various pressing needs of urban modeling. Strategically, urban modeling research nowadays focuses on the usage of specialized priors, such as global regularity, Manhattan-geometry or symmetry to aid the reconstruction, or efficient adaptation of existing reconstruction techniques to the urban modeling pipeline. Aimed at an in-depth exploration of further possibilities, we review the existing urban reconstruction algorithms, prevalent in computer graphics, computer vision and photogrammetry disciplines, evaluate their performance in the architectural modeling context, and discuss the adaptability of generic mesh reconstruction techniques to the urban modeling pipeline. In the end, we suggest a few directions of research that may be adopted to close in the technology gaps.

Recently, approaches based on fully convolutional networks (FCN) have achieved state-of-the-art performance in the semantic segmentation of very high resolution (VHR) remotely sensed images. One central issue in this method is the loss of detailed information due to downsampling operations in FCN. To solve this problem, we introduce the maximum fusion strategy that effectively combines semantic information from deep layers and detailed information from shallow layers. Furthermore, this letter develops a powerful backend to enhance the result of FCN by leveraging the digital surface model, which provides height information for VHR images. The proposed semantic segmentation scheme has achieved an overall accuracy of 90.6% on the ISPRS Vaihingen benchmark.

The iterative closest point (ICP) algorithm is widely used in three-dimensional (3D) point cloud registration, and it is very stable and robust. However, its biggest drawback is being easily trapped in a local optimal solution, which results in the incorrect registration result. Currently, there is neither a clear effective range to define whether the ICP algorithm will fall into a local optimum nor a study providing a comprehensive evaluation of the ICP algorithm. In this paper, we take the overlap ratio, angle, distance, and noise as the influencing factors of ICP and evaluate the validity, robustness, accuracy, and efficiency of point-to-point and point-to-plane ICP by using four datasets. We first analyze the effective ranges of the two ICP algorithms with respect to overlap ratio, angle, and distance and then propose a universal effective range for the three factors. Next, the effect of Gaussian noise on the validity and accuracy of two ICPs is evaluated. We also analyze the factors influencing ICP accuracy and explain their changing rules. We finally study the effect of different parameters on the efficiency. All results are compared by using point-to-point and point-to-plane ICP algorithms. The results show that the overlap ratio has no effect on validity, but it has a significant influence on accuracy. The angle has a great impact on the validity and efficiency of ICP but has no effect on accuracy. The distance only affects validity, has a limited effect on efficiency and no effect on accuracy. Meanwhile, Gaussian noise has a little effect on the validity. In addition, the general effective range of point-to-point ICP is larger than that of point-to-plane ICP, but the point-to-plane ICP algorithm presents a better efficiency. The point-to-point ICP is more robust to Gaussian noise with respect to validity, while the point-to-plane is more resistant in terms of accuracy.

The mobile laser scanning (MLS) technique has attracted considerable attention for providing high-density, high-accuracy, unstructured, three-dimensional (3D) geo-referenced point-cloud coverage of the road environment. Recently, there has been an increasing number of applications of MLS in the detection and extraction of urban objects. This paper presents a systematic review of existing MLS related literature. This paper consists of three parts. Part 1 presents a brief overview of the state-of-the-art commercial MLS systems. Part 2 provides a detailed analysis of on-road and off-road information inventory methods, including the detection and extraction of on-road objects (e.g., road surface, road markings, driving lines, and road crack) and off-road objects (e.g., pole-like objects and power lines). Part 3 presents a refined integrated analysis of challenges and future trends. Our review shows that MLS technology is well proven in urban object detection and extraction, since the improvement of hardware and software accelerate the efficiency and accuracy of data collection and processing. When compared to other review papers focusing on MLS applications, we review the state-of-the-art road object detection and extraction methods using MLS data and discuss their performance and applicability. The main contribution of this review demonstrates that the MLS systems are suitable for supporting road asset inventory, ITS-related applications, high-definition maps, and other highly accurate localization services.

3D modeling from images and LiDAR (Light Detection And Ranging) has been an active research area in the photogrammetry, computer vision, and computer graphics communities. In terms of literature review, a comprehensive survey on 3D building modeling that contains methods from all these fields will be beneficial. This article attempts to survey the state-of-the-art 3D building modeling methods in the areas of photogrammetry, computer vision, and computer graphics. The existing methods are grouped into three categories: 3D reconstruction from images, 3D modeling using range data, and 3D modeling using images and range data. The use of both data for 3D modeling is a sensor fusion approach, in which methods of image-to-LiDAR registration, upsampling, and image-guided segmentation are reviewed. For each category, the key problems are identified and solutions are addressed.

Geometric primitives that consist of a group of discrete points may be viewed as one kind of abstraction and representation of lidar data at the entity level. In recent years, many efforts from different scientific communities, such as photogrammetry, computer vision and computer graphics, have been made into geometric primitive detection, regularization, and in-depth applications. The most recent lidar-based surveys focus only on reconstruction, object segmentation, and recognition, as well as the data processing techniques based on a specific platform such as mobile LiDAR. However, in this paper, lidar point clouds are understood from a new perspective, i.e., geometric primitives embedded in versatile objects in physical world. We categorize geometric primitives into two classes: shape primitives, e.g., lines, surfaces, and volumetric shapes, as well as structure primitives, represented by skeletons and edges. Interpretations of geometric primitive from multiple disciplines try to convey the significance of geometric primitives, the latest processing techniques regarding geometric primitives, and their potential possibilities in the context of lidar point clouds. To this end, applications of these primitives are reviewed with an emphasis on object extraction and reconstruction to clearly show the significances of this paper. Next, we survey and compare methods for geometric primitive extraction and then survey primitive regularization methods that add real-world geometry constrains to detected primitives. Finally, we summarize the problems and challenges and describe possible future for primitive extraction methods that can be achieved globally optimal results efficiently, even with disorganized, uneven, noisy, incomplete, and large-scale lidar point clouds.