Semantic segmentation is used to enable a computer to understand its surrounding environment. In image processing, images are partitioned into segments for this purpose. State-of-the-art methods make use of Convolutional Neural Networks to segment a 2D image. Compared to that, 3D approaches suffer from computational cost and are not applicable without any further steps. In this work, we focus on semantic segmentation based on 3D point clouds. We use the idea to project the 3D data into a 2D image to accelerate the segmentation process. Afterward, the processed image gets re-projected to receive the desired result. We investigate different projection views and compare them to clarify their strengths and weaknesses. To compensate for projection errors and the loss of geometrical information, we evolve the approach and show how to fuse different views. We have decided to fuse the bird’s-eye and the spherical projection as each of them achieves reasonable results, and the two perspectives complement each other best. For training and evaluation, we use the real-world datasets SemanticKITTI. Further, we use the ParisLille and synthetic data generated by the simulation framework Carla to analyze the approaches in more detail and clarify their strengths and weaknesses. Although these methods achieve reasonable and competitive results, they lack flexibility. They depend on the sensor used and the setup in which the sensor is used.

In this paper, we present a workflow that combines supervised and unsupervised methods for the reconstruction of geometric models with architectural information from unordered 3D data. Our method uses a downsampling strategy to enrich features to provide scalability for large datasets, increase robustness, and be independent of the sensor used. A Neural Network is then used to segment the resulting point cloud into basic structures. This removes furniture and clutter and preserves the relevant walls, ceilings, floors, and openings. A 2D projection combined with a graph structure is used to find a Region of Interest within the cleaned point cloud, indicating a potential room. Each detected region is projected back into a 3D data patch to refine the room candidates and allow for more complex room structures. The resulting patches are fitted with a polygon using geometric approaches. In addition, architectural features, such as windows and doors, are added to the polygon. To demonstrate that the presented approach works and that the network provides usable results, even with changing data sources, we tested the approach in different real-world scenarios with different sensor systems.

Mobile mapping is an application field of ever-increasing relevance. Data of the surrounding environment is typically captured using combinations of LiDAR systems and cameras. The large amounts of measurement data are then processed and interpreted, which is often done automated using neural networks. For the evaluation the data of the LiDAR and the cameras needs to be fused, which requires a reliable calibration of the sensors. Segmentation solemnly on the LiDAR data drastically decreases the amount of data and makes the complex data fusion process obsolete but on the other hand often performs poorly due to the lack of information about the surface remission properties. The work at hand evaluates the effect of a novel multispectral LiDAR system on automated semantic segmentation of 3D-point clouds to overcome this downside. Besides the presentation of the multispectral LiDAR system and its implementation on a mobile mapping vehicle, the point cloud processing and the training of the CNN are described in detail. The results show a significant increase in the mIoU when using the additional information from the multispectral channel compared to just 3D and intensity information. The impact on the IoU was found to be strongly dependent on the class.

In recent decades, the condition of many bridge structures has deteriorated and the need for maintenance measures has increased. Until now, these maintenance measures have mainly been carried out manually and reactively. The use of digital 3D models based on Building Information Modelling (BIM) can remedy this situation and create the basis for predictive maintenance management. While the generation of 3D models of new bridge structures is simple, the digitization of existing structures can be a complex process. This article provides an overview of the state of the scientific practice with regard to the procedures, technologies and data used to generate 3D models of existing bridge structures using the BIM method. Using a systematic literature analysis, scientific databases are searched for suitable literature and analysed with predefined filtering parameters. The results provide a uniform understanding of the current status quo of the digitisation of existing bridge structures and show existing degrees of digitisation and automation.

While most recent work in room instance segmentation relies on orthographic top-down projections of 3D point clouds to 2D density maps, leading to information loss of one dimension, 3D instance segmentation methods based on deep learning were rarely considered. We explore the potential of the general 3D instance segmentation deep learning model Mask3D for room instance segmentation in indoor building point clouds. We show that Mask3D generates meaningful predictions for multi-floor scenes. After hyperparameter optimization, Mask3D outperforms the current state-of-the-art method RoomFormer evaluated in 3D on the synthetic Structured3D dataset. We provide generalization results of Mask3D trained on Structured3D to the real-world S3DIS and Matterport3D datasets, showing a domain gap. Fine-tuning improves the results. In contrast to related work in room instance segmentation, we employ the more expressive mean average precision (mAP) metric, and we propose the more intuitive successfully detected rooms (SDR) metric, which is an absolute recall measure. Our results indicate potential for the digitization of the construction industry.