Digital twins are increasingly gaining popularity as a method for simulating intricate natural and urban environments, with the precise segmentation of 3D objects playing an important role. This study focuses on developing a methodology for extracting buildings from textured 3D meshes, employing the PicassoNet-II semantic segmentation architecture. Additionally, we integrate Markov field-based contextual analysis for post-segmentation assessment and cluster analysis algorithms for building instantiation. Training a model to adapt to diverse datasets necessitates a substantial volume of annotated data, encompassing both real data from Quebec City, Canada, and simulated data from Evermotion and Unreal Engine. The experimental results indicate that incorporating simulated data improves segmentation accuracy, especially for under-represented features, and the DBSCAN algorithm proves effective in extracting isolated buildings. We further show that the model is highly sensible for the method of creating 3D meshes.

Digital twins are gaining in popularity for simulating complex natural and urban environments. In this context, accurate segmentation of objects within 3D urban environments is of crucial importance. The aim of this project is to develop a methodology for extracting buildings from textured 3D meshes. To this end, PicassoNet-II, a semantic segmentation architecture is employed. The methodology also incorporates Markov field-based contextual analysis to assess post-segmentation features. In addition, building instantiation is performed using cluster analysis algorithms. Training this model to fit various datasets requires a large amount of annotated data, both from Quebec City, Canada, and from simulated data. Experimental results show that the use of simulated data improves segmentation accuracy, and the DBScan algorithm proves effective in extracting isolated buildings. This project paves the way for improved applications in 3D urban modeling, offering opportunities in fields ranging from urban planning to resource management. The positive results with simulated data reinforce the impact of this research on improving digital models of our ever-changing urban environments.