The capture of 3D reality has demonstrated increased efficiency and consistently accurate outcomes in architectural digitisation. Nevertheless, despite advancements in data collection, 3D reality-based modelling still lacks full automation, especially in the post-processing and modelling phase. Artificial intelligence (AI) has been a significant focus, especially in computer vision, and tasks such as image classification and object recognition might be beneficial for the digitisation process and its subsequent utilisation. This study aims to examine the potential outcomes of integrating AI technology into the field of 3D reality-based modelling, with a particular focus on its use in architecture and cultural-heritage scenarios. The main methods used for data collection are laser scanning (static or mobile) and photogrammetry. As a result, image data, including RGB-D data (files containing both RGB colours and depth information) and point clouds, have become the most common raw datasets available for object mapping. This study comprehensively analyses the current use of 2D and 3D deep learning techniques in documentation tasks, particularly downstream applications. It also highlights the ongoing research efforts in developing real-time applications with the ultimate objective of achieving generalisation and improved accuracy.

In recent years, the investigation and 3D documentation of architectural heritage has made an efficient digitalization process possible and allowed for artificial intelligence post-processing on point clouds. This article investigates the multilevel multiresolution methodology using machine learning classification algorithms on three point-cloud projects in China: Nanchan Ssu, Fokuang Ssu, and Kaiyuan Ssu. The performances obtained by extending the prediction to datasets other than those used to train the machine learning algorithm are compared against those obtained with a standard approach. Furthermore, the classification results obtained with an MLMR approach are compared against a standard single-pass classification. This work proves the reliability of the MLMR classification of heritage point clouds and its good generalizability across scenarios with similar geometrical characteristics. The pros and cons of the different approaches are highlighted.

Image classification and object detection techniques have been widely discussed and developed in recent years; they are the basis of various prosperous applications, for example, real-time mapping. Promising as it is, the practical test in the cultural heritage field encountered multiple problems. In this paper, the authors attempt to share the research experimentations and the empirical knowledge focusing on the classification and detection of architectural pathology. The tests are built on elaborated training sets annotated with analysed and in-advance defined categories. The trained models were examined from the perspective of evaluation sets, model explanation and unseen datasets. The outcomes indicated the mistakes and confusions behind things and stuff in the object detection efforts, to which cultural heritage and architectural field are closely related. The model also reveals specific visual patterns for recognition from thousands of instances in the training set. By digging into different aspects of model performance, the potential and limitations of these techniques in practical applications can be better understood.

Historical buildings and monuments are typically subject to degradation over time due to the passage of time and constant exposure to external agents. The use of artificial intelligence (AI) to support the work of conservation and restoration specialists in identifying surface decay is a research topic of considerable interest at present. This study presents two approaches: ChatGPT and an object detection architecture (YOLOv5). Specifically, this investigation sought to evaluate the ChatGPT’s ability to identify and describe surface degradation pathologies by exploiting its pre-trained models for image analysis. The ICOMOS-ISCS: Illustrated Glossary on Stone Deterioration Patterns (2008) was provided as a reference to guide the use of specific terminology. In the first test phase, to verify the accuracy of the ChatGPT results, benchmark images (depicting different types of damage) extracted from the UNI 11182 (2006) standard referring to the definition of degradation types were used. Only later were images from literature studies and other photographic datasets also used. In general, the results of the analysis were validated with the conclusions of professionals and with the conclusions of other AI techniques, as well as with the descriptions provided by reference manuals in the literature. In particular, the decay annotations predicted by the pre-trained object detection model were compared with those made by human experts. The capabilities and limitations of both approaches as tools for identifying deterioration pathologies are illustrated.

3D reality capturing has demonstrated increased efficiency and consistently accurate outcomes in architectural digitisation. Nevertheless, despite advancements in data collecting, 3D reality capturing still lacks full automation, especially in the post-processing and modelling phase. Artificial intelligence (AI) has been a significant focus, especially in computer vision, and tasks such as image classification and object recognition might be beneficial to the digitisation process and its subsequent utilisation. This article aims to examine the potential outcomes of integrating AI technology into the field of 3D reality-capturing, with a particular focus on its use in architectural scenarios. The main methods used for data collection are laser scanning (static or mobile) and photogrammetry. As a result, image data including RGB-D data (files containing both RGB colours and Depth information) and point clouds have become the most common raw datasets available for object mapping. The study comprehensively analyses the current use of 2D and 3D deep learning techniques in documentation tasks, particularly downstream applications. It also highlights the ongoing research efforts in developing real-time applications with the ultimate objective of achieving generalisation and improved accuracy.

3D digitization has become an essential tool in cultural heritage documentation, offering unprecedented opportunities for preservation, analysis, and dissemination. Beyond only capturing 3D spatial geometry, the semantic enrichment of 3D models is rapidly evolving offering a more efficient interpretation and usage of 3D data. Traditionally, 3D semantic enrichment has relied on point cloud-based segmentation. However, 3D point cloud-based segmentation approaches can struggle with the efficient identification of small-scale, geometric elements, or visually ambiguous classes, limiting their applicability in such contexts. This study leverages the rich contextual and textural information of 2D imagery to detect challenging semantic categories, such as fine architectural elements (e.g., individual stone blocks) and material decay (e.g., material detachment and material loss), using deep learning-based 2D semantic segmentation techniques. These detections are then projected into 3D space through a 2D-to-3D semantic segmentation framework that couples V-SLAM and 3D results with the 2D predictions. The framework is evaluated on data acquired using the fish-eye multi-camera mobile mapping system ATOM-ANT3D in two challenging case study environments. Achieved results demonstrate a reliable level of accu-racy given the inherent complexity of targeted classes, enhancing the interpretability of 3D models by providing meaningful and met-rically interpreted objects classifications in 3D models. (Demonstration video: https://youtu.be/GidxhNS7ECc)