This work presents a semi-automatic approach to the 3D reconstruction of Heritage-Building Information Models from point clouds based on machine learning techniques. The use of digital information systems leveraging on three-dimensional (3D) representations in architectural heritage documentation and analysis is ever increasing. For the creation of such repositories, reality-based surveying techniques, such as photogrammetry and laser scanning, allow the fast collection of reliable digital replicas of the study objects in the form of point clouds. Besides, their output is raw and unstructured, and the transition to intelligible and semantic 3D representations is still a scarcely automated and time-consuming process requiring considerable human intervention. More refined methods for 3D data interpretation of heritage point clouds are therefore sought after. In tackling these issues, the proposed approach relies on (i) the application of machine learning techniques to semantically label 3D heritage data by identification of relevant geometric, radiometric and intensity features, and (ii) the use of the annotated data to streamline the construction of Heritage-Building Information Modeling (H-BIM) systems, where purely geometric information derived from surveying is associated with semantic descriptors on heritage documentation and management. The “Grand-Ducal Cloister” dataset, related to the emblematic case study of the Pisa Charterhouse, is discussed.

We propose a semi-automatic Scan-to-BIM reconstruction approach, making the most of Artificial Intelligence (AI) techniques, for the classification of digital architectural heritage data. Nowadays, Heritage- or Historic-Building Information Modeling (H-BIM) reconstruction from laser scanning or photogrammetric surveys is a manual, time-consuming, overly subjective process, but the emergence of AI techniques, applied to the realm of existing architectural heritage, is offering new ways to interpret, process and elaborate raw digital surveying data, as point clouds. The proposed methodological approach for higher-level automation in Scan-to-BIM reconstruction is threaded as follows: (i) semantic segmentation via Random Forest and import of annotated data in 3D modeling environment, broken down class by class; (ii) reconstruction of template geometries of classes of architectural elements; (iii) propagation of template reconstructed geometries to all elements belonging to a typological class. Visual Programming Languages (VPLs) and reference to architectural treatises are leveraged for the Scan-to-BIM reconstruction. The approach is tested on several significant heritage sites in the Tuscan territory, including charterhouses and museums. The results suggest the replicability of the approach to other case studies, built in different periods, with different construction techniques or under different states of conservation.

This paper conducts a comparative evaluation between Neural Radiance Fields (NeRF) and photogrammetry for 3D reconstruction in the cultural heritage domain. Focusing on three case studies, of which the Terpsichore statue serves as a pilot case, the research assesses the quality, consistency, and efficiency of both methods. The results indicate that, under conditions of reduced input data or lower resolution, NeRF outperforms photogrammetry in preserving completeness and material description for the same set of input images (with known camera poses). The study recommends NeRF for scenarios requiring extensive area mapping with limited images, particularly in emergency situations. Despite NeRF’s developmental stage compared to photogrammetry, the findings demonstrate higher potential for describing material characteristics and rendering homogeneous textures with enhanced visual fidelity and accuracy; however, NeRF seems more prone to noise effects. The paper advocates for the future integration of NeRF with photogrammetry to address respective limitations, offering more comprehensive representation for cultural heritage preservation tasks. Future developments include extending applications to planar surfaces and exploring NeRF in virtual and augmented reality, as well as studying NeRF evolution in line with emerging trends in semantic segmentation and in-the-wild scene reconstruction.

Neural Radiance Fields (NeRF or NeRFs) are to date emerging as a novel method for synthesizing novel views of complex 3D scenes, leveraging an artificial neural network to optimize a volumetric scene function using a set of input views. We conduct a preliminary critical review of the scientific and technical literature on NeRFs, and we highlight possible applications of the latter in the Cultural Heritage domain, for the image-based reconstruction of 3D models of real, multi-scale objects, even in combination with the more well-established photogrammetric techniques. A comparison is made between NeRFs and photogrammetry in terms of operating procedures and outputs (volumetric renderings vs. point clouds or meshes). It is demonstrated that NeRFs could be conveniently used for rendering objects (sculptures, archaeological remains, sites, paintings etc.) that are challenging for photogrammetry, typically: i) metallic, translucent, and/or transparent surfaces; ii) objects that present homogeneous textures; iii) occlusions, vegetation, and elements of very fine detail.

Research in the field of Cultural Heritage is increasingly moving towards the creation of digital information systems, in which the geometric representation of an artifact is linked to some external information, through meaningful tags. The process of attributing additional and structured information to various elements in a given digital model is customarily identified with the term semantic annotation; the added contextual information is associated, for instance, to analysis and conservation terms. Starting from the existing literature, aim of this work is to discuss how semantic annotations are used, in digital architectural heritage models, to link the geometrical representation of an artefact with knowledge-related information. Most consolidated methods -such as traditional mapping on 2D media, are compared with more recent approaches making the most of 3D representation. Reference is made, in particular, to Heritage-BIM techniques and to collaborative reality-based platforms, such as Aïoli (http://aioli.cloud). Potentialities and limits of the different solutions proposed in literature are critically discussed, also addressing future research challenges in Cultural Heritage application fields.

The evolution of Structure from Motion (SfM) techniques and their integration with the established procedures of classic stereoscopic photogrammetric survey have provided a very effective tool for the production of three-dimensional textured models. Such models are not only aesthetically pleasing but can also contain metric information, the quality of which depends on both survey type and applied processing methodologies. An open research topic in this area refers to checking attainable accuracy levels. The knowledge of such accuracy is essential, especially in the integration of models obtained through SfM with other models derived from different sensors or methods (laser scanning, classic photogrammetry ...). Accuracy checks may be conducted by either comparing SfM models against a reference one or measuring the deviation of control points identified on models and measured with classic topographic instrumentation and methodologies. This paper presents an analysis of attainable accuracy levels, according to different approaches of survey and data processing. For this purpose, a survey of the Church of San Miniato in Marcianella (Pisa, Italy), has been used. The dataset is an integration of laser scanning with terrestrial and UAV-borne photogrammetric surveys; in addition, a high precision topographic network was established for the specific purpose. In particular, laser scanning has been used for the interior and the exterior of the church, with the exclusion of the roof, while UAVs have been used for the photogrammetric survey of both roof, with horizontal strips, and façade, with vertical strips.