This work presents a semi-automatic approach to the 3D reconstruction of Heritage-Building Information Models from point clouds based on machine learning techniques. The use of digital information systems leveraging on three-dimensional (3D) representations in architectural heritage documentation and analysis is ever increasing. For the creation of such repositories, reality-based surveying techniques, such as photogrammetry and laser scanning, allow the fast collection of reliable digital replicas of the study objects in the form of point clouds. Besides, their output is raw and unstructured, and the transition to intelligible and semantic 3D representations is still a scarcely automated and time-consuming process requiring considerable human intervention. More refined methods for 3D data interpretation of heritage point clouds are therefore sought after. In tackling these issues, the proposed approach relies on (i) the application of machine learning techniques to semantically label 3D heritage data by identification of relevant geometric, radiometric and intensity features, and (ii) the use of the annotated data to streamline the construction of Heritage-Building Information Modeling (H-BIM) systems, where purely geometric information derived from surveying is associated with semantic descriptors on heritage documentation and management. The “Grand-Ducal Cloister” dataset, related to the emblematic case study of the Pisa Charterhouse, is discussed.

We propose a semi-automatic Scan-to-BIM reconstruction approach, making the most of Artificial Intelligence (AI) techniques, for the classification of digital architectural heritage data. Nowadays, Heritage- or Historic-Building Information Modeling (H-BIM) reconstruction from laser scanning or photogrammetric surveys is a manual, time-consuming, overly subjective process, but the emergence of AI techniques, applied to the realm of existing architectural heritage, is offering new ways to interpret, process and elaborate raw digital surveying data, as point clouds. The proposed methodological approach for higher-level automation in Scan-to-BIM reconstruction is threaded as follows: (i) semantic segmentation via Random Forest and import of annotated data in 3D modeling environment, broken down class by class; (ii) reconstruction of template geometries of classes of architectural elements; (iii) propagation of template reconstructed geometries to all elements belonging to a typological class. Visual Programming Languages (VPLs) and reference to architectural treatises are leveraged for the Scan-to-BIM reconstruction. The approach is tested on several significant heritage sites in the Tuscan territory, including charterhouses and museums. The results suggest the replicability of the approach to other case studies, built in different periods, with different construction techniques or under different states of conservation.

This paper conducts a comparative evaluation between Neural Radiance Fields (NeRF) and photogrammetry for 3D reconstruction in the cultural heritage domain. Focusing on three case studies, of which the Terpsichore statue serves as a pilot case, the research assesses the quality, consistency, and efficiency of both methods. The results indicate that, under conditions of reduced input data or lower resolution, NeRF outperforms photogrammetry in preserving completeness and material description for the same set of input images (with known camera poses). The study recommends NeRF for scenarios requiring extensive area mapping with limited images, particularly in emergency situations. Despite NeRF’s developmental stage compared to photogrammetry, the findings demonstrate higher potential for describing material characteristics and rendering homogeneous textures with enhanced visual fidelity and accuracy; however, NeRF seems more prone to noise effects. The paper advocates for the future integration of NeRF with photogrammetry to address respective limitations, offering more comprehensive representation for cultural heritage preservation tasks. Future developments include extending applications to planar surfaces and exploring NeRF in virtual and augmented reality, as well as studying NeRF evolution in line with emerging trends in semantic segmentation and in-the-wild scene reconstruction.

Neural Radiance Fields (NeRF or NeRFs) are to date emerging as a novel method for synthesizing novel views of complex 3D scenes, leveraging an artificial neural network to optimize a volumetric scene function using a set of input views. We conduct a preliminary critical review of the scientific and technical literature on NeRFs, and we highlight possible applications of the latter in the Cultural Heritage domain, for the image-based reconstruction of 3D models of real, multi-scale objects, even in combination with the more well-established photogrammetric techniques. A comparison is made between NeRFs and photogrammetry in terms of operating procedures and outputs (volumetric renderings vs. point clouds or meshes). It is demonstrated that NeRFs could be conveniently used for rendering objects (sculptures, archaeological remains, sites, paintings etc.) that are challenging for photogrammetry, typically: i) metallic, translucent, and/or transparent surfaces; ii) objects that present homogeneous textures; iii) occlusions, vegetation, and elements of very fine detail.

The digitization of Cultural Heritage paves the way for new approaches to surveying and restitution of historical sites. With a view to the management of integrated programs of documentation and conservation, the research is now focusing on the creation of information systems where to link the digital representation of a building to semantic knowledge. With reference to the emblematic case study of the Calci Charterhouse, also known as Pisa Charterhouse, this contribution illustrates an approach to be followed in the transition from 3D survey information, derived from laser scanner and photogrammetric techniques, to the creation of semantically enriched 3D models. The proposed approach is based on the recognition -segmentation and classification- of elements on the original raw point cloud, and on the manual mapping of NURBS elements on it. For this shape recognition process, reference to architectural treatises and vocabularies of classical architecture is a key step. The created building components are finally imported in a H-BIM environment, where they are enriched with semantic information related to historical knowledge, documentary sources and restoration activities.