The understanding of how users interact with the virtual cultural heritage could provide digital curators valuable insights into user behaviors and also improve the overall user experience through the ability to observe and record interactions of virtual visitors. This article introduces the new User Behavior (UB) tracking algorithm that we developed investigating a salience of the Virtual Reality (VR) panoramic regions. The algorithm extracts the importance of Region of Interest (ROI) determining patterns of the visitors’ virtual movement and interest in combination with statistics of captured browser activity. The input of our algorithm is the virtual online interactive platform (Virtual Museum of the Civic Art Gallery of Ascoli Piceno in Italy) with 81 16,386 × 8,192 pixels panoramic images and several interactive features including maps, thumbnails, and menus. The software engine of the tracking model “Vice Versa” VR operates on inverse functions of all descriptive functions (descriptors), which are assigned particularly to each interactive feature such as viewing multimedia content and observing the panoramic environment. The tracking experiment was performed online and the web virtual museum key study collected behavior information from 171 visitors around the world. Collected data, multimedia and textual content, and the coordinates of the ROIs are then subjected to standard statistics operations to define common patterns of UBs. Thus, we have discovered that the ROIs are mostly mapped onto the artworks and it is possible to obtain patterns about the main interests of users. The developed tool offers a guideline for the panoramic tours design and the potential benefits for museums are to understand the public, verify the effectiveness of choices, and re-shape a cultural offer based on visitors’ needs. Exploiting this kind of user experience, our algorithm ensures relevant feedback during virtual visits and thus paves the way for further development of the recommender system.

Virtual museums are factual means for the dissemination and documentation of Cultural Heritage (CH) content. They are suitable environments for the semantic annotation of artifacts and automatic virtual guides. To this end, we identify and compare Traditional (ontology-based), Large Language Model (LLM)-extended, and LLM-pure methods for the semantic information strategies of digital CH. The traditional method is described through an application prototype, while the methods that involve LLM are tested experimentally. To investigate the integral tasks related to LLMs, our experiments include (i) semantic annotation using the CIDOC Conceptual Reference Model (CRM) and Knowledge Graph (KG) generation with LLMs for a painting sample, and (ii) painting ranking relying solely on LLMs using catalog descriptions as input. The experiments demonstrate the potential of these methods to enhance artwork interpretation, description, and refinement of the results. Based on the relevant literature on traditional semantic annotation and conducted experiments with LLMs, a combination of ontologies and LLMs may provide an optimal approach, as it offers the accuracy of structured knowledge while providing a tool that interprets these elements into natural language and vice versa. Relying solely on LLMs may be risky due to the lack of domain-specific knowledge in the training data of LLMs, whereas traditional methods demand expertise in a specific domain and are more time-consuming. Our approach shows potential in use cases such as guiding museum visitors to artifacts that match their interests, assisting museum curators with documentation, or helping CH researchers identify similarities in artifact collections.

This paper introduces a new method for determining the shape similarity of complex three-dimensional (3D) mesh structures based on extracting a vector of important vertices, ordered according to a matrix of their most important geometrical and topological features. The correlation of ordered matrix vectors is combined with perceptual definition of salient regions in order to aid detection, distinguishing, measurement and restoration of real degradation and digitization errors. The case study is the digital 3D structure of the Camino Degli Angeli, in the Urbino’s Ducal Palace, acquired by the structure from motion (SfM) technique. In order to obtain an accurate, featured representation of the matching shape, the strong mesh processing computations are performed over the mesh surface while preserving real shape and geometric structure. In addition to perceptually based feature ranking, the new theoretical approach for ranking the evaluation criteria by employing neural networks (NNs) has been proposed to reduce the probability of deleting shape points, subject to optimization. Numerical analysis and simulations in combination with the developed virtual reality (VR) application serve as an assurance to restoration specialists providing visual and feature-based comparison of damaged parts with correct similar examples. The procedure also distinguishes mesh irregularities resulting from the photogrammetry process.