The virtual inpainting of artworks provides a nondestructive mode of hypothesis visualization, and it is especially attractive when physical restoration raises too many methodological and ethical concerns. At the same time, in Cultural Heritage applications, the level of details in virtual reconstruction and their accuracy are crucial. We propose an inpainting algorithm that is based on generative adversarial network, with two generators: one for edges and another one for colors. The color generator rebalances chromatically the result by enforcing a loss in the discretized gamut space of the dataset. This way, our method follows the modus operandi of an artist: edges first, then color palette, and, at last, color tones. Moreover, we simulate the stochasticity of the lacunae in artworks with morphological variations of a random walk mask that recreate various degradations, including craquelure. We showcase the performance of our model on a dataset of digital images of wall paintings from the Dunhuang UNESCO heritage site. Our proposals of restored images are visually satisfactory and they are quantitatively comparable to state-of-the-art approaches.

Photo-sensitive materials tend to change with exposure to light. Often, this change is visible when it affects the reflectance of the material in the visible range of the electromagnetic spectrum. In order to understand the photo-degradation mechanisms and their impact on fugitive materials, high-end scientific analysis is required. In a two-part article, we present a multi-modal approach to model fading effects in the spectral, temporal (first part) and spatial dimensions (second part). Specifically, we collect data from the same artwork, namely “A Japanese Lantern” by Norwegian artist, Oda Krohg, with two techniques, point-based microfading spectroscopy and hyperspectral imaging. In this first part, we focus on characterizing the pigments in the painting based on their spectral and fading characteristics. To begin with, using microfading data of a region in the painting, we analyze the color deterioration of the measured points. Then, we train a tensor decomposition model to reduce the measured materials to a spectral basis of unmixed pigments and, at the same time, to recover the fading rate of these endmembers (i.e. pure, unmixed chemical signals). Afterwards, we apply linear regression to predict the fading rate in the future. We validate the quality of these predictions by spectrally comparing them with temporal observations not included in the training part. Furthermore, we statistically assess the goodness of our model in explaining new data, collected from another region of the painting. Finally, we propose a visual way to explore the artist’s palette, where potential matches between endmembers and reference spectral libraries can be evaluated based on three metrics at once.

In a previous article, we modelled the spectral and temporal dimensions of the photodegradation behaviour of pigments in the painting “A Japanese Lantern” by Oda Krohg. In particular, we extracted the endmembers and spectral fading rate of pigments by applying tensor decomposition on a time-series of spectroscopic point measurements. Now, we capture the same painting with a hyperspectral imaging setup and propose an approach to render the fading effects as 2D images. More precisely, from the hyperspectral image, we compute the concentration maps of each previously identified endmember with a least-squares unmixing method. Subsequently, by using tensor algebra, we multiply the concentration maps with the endmembers and their corresponding fading rate and obtain a 4D tensor where each pixel in the image is described by a spectrum and a fading function. This way, we generate past and future spatio-temporal simulations of the painting’s appearance by reversing and elevating light exposure, respectively.

This chapter explores the concept of authentic experiences and how it could be applied to design new engaging applications and user experiences dedicated to coloured collections. The authors analyse how the concept was used in different domains. They then explore users’ behaviours and their understanding of this concept by designing a Cultural Probe Kit and involving a group of participants into an experiment. They then summarise the results and propose a theoretical framework in which authenticity is made of three interconnected dimensions: the ‘Self’ (representing cognitive, emotional and sensory engagement characteristics of individual users), the ‘Others’ (dealing with social relationships) and the ‘World’ (as physical or digital environment). As a conclusion, the chapter presents examples of how such framework can be practically applied to Interactive Media projects.