Despite the use of tactile graphics and audio guides, blind and visually impaired people still face challenges to experience and understand visual artworks independently at art exhibitions. Art museums and other art places are increasingly exploring the use of interactive guides to make their collections more accessible. In this work, we describe our approach to an interactive multimodal guide prototype that uses audio and tactile modalities to improve the autonomous access to information and experience of visual artworks. The prototype is composed of a touch-sensitive 2.5D artwork relief model that can be freely explored by touch. Users can access localized verbal descriptions and audio by performing touch gestures on the surface while listening to themed background music along. We present the design requirements derived from a formative study realized with the help of eight blind and visually impaired participants, art museum and gallery staff, and artists. We extended the formative study by organizing two accessible art exhibitions. There, eighteen participants evaluated and compared multimodal and tactile graphic accessible exhibits. Results from a usability survey indicate that our multimodal approach is simple, easy to use, and improves confidence and independence when exploring visual artworks.

Visually impaired people can take advantage of multimodal systems in which visual information is communicated through different modes of interaction and types of feedback. Among the possible interaction modes, thermal interaction in the context of assistive devices for visually impaired people lacks research in spite of its potential. In this paper, we propose a temperature-depth mapping algorithm and a thermal display system to convey depth and depth-color of artworks’ features in the context of tactile exploration by visually impaired people. Tests with a total of 18 sighted users and six visually impaired users were performed both during the mapping algorithm design and after developing a tactile temperature prototype artwork model to assess the potentials of thermal interaction for recognizing depth and color-depth in tactile art appreciation. These tests showed both an existing correlation between depth and temperature and that the mapping based on that correlation is appropriate for conveying depth during artwork tactile exploration.

For years the HCI community’s research has been focused on the hearing and sight senses. However, in recent times, there has been an increased interest in using other types of senses, such as smell or touch. Moreover, this has been accompanied with growing research related to sensory substitution techniques and multi-sensory systems. Similarly, contemporary art has also been influenced by this trend and the number of artists interested in creating novel multi-sensory works of art has increased substantially. As a result, the opportunities for visually impaired people to experience artworks in different ways are also expanding. In spite of all this, the research focusing on multimodal systems for experiencing visual arts is not large and user tests comparing different modalities and senses, particularly in the field of art, are insufficient. This paper attempts to design a multi-sensory mapping to convey color to visually impaired people employing musical sounds and temperature cues. Through user tests and surveys with a total of 18 participants, we show that this multi-sensory system is properly designed to allow the user to distinguish and experience a total of 24 colors. The tests consist of several semantic correlational adjective-based surveys for comparing the different modalities to find out the best way to express colors through musical sounds and temperature cues based on previously well-established sound-color and temperature-color coding algorithms. In addition, the resulting final algorithm is also tested with 12 more users.