Human haptic perception system is complex, involving both cutaneous and kinesthetic receptors. These receptors work together and enable human to perceive the external world. To simulate immersive interaction with virtual objects in virtual reality scenarios haptic devices are desired to reproduce multi-properties of virtual objects, support multi-gestures of human hands to perform fine manipulation, produce haptic stimuli for simultaneously stimulating multi-receptors (including cutaneous and kinesthetic receptors) of human haptic channel, and thus invoke realistic compound haptic sensations. In recent years such multimodal haptic devices have emerged. In this paper, we survey the latest progress on multimodal haptic devices, identify the gaps, and put forward future research directions on the topic.

Interacting with virtual objects with haptic feedback directly using the user's hand (hand-based haptic interaction) has attracted increasing attention. Due to the high degrees of freedom of the hand, compared with tool-based interactive simulation using a pen-like haptic proxy, hand-based haptic simulation faces greater challenges, mainly including higher motion mapping and modeling difficulty of deformable hand avatars, higher computational complexity of contact dynamics, and nontrivial multi-modal fusion feedback. In this paper, we aim to review key computing components for hand-based haptic simulation, and draw out major findings in this direction while analyzing the gaps toward immersive and natural hand-based haptic interaction. To this end, we investigate existing relevant studies on hand-based interaction with kinesthetic and/or cutaneous display in terms of virtual hand modeling, hand-based haptic rendering, and visuo-haptic fusion feedback. By identifying current challenges, we finally highlight future perspectives in this field.

In the era of ubiquitous computing with flourished visual displays in our surroundings, the application of haptic feedback technology still remains in its infancy. Bridging the gap between haptic technology and the real world to enable ambient haptic feedback on various physical surfaces is a grand challenge in the field of human-computer interaction. This paper presents the concept of an active electronic skin, characterized by three features: richness (multi-modal haptic stimuli), interactivity (bi-directional sensing and actuation capabilities), and invisibility (transparent, ultra-thin, flexible, and stretchable). By deploying this skin on physical surfaces, dynamic and versatile multi-modal haptic display, as well as tactile sensing, can be achieved. The potential applications of this skin include two categories: skin for the physical world (such as intelligent home, intelligent car, and intelligent museum), and skin for the digital world (such as haptic screen, wearable device, and bare-hand device). Furthermore, existing skin-based haptic display technologies including texture, thermal, and vibrotactile feedback are surveyed, as well as multidimensional tactile sensing techniques. By analyzing the gaps between current technologies and the goal of ambient haptics, future research topics are proposed, encompassing fundamental theoretical research on the physiological and psychological perception mechanisms of human skin, spatial-temporal registration among multimodal haptic stimuli, integration between sensing and actuation, and spatial-temporal registration between visual and haptic display. This concept of active electronic skin is promising for advancing the field of ambient haptics, enabling seamless integration of touch into our digital and physical surroundings.