This paper reports about the effects of vibration direction and finger-pressing force on vibrotactile perception, with the goal of improving the effectiveness of haptic feedback on interactive surfaces. An experiment was conducted to assess the sensitivity to normal or tangential vibration at 250 Hz of a finger exerting constant pressing forces of 0.5 or 4.9 N. Results show that perception thresholds for normal vibration depend on the applied pressing force, significantly decreasing for the stronger force level. Conversely, perception thresholds for tangential vibrations are independent of the applied force, and approximately equal the lowest thresholds measured for normal vibration.
An advanced multi-touch sensor surface aimed at musical expression was recently equipped by the authors with interactive multi-point localized vibrotactile feedback. Using such interface, a subjective assessment was conducted that measured how the presence and type of vibration affect the perceived quality of the device and various attributes related to the playing experience. Two clearly distinct sound settings each with three vibrotactile feedback strategies were tested. At each trial, the task was to play freely while comparing two related setups which used the same sound setting and differed only in the presence/absence of vibration. Independent of the sound setting, as compared to the respective non-vibrating setups, vibrations conveying frequency and amplitude dynamics cues coherent with the player's gesture and/or sonic feedback had the most positive effect. Vibrotactile feedback especially improved the enjoyment of playing and the perceived potential for musical expressivity.
We suggest that studies on active touch psychophysics are needed to inform the design of haptic musical interfaces and better understand the relevance of haptic cues in musical performance. Following a review of the previous literature on vibrotactile perception in musical performance, two recent experiments are reported. The first experiment investigated how active finger-pressing forces affect vibration perception, finding significant effects of vibration type and force level on perceptual thresholds. Moreover, the measured thresholds were considerably lower than those reported in the literature, possibly due to the concurrent effect of large (unconstrained) finger contact areas, active pressing forces, and long-duration stimuli. The second experiment assessed the validity of these findings in a real musical context by studying the detection of vibrotactile cues at the keyboard of a grand and an upright piano. Sensitivity to key vibrations in fact not only was highest at the lower octaves and gradually decreased toward higher pitches; it was also significant for stimuli having spectral peaks of acceleration similar to those of the first experiment, i.e., below the standard sensitivity thresholds measured for sinusoidal vibrations under passive touch conditions.
Touch has a pivotal importance in determining the expressivity of musical performance for a number of musical instruments. However, most digital musical devices provide no interactive force and/or vibratory feedback to the performer, thus depriving the somatosensory channel of a number of cues. Is the lack of haptic feedback only an aesthetic issue, or does it remove cues essential for digital instrument playing? If so, at which level is the interaction objectively impoverished? What are the effects on musical performance? In this survey article we illustrate our recent research about the use of vibrotactile feedback in three digital instrument interfaces, using tools that we developed over several years and made available to the community in open-source form. These interfaces span a wide range of familiarity and gestural opportunities, enabling us to explore the impact of haptic feedback on different types of digital instruments. We conducted experiments with professional musicians to assess the impact of vibratory cues on both the perceived quality of the instrument and the playing experience, as well as on musical performance. Particular attention was paid to scientific rigor and repeatability of the results, so as to serve as a reference for researchers and practitioners of the musical haptics community. Our results suggest a significant role of vibrotactile feedback in shaping the perception of digital musical instruments, although the effects on musical performance varied depending on the interfaces tested.
13 July 2020 in Journal on Multimodal User Interfaces
An experiment is described that tested the possibility to classify wooden, plastic, and metallic objects based on reproduced auditory and vibrotactile stimuli. The results show that recognition rates are considerably above chance level with either unimodal auditory or vibrotactile feedback. Supported by those findings, the possibility to render virtual buttons for professional appliances with different tactile properties was tested. To this end, a touchscreen device was provided with various types of vibrotactile feedback in response to the sensed pressing force and location of a finger. Different virtual buttons designs were tested by user panels who performed a subjective evaluation on perceived tactile properties and materials. In a first implementation, virtual buttons were designed reproducing the vibration recordings of real materials used in the classification experiment: mainly due to hardware limitations of our prototype and the consequent impossibility to render complex vibratory signals, this approach did not prove successful. A second implementation was then optimized for the device capabilities, moreover introducing surface compliance effects and button release cues: the new design led to generally high quality ratings, clear discrimination of different buttons and unambiguous material classification. The lesson learned was that various material and physical properties of virtual buttons can be successfully rendered by characteristic frequency and decay cues if correctly reproduced by the device.