Emotions play a critical role in rational and intelligent behavior; a better fundamental knowledge of them is indispensable for understanding higher order brain function. We propose a non-invasive brain-computer interface (BCI) system to feedback a person’s affective state such that a closed-loop interaction between the participant’s brain responses and the musical stimuli is established. We realized this concept technically in a functional prototype of an algorithm that generates continuous and controllable patterns of synthesized affective music in real-time, which is embedded within a BCI architecture. We evaluated our concept in two separate studies. In the first study, we tested the efficacy of our music algorithm by measuring subjective affective responses from 11 participants. In a second pilot study, the algorithm was embedded in a real-time BCI architecture to investigate affective closed-loop interactions in 5 participants. Preliminary results suggested that participants were able to intentionally modulate the musical feedback by self-inducing emotions (e.g., by recalling memories), suggesting that the system was able not only to capture the listener’s current affective state in real-time, but also potentially provide a tool for listeners to mediate their own emotions by interacting with music. The proposed concept offers a tool to study emotions in the loop, promising to cast a complementary light on emotion-related brain research, particularly in terms of clarifying the interactive, spatio-temporal dynamics underlying affective processing in the brain.
28 September 2018 in Journal of Neural Engineering
Objective . Error-related potentials (ErrP) have been proposed as an intuitive feedback signal decoded from the ongoing electroencephalogram (EEG) of a human observer for improving human-robot interaction (HRI). While recent demonstrations of this approach have successfully studied the use of ErrPs as a teaching signal for robot skill learning, so far, no efforts have been made towards HRI scenarios where mutual adaptations between human and robot are expected or required. These are collaborative or social interactive scenarios without predefined dominancy of the human partner and robots being perceived as intentional agents. Here we explore the usability of ErrPs as a feedback signal from the human for mediating co-adaptation in human-robot interaction. Approach . We experimentally demonstrate ErrPs-based mediation of co-adaptation in a human-robot interaction study where successful interaction depended on co-adaptive convergence to a consensus between them. While subjects adapted to the robot by reflecting upon its behavior, the robot adapted its behavior based on ErrPs decoded online from the human partner’s ongoing EEG. Main results . ErrPs were decoded online in single trial with an avg. accuracy of 81.8% ± 8.0% across 13 subjects, which was sufficient for effective adaptation of robot behavior. Successful co-adaptation was demonstrated by significant improvements in human-robot interaction efficacy and efficiency, and by the robot behavior that emerged during co-adaptation. These results indicate the potential of ErrPs as a useful feedback signal for mediating co-adaptation in human-robot interaction as demonstrated in a practical example. Significance . As robots become more widely embedded in society, methods for aligning them to human expectations and conventions will become increasingly important in the future. In this quest, ErrPs may constitute a promising complementary feedback signal for guiding adaptations towards human preferences. In this paper we extended previous research to less constrained HRI scenarios where mutual adaptations between human and robot are expected or required.
2 November 2018 in International Journal of Social Robotics
Playing a musical instrument demands the integration of sensory and perceptual information with motor processes in order to produce a harmonic musical piece. The diversity of brain mechanisms involved and the joyful character of playing an instrument make musical instrument training a potential vehicle for neurorehabilitation of motor skills in patients with cerebral palsy (CP). This clinical condition is characterized by motor impairments that can affect, among others, manual function, and limit severely the execution of basic daily activities. In this study, adolescents and adult patients with CP, as well as a group of typically developing children learned to play piano for 4 consecutive weeks, having completed a total of 8 hours of training. For ten of the participants, learning was supported by a special technical system aimed at helping people with sensorimotor deficits to better discriminate fingers and orient themselves along the piano keyboard. Potential effects of piano training were assessed with tests of finger tapping at the piano and tests of perception of vibratory stimulation of fingers, and by measuring neuronal correlates of motor learning in the absence of and after piano training. Results were highly variable especially among participants with CP. Nevertheless, a significant effect of training on the ability to perceive the localization of vibrations over fingers was found. No effects of training on the performance of simple finger tapping sequences at the piano or on motor-associated brain responses were registered. Longer periods of training are likely required to produce detectable changes.
This study differentiates the dynamic temporal-spectral effects during listening to 10 s of familiar music compared with unfamiliar music. This study highlights that listening to familiar music leads to continuous suppression in the alpha and low-beta bands. This suppression starts ∼800 ms after the stimulus onset.