7 August 2018 in Frontiers in Neuroscience
The decoding of selective auditory attention from noninvasive electroencephalogram (EEG) data is of interest in brain computer interface and auditory perception research. The current state-of-the-art approaches for decoding the attentional selection of listeners are based on linear mappings between features of sound streams and EEG responses (forward model), or vice versa (backward model). It has been shown that when the envelope of attended speech and EEG responses are used to derive such mapping functions, the model estimates can be used to discriminate between attended and unattended talkers. However, the predictive/reconstructive performance of the models is dependent on how the model parameters are estimated. There exist a number of model estimation methods that have been published, along with a variety of datasets. It is currently unclear if any of these methods perform better than others, as they have not yet been compared side by side on a single standardized dataset in a controlled fashion. Here, we present a comparative study of the ability of different estimation methods to classify attended speakers from multi-channel EEG data. The performance of the model estimation methods is evaluated using different performance metrics on a set of labeled EEG data from 18 subjects listening to mixtures of two speech streams. We find that when forward models predict the EEG from the attended audio, regularized models do not improve regression or classification accuracies. When backward models decode the attended speech from the EEG, regularization provides higher regression and classification accuracies.
1 February 2025 in Journal of Neural Engineering
JMJonatan Märcher-Rørsted
+6
Enhancing speech perception in everyday noisy acoustic environments remains an outstanding challenge for hearing aids. Speech separation technology is improving rapidly, but hearing devices cannot fully exploit this advance without knowing which sound sources the user wants to hear. Even with high-quality source separation, the hearing aid must know which speech streams to enhance and which to suppress. Advances in EEG-based decoding of auditory attention raise the potential of neurosteering, in which a hearing instrument selectively enhances the sound sources that a hearing-impaired listener is focusing their attention on. Here, we present and discuss a real-time brain–computer interface system that combines a stimulus–response model based on canonical correlation analysis for real-time EEG attention decoding, coupled with a multi-microphone hardware platform enabling low-latency real-time speech separation through spatial beamforming. We provide an overview of the system and its various components, discuss prospects and limitations of the technology, and illustrate its application with case studies of listeners steering acoustic feedback of competing speech streams via real-time attention decoding. A software implementation code of the system is publicly available for further research and explorations.
24 December 2024 in Brain Communications
Former studies have established that individuals with a cochlear implant (CI) for treating single-sided deafness experience improved speech processing after implantation. However, it is not clear how each ear contributes separately to improve speech perception over time at the behavioural and neural level. In this longitudinal EEG study with four different time points, we measured neural activity in response to various temporally and spectrally degraded spoken words presented monaurally to the CI and non-CI ears (5 left and 5 right ears) in 10 single-sided CI users and 10 age- and sex-matched individuals with normal hearing. Subjective comprehension ratings for each word were also recorded. Data from single-sided CI participants were collected pre-CI implantation, and at 3, 6 and 12 months after implantation. We conducted a time-resolved representational similarity analysis on the EEG data to quantify whether and how neural patterns became more similar to those of normal hearing individuals. At 6 months after implantation, the speech comprehension ratings for the degraded words improved in both ears. Notably, the improvement was more pronounced for the non-CI ears than the CI ears. Furthermore, the enhancement in the non-CI ears was paralleled by increased similarity to neural representational patterns of the normal hearing control group. The maximum of this effect coincided with peak decoding accuracy for spoken-word comprehension (600–1200 ms after stimulus onset). The present data demonstrate that cortical processing gradually normalizes within months after CI implantation for speech presented to the non-CI ear. CI enables the deaf ear to provide afferent input, which, according to our results, complements the input of the non-CI ear, gradually improving its function. These novel findings underscore the feasibility of tracking neural recovery after auditory input restoration using advanced multivariate analysis methods, such as representational similarity analysis.
31 December 2018 in bioRxiv
Perceptual processes can be probed by fitting stimulus-response models that relate measured brain signals such as electroencephalography (EEG) to the stimuli that evoke them. These models have also found application for the control of devices such as hearing aids. The quality of the fit, as measured by correlation, classification, or information rate metrics, indicates the value of the model and the usefulness of the device. Models based on Canonical Correlation Analysis (CCA) achieve a quality of fit that surpasses that of commonly-used linear forward and backward models. Here, we show that their performance can be further improved using several techniques that capture the time-varying and context-dependent relationships within the data, including adaptive beamforming, CCA weight optimization, and recurrent neural networks that capture the time-varying and context-dependent relationships within the data. We demonstrate these results using a match-vs-mismatch classification paradigm, in which the classifier must decide which of two stimulus samples produced a given EEG response and which is a randomly chosen stimulus sample. This task captures the essential features of the more complex auditory attention decoding (AAD) task explored in many other studies.