19 December 2022 in IEEE Access
Neuroprosthetics have demonstrated the potential to decode speech from intracranial brain signals, and hold promise for one day returning the ability to speak to those who have lost it. However, data in this domain is scarce, highly variable, and costly to label for supervised modeling. In order to address these constraints, we present brain2vec, a transformer-based approach for learning feature representations from intracranial electroencephalogram data. Brain2vec combines a self-supervised learning methodology, neuroanatomical positional embeddings, and the contextual representations of transformers to achieve three novelties: (1) learning from unlabeled intracranial brain signals, (2) learning from multiple participants simultaneously, all while (3) utilizing only raw unprocessed data. To assess our approach, we use a leave-one-participant-out validation procedure to separate brain2vec’s feature learning from the holdout participant’s speech-related supervised classification tasks. With only two linear layers, we achieve 90% accuracy on a canonical speech detection task, 42% accuracy on a more challenging 4-class speech-related behavior recognition, and 53% accuracy when applied to a 10-class, few-shot word classification task. Combined with visualizations of unsupervised class separation in the learned features, our results evidence brain2vec’s ability to learn highly generalized representations of neural activity without the need for labels or consistent sensor location.
19 September 2022 in IEEE Transactions on Neural Systems and Rehabilitation Engineering
Numerous state-of-the-art solutions for neural speech decoding and synthesis incorporate deep learning into the processing pipeline. These models are typically opaque and can require significant computational resources for training and execution. A deep learning architecture is presented that learns input bandpass filters that capture task-relevant spectral features directly from data. Incorporating such explainable feature extraction into the model furthers the goal of creating end-to-end architectures that enable automated subject-specific parameter tuning while yielding an interpretable result. The model is implemented using intracranial brain data collected during a speech task. Using raw, unprocessed timesamples, the model detects the presence of speech at every timesample in a causal manner, suitable for online application. Model performance is comparable or superior to existing approaches that require substantial signal preprocessing and the learned frequency bands were found to converge to ranges that are supported by previous studies.
4 March 2021 in IEEE Access
Classification of human activities from wearable sensor data is challenged by inter-subject variance and resource-constrained platforms. We address these issues with SincEMG, a deep neural network that exploits digital signal processing concepts and transfer learning to reduce model size for activity recognition on raw sensor data. The model’s first layer decomposes signals into frequency bands using finite impulse response filters optimized directly from the data. The subsequent convolutional layers downsample across time and aggregate the first layer’s band data. Batch normalization and dropout help to regularize intermediate layer outputs. This approach reduces compute requirements by decreasing the number of learned parameters and eliminating any significant data pre-processing. In addition to these improvements, the model’s first layer learns a set of bandpass filters, which provide insight into predictive regions of the source spectrum. We evaluate SincEMG using two publicly available surface electromyography datasets. Our model uses far fewer parameters and achieves state-of-the-art results with 98.53% accuracy for 7-classes and 68.45% accuracy for 18-classes.
1 November 2025 in IEEE Journal of Biomedical and Health Informatics
As Deep Learning (DL) algorithms become more widely adopted in healthcare applications, there is a greater emphasis on understanding and addressing potential privacy risks associated with these models. The purpose of this study is to investigate the privacy vulnerabilities of the Convolutional Neural Network (CNN) classifiers for Electroencephalogram (EEG) data in the Brain-Computer Interfaces (BCIs). Specifically, it focuses on the Membership Inference Attack (MIA), which seeks to determine if data from an individual were used in model training. The novelty of this work lies in its empirical analysis of MIA, specifically by addressing two key challenges that are less common in other domains: 1) datasets that are heterogeneous and 2) spatial-temporal design choices. Motivated by these challenges, we investigate the susceptibility to MIA based on: 1) specifics of the training dataset (number of participants, demographics), and 2) specifics of the CNN (such as architecture, regularization). Our experiments revealed that an adversary with limited knowledge of the model and its training process can compromise the privacy of training participants, noting that the same attack is not effective against deep learning models trained on image and tabular datasets. Some of our findings are: 1) training on diverse participant datasets improves the privacy of the most participants but increases risks of memorization and vulnerabilities for underrepresented groups; 2) regularization is less effective in defending against the MIA for EEG data CNN classifiers when compared to other types of input data; 3) depth and width of model architecture has no impact on membership attack effectiveness. We hope that the presented insights will assist future researchers develop more privacy-aware deep learning based BCI systems.
1 January 2022 in Complexity
We present a perspective of the national transplant program based on organizational theory and complexity theory, framing the system’s allocation of donor organs as an interorganizational directed multiplex of agents with diverse belief formation in a cooperative‐competitive environment. Simulation and analysis of this macroscale complexity may help explain known behavioural variations across member organizations. However, the transplant community still relies on system‐scale simulations since effective macroscale methodologies are not well established. Therefore, we offer this perspective of the national transplant program as a means to stimulate new methods that capture macroscale impacts of policy development for deceased donor organ allocation.