28 May 2020 in IEEE Transactions on Neural Systems and Rehabilitation Engineering
Visual information plays an essential role in enhancing neural activity during mental practices. Previous research has shown that using different visual scenarios during mental practices that involve imagining the movement of a specific body part may result in differences in performance. Many of these scenarios utilize the concept of embodiment, or one’s observation of another entity to be a part of oneself, to improve practice quality of the imagined body movement. We therefore hypothesized that applying immersive virtual reality headsets, with their ability to provide rich immersion and illusion by presenting egocentrically simulated virtual scenarios, and action observation to motor imagery practice will result in significant improvement. To explore the possible synergy between immersive systems and motor imagery, we analyzed the electroencephalogram signals of our participants as they were presented the same virtual hand movement scenario with two different mediums: an immersive virtual reality headset and a monitor display. Our experimental results provide evidence that the immersive virtual reality headsets induced improved rhythmic patterns with better discriminating spatial features from the brain compared to the monitor display. These findings suggest that the use of immersive virtual reality headsets, with the illusion and embodiment they provide, can effectively improve motor imagery training.
9 March 2023 in IEEE Transactions on Neural Systems and Rehabilitation Engineering
Recent advancements in immersive virtual reality head-mounted displays allowed users to better engage with simulated graphical environments. Having the screen egocentrically stabilized in a way such that the users may freely rotate their heads to observe virtual surroundings, head-mounted displays present virtual scenarios with rich immersion. With such an enhanced degree of freedom, immersive virtual reality displays have also been integrated with electroencephalograms, which make it possible to study and utilize brain signals non-invasively, to analyze and apply their capabilities. In this review, we introduce recent progress that utilized immersive head-mounted displays along with electroencephalograms across various fields, focusing on the purposes and experimental designs of their studies. The paper also highlights the effects of using immersive virtual reality discovered through the electroencephalogram analysis and discusses existing limitations, current trends as well as future research opportunities that may hopefully act as a useful source of information for further improvement of electroencephalogram-based immersive virtual reality applications.
1 December 2020 in Computers in Biology and Medicine
5 June 2023 in IEEE Sensors Journal
Mapping drivers’ thoughts directly to mobility system control would make driving more intuitive as if the mobility system is an extension of their own body. Such a system would allow patients with motor disabilities to drive, as it would not require any physical movement. In this paper, we therefore propose a brain-controlled mobility system that analyzes real-time neural signals elicited from motor imagery, an imagination of different body movements. As such asynchronous brain-computer interfaces (BCIs) are prone to error, our system contains shared control capabilities that take into consideration continuously updated information of the surrounding environment along with electroencephalogram (EEG) signals to improve navigating performance without precise and accurate control from the driver. With our shared control method that uses a wheelchair with light detection and ranging (LiDAR) and inertial measurement unit (IMU) sensors, we held a comparative study in which participants drove our wheelchair with and without our shared control approach using either our brain-controlled system or a keyboard in a physical environment. The experimental results show that among the five participants, the three participants that failed the driving task with the asynchronous BCI-based system could also successfully complete it using our shared control approach. Furthermore, our approach narrows the gap between driving with neural signals and driving with a widely used interface in terms of both elapsed time and safety. These results show not only the potential of brain signals for driving, but also the applicability of BCIs to real-life situations.
12 September 2023 in IEEE Transactions on Cognitive and Developmental Systems
Discriminating motor imagery with electroencephalogram (EEG)-based brain-computer interface (BCI) poses a challenge as it involves an extensive data acquisition phase that demands a substantial amount of effort from the user. To address this issue, one approach is to use unsupervised domain adaptation, where classification models are constructed using data from multiple subjects, and only the unlabeled data from the target user is used for model calibration. However, since brain patterns from motor imagery vary between individuals, the reliability of each subject must be considered when multiple subjects are used to build the classification model. Thus in this paper, we propose Selective-MDA that performs domain adaptation on each source subject and selectively limits influences based on their domain discrepancies. To evaluate our approach, we assess our results with two public datasets, BCI Competition IV IIa and the Autocalibration and Recurrent Adaptation datasets. We further investigate the effect of source selection by comparing the discrimination performance when different numbers of source domains are selected based on discrepancy measures. Our results demonstrate that Selective-MDA not only integrates multi-source domain adaptation to cross-subject motor imagery discrimination but also highlights the impact of source domain selection when using data from multiple subjects for model training.