A common assumption in traditional supervised learning is the similar probability distribution of data between the training phase and the testing/operating phase. When transitioning from the training to testing phase, a shift in the probability distribution of input data is known as a covariate shift. Covariate shifts commonly arise in a wide range of real-world systems such as electroencephalogram-based brain–computer interfaces (BCIs). In such systems, there is a necessity for continuous monitoring of the process behavior, and tracking the state of the covariate shifts to decide about initiating adaptation in a timely manner. This paper presents a covariate shift-detection and -adaptation methodology, and its application to motor imagery-based BCIs. A covariate shift-detection test based on an exponential weighted moving average model is used to detect the covariate shift in the features extracted from motor imagery-based brain responses. Following the covariate shift-detection test, the methodology initiates an adaptation by updating the classifier during the testing/operating phase. The usefulness of the proposed method is evaluated using real-world BCI datasets (i.e. BCI competition IV dataset 2A and 2B). The results show a statistically significant improvement in the classification accuracy of the BCI system over traditional learning and semi-supervised learning methods.

The non-stationary nature of electroencephalography (EEG) signals makes an EEG-based brain-computer interface (BCI) a dynamic system, thus improving its performance is a challenging task. In addition, it is well-known that due to non-stationarity based covariate shifts, the input data distributions of EEG-based BCI systems change during inter- and intra-session transitions, which poses great difficulty for developments of online adaptive data-driven systems. Ensemble learning approaches have been used previously to tackle this challenge. However, passive scheme based implementation leads to poor efficiency while increasing high computational cost. This paper presents a novel integration of covariate shift estimation and unsupervised adaptive ensemble learning (CSE-UAEL) to tackle non-stationarity in motor-imagery (MI) related EEG classification. The proposed method first employs an exponentially weighted moving average model to detect the covariate shifts in the common spatial pattern features extracted from MI related brain responses. Then, a classifier ensemble was created and updated over time to account for changes in streaming input data distribution wherein new classifiers are added to the ensemble in accordance with estimated shifts. Furthermore, using two publicly available BCI-related EEG datasets, the proposed method was extensively compared with the state-of-the-art single-classifier based passive scheme, single-classifier based active scheme and ensemble based passive schemes. The experimental results show that the proposed active scheme based ensemble learning algorithm significantly enhances the BCI performance in MI classifications.

Fully immersive virtual reality (VR) applications have modified the way people access cultural heritage—from the visiting of virtual museums containing large collections of paintings to the visiting of ancient buildings. In this paper, we propose to review the software that are currently available that deal with cultural heritage in fully immersive virtual reality. It goes beyond technologies that were available prior to virtual reality headsets, at a time where virtual was simply the synonym of the application of digital technologies to cultural heritage. We propose to group these applications depending on their content—from generic art galleries and museums to applications that focus on a single artwork or single artist. Furthermore, we review different ways to assess the performance of such applications with workload, usability, flow, and potential VR symptoms surveys. This paper highlights the progress in the implementation of applications that provide immersive learning experiences related to cultural heritage, from 360 images to photogrammetry and 3D models. The paper shows the discrepancy between available software to the general audience on various VR headsets and scholarship activities dealing with cultural heritage in VR.

A brain-computer interface (BCI) is a specific type of human-computer interface that enables direct communication between human and computer through decoding of brain activity. As such, event-related potentials like the P300 can be obtained with an oddball paradigm whose targets are selected by the user. This paper deals with methods to reduce the needed set of EEG sensors in the P300 speller application. A reduced number of sensors yields more comfort for the user, decreases installation time duration, may substantially reduce the financial cost of the BCI setup and may reduce the power consumption for wireless EEG caps. Our new approach to select relevant sensors is based on backward elimination using a cost function based on the signal to signal-plus-noise ratio, after some spatial filtering. We show that this cost function selects sensors' subsets that provide a better accuracy in the speller recognition rate during the test sessions than selected subsets based on classification accuracy. We validate our selection strategy on data from 20 healthy subjects.

New portable and noninvasive eye-trackers allow the creation of robust virtual keyboards that aim to improve the life of disabled people who are unable to communicate. This paper presents a novel multimodal virtual keyboard and evaluates the performance changes that occur with the use of different modalities. The virtual keyboard is based on a menu selection with eight main commands that allow us to spell 30 different characters and correct errors with a delete button. The system has been evaluated with 18 adult participants in three conditions corresponding to three modalities: direct selection using a mouse, with the eye-tracker to point at the desired command and a switch to select it, and with only the eye-tracker for command selection. The performance of the proposed virtual keyboard was evaluated by the speed and information transfer rate (ITR) at both the command and application levels. The average speed across subjects was 18.43 letters/min with the mouse only, 15.26 letters/min with the eye-tracker and the switch, and 9.30 letters/min with only the eye-tracker. The later provided an ITR of 44.96 and 57.46 bits/min at the letter and command levels, respectively. The results show to what extent a drop of performance can occur when switching between several modalities. While the speed decreases when controlling the virtual keyboard with the eye-tracker only, the system's performance remains functioning for severely disabled people who have their gaze as one of their only means of communication.