Wearable sensors have increasingly been applied in healthcare to generate data and monitor patients unobtrusively. Their application for Brain–Computer Interfaces (BCI) allows for unobtrusively monitoring one’s cognitive state over time. A particular state relevant in multiple domains is cognitive fatigue, which may impact performance and attention, among other capabilities. The monitoring of this state will be applied in real learning settings to detect and advise on effective break periods. In this study, two functional near-infrared spectroscopy (fNIRS) wearable devices were employed to build a BCI to automatically detect the state of cognitive fatigue using machine learning algorithms. An experimental procedure was developed to effectively induce cognitive fatigue that included a close-to-real digital lesson and two standard cognitive tasks: Corsi-Block task and a concentration task. Machine learning models were user-tuned to account for the individual dynamics of each participant, reaching classification accuracy scores of around 70.91 ± 13.67 %. We concluded that, although effective for some subjects, the methodology needs to be individually validated before being applied. Moreover, time on task was not a particularly determining factor for classification, i.e., to induce cognitive fatigue. Further research will include other physiological signals and human–computer interaction variables.
Affective computing has been widely used to detect and recognize emotional states. The main goal of this study was to detect emotional states using machine learning algorithms automatically. The experimental procedure involved eliciting emotional states using film clips in an immersive and non-immersive virtual reality setup. The participants’ physiological signals were recorded and analyzed to train machine learning models to recognize users’ emotional states. Furthermore, two subjective ratings emotional scales were provided to rate each emotional film clip. Results showed no significant differences between presenting the stimuli in the two degrees of immersion. Regarding emotion classification, it emerged that for both physiological signals and subjective ratings, user-dependent models have a better performance when compared to user-independent models. We obtained an average accuracy of 69.29 ± 11.41% and 71.00 ± 7.95% for the subjective ratings and physiological signals, respectively. On the other hand, using user-independent models, the accuracy we obtained was 54.0 ± 17.2% and 24.9 ± 4.0%, respectively. We interpreted these data as the result of high inter-subject variability among participants, suggesting the need for user-dependent classification models. In future works, we intend to develop new classification algorithms and transfer them to real-time implementation. This will make it possible to adapt to a virtual reality environment in real-time, according to the user’s emotional state.
Virtual reality has increasingly been integrated with biofeedback systems for treating affective states such as depression and anxiety. The implementation of stress exposure therapies that adapt in real-time to individuals' physiological signals improves patients' ability to deal with real life stressors. This work evaluates the effect of a real-time adaptive closed-loop system using breathing rate as a biofeedback visual cue in an immersive, nature-based restorative virtual environment. Participants were divided into two groups and exposed to a baseline condition and different stressors: two Stroop Color-Word tests and a restorative virtual reality exploration task in the Virtual Levada. Their physiological signals (electrocardiography, electrodermal activity, and respiration) were recorded and subjective stress levels were annotated . The adaptive group received biofeedback based on their breathing rate, using the field of view as a representation of their stress levels, which was adapted using rule-based changes. Results revealed significant differences in physiological responses across experimental conditions. Specifically, the adaptive biofeedback group reported lower levels of stress during the immersive virtual reality task compared to the Stroop tests, highlighting the restorative effect of the Virtual Levada. These findings suggest that real-time adaptive biofeedback systems, when integrated with restorative elements, enhances users' ability to regulate their stress responses.
Introduction: Serious games are an important tool to overcome the low engagement and adherence to rehabilitation programs due to their repetitive nature and lack of positive reinforcement. Dynamic difficulty adjustment (DDA) systems can contribute by providing algorithms to adapt serious games, keeping players engaged and in a flow state. However, these systems are generally custom-made for specific purposes and goals, lacking the adaptability to be easily integrated into serious games. In response to this problem, we introduced the Flow Optimizer Framework (FOF), a game-agnostic DDA system developed for Unity. This framework facilitates the integration of DDA algorithms with serious games in Unity, enabling real-time monitoring and adaptation based on player state through data processing, rule-setting, and decision-making. Materials and Methods: First, we conducted a technical validation of the framework, assessing its performance in handling real-time data streams and its responsiveness to different scenarios. Following this validation, we evaluated its effectiveness in enhancing the flow state by conducting a usability study. Participants were presented with three different types of DDA paradigms implemented in FOF (Implicit, Explicit, and Subjective), each with different algorithms to adjust the game’s difficulty. Results: The results obtained showed that the implementation of a biofeedback paradigm using the player’s heart rate was the one that increased game performance the most, and participants reported this condition as the most enjoyable and fitting to their skills. Discussion: Overall, participants reported a high usability and a high presence experienced in the serious games implemented with FOF.