The commercial flightdeck is a naturally multi-tasking work environment, one in which interruptions are frequent come in various forms, contributing in many cases to aviation incident reports. Automatic characterization of pilots’ workloads is essential to preventing these kind of incidents. In addition, minimizing the physiological sensor network as much as possible remains both a challenge and a requirement. Electroencephalogram (EEG) signals have shown high correlations with specific cognitive and mental states, such as workload. However, there is not enough evidence in the literature to validate how well models generalize in cases of new subjects performing tasks with workloads similar to the ones included during the model’s training. In this paper, we propose a convolutional neural network to classify EEG features across different mental workloads in a continuous performance task test that partly measures working memory and working memory capacity. Our model is valid at the general population level and it is able to transfer task learning to pilot mental workload recognition in a simulated operational environment.

High mental workload reduces human performance and the ability to correctly carry out complex tasks. In particular, aircraft pilots enduring high mental workloads are at high risk of failure, even with catastrophic outcomes. Despite progress, there is still a lack of knowledge about the interrelationship between mental workload and brain functionality, and there is still limited data on flight-deck scenarios. Although recent emerging deep-learning (DL) methods using physiological data have presented new ways to find new physiological markers to detect and assess cognitive states, they demand large amounts of properly annotated datasets to achieve good performance. We present a new dataset of electroencephalogram (EEG) recordings specifically collected for the recognition of different levels of mental workload. The data were recorded from three experiments, where participants were induced to different levels of workload through tasks of increasing cognition demand. The first involved playing the N-back test, which combines memory recall with arithmetical skills. The second was playing Heat-the-Chair, a serious game specifically designed to emphasize and monitor subjects under controlled concurrent tasks. The third was flying in an Airbus320 simulator and solving several critical situations. The design of the dataset has been validated on three different levels: (1) correlation of the theoretical difficulty of each scenario to the self-perceived difficulty and performance of subjects; (2) significant difference in EEG temporal patterns across the theoretical difficulties and (3) usefulness for the training and evaluation of AI models.

Continuous Sign Recognition of Brazilian Sign Language in a Healthcare Setting DOI: 10.14209/jcis.2015.10

In this paper we explore the architecture of GPU-based volume ray-casting to control the cursor movements on surfaces of volume data only on the basis of their sampled scalar values. This method does not rely on building a tight proxy geometry nor computing local iso-surface parametrization. In this setting only a few instructions should be included in a ray-casting shader: at every fragment the depth of the closest non-transparent voxel along each viewing ray is calculated and stored in the depth buffer. Its application in the manipulation of 3D medical images has, however, a broad spectrum. Two of them are presented. The proposed technique is very simple and fast, yet produces very nice and intuitive visual feedbacks.

Dynamic queries continuously update the data that is visualized in accordance with the user actions. They are typically applied for visual information seeking. This paper proposes to introduce this interaction style for exploring 3D medical neuroimages in its original form, enhancing visual seeking technology in a medical diagnostic procedure. More precisely, we present three dynamic query tools that allow the user to change the focus on-the-fly, while the surrounding tissue is preserved. They are a curvilinear cropper, a volumetric probe and a movable magnifying lens. Once information-preserving visualization is essential for accurate diagnosis and legal protection, the dataset is in its original form. The originality of our work relies on the input interface through which an expert can directly manipulate those tools on the raw data and the responsiveness of each displayed voxel by exploiting the power of GPUs. The proposed techniques have been integrated in a visualization prototype and were assessed by the neuroimaging experts, who were be able to identify subtle lesions in the brain.