Modern systems (e.g., assistive technology and self-driving) can place significant demands on the user’s working memory (WM), which can adversely impact performance (i.e., elevated risk of errors) and increase cognitive load (CL). Robust prediction of CL from EEG remains a challenge due to the small sample problem, noisy recordings, ineffective data representation, and lack of robust models. This paper presents a holistic approach to developing a reliable prediction of CL. We used EEG data recorded following a modified Stenberg WM task in which four levels of CL were defined based on the encoding of 2, 4, 6, and 8 English characters. First, we address the problem of noise and “small sample” by generating large low noise data using eigenspace-based bootstrap sampling and generative adversarial network (GAN). Second, we transform EEG recordings into spatial-spectral images to capture spatial information. Third, we built parameter-optimized CNN models to predict four levels of CL using single-frequency bands (i.e., θ, α, β) and stacked (i.e., all three bands) representations. In our quest to provide interpretable models, we applied Gradient-weighted Class Activation Mapping (Grad-CAM) to our models to localize the brain regions responsible for the prediction of CL. Empirical analysis of models trained using θ, α, β, and stacked representation show accuracy of 90%, 89%, 91%, and 94%, respectively. Grad-CAM visualizations showed that the prefrontal, cerebellum, frontal, and parietal areas have the highest contribution to the prediction of CL.

Background: Lumbar spinal stenosis (LSS) is a major cause of chronic lower back and leg pain, and is traditionally diagnosed through labor-intensive analysis of magnetic resonance imaging (MRI) scans by radiologists. This study aims to streamline the diagnostic process by developing an automated radiology report generation (ARRG) system using a vision–language (VL) model. Methods: We utilized a Generative Image-to-Text (GIT) model, originally designed for visual question answering (VQA) and image captioning. The model was fine-tuned to generate diagnostic reports directly from lumbar spine MRI scans using a modest set of annotated data. Additionally, GPT-4 was used to convert semistructured text into coherent paragraphs for better comprehension by the GIT model. Results: The model effectively generated semantically accurate and grammatically coherent reports. The performance was evaluated using METEOR (0.37), BERTScore (0.886), and ROUGE-L (0.3), indicating its potential to produce clinically relevant content. Conclusions: This study highlights the feasibility of using vision–language models to automate report generation from medical imaging, potentially reducing the diagnostic workload for radiologists.

The process of categorizing sounds into distinct phonetic categories is known as categorical perception (CP). Response times (RTs) provide a measure of perceptual difficulty during labeling decisions (i.e., categorization). The RT is quasi-stochastic in nature due to individuality and variations in perceptual tasks. To identify the source of RT variation in CP, we have built models to decode the brain regions and frequency bands driving fast, medium and slow response decision speeds. In particular, we implemented a parameter optimized convolutional neural network (CNN) to classify listeners’ behavioral RTs from their neural EEG data. We adopted visual interpretation of model response using Guided-GradCAM to identify spatial-spectral correlates of RT. Our framework includes (but is not limited to): (i) a data augmentation technique designed to reduce noise and control the overall variance of EEG dataset; (ii) bandpower topomaps to learn the spatial-spectral representation using CNN; (iii) large-scale Bayesian hyper-parameter optimization to find best performing CNN model; (iv) ANOVA and posthoc analysis on Guided-GradCAM activation values to measure the effect of neural regions and frequency bands on behavioral responses. Using this framework, we observe that $\alpha -\beta$ (10–20 Hz) activity over left frontal, right prefrontal/frontal, and right cerebellar regions are correlated with RT variation. Our results indicate that attention, template matching, temporal prediction of acoustics, motor control, and decision uncertainty are the most probable factors in RT variation.

Modern systems (e.g., assistive technology and self-driving) can place significant demands on the user’s working memory (WM), which can adversely impact performance (i.e., elevated risk of errors) and increase the cognitive load (CL). Robust prediction of CL from EEG remains a challenge due to the small sample problem, noisy recordings, ineffective data representation, and lack of robust models. This paper presents a holistic approach to developing a reliable prediction of CL. We used EEG data recorded following a modified Stenberg WM task in which four levels of CL were defined based on the encoding of 2, 4, 6, and 8 English characters. First, we address the problem of noise and “small sample” by generating large low noise data using eigenspace-based bootstrap sampling and generative adversarial network (GAN). Second, we transform EEG recordings into spatial-spectral images to capture spatial information. Third, we built parameter-optimized CNN models to predict four levels of CL using single-frequency bands (i.e., θ, α, β) and stacked (i.e., all three bands) representations. In our quest to provide interpretable models, we applied Gradientweighted Class Activation Mapping (Grad-CAM) to our models to localize the brain regions responsible for the prediction of CL. Empirical analysis of models trained using θ, α, β, and stacked representation show accuracy of 90%, 89%, 91%, and 94%, respectively. Grad-CAM visualizations showed that the prefrontal, cerebellum, frontal, and parietal areas have the highest contribution to the prediction of CL.

Learning representations and extracting meaningful patterns from electroencephalogram (EEG) recordings is critical for analyzing cognitive events (e.g., predicting cognitive load). The primary challenges include individual variability, technical noise from unreliable sensor-skin contacts, and rapid temporal changes in the EEG recordings. Given the multi-factorial nature of the problems, deep learning models are natural choices for learning representations from the data. However, the extensive time required for data collection limits the number of subjects (samples) available, which is essential for building robust deep learning models. We introduce an ensemble generative adversarial network (EGAN) to generate high-fidelity EEG data. The EGAN generates multichannel EEG recordings and their spatial-spectral representation. Key design constraints were preserving topological structure and maintaining proper bias-variance trade-offs for building robust models. To ensure the quality of the synthetic data, we visually inspected the data generated by EGAN. We conducted spectral analyses to confirm that the quality and spectral similarity were comparable to EEG recordings. To illustrate the efficacy of data generated by EGAN, we developed a convolutional neural network (CNN) model to predict four levels of cognitive load. We used spatial-spectral representations (Topomap) from three frequency bands (i.e., $$\theta $$ θ , $$\alpha $$ α , and $$\beta $$ β ). We trained our model on real data and a mixture of original and EGAN-generated data with varying proportions (e.g., 50%, 70%, 100%) of real data for training. We compared the performance of models trained solely on original data to those trained on mixed data. Our results indicate that CNN models trained on EGAN-supplemented data significantly outperformed those trained using only real data across all three frequency bands and all training set proportions. For example, the model trained on the mixed data with 50% real data achieved classification F1 and accuracy scores of approximately 71%, 69%, and 72% for $$\theta $$ θ , $$\alpha $$ α , and $$\beta $$ β bands, respectively. The same model trained on only real data achieved approximately 52%, 57%, and 59% accuracy and F1 scores for the respective bands. These results demonstrate the utility of EGAN in generating high-fidelity EEG data, significantly advancing cognitive load analysis and modeling other cognitive events.