In this research, we develop an affective computing method based on machine learning for emotion recognition using a wireless protocol and a wearable electroencephalography (EEG) custom-designed device. The system collects EEG signals using an eight-electrode placement on the scalp; two of these electrodes were placed in the frontal lobe, and the other six electrodes were placed in the temporal lobe. We performed experiments on eight subjects while they watched emotive videos. Six entropy measures were employed for extracting suitable features from the EEG signals. Next, we evaluated our proposed models using three popular classifiers: a support vector machine (SVM), multi-layer perceptron (MLP), and one-dimensional convolutional neural network (1D-CNN) for emotion classification; both subject-dependent and subject-independent strategies were used. Our experiment results showed that the highest average accuracies achieved in the subject-dependent and subject-independent cases were 85.81% and 78.52%, respectively; these accuracies were achieved using a combination of the sample entropy measure and 1D-CNN. Moreover, our study investigates the T8 position (above the right ear) in the temporal lobe as the most critical channel among the proposed measurement positions for emotion classification through electrode selection. Our results prove the feasibility and efficiency of our proposed EEG-based affective computing method for emotion recognition in real-world applications.

This study presents a novel approach for affective computing through deep learning-assisted electroencephalography (EEG) analysis aimed at enhancing the detection, processing, interpretation, and emulation of human emotions. The proposed methodology introduces an Ear-EEG emotion recognition (EEER) system, leveraging a deep neural network and integrating the Internet of Things (IoT) capabilities. The key contributions of this study include (1) a comprehensive design framework encompassing both hardware and software aspects of the Ear-EEG system, as well as (2) a signals-to-three-channel image conversion method employing time-frequency super-resolution with superlets as inputs for deep learning models. Moreover, (3) a modified Vision Transformer (ViT) architecture is introduced, incorporating Shifted Patch Tokenization (SPT) and Locality Self-Attention (LSA) techniques, addressing challenges associated with limited and imbalanced small datasets, as well as the lack of locality inductive bias for emotion recognition. Additionally, (4) an IoT-assisted EEER platform is proposed, enabling remote monitoring and management. Experimental results indicate that the trained ViT model with SPT and LSA surpasses recent models in terms of performance on untrained datasets, achieving an average accuracy of 92.39%. The findings highlight the efficacy of the proposed EEER system in accurately detecting emotional states, encompassing positive and negative affective states. The integration of artificial intelligence and IoT-based healthcare platforms represents a significant advancement in the development of medical assistant tools with broad implications for future applications.

Automated monitoring technology is necessary to ensure the safe consumption and management of meat. Here we propose a system for determining food freshness on the basis of data collected by embedded cameras and gas sensors using artificial intelligence of things (AIoT). Using the DeepLab V3+ model, images of meat obtained from embedded cameras underwent semantic segmentation into lean meat and fat, revealing the nature of rotting over time. Using convolutional neural network (CNN) learning, we classified meat into fresh, semi-fresh, and rotten and detected the transition among these. The Raspberry Pi server collected online images and gas sensor data, which was then processed by a mobile application. The sensor data were collected at 1-minute intervals, and the means and standard deviations of the valuable features were extracted. Meanwhile, the camera images were preprocessed, and their features were obtained using the transfer-learning approach and MobileNet as the feature extractor. Next, a proposed one-dimensional CNN was trained on the combined features for predicting different meat quality states. The accuracy of meat freshness detection using the proposed deep learning model reaches up to 99.44%. Our Internet of Things (IoT)−connected monitoring platform, including an embedded device, database server, and smartphone application, allows for quick and easy monitoring of meat quality. The experimental results confirmed the viability of our deep-learning−based meat-monitoring system with IoT applications in the real world.

The study introduces an innovative approach to efficient mental stress detection by combining electroencephalography (EEG) analysis with on-chip neural networks, taking advantage of EEG's temporal resolution and the computational capabilities of embedded neural networks. The proposed system utilizes behind-the-ear (BTE) EEG signals and on-chip neural networks for mental stress detection. A wearable custom-designed device captures EEG signals from a single BTE channel, performs on-chip signal-to-spectrogram conversion, and integrates a compact convolutional neural network (CNN) for stress classification. The system systematically identifies key EEG frequency bands associated with stress and includes a user-friendly smartphone application for intuitive stress monitoring. EEG data were collected from 15 participants during stress-inducing tasks, such as Stroop and Mental Arithmetic tests. On-chip processing is essential for filtering EEG noise, converting signals into spectrogram images, and using these images as inputs for stress detection through the proposed on-chip CNN model. The experimental results demonstrate strong performance: leave-one-out cross-validation (LOOCV) achieves 91.72% accuracy, 93.74% specificity, 88.69% sensitivity, 90.43% precision, and an F1-score of 0.8955; while 10-fold cross-validation (CV) yields 95.32% accuracy, 95.89% specificity, 94.47% sensitivity, 93.95% precision, and an F1-score of 0.9421 on untrained datasets. The Beta band (13 Hz to 30 Hz) is identified as the most significant frequency band for detecting mental stress. This integration of BTE EEG analysis with on-chip CNNs represents a significant advancement in mental stress detection and has potential applications in medical assistance tools.

Integrating computer vision and physiological analysis in driver drowsiness detection (DDD) is a promising technology for accurately identifying drowsy states while driving, thereby preventing potentially dangerous accidents. This study proposes a multimodal DDD system with a deep neural network that combines computer vision-based face expression analysis and electroencephalogram (EEG) data analysis. Key contributions include: 1) providing a comprehensive hardware, firmware, and software design for the DDD system to acquire behind-the-ear (BTE) EEG signals, rather than conventional scalp EEGs, due to their convenience and practicality; 2) proposing a powerful and lightweight GAN-based auto-denoising method to eliminate artifacts from EEG signals during signal acquisition, significantly influencing the quality of the obtained result; 3) developing a multimodal DDD network by combining EEG analysis and computer vision-based face expression identification to improve performance in monitoring and early detection of the driver’s drowsiness while engaging in traffic. The study employs the relative root mean squared error (RRMSE) in both temporal and spectral domains to quantitatively assess the performance of the proposed approaches in artifact removal. The proposed GAN-based auto-denoising network outperforms other comparable approaches, with an RRMSE (temporal) of 0.210 and RRMSE (spectral) of 0.161. The proposed trained multimodal model with GAN-based auto-denoising is superior to other models with different denoising approaches in driver drowsiness detection across all five-evaluation metrics, with an accuracy of 95.33%, specificity of 95.48%, sensitivity of 95.17%, precision of 95.47%, and an F1-score of 95.32%. The experimental results demonstrate the practicality and feasibility of our proposed DDD system.

Monitoring mental states is essential for understanding physiological and neurological health, offering insights that enhance diagnostics and therapy. However, current systems are often bulky, immobile, and reliant on external devices for deep learning (DL), limiting real-time applications. This study introduces a compact, real-time mental state monitoring system with four innovations. First, a wearable ear-centric device captures multimodal biosignals-behind-the-ear (BTE) electroencephalography (EEG) and in-ear photoplethysmography (PPG)-enhancing portability. Second, a high-resolution superlet transform (HRST) converts these signals into spectrogram images to enrich input features for DL models. Third, the PhysAttenConvLSTM model, designed for edge deployment using tiny machine learning (TinyML), integrates convolutional neural networks (CNNs), long short-term memory (LSTM), self-attention, and a physics-informed merge layer (PIML) to classify affective states efficiently. All processes—from signal acquisition to classification—occur directly on the edge device, enabling low-latency, real-time operation without external computation. Fourth, the Internet of Medical Things (IoMT) framework is implemented for data storage, remote access, and system management. Multimodal data were collected from 25 participants during social media-based mental state induction tasks. The system achieved 92.07% accuracy in leave-one-out cross-validation (LOOCV) and 95.05% in ten-fold cross-validation (CV), with strong specificity, sensitivity, precision, and $F1$ scores. This research advances edge DL, multimodal biosignal fusion, and IoMT integration, offering an ultraportable, energy-efficient solution for real-time mental state monitoring, with potential in affective computing and personalized mental healthcare.