Background: Stress recognition using electrocardiogram (ECG) signals requires the intractable long-term heart rate variability (HRV) parameter extraction process. This study proposes a novel deep learning framework to recognize the stressful states, the Deep ECGNet, using ultra short-term raw ECG signals without any feature engineering methods. Methods: The Deep ECGNet was developed through various experiments and analysis of ECG waveforms. We proposed the optimal recurrent and convolutional neural networks architecture, and also the optimal convolution filter length (related to the P, Q, R, S, and T wave durations of ECG) and pooling length (related to the heart beat period) based on the optimization experiments and analysis on the waveform characteristics of ECG signals. The experiments were also conducted with conventional methods using HRV parameters and frequency features as a benchmark test. The data used in this study were obtained from Kwangwoon University in Korea (13 subjects, Case 1) and KU Leuven University in Belgium (9 subjects, Case 2). Experiments were designed according to various experimental protocols to elicit stressful conditions. Results: The proposed framework to recognize stress conditions, the Deep ECGNet, outperformed the conventional approaches with the highest accuracy of 87.39% for Case 1 and 73.96% for Case 2, respectively, that is, 16.22% and 10.98% improvements compared with those of the conventional HRV method. Conclusions: We proposed an optimal deep learning architecture and its parameters for stress recognition, and the theoretical consideration on how to design the deep learning structure based on the periodic patterns of the raw ECG data. Experimental results in this study have proved that the proposed deep learning model, the Deep ECGNet, is an optimal structure to recognize the stress conditions using ultra short-term ECG data.

Blood pressure (BP) is a vital sign that provides fundamental health information regarding patients. Continuous BP monitoring is important for patients with hypertension. Various studies have proposed cuff-less BP monitoring methods using pulse transit time. We propose an end-to-end deep learning architecture using only raw signals without the process of extracting features to improve the BP estimation performance using the attention mechanism. The proposed model consisted of a convolutional neural network, a bidirectional gated recurrent unit, and an attention mechanism. The model was trained by a calibration-based method, using the data of each subject. The performance of the model was compared to the model that used each combination of the three signals, and the model with the attention mechanism showed better performance than other state-of-the-art methods, including conventional linear regression method using pulse transit time (PTT). A total of 15 subjects were recruited, and electrocardiogram, ballistocardiogram, and photoplethysmogram levels were measured. The 95% confidence interval of the reference BP was [86.34, 143.74] and [51.28, 88.74] for systolic BP (SBP) and diastolic BP (DBP), respectively. The $R^2$ values were 0.52 and 0.49, and the mean-absolute-error values were 4.06 ± 4.04 and 3.33 ± 3.42 for SBP and DBP, respectively. In addition, the results complied with global standards. The results show the applicability of the proposed model as an analytical metric for BP estimation.

Recent studies have demonstrated the disassociation between the mu and beta rhythms of electroencephalogram (EEG) during motor imagery tasks. The proposed algorithm in this paper uses a fully data-driven multivariate empirical mode decomposition (MEMD) in order to obtain the mu and beta rhythms from the nonlinear EEG signals. Then, the strong uncorrelating transform complex common spatial patterns (SUTCCSP) algorithm is applied to the rhythms so that the complex data, constructed with the mu and beta rhythms, becomes uncorrelated and its pseudocovariance provides supplementary power difference information between the two rhythms. The extracted features using SUTCCSP that maximize the interclass variances are classified using various classification algorithms for the separation of the left- and right-hand motor imagery EEG acquired from the Physionet database. This paper shows that the supplementary information of the power difference between mu and beta rhythms obtained using SUTCCSP provides an important feature for the classification of the left- and right-hand motor imagery tasks. In addition, MEMD is proved to be a preferred preprocessing method for the nonlinear and nonstationary EEG signals compared to the conventional IIR filtering. Finally, the random forest classifier yielded a high performance for the classification of the motor imagery tasks.

Work-related stress causes serious negative physiological and socioeconomic effects on employees. Detecting stress levels in a timely manner is important for appropriate stress management; therefore, this study proposes a deep learning (DL) approach that accurately detects work-related stress by using multimodal signals. We designed a protocol that simulates stressful situations and recruited 24 subjects for the experiments. Then, we collected electrocardiogram (ECG), respiration (RESP), and video data. The datasets were pre-processed and 10-s ECG and RESP signals and a sequence of facial features were fed into our deep neural network. Sixty-eight facial landmarks’ coordinates were extracted, and facial textures were extracted from a pre-trained network based on facial expression recognition. Each signal was processed by each of its network branch, and data were fused at two different levels: 1) feature-level and 2) decision-level. The feature-level fusion that used RESP and facial landmarks’ coordinates showed average accuracy of 73.3%, AUC of 0.822, and F1 score of 0.700 in two-level stress classification, and the feature-level fusion that used ECG, RESP, and the coordinates showed average accuracy of 54.4%, AUC of 0.727, and F1 score of 0.508 in three-level stress classification. When analyzing the weights in the decision-level fusion, we found that the importance of each information item varied according to the stress classification problem. When comparing t-stochastic neighbor embedding results, we observed that overlapped samples of different classes caused performance degradation in both classifications. Our findings suggest that the proposed DL approach fusing multimodal and heterogeneous signals can enhance stress detection.

We report printed flexible optoelectronic sensors composed of red organic light-emitting diodes (OLEDs) and organic photodiodes (OPDs) for detection of various biological signals in a photoplethysmograph (PPG) device. Fabricated flexible OLEDs achieved maximum luminance >1000 cd/m² at 9 V, with peak at 640 nm. Maximum flexible OPD photosensitivity for the poly(3-hexylthiophene-2, 5-diyl) and phenyl-C61-butyric acid methyl ester (PCBM) heterojunction is $ {2} \times {10}^{{2}}$ at 0 V and 1.76 at −1 V, irradiated with 1.2 mW/cm² at 660 nm. The diketopyrrolopyrrole thieno [3,2-b]thiophene blended with PCBM OPDs with poly (3,4-ethylenedioxythiophene):polystyrene sulfonate anode showed photosensitivity = 84 at −1 V bias to almost ${6} \times {10}^{{4}}$ at 0 V accompanied by low dark current ( ${9.5} \times {10}^{-{8}}$ A/cm² at −1 V). PPG signals were successfully detected using the developed flexible PPG sensor and the conventional driving circuit. Human studies were conducted to evaluate the flexible PPG sensor performance in practical applications. Subject drowsiness was estimated from heart rate variability, extracted from the PPG signals, using machine learning algorithms. The flexible PPG sensor achieved 79.2% accuracy and 72.1% area under the receiver (AUC) to predict drowsiness (60-s window), which are meaningful results compared with conventional PPG sensors (83.3% accuracy and 69% AUC). Drowsiness estimation experiments using two PPG signals showed that the flexible PPG sensor achieved similar or better performance compared to conventional PPG sensors.