This paper aims to investigate the robust and distinguishable pattern of heart rate variability (HRV) signals, acquired from wearable electrocardiogram (ECG) or photoplethysmogram (PPG) sensors, for driver drowsiness detection. As wearable sensors are so vulnerable to slight movement, they often produce more noise in signals. Thus, from noisy HRV signals, we need to find good traits that differentiate well between drowsy and awake states. To this end, we explored three types of recurrence plots (RPs) generated from the R–R intervals (RRIs) of heartbeats: Bin-RP, Cont-RP, and ReLU-RP. Here Bin-RP is a binary recurrence plot, Cont-RP is a continuous recurrence plot, and ReLU-RP is a thresholded recurrence plot obtained by filtering Cont-RP with a modified rectified linear unit (ReLU) function. By utilizing each of these RPs as input features to a convolutional neural network (CNN), we examined their usefulness for drowsy/awake classification. For experiments, we collected RRIs at drowsy and awake conditions with an ECG sensor of the Polar H7 strap and a PPG sensor of the Microsoft (MS) band 2 in a virtual driving environment. The results showed that ReLU-RP is the most distinct and reliable pattern for drowsiness detection, regardless of sensor types (i.e., ECG or PPG). In particular, the ReLU-RP based CNN models showed their superiority to other conventional models, providing approximately 6–17% better accuracy for ECG and 4–14% for PPG in drowsy/awake classification.

Mental stress can lead to traffic accidents by reducing a driver’s concentration or increasing fatigue while driving. In recent years, demand for methods to detect drivers’ stress in advance to prevent dangerous situations increased. Thus, we propose a novel method for detecting driving stress using nonlinear representations of short-term (30 s or less) physiological signals for multimodal convolutional neural networks (CNNs). Specifically, from hand/foot galvanic skin response (HGSR, FGSR) and heart rate (HR) short-term input signals, first, we generate corresponding two-dimensional nonlinear representations called continuous recurrence plots (Cont-RPs). Second, from the Cont-RPs, we use multimodal CNNs to automatically extract FGSR, HGSR, and HR signal representative features that can effectively differentiate between stressed and relaxed states. Lastly, we concatenate the three extracted features into one integrated representation vector, which we feed to a fully connected layer to perform classification. For the evaluation, we use a public stress dataset collected from actual driving environments. Experimental results show that the proposed method demonstrates superior performance for 30-s signals, with an overall accuracy of 95.67%, an approximately 2.5–3% improvement compared with that of previous works. Additionally, for 10-s signals, the proposed method achieves 92.33% classification accuracy, which is similar to or better than the performance of other methods using long-term signals (over 100 s).

A method for accurately analyzing electrocardiograms (ECGs), which are obtained from electrical signals generated by cardiac activity, is essential in heart disease diagnosis. However, rhythms are typically obtained with relatively few data samples and similar characteristics, making them difficult to classify. To solve these issues, we proposed a novel method that distinguishes a given ECG rhythm using a beat score map (BSM) image. Through the proposed method, the associations between beats and previously used features, such as the R–R interval, were considered. Rhythm classification was implemented by training a convolutional neural network model and using transfer learning with the created BSM image. As a result, the proposed method for ECG rhythms with small data samples showed significant results. It also showed good performance in differentiating atrial fibrillation (AFIB) and atrial flutter (AFL) rhythms, which are difficult to distinguish due to their similar characteristics. The performance for rhythms with a small number of samples of the proposed method is 20% better than an existing method. In addition, the performance based on the F-1 score for classifying AFIB and AFL of the proposed method is 30% better than the existing method. This study solved the previous limitations caused by small sample numbers and similar rhythms.

Research on computer-aided arrhythmia classification is actively conducted, but the limited generalization capacity constrains its applicability in practical clinical settings. One of the primary challenges in deploying such techniques in real-world scenarios is the inter-patient variability and the consequent performance degradation. In this study, we leverage our previous innovation, the n-beat-score map (n-BSM), to introduce an adversarial framework to mitigate the issue of poor performance in arrhythmia classification within the inter-patient paradigm. The n-BSM is a 2D representation of the ECG signal, capturing its constituent beat characteristics through beat-score vectors derived from a pre-trained beat classifier. We employ adversarial learning to eliminate patient-dependent features during the training of the beat classifier, thereby generating the patient-independent n-BSM (PI-BSM). This approach enables us to concentrate primarily on the learning characteristics associated with beat type rather than patient-specific features. Through a beat classifier pre-trained with adversarial learning, a series of beat-score vectors are generated for the beat segments that make up a given ECG signal. These vectors are then concatenated chronologically to form a PI-BSM. Utilizing PI-BSMs as the input, an arrhythmia classifier is trained to differentiate between distinct types of rhythms. This approach yields a 14.27% enhancement in the F1-score in the MIT-BIH arrhythmia database and a 4.97% improvement in cross-database evaluation using the Chapman–Shaoxing 12-lead ECG database.

Cross-database electrocardiogram (ECG) classification remains a critical challenge due to variations in patient populations, recording conditions, and annotation granularity. Existing methodologies for ECG arrhythmia classification have primarily utilized datasets with either fine-grained or coarse-grained labels, but seldom both simultaneously. Fine-grained labels provide beat-level annotations, whereas coarse-grained labels offer only record-level labels. In this study, we propose an innovative cross-database learning framework that utilizes both fine-grained and coarse-grained labels in tandem, thereby enhancing classification performance across heterogeneous datasets. Specifically, our approach begins with the pretraining of a CNN-based beat classifier that takes ECG signals as the input and predicts beat types on a finely labeled dataset, namely the MIT-BIH Arrhythmia Database (MITDB). The pretrained model is then fine-tuned using weakly supervised learning on two coarsely labeled datasets: the SPH one, which contains four rhythm classes, and the PTB-XL one, which involves binary classification between the sinus rhythm (SR) and atrial fibrillation (AFIB). Once the beat classifier is adapted to a new dataset, it generates a two-dimensional beat-score-map (BSM) representation from the input ECG signal. This 2D BSM is subsequently utilized as the input for arrhythmia rhythm classification. The proposed method achieves F1 scores of 0.9301 on the SPH dataset and 0.9267 on the PTB-XL dataset, corresponding to the multi-class and binary rhythm classification tasks described above. These results demonstrate a robust cross-database classification of complex cardiac arrhythmia rhythms. Furthermore, t-SNE visualizations of the 2D BSM representations, after adaptation to the coarsely labeled SPH and PTB-XL datasets, validate how our method significantly enhances the ability to differentiate between various arrhythmia rhythm types, thus highlighting its effectiveness in cross-database ECG analysis.