Continuous cardiac monitoring has been developed to evaluate cardiac activity outside of clinical environments due to the advancement of novel instruments. Seismocardiography (SCG) is one of the vital components that could develop such a monitoring system. Although SCG has been presented with a lower accuracy, this novel cardiac indicator has been steadily proposed over traditional methods such as electrocardiography (ECG). Thus, it is necessary to develop an enhanced method by combining the significant cardiac indicators. In this study, the six-axis signals of accelerometer and gyroscope were measured and integrated by the L2 normalization and multi-dimensional kineticardiography (MKCG) approaches, respectively. The waveforms of accelerometer and gyroscope were standardized and combined via ensemble averaging, and the heart rate was calculated from the dominant frequency. Thirty participants (15 females) were asked to stand or sit in relaxed and aroused conditions. Their SCG was measured during the task. As a result, proposed method showed higher accuracy than traditional SCG methods in all measurement conditions. The three main contributions are as follows: (1) the ensemble averaging enhanced heart rate estimation with the benefits of the six-axis signals; (2) the proposed method was compared with the previous SCG method that employs fewer-axis; and (3) the method was tested in various measurement conditions for a more practical application.

Heart rate has been measured comfortably using a camera without the skin-contact by the development of vision-based measurement. Despite the potential of the vision-based measurement, it has still presented limited ability due to the noise of illumination variance and motion artifacts. Remote ballistocardiography (BCG) was used to estimate heart rate from the ballistocardiographic head movements generated by the flow of blood through the carotid arteries. It was robust to illumination variance but still limited in the motion artifacts such as facial expressions and voluntary head motions. Recent studies on remote BCG focus on the improvement of signal extraction by minimizing the motion artifacts. They simply estimated the heart rate from the cardiac signal using peak detection and fast fourier transform (FFT). However, the heart rate estimation based on peak detection and FFT depend on the robust signal estimation. Thus, if the cardiac signal is contaminated with some noise, the heart rate cannot be estimated accurately. This study aimed to develop a novel method to improve heart rate estimation from ballistocardiographic head movements using the unsupervised clustering. First, the ballistocardiographic head movements were measured from facial video by detecting facial points using the good-feature-to-track (GFTT) algorithm and by tracking using the Kanade–Lucas–Tomasi (KLT) tracker. Second, the cardiac signal was extracted from the ballistocardiographic head movements by bandpass filter and principal component analysis (PCA). The relative power density (RPD) was extracted from its power spectrum between 0.75 Hz and 2.5 Hz. Third, the unsupervised clustering was performed to construct a model to estimate the heart rate from the RPD using the dataset consisting of the RPD and the heart rate measured from electrocardiogram (ECG). Finally, the heart rate was estimated from the RPD using the model. The proposed method was verified by comparing it with previous methods using the peak detection and the FFT. As a result, the proposed method estimated a more accurate heart rate than previous methods in three experiments by levels of the motion artifacts consisting of facial expressions and voluntary head motions. The four main contributions are as follows: (1) the unsupervised clustering improved the heart rate estimation by overcoming the motion artifacts (i.e., facial expressions and voluntary head motions); (2) the proposed method was verified by comparing with the previous methods using the peak detection and the FFT; (3) the proposed method can be combined with existing vision-based measurement and can improve their performance; (4) the proposed method was tested by three experiments considering the realistic environment including the motion artifacts, thus, it increases the possibility of the non-contact measurement in daily life.

Remote sensing of vital signs has been developed to improve the measurement environment by using a camera without a skin-contact sensor. The camera-based method is based on two concepts, namely color and motion. The color-based method, remote photoplethysmography (RPPG), measures the color variation of the face generated by reflectance of blood, whereas the motion-based method, remote ballistocardiography (RBCG), measures the subtle motion of the head generated by heartbeat. The main challenge of remote sensing is overcoming the noise of illumination variance and motion artifacts. The studies on remote sensing have focused on the blind source separation (BSS) method for RGB colors or motions of multiple facial points to overcome the noise. However, they have still been limited in their real-world applications. This study hypothesized that BSS-based combining of colors and the motions can improve the accuracy and feasibility of remote sensing in daily life. Thus, this study proposed a fusion method to estimate heart rate based on RPPG and RBCG by the BSS methods such as ensemble averaging (EA), principal component analysis (PCA), and independent component analysis (ICA). The proposed method was verified by comparing it with previous RPPG and RBCG from three datasets according to illumination variance and motion artifacts. The three main contributions of this study are as follows: (1) the proposed method based on RPPG and RBCG improved the remote sensing with the benefits of each measurement; (2) the proposed method was demonstrated by comparing it with previous methods; and (3) the proposed method was tested in various measurement conditions for more practical applications.

Cardiac activity has been monitored continuously in daily life by virtue of advanced medical instruments with microelectromechanical system (MEMS) technology. Seismocardiography (SCG) has been considered to be free from the burden of measurement for cardiac activity, but it has been limited in its application in daily life. The most important issues regarding SCG are to overcome the limitations of motion artifacts due to the sensitivity of motion sensor. Although novel adaptive filters for noise cancellation have been developed, they depend on the researcher’s subjective decision. Convolutional neural networks (CNNs) can extract significant features from data automatically without a researcher’s subjective decision, so that signal processing has been recently replaced as CNNs. Thus, this study aimed to develop a novel method to enhance heart rate estimation from thoracic movement by CNNs. Thoracic movement was measured by six-axis accelerometer and gyroscope signals using a wearable sensor that can be worn by simply clipping on clothes. The dataset was collected from 30 participants (15 males, 15 females) using 12 measurement conditions according to two physical conditions (i.e., relaxed and aroused conditions), three body postures (i.e., sitting, standing, and supine), and six movement speeds (i.e., 3.2, 4.5, 5.8, 6.4, 8.5, and 10.3 km/h). The motion data (i.e., six-axis accelerometer and gyroscope) and heart rate (i.e., electrocardiogram (ECG)) were determined as the input data and labels in the dataset, respectively. The CNN model was developed based on VGG Net and optimized by testing according to network depth and data augmentation. The ensemble network of the VGG-16 without data augmentation and the VGG-19 with data augmentation was determined as optimal architecture for generalization. As a result, the proposed method showed higher accuracy than the previous SCG method using signal processing in most measurement conditions. The three main contributions are as follows: (1) the CNN model enhanced heart rate estimation with the benefits of automatic feature extraction from the data; (2) the proposed method was compared with the previous SCG method using signal processing; (3) the method was tested in 12 measurement conditions related to daily motion for a more practical application.

As media consumption through electronic devices increases, there is growing interest in ASMR videos, known for inducing relaxation and positive emotional states. However, the effectiveness of ASMR varies depending on each video’s characteristics. This study identifies key sound and image features that evoke specific emotional responses. ASMR videos were categorized into two groups: high valence–low relaxation (HVLR) and low valence–high relaxation (LVHR). Subjective evaluations, along with physiological data such as electroencephalography (EEG) and heart rate variability (HRV), were collected from 31 participants to provide objective evidence of emotional and physiological responses. The results showed that both HVLR and LVHR videos can induce relaxation and positive emotions, but the intensity varies depending on the video’s characteristics. LVHR videos have sound frequencies between 50 and 500 Hz, brightness levels of 20 to 30%, and a higher ratio of green to blue. These videos led to 45% greater delta wave activity in the frontal lobe and a tenfold increase in HF HRV, indicating stronger relaxation. HVLR videos feature sound frequencies ranging from 500 to 10,000 Hz, brightness levels of 60 to 70%, and a higher ratio of yellow to green. These videos resulted in 1.2 times higher beta wave activity in the frontal lobe and an increase in LF HRV, indicating greater cognitive engagement and positive arousal. Participants’ subjective reports were consistent with these physiological responses, with LVHR videos evoking feelings of calmness and HVLR videos inducing more vibrant emotions. These findings provide a foundation for creating ASMR content with specific emotional outcomes and offer a framework for researchers to achieve consistent results. By defining sound and image characteristics along with emotional keywords, this study provides practical guidance for content creators and enhances user understanding of ASMR videos.

Facial expression recognition (FER) plays a pivotal role in affective computing and human–computer interaction by enabling machines to interpret human emotions. However, conventional FER models often overlook individual differences in emotional intelligence (EI), which may significantly influence how emotions are perceived and expressed. This study investigates the effect of EI on facial expression recognition accuracy within the valence–arousal space. Participants were divided into high and low EI groups based on a composite score derived from the Tromsø Social Intelligence Scale and performance-based emotion tasks. Five deep learning models (EfficientNetV2-L/S, MaxViT-B/T, and VGG16) were trained on the AffectNet dataset and evaluated using facial expression data collected from participants. Emotional states were predicted as continuous valence and arousal values, which were then mapped onto discrete emotion categories for interpretability. The results indicated that individuals with higher EI achieved significantly greater recognition accuracy, particularly for emotions requiring contextual understanding (e.g., anger, sadness, and happiness), while fear was better recognized by individuals with lower EI. These findings highlight the role of emotional intelligence in modulating FER performance and suggest that integrating EI-related features into valence–arousal-based models could enhance the adaptiveness of affective computing systems.