Brain disease can be screened using eye movements. Degenerative brain disorders change eye movement because they affect not only memory and cognition but also the cranial nervous system involved in eye movement. We compared the facial and eye movement patterns of patients with mild Alzheimer’s disease and cognitively normal people to analyze the neurological signs of dementia. After detecting the facial landmarks, the coordinate values for the movements were extracted. We used Spearman’s correlation coefficient to examine associations between horizontal and vertical facial and eye movements. We analyzed the correlation between facial and eye movements without using special eye-tracking equipment or complex conditions in order to measure the behavioral aspect of the natural human gaze. As a result, we found differences between patients with Alzheimer’s disease and cognitively normal people. Patients suffering from Alzheimer’s disease tended to move their face and eyes simultaneously in the vertical direction, whereas the cognitively normal people did not, as confirmed by a Mann–Whitney–Wilcoxon test. Our findings suggest that objective and accurate measurement of facial and eye movements can be used to screen such patients quickly. The use of camera-based testing for the early detection of patients showing signs of neurodegeneration can have a significant impact on the public care of dementia.

Research on emotion recognition from facial expressions has found evidence of different muscle movements between genuine and posed smiles. To further confirm discrete movement intensities of each facial segment, we explored differences in facial expressions between spontaneous and posed smiles with three-dimensional facial landmarks. Advanced machine analysis was adopted to measure changes in the dynamics of 68 segmented facial regions. A total of 57 normal adults (19 men, 38 women) who displayed adequate posed and spontaneous facial expressions for happiness were included in the analyses. The results indicate that spontaneous smiles have higher intensities for upper face than lower face. On the other hand, posed smiles showed higher intensities in the lower part of the face. Furthermore, the 3D facial landmark technique revealed that the left eyebrow displayed stronger intensity during spontaneous smiles than the right eyebrow. These findings suggest a potential application of landmark based emotion recognition that spontaneous smiles can be distinguished from posed smiles via measuring relative intensities between the upper and lower face with a focus on left-sided asymmetry in the upper region.

As emotional states are diverse, simply classifying them through discrete facial expressions has its limitations. Therefore, to create a facial expression recognition system for practical applications, not only must facial expressions be classified, emotional changes must be measured as continuous values. Based on the knowledge distillation structure and the teacher-bounded loss function, we propose a method to maximize the synergistic effect of jointly learning discrete and continuous emotional states of eight expression classes, valences, and arousal levels. The proposed knowledge distillation model uses Emonet, a state-of-the-art continuous estimation method, as the teacher model, and uses a lightweight network as the student model. It was confirmed that performance degradation can be minimized even though student models have multiply-accumulate operations of approximately 3.9 G and 0.3 G when using EfficientFormer and MobileNetV2, respectively, which is much less than the amount of computation required by the teacher model (16.99 G). Together with the significant improvements in computational efficiency (by 4.35 and 56.63 times using EfficientFormer and MobileNetV2, respectively), the decreases in facial expression classification accuracy were approximately 1.35% and 1.64%, respectively. Therefore, the proposed method is optimized for application-level interaction systems in terms of both the amount of computation required and the accuracy.

Human respiration reflects meaningful information, such as one’s health and psychological state. Rates of respiration are an important indicator in medicine because they are directly related to life, death, and the onset of a serious disease. In this study, we propose a noncontact method to measure respiration. Our proposed approach uses a standard RGB camera and does not require any special equipment. Measurement is performed automatically by detecting body landmarks to identify regions of interest (RoIs). We adopt a learning model trained to measure motion and respiration by analyzing movement from RoI images for high robustness to background noise. We collected a remote respiration measurement dataset to train the proposed method and compared its measurement performance with that of representative existing methods. Experimentally, the proposed method showed a performance similar to that of existing methods in a stable environment with restricted motion. However, its performance was significantly improved compared to existing methods owing to its robustness to motion noise. In an environment with partial occlusion and small body movement, the error of the existing methods was 4–8 bpm, whereas the error of our proposed method was around 0.1 bpm. In addition, by measuring the time required to perform each step of the respiration measurement process, we confirmed that the proposed method can be implemented in real time at over 30 FPS using only a standard CPU. Since the proposed approach shows state-of-the-art accuracy with the error of 0.1 bpm in the wild, it can be expanded to various applications, such as medicine, home healthcare, emotional marketing, forensic investigation, and fitness in future research.

Therapeutic communication, of which nonverbal communication is a vital component, is an essential skill for professional nurses. The aim of this study is to assess the possibility of incorporating computer analysis programs into nursing education programs to improve the nonverbal communication skills of those preparing to become professional nurses. In this pilot observational study, the research team developed a computer program for nonverbal communication analysis including facial expressions and poses. The video clip data captured during nursing simulation practice by 10 3rd- and 4th-grade nursing students at a university in South Korea involved two scenarios of communication with a child’s mother regarding the child’s pre- and post-catheterization care. The dominant facial expressions varied, with sadness (30.73%), surprise (30.14%), and fear (24.11%) being the most prevalent, while happiness (7.96%) and disgust (6.79%) were less common. The participants generally made eye contact with the mother, but there were no instances of light touch by hand and the physical distance for nonverbal communication situations was outside the typical range. These results confirm the potential use of facial expression and pose analysis programs for communication education in nursing practice.