A significant challenge that hinders advancements in medical research is the sensitive and confidential nature of patient data in available datasets. In particular, sharing patients’ facial images poses considerable privacy risks, especially with the rise of generative artificial intelligence (AI), which could misuse such data if accessed by unauthorized parties. However, facial expressions are a valuable source of information for doctors and researchers, which creates a need for methods to derive them without compromising patient privacy or safety by exposing identifiable facial images. To address this, we present a quick, computationally efficient method for detecting action units (AUs) and their intensities—key indicators of health and emotion—using only 3D facial landmarks. Our proposed framework extracts 3D face landmarks from video recordings and employs a lightweight neural network (NN) to identify AUs and estimate AU intensities based on these landmarks. Our proposed method reaches a 79.25% F1-score in AU detection for the main AUs, and 0.66 in AU intensity estimation Root Mean Square Error (RMSE). This performance shows that it is possible for researchers to share 3D landmarks, which are far less intrusive, instead of facial images while maintaining high accuracy in AU detection. Moreover, to showcase the usefulness of our AU detection model, using the detected AUs and estimated intensities, we trained state-of-the-art Deep Learning (DL) models to detect pain. Our method reaches 91.16% accuracy in pain detection, which is not far behind the 93.14% accuracy obtained when employing a convolutional neural network (CNN) with residual blocks trained on actual images and the 92.11% accuracy obtained when employing all the ground-truth AUs.
This study introduces a radar-based model for estimating blood pressure (BP) in a touch-free manner. The model accurately detects cardiac activity, allowing for contactless and continuous BP monitoring. Cardiac motions are considered crucial components for estimating blood pressure. Unfortunately, because these movements are extremely subtle and can be readily obscured by breathing and background noise, accurately detecting these motions with a radar system remains challenging. Our approach to radar-based blood pressure monitoring in this research primarily focuses on cardiac feature extraction. Initially, an integrated-spectrum waveform is implemented. The method is derived from the short-time Fourier transform (STFT) and has the ability to capture and maintain minute cardiac activities. The integrated spectrum concentrates on energy changes brought about by short and high-frequency vibrations, in contrast to the pulse-wave signals used in previous works. Hence, the interference caused by respiration, random noise, and heart contractile activity can be effectively eliminated. Additionally, we present two approaches for estimating cardiac characteristics. These methods involve the application of a hidden semi-Markov model (HSMM) and a U-net model to extract features from the integrated spectrum. In our approach, the accuracy of extracted cardiac features is highlighted by the notable decreases in the root mean square error (RMSE) for the estimated interbeat intervals (IBIs), systolic time, and diastolic time, which were reduced by 87.5%, 88.7%, and 73.1%. We reached a comparable prediction accuracy even while our subject was breathing normally, despite previous studies requiring the subject to hold their breath. The diastolic BP (DBP) error of our model is 3.98±5.81 mmHg (mean absolute difference ± standard deviation), and the systolic BP (SBP) error is 6.52±7.51 mmHg.
A significant challenge that hinders advancements in medical research is the sensitive and confidential nature of patient data in available datasets. In particular, sharing patients’ facial images poses considerable privacy risks, especially with the rise of generative artificial intelligence (AI), which could misuse such data if accessed by unauthorized parties. However, facial expressions are a valuable source of information for doctors and researchers, which creates a need for methods to derive them without compromising patient privacy or safety by exposing identifiable facial images. To address this, we present a quick, computationally efficient method for detecting Action Units (AUs) and their intensities—key indicators of health and emotion—using only 3D facial landmarks. Our proposed framework extracts 3D face landmarks from video recordings and employs a lightweight neural network to identify AUs and estimate AU intensities based on these landmarks. Our proposed method reaches 79.25% in AU detection F1-score for the main AUs, and 0.66 in AU intensity estimation Root Mean Square Error (RMSE). This performance shows that it is possible for researchers to share 3D landmarks, which are far less intrusive, instead of facial images while maintaining high accuracy in AU detection. Moreover, to showcase the usefulness of our AU detection model, using the detected AUs and estimated intensities, we trained state-of-the-art deep learning models to detect pain. Our method reaches 91.16% accuracy in pain detection, which is not far behind the 93.14% accuracy obtained when employing a convolutional neural network (CNN) with residual blocks trained on actual images and the 92.11% accuracy obtained when employing all the ground-truth AUs.