9 February 2024 in IEEE Transactions on Multimedia
Remote photoplethysmography (rPPG) is an important technique for detecting human vital signs and has received extensive attention. For a long time, researchers have focused attention on supervised methods that rely on large amounts of labeled data. These methods are limited by their need for large amounts of data and the difficulty of acquiring ground truth physiological signals. To address these issues, several self-supervised methods based on contrastive learning have been proposed. However, they focus on contrastive learning between samples, which neglects inherent self-similar priors in physiological signals and seems to have a limited ability to cope with noise. In this paper, a linear self-supervised reconstruction task was designed for extracting the inherent self-similar priors in physiological signals. In addition, a specific noise-insensitive strategy was explored for reducing the interference of motion and illumination. The framework proposed in this paper, rPPG-MAE, demonstrates excellent performance even on the challenging VIPL-HR dataset. We also evaluate the proposed method on two public datasets, namely, PURE and UBFC-rPPG. The results show that our method not only outperforms existing self-supervised methods but also outperforms state-of-the-art (SOTA) supervised methods. One important observation is that the quality of the dataset appears to be more important than the size of the dataset used in self-supervised pretraining of the rPPG. The source code is available at https://github.com/linuxsino/rPPG-MAE .
21 November 2024 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Arterial blood pressure waveform (ABPW) offers comprehensive insights into cardiovascular health compared to discrete blood pressure measurements. However, accurately estimating shapes and pressure values of ABPW points in a beat-to-beat manner poses significant challenges. Current ABPW monitoring methods require invasive procedures or continuous skin contact, which are inconvenient and unsatisfactory. Thus, we propose WaveBP, the first contactless ABPW monitoring system utilizing a commercial mmWave radar, driven by the understanding that cardiac information serves as an implicit bridge between mmWave signals and ABPW based on a hemodynamics analysis model. To preserve waveform details, we design a hybrid Transformer model called mmFormer, incorporated with spatially-informed shortcuts. mmFormer enables consistent sequence-to-sequence transformations while accommodating different levels of personalization efforts. To mitigate the inherent instability of mmWave signals, we develop a beamforming-based data augmentation approach that has been empirically and theoretically proven to enhance robustness with multiple spatial observations. Additionally, we introduce a cross-modality knowledge transfer framework to fuse knowledge from cardiac modalities (ECG/PPG) with vibrations captured in mmWave reflections, improving accuracy without requiring extra deployment overhead. Extensive evaluations conducted on 43 subjects using a leave-one-subject-out setup validate that WaveBP achieves a high waveform correlation of 0.903 and exhibits a low (mean±standard deviation) error of point-level measurements at (-0.14±7.48) mmHg, which could be further reduced by subject-specific specialization. WaveBP demonstrates remarkable performance under challenging scenarios and exhibits potential for detailed cardiac estimations, as evidenced by our case studies on relative cardiac output estimation and cardiac abnormality detection.
22 December 2023 in IEEE Journal of Biomedical and Health Informatics
The utilization of remote photoplethysmography (rPPG) technology has gained attention in recent years due to its ability to extract blood volume pulse (BVP) from facial videos, making it accessible for various applications such as health monitoring and emotional analysis. However, the BVP signal is susceptible to complex environmental changes or individual differences, causing existing methods to struggle in generalizing for unseen domains. This paper addresses the domain shift problem in rPPG measurement and shows that most domain generalization methods fail to work well in this problem due to ambiguous instance-specific differences. To address this, the paper proposes a novel approach called Hierarchical Style-aware Representation Disentangling (HSRD). HSRD improves generalization capacity by separating domain-invariant and instance-specific feature space during training, which increases the robustness of out-of-distribution samples during inference. This work presents state-of-the-art performance against several methods in both cross and intra-dataset settings.
25 February 2025 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Remote photoplethysmography (rPPG) has been widely applied to measure heart rate from face videos. To increase the generalizability of the algorithms, domain generalization (DG) attracted increasing attention in rPPG. However, when rPPG is extended to simultaneously measure more vital signs (e.g., respiration and blood oxygen saturation), achieving generalizability brings new challenges. Although partial features shared among different physiological signals can benefit multi-task learning, the sparse and imbalanced target label space brings the seesaw effect over task-specific feature learning. To resolve this problem, we designed an end-to-end Mixture of Low-rank Experts for multi-task remote Physiological measurement (PhysMLE), which is based on multiple low-rank experts with a novel router mechanism, thereby enabling the model to adeptly handle both specifications and correlations within tasks. Additionally, we introduced prior knowledge from physiology among tasks to overcome the imbalance of label space under real-world multi-task physiological measurement. For fair and comprehensive evaluations, this paper proposed a large-scale multi-task generalization benchmark, named Multi-Source Synsemantic Domain Generalization (MSSDG) protocol. Extensive experiments with MSSDG and intra-dataset have shown the effectiveness and efficiency of PhysMLE. In addition, a new dataset was collected and made publicly available to meet the needs of the MSSDG. The code is available at https://github.com/WJULYW/PhysMLE.
11 June 2025 in IEEE Transactions on Image Processing
Remote photoplethysmography (rPPG) is a promising technology for capturing physiological signals from facial videos, with potential applications in medical health, affective computing, and biometric recognition. The demand for rPPG tasks has evolved from achieving high performance in intra-dataset testing to excelling in cross-dataset testing (i.e., domain generalization). However, most existing methods have overlooked the incorporation of prior knowledge specific to rPPG, leading to limited generalization capabilities. In this paper, we propose a novel framework that effectively integrates both explicit and implicit prior knowledge into the rPPG task. Specifically, we conduct a systematic analysis of noise sources (e.g., variations in cameras, lighting conditions, skin types, and motion) across different domains and embed this prior knowledge into the network design. Furthermore, we employ a two-branch network to disentangle physiological feature distributions from noise through implicit label correlation. Extensive experiments demonstrate that the proposed method not only surpasses state-of-the-art approaches in RGB cross-dataset evaluation but also exhibits strong generalization from RGB datasets to NIR datasets. The code is publicly available at https://github.com/keke-nice/Greip.