Public databases are important for evaluating and comparing different methods and algorithms for camera-based heart rate estimation. Because uncompressed video requires huge file sizes, a need for compression algorithms exists to store and share video data. Due to the optimization of modern video codecs for human perception, video compression can influence heart rate estimation negatively by reducing or eliminating small color changes of the skin (PPG) that are needed for camera based heart rate estimation. In this paper, we contribute a comprehensive analysis to answer the question of how to compress video without compromising PPG information. To analyze the influence of video compression, we compare the effect of several encoding parameters: two modern encoders (H264, H265), compression rate, resolution changes using different scaling algorithms, color subsampling, and file size on two publicly available datasets. We show that increasing the compression rate decreases the accuracy of heart rate estimation, but that resolution can be reduced (up to a cutoff point) and color subsampling can be applied for reducing file size without a big impact on heart rate estimation. From the results, we derive and propose guidelines for the recording and encoding of video data for camera-based heart rate estimation. The paper can sensitize the research community toward the problems of video encoding, and the proposed recommended practices can help with conducting future experiments and creating valuable datasets that can be shared publicly. Such datasets would improve comparability and reproducibility in the research field.

Vision-based 3D human pose estimation approaches are typically evaluated on datasets that are limited in diversity regarding many factors, e.g., subjects, poses, cameras, and lighting. However, for real-life applications, it would be desirable to create systems that work under arbitrary conditions (“in-the-wild”). To advance towards this goal, we investigated the commonly used datasets HumanEva-I, Human3.6M, and Panoptic Studio, discussed their biases (that is, their limitations in diversity), and illustrated them in cross-database experiments (for which we used a surrogate for roughly estimating in-the-wild performance). For this purpose, we first harmonized the differing skeleton joint definitions of the datasets, reducing the biases and systematic test errors in cross-database experiments. We further proposed a scale normalization method that significantly improved generalization across camera viewpoints, subjects, and datasets. In additional experiments, we investigated the effect of using more or less cameras, training with multiple datasets, applying a proposed anatomy-based pose validation step, and using OpenPose as the basis for the 3D pose estimation. The experimental results showed the usefulness of the joint harmonization, of the scale normalization, and of augmenting virtual cameras to significantly improve cross-database and in-database generalization. At the same time, the experiments showed that there were dataset biases that could not be compensated and call for new datasets covering more diversity. We discussed our results and promising directions for future work.

The respiratory rate is an important vital parameter that provides information about persons’ physical condition. In clinical practice it is currently only monitored using contact-based techniques, which can have negative effects on patients. In this study, a new algorithm for remote respiratory rate recognition is presented using photoplethysmographic signals derived from facial video images in the visible light spectrum. The effects of different implementation steps in the presented algorithm are investigated in order to optimize the approach and gain new findings in this research field. In addition, a detailed examination of already implemented procedures is performed and the results are compared on two different databases. We show that by fusing the results of seven different respiratory-induced modulations in combination with other processing steps, very good estimates for the respiratory rate on both moving and non-moving data are achieved. The obtained detection rates of 72.16 % and 87.68 % are significantly higher than those of the best comparison algorithm with 37.37 % and 59.13 %. The comparison algorithms developed so far are not competitive with the newly designed method, especially for video recordings involving persons in motion. This paper provides important new findings in the field of facial video-based respiratory rate recognition for the research community. A new method has been created that delivers significantly better estimates of the respiratory rate than previously developed techniques.