Monitoring vital signs through unobtrusive means is a goal which has attracted a lot of attention in the past decade. This review provides a systematic and comprehensive review over the current state of the field of ambient and unobtrusive cardiorespiratory monitoring. To this end, nine different sensing modalities which have been in the focus of current research activities are covered: capacitive electrocardiography, seismo- and ballistocardiography, reflective photoplethysmography (PPG) and PPG imaging, thermography, methods relying on laser or radar for distance-based measurements, video motion analysis, as well as methods using high-frequency electromagnetic fields. Current trends in these subfields are reviewed. Moreover, we systematically analyze similarities and differences between these methods with respect to the physiological and physical effects they sense as well as the resulting implications. Finally, future research trends for the field as a whole are identified.

Optical heart rate monitoring (OHR) with reflective wrist photoplethysmography is a technique mainly used in the wellness application domain for monitoring heart rate levels during exercise. In the absence of motion, OHR technique is also able to estimate individual beat-to-beat intervals relatively well and can therefore also be used, for example, in monitoring of cardiac arrhythmias, stress, or sleep quality through heart rate variability (HRV) analysis. HRV analysis has also potential in monitoring the recovery of patients, e.g. after a medical intervention. However, in order to detect subtle changes, the calculated HRV parameters should be sufficiently accurate and very few studies exist that asses the accuracy of OHR derived HRV in non-healthy subjects. In this paper, we present a method to estimate beat-to-beat-intervals (BBIs) from reflective wrist PPG signal and evaluated the accuracy of the proposed method in estimating BBIs in a cross-sectional study with 29 hospitalized patients (mean age 70.6 years) in 24-h recordings performed after peripheral vascular surgery or endovascular interventions. Finally, we evaluate the accuracy of more than 30 commonly used HRV parameters and find that the accuracy of certain metrics, for example SDNN and triangular index, shown in the literature to be associated with the deterioration of the status of the patients during recovery from surgical intervention, could be adequate for patient monitoring. On the other hand, the parameters more affected by the high-frequency content of the HRV and especially the LF/HF-ratio should be used with caution.

Tremor is one of the common symptoms of Parkinson’s disease (PD). Thanks to the recent evolution of digital technologies, monitoring of PD patients’ hand movements employing contactless methods gained momentum. Objective: We aimed to quantitatively assess hand movements in patients suffering from PD using the artificial intelligence (AI)-based hand-tracking technologies of MediaPipe. Method: High-frame-rate videos and accelerometer data were recorded from 11 PD patients, two of whom showed classical Parkinsonian-type tremor. In the OFF-state and 30 Minutes after taking their standard oral medication (ON-state), video recordings were obtained. First, we investigated the frequency and amplitude relationship between the video and accelerometer data. Then, we focused on quantifying the effect of taking standard oral treatments. Results: The data extracted from the video correlated well with the accelerometer-based measurement system. Our video-based approach identified the tremor frequency with a small error rate (mean absolute error 0.229 (±0.174) Hz) and an amplitude with a high correlation. The frequency and amplitude of the hand movement before and after medication in PD patients undergoing medication differ. PD Patients experienced a decrease in the mean value for frequency from 2.012 (±1.385) Hz to 1.526 (±1.007) Hz and in the mean value for amplitude from 8.167 (±15.687) a.u. to 4.033 (±5.671) a.u. Conclusions: Our work achieved an automatic estimation of the movement frequency, including the tremor frequency with a low error rate, and to the best of our knowledge, this is the first paper that presents automated tremor analysis before/after medication in PD, in particular using high-frame-rate video data.

Infrared thermography for camera-based skin temperature measurement is increasingly used in medical practice, e.g., to detect fevers and infections, such as recently in the COVID-19 pandemic. This contactless method is a promising technology to continuously monitor the vital signs of patients in clinical environments. In this study, we investigated both skin temperature trend measurement and the extraction of respiration-related chest movements to determine the respiratory rate using low-cost hardware in combination with advanced algorithms. In addition, the frequency of medical examinations or visits to the patients was extracted. We implemented a deep learning-based algorithm for real-time vital sign extraction from thermography images. A clinical trial was conducted to record data from patients on an intensive care unit. The YOLOv4-Tiny object detector was applied to extract image regions containing vital signs (head and chest). The infrared frames were manually labeled for evaluation. Validation was performed on a hold-out test dataset of 6 patients and revealed good detector performance (0.75 intersection over union, 0.94 mean average precision). An optical flow algorithm was used to extract the respiratory rate from the chest region. The results show a mean absolute error of 2.69 bpm. We observed a computational performance of 47 fps on an NVIDIA Jetson Xavier NX module for YOLOv4-Tiny, which proves real-time capability on an embedded GPU system. In conclusion, the proposed method can perform real-time vital sign extraction on a low-cost system-on-module and may thus be a useful method for future contactless vital sign measurements.

Objective: Geriatric patients, especially those with dementia or in a delirious state, do not accept conventional contact-based monitoring. Therefore, we propose to measure heart rate (HR) and heart rate variability (HRV) of geriatric patients in a noncontact and unobtrusive way using photoplethysmography imaging (PPGI). Methods: PPGI video sequences were recorded from 10 geriatric patients and 10 healthy elderly people using a monochrome camera operating in the near-infrared spectrum and a colour camera operating in the visible spectrum. PPGI waveforms were extracted from both cameras using superpixel-based regions of interests (ROI). A classifier based on bagged trees was trained to automatically select artefact-free ROIs for HR estimation. HRV was calculated in the time-domain and frequency-domain. Results: an RMSE of 1.03 bpm and a correlation of 0.8 with the reference was achieved using the NIR camera for HR estimation. Using the RGB camera, RMSE and correlation improved to 0.48 bpm and 0.95, respectively. Correlation for HRV in the frequency-domain (LF/HF-ratio) was 0.50 using the NIR camera and 0.70 using the RGB camera. Conclusion: We were able to demonstrate that PPGI is very suitable to measure HR and HRV in geriatric patients. We strongly believe that PPGI will become clinically relevant in monitoring of geriatric patients. Significance: we are the first group to measure both HR and HRV in awake geriatric patients using PPGI. Moreover, we systematically evaluate the effects of the spectrum (near-infrared vs. visible), ROI, and additional motion artefact reduction algorithms on the accuracy of estimated HR and HRV.

Objectives: Camera-based vital sign estimation allows the contactless assessment of important physiological parameters. Seminal contributions were made in the 1930s, 1980s, and 2000s, and the speed of development seems ever increasing. In this suivey, we aim to overview the most recent works in this area, describe their common features as well as shortcomings, and highlight interesting “outliers”. Methods: We performed a comprehensive literature research and quantitative analysis of papers published between 2016 and 2018. Quantitative information about the number of subjects, studies with healthy volunteers vs. pathological conditions, public datasets, laboratory vs. real-world works, types of camera, usage of machine learning, and spectral properties of data was extracted. Moreover, a qualitative analysis of illumination used and recent advantages in terms of algorithmic developments was also performed. Results: Since 2016, 116 papers were published on camera-based vital sign estimation and 59% of papers presented results on 20 or fewer subjects. While the average number of participants increased from 15.7 in 2016 to 22.9 in 2018, the vast majority of papers (n=100) were on healthy subjects. Four public datasets were used in 10 publications. We found 27 papers whose application scenario could be considered a real-world use case, such as monitoring during exercise or driving. These include 16 papers that dealt with non-healthy subjects. The majority of papers (n=61) presented results based on visual, red-green-blue (RGB) information, followed by RGB combined with other parts of the electromagnetic spectrum (n=18), and thermography only (n=12), while other works (n=25) used other mono- or polychromatic non-RGB data. Surprisingly, a minority of publications (n=39) made use of consumer-grade equipment. Lighting conditions were primarily uncontrolled or ambient. While some works focused on specialized aspects such as the removal of vital sign information from video streams to protect privacy or the influence of video compression, most algorithmic developments were related to three areas: region of interest selection, tracking, or extraction of a one-dimensional signal. Seven papers used deep learning techniques, 17 papers used other machine learning approaches, and 92 made no explicit use of machine learning. Conclusion: Although some general trends and frequent shortcomings are obvious, the spectrum of publications related to camera-based vital sign estimation is broad. While many creative solutions and unique approaches exist, the lack of standardization hinders comparability of these techniques and of their performance. We believe that sharing algorithms and/ or datasets will alleviate this and would allow the application of newer techniques such as deep learning.