The monitoring of physiological parameters is a crucial topic in promoting human health and an indispensable approach for assessing physiological status and diagnosing diseases. Particularly, it holds significant value for patients who require long-term monitoring or with underlying cardiovascular disease. To this end, Visual Contactless Physiological Monitoring (VCPM) is capable of using videos recorded by a consumer camera to monitor blood volume pulse (BVP) signal, heart rate (HR), respiratory rate (RR), oxygen saturation (SpO₂) and blood pressure (BP). Recently, deep learning-based pipelines have attracted numerous scholars and achieved unprecedented development. Although VCPM is still an emerging digital medical technology and presents many challenges and opportunities, it has the potential to revolutionize clinical medicine, digital health, telemedicine as well as other areas. The VCPM technology presents a viable solution that can be integrated into these systems for measuring vital parameters during video consultation, owing to its merits of contactless measurement, cost-effectiveness, user-friendly passive monitoring and the sole requirement of an off-the-shelf camera. In fact, the studies of VCPM technologies have been rocketing recently, particularly AI-based approaches, but few are employed in clinical settings. Here we provide a comprehensive overview of the applications, challenges, and prospects of VCPM from the perspective of clinical settings and AI technologies for the first time. The thorough exploration and analysis of clinical scenarios will provide profound guidance for the research and development of VCPM technologies in clinical settings.

Remote photoplethysmography (rPPG) has been an active research topic in recent years. While most existing methods are focusing on eliminating motion artifacts in the raw traces obtained from single-scale region-of-interest (ROI), it is worth noting that there are some noise signals that cannot be effectively separated in single-scale space but can be separated more easily in multi-scale space. In this paper, we analyze the distribution of pulse signal and motion artifacts in different layers of a Gaussian pyramid. We propose a method that combines multi-scale analysis and neural network for pulse extraction in different scales, and a layer-wise attention mechanism to adaptively fuse the features according to signal strength. In addition, we propose spatial-temporal joint attention module and channel-temporal joint attention module to learn and exaggerate pulse features in the joint spaces, respectively. The proposed remote pulse extraction network is called Joint Attention and Multi-Scale fusion Network (JAMSNet). Extensive experiments have been conducted on two publicly available datasets and one self-collected dataset. The results show that the proposed JAMSNet shows better performance than state-of-the-art methods.

Deep learning has recently been intensively studied in the context of image compressive sensing (CS) to discover and represent complicated image structures. These approaches, however, either suffer from nonflexibility for an arbitrary sampling ratio or lack an explicit deep-learned regularization term. This paper aims to solve the CS reconstruction problem by combining the deep-learned regularization term and proximal operator. We first introduce a regularization term using a carefully designed residual-regressive net, which can measure the distance between a corrupted image and a clean image set and accurately identify to which subspace the corrupted image belongs. We then address a proximal operator with a tailored dilated residual channel attention net, which enables the learned proximal operator to map the distorted image into the clean image set. We adopt an adaptive proximal selection strategy to embed the network into the loop of the CS image reconstruction algorithm. Moreover, a self-ensemble strategy is presented to improve CS recovery performance. We further utilize state evolution to analyze the effectiveness of the designed networks. Extensive experiments also demonstrate that our method can yield superior accurate reconstruction (PSNR gain over 1 dB) compared to other competing approaches while achieving the current state-of-the-art image CS reconstruction performance. The test code is available at https://github.com/zjut-gwl/CSDRCANet.

Recent studies have demonstrated the benefit of extracting and fusing pulse signals from multi-scale region-of-interests (ROIs). However, these methods suffer from heavy computational load. This paper aims to effectively utilize multi-scale rPPG features with a more compact architecture. Inspired by recent research works exploring two-path architecture that leverages global and local information with bidirectional bridge in between. This paper designs a novel architecture Global-Local Interaction and Supervision Network (GLISNet), which uses a local path to learn representations in the original scale and a global path to learn representations in the other scale capturing multi-scale information. A light-weight rPPG signal generation block is attached to the output of each path that maps the pulse representation to the pulse output. A hybrid loss function is utilized enabling the local and global representations to learn directly from the training data. Extensive experiments are conducted on two publicly available datasets, and results demonstrate that GLISNet achieves superior performance in terms of signal-to-noise ratio (SNR), mean absolute error (MAE), and root mean squared error (RMSE). In terms of SNR, GLISNet has an increase of 4.41% compared with the second best algorithm PhysNet on PURE dataset. The MAE has a decrease of 13.16% compared with the second best algorithm DeeprPPG on UBFC-rPPG dataset. The RMSE has a decrease of 26.29% compared with the second best algorithm PhysNet on UBFC-rPPG dataset. Experiments on MIHR dataset demonstrates the robustness of GLISNet under low-light environment.

Remote detection of human heart rate (HR) in a non-contact manner has become an active research topic in recent years. Most existing works focus on signal separation of raw traces obtained from the facial region of interest (ROI), whereas the ROI itself has a crucial impact on the measurement accuracy. The ROI tracking issues such as not fitting the subject’s face, pixel shift when there is head motion, and fluctuation when keeping still cannot be effectively dealt with by existing remote photoplethysmography (rPPG) algorithms, which seriously affects the pulse measurement accuracy. This article addresses these issues by proposing a novel pulse signal extraction framework based on generating region proposals and multi-scale ROIs. The framework consists of two signal selection modules and one signal fusion module. First, the multiple aspect ratio sampling operation is proposed to generate bounding boxes with various aspect ratios to remove background pixels as well as to fit different subjects’ faces. Second, the bounding box perturbation strategy is proposed to generate bounding boxes around the selected position to reduce the tracking error caused by head translation and rotation. Finally, multi-scale signal fusion (MSSF) is proposed for fusion signals from multi-scale ROI. Extensive experiments are conducted on two publicly available datasets and one self-collected dataset. The results show that the proposed framework achieves significant performance improvement in a signal-to-noise ratio (SNR), mean absolute error (MAE), and root mean squared error (RMSE). On pulse rate detection dataset (PURE), Université Bourgogne Franche-Comté (UBFC)-rPPG, and Self-rPPG datasets, the SNR has a maximum increase of 2.848 dB [compared with chrominance-based rPPG (CHROM)], 1.690 dB [compared with plane-orthogonal-to-skin (POS)], and 1.806 dB (compared with CHROM), respectively. MAE has a maximum decrease of 1.947 bpm (compared with CHROM), 1.610 bpm [compared with pulse blood volume (PBV)], and 1.577 bpm (compared with CHROM), respectively. Moreover, the proposed framework is independent of existing signal extraction algorithms. They can be easily embedded in the proposed framework.