Video-based heartbeat rate measurement is a rapidly growing application in remote health monitoring. Video-based heartbeat rate measuring methods operate mainly by estimating photoplethysmography or ballistocardiography signals. These methods operate by estimating the microscopic color change in the face or by estimating the microscopic rigid motion of the head/ facial skin. However, the robustness to motion artifacts caused by illumination variance and motion variance of the subject poses main challenge. We present a video-based heartbeat rate measuring framework to overcome these problems by using the principle of ballistocardiography. In this paper, we proposed a ballistocardiography model based on Newtons third law of force and dynamics of harmonic oscillation. We formulate a framework based on the ballistocardiography model to measure the rigid involuntary head motion caused by the ejection of the blood from the heart. Our proposed framework operates by estimating the motion of multivariate feature points to estimate the heartbeat rate autonomously. We evaluated our proposed framework along with existing video-based heartbeat rate measuring methods with three databases, namely; MAHNOB HCI database, human-computer interaction database, and driver health monitoring database. Our proposed framework outperformed existing methods by reporting a low mean error rate of 4.34 bpm with a standard deviation of 3.14 bpm, root mean square error of 5.29 with a high Pearson correlation coefficient of 0.91. The proposed method also operated robustly in the human-computer interaction database and driver health monitoring database by overcoming the issues related to illumination and motion variance.

Objective: Facial weakness is a common sign of neurological diseases such as Bell's palsy and stroke. However, recognizing facial weakness still remains as a challenge, because it requires experience and neurological training. Methods: We propose a framework for facial weakness detection, which models the temporal dynamics of both shape and appearance-based features of each target frame through a bi-directional long short-term memory network (Bi-LSTM). The system is evaluated on a “in-the-wild”video dataset that is verified by three board-certified neurologists. In addition, three emergency medical services (EMS) personnel and three upper level residents rated the dataset. We compare the evaluation of the proposed algorithm with other comparison methods as well as the human raters. Results: Experimental evaluation demonstrates that: (1) the proposed algorithm achieves the accuracy, sensitivity, and specificity of 94.3%, 91.4%, and 95.7%, which outperforms other comparison methods and achieves the equal performance to paramedics; (2) the framework can provide visualizable and interpretable results that increases model transparency and interpretability; (3) a prototype is implemented as a proof-of-concept showcase to show the feasibility of an inexpensive solution for facial weakness detection. Conclusion: The experiment results suggest that the proposed framework can identify facial weakness effectively. Significance: We provide a proof-of-concept study, showing that such technology could be used by non-neurologists to more readily identify facial weakness in the field, leading to increasing coverage and earlier treatment.

Intraoperative identification of head and neck cancer tissue is essential to achieve complete tumor resection and mitigate tumor recurrence. Mesoscopic fluorescence lifetime imaging (FLIm) of intrinsic tissue fluorophores emission has demonstrated the potential to demarcate the extent of the tumor in patients undergoing surgical procedures of the oral cavity and the oropharynx. Here, we report FLIm-based classification methods using standard machine learning models that account for the diverse anatomical and biochemical composition across the head and neck anatomy to improve tumor region identification. Three anatomy-specific binary classification models were developed (i.e., “base of tongue,” “palatine tonsil,” and “oral tongue”). FLIm data from patients (N = 85) undergoing upper aerodigestive oncologic surgery were used to train and validate the classification models using a leave-one-patient-out cross-validation method. These models were evaluated for two classification tasks: (1) to discriminate between healthy and cancer tissue, and (2) to apply the binary classification model trained on healthy and cancer to discriminate dysplasia through transfer learning. This approach achieved superior classification performance compared to models that are anatomy-agnostic; specifically, a ROC-AUC of 0.94 was for the first task and 0.92 for the second. Furthermore, the model demonstrated detection of dysplasia, highlighting the generalization of the FLIm-based classifier. Current findings demonstrate that a classifier that accounts for tumor location can improve the ability to accurately identify surgical margins and underscore FLIm's potential as a tool for surgical guidance in head and neck cancer patients, including those subjects of robotic surgery

Automated eye-tracking technology could enhance diagnosis for many neurological diseases, including stroke. Current literature focuses on gaze estimation through a form of calibration. However, patients with neuro-ocular abnormalities may have difficulty completing a calibration procedure due to inattention or other neurological deficits. Objective: We investigated 1) the need for calibration to measure eye movement symmetry in healthy controls and 2) the potential of eye movement symmetry to distinguish between healthy controls and patients. Methods: We analyzed fixations, smooth pursuits, saccades, and conjugacy measured by a Spearman correlation coefficient and utilized a linear mixed-effects model to estimate the effect of calibration. Results: Healthy participants (n = 18) did not differ in correlations between calibrated and non-calibrated conditions for all tests. The calibration condition did not improve the linear mixed effects model (log-likelihood ratio test p = 0.426) in predicting correlation coefficients. Interestingly, the patient group (n = 17) differed in correlations for the DOT (0.844 [95% CI 0.602, 0.920] vs. 0.98 [95% CI 0.976, 0.985]), H (0.903 [95% CI 0.746, 0.958] vs. 0.979 [95% CI 0.971, 0.986]), and OKN (0.898 [95% CI 0.785, 0.958] vs. 0.993 [95% CI 0.987, 0.996]) tests compared to healthy controls along the x-axis. These differences were not observed along the y-axis. Significance: This study suggests that automated eye tracking can be deployed without calibration to measure eye movement symmetry. It may be a good discriminator between normal and abnormal eye movement symmetry. Validation of these findings in larger populations is required.

Automatic Ingestion Monitor v2 (AIM-2) is an egocentric camera and sensor that aids monitoring of individual diet and eating behavior by capturing still images throughout the day and using sensor data to detect eating. The images may be used to recognize foods being eaten, eating environment, and other behaviors and daily activities. At the same time, captured images may carry privacy concerning content such as (1) people in social eating and/or bystanders (i.e., bystander privacy); (2) sensitive documents that may appear on a computer screen in the view of AIM-2 (i.e., context privacy). In this paper, we propose a novel approach based on automatic, image redaction for privacy protection by selective content removal by semantic segmentation using a deep learning neural network. The proposed method reported a bystander privacy removal with precision of 0.87 and recall of 0.94 and reported context privacy removal by precision and recall of 0.97 and 0.98. The results of the study showed that selective content removal using deep learning neural network is a much more desirable approach to address privacy concerns for an egocentric wearable camera for nutritional studies.

The ability to perform quantitative and automated neurological assessment could enhance diagnosis and treatment in the pre-hospital setting, such as during telemedicine or emergency medical services (EMS) encounters. Such a tool could be developed by adapting clinically significant information such as symmetry of eye movement or conjugate eye movement. Here we describe a digital camera-based eye tracking method “NeuroGaze” to capture the symmetry of eye movement while performing neurological eye examination. The proposed method was developed based on detecting the center of the pupil for both eyes from a given video and measuring eye conjugacy by transforming the pupil center coordinates to relative gaze. The method was tested on healthy volunteers while performing three neurological eye examinations ^{1 1}The NeuroEye dataset is made available at https://www.kaggle.com/datasets/mahassan8/neuroeye . We also compared our proposed approach to state-of-the-art digital camera-based eye-tracking methods and commercial off-the-shelf (COTS) eye trackers. NeuroGaze outperformed digital camera-based eye tracking methods by reporting a mean Spearman rank-order correlation coefficient of 0.86 for the H-test, 0.87 for the Dot-test, and 0.56 for the OKN-Test, and shows similarity in trends for the relative gaze trajectories with a noticeable offset in the scale of the relative gaze angle compared to COTS eye tracker (see Fig. 1). The study demonstrates that by using a pupil-center-based eye-tracking method, a digital camera can measure clinically relevant information regarding eye movement.