Computer vision (CV), a type of artificial intelligence (AI) that uses digital videos or a sequence of images to recognize content, has been used extensively across industries in recent years. However, in the healthcare industry, its applications are limited by factors like privacy, safety, and ethical concerns. Despite this, CV has the potential to improve patient monitoring, and system efficiencies, while reducing workload. In contrast to previous reviews, we focus on the end-user applications of CV. First, we briefly review and categorize CV applications in other industries (job enhancement, surveillance and monitoring, automation, and augmented reality). We then review the developments of CV in the hospital setting, outpatient, and community settings. The recent advances in monitoring delirium, pain and sedation, patient deterioration, mechanical ventilation, mobility, patient safety, surgical applications, quantification of workload in the hospital, and monitoring for patient events outside the hospital are highlighted. To identify opportunities for future applications, we also completed journey mapping at different system levels. Lastly, we discuss the privacy, safety, and ethical considerations associated with CV and outline processes in algorithm development and testing that limit CV expansion in healthcare. This comprehensive review highlights CV applications and ideas for its expanded use in healthcare.

In surveillance and search and rescue applications, it is important to perform multi-target tracking (MOT) in real-time on low-end devices. Today's MOT solutions employ deep neural networks, which tend to have high computation complexity. Recognizing the effects of frame sizes on tracking performance, we propose DeepScale, a model agnostic frame size selection approach that operates on top of existing fully convolutional network-based trackers to accelerate tracking throughput. In the training stage, we incorporate detectability scores into a one-shot tracker architecture so that DeepScale can learn representation estimations for different frame sizes in a self-supervised manner. During inference, it can adapt frame sizes according to the complexity of visual contents based on user-controlled parameters. To leverage computation resources on edge servers, we propose two computation partition schemes tailored for MOT, namely, edge server only with adaptive frame-size transmission and edge server-assisted tracking. Extensive experiments and benchmark tests on MOT datasets demonstrate the effectiveness and flexibility of DeepScale. Compared to a state-of-the-art tracker, DeepScale++, a variant of DeepScale achieves 1.57X accelerated with only moderate degradation (∼2.3%) in tracking accuracy on the MOT15 dataset in one configuration. We have implemented and evaluated DeepScale++ and the proposed computation partition schemes on a small-scale testbed consisting of an NVIDIA Jetson TX2 board and a GPU server. The experiments reveal non-trivial trade-offs between tracking performance and latency compared to server-only or smart camera-only solutions.

Convolutional Neural Networks (CNNs) are multi-layer deep structures that have been very successful in visual recognition tasks. These networks basically consist of the convolution, pooling, and the nonlinearity layers, each of which operates on the representation produced by the preceding layer and generates a new representation. Convolution layers naturally compute some inner product between a plane represented by the weight parameters and input patches. Recently, Generalized Convolutional Neural Networks (GCNN) have been introduced which justify the use of some kernels or distance functions in place of the inner product operator inside the convolution layers. Although GCNNs gained interesting results on the MNIST dataset, their application to more challenging datasets is hindered by lack of an efficient implementation. In this paper, we focus on a specific generalized convolution operator which is based on the weighted L2 norm distance (WL2Dist). By replacing the nonlinear part with three convolution operators and using effective matrix-matrix multiplications, we were able to efficiently compute the WL2Dist convolution layer both on CPU and GPU. Our experiments show that, on CPU (GPU), the proposed implementation of the WL2Dist layer achieves a 5.5x (21x) speed-up over the initial BLAS-based (CUDA-based) implementations.

Multi-object tracking (MOT) is a vital component of intelligent video analytics applications such as surveillance and autonomous driving. The time and storage complexity required to execute deep learning models for visual object tracking hinder their adoption on embedded devices with limited computing power. In this paper, we aim to accelerate MOT by transferring the knowledge from high-level features of a complex network (teacher) to a lightweight network (student) at both training and inference times. The proposed AttTrack framework has three key components: 1) cross-model feature learning to align intermediate representations from the teacher and student models, 2) interleaving the execution of the two models at inference time, and 3) incorporating the updated predictions from the teacher model as prior knowledge to assist the student model. Experiments on pedestrian tracking tasks are conducted on the MOT17 and MOT15 datasets using two different object detection backbones YOLOv5 and DLA34 show that AttTrack can significantly improve student model tracking performance while sacrificing only minor degradation of tracking speed.

Background: Delirium, an acute brain dysfunction, is a complication in up to 50% of patients in the intensive care unit (ICU). Measuring and mitigating delirium severity can reduce associated morbidity and improve long-term health outcomes post discharge. However, the perceived complexity of the available delirium detection tools and clinical workload limits the routine assessment of delirium severity. Developing a passive digital marker for delirium severity, combining routine electronic health record (EHR) and computer vision technology data, could be an implementable, scalable, and sustainable approach. Objective: Our primary objective is to develop a passive digital marker for delirium severity (PDM-Del) and examine its performance in comparison to validated delirium severity tools. Our secondary objective is to evaluate the acceptability and usability of the PDM-Del by patients, families, and clinicians. Methods: We will conduct a prospective, longitudinal cohort study to develop a PDM-Del using computer vision data and routinely collected EHR data. Following informed consent, the study team will collect image data through continuous digital video recordings of adult patients (>50 years) in their ICU room, routine EHR data (demographic and clinical variables), and administer delirium severity assessments (4 times daily) until ICU discharge or death. We will examine the usability and acceptability of the developed PDM-Del by patients, families, and direct care clinicians in a pilot randomized controlled clinical trial (aim 3). Descriptive statistics (means, SDs, medians, IQRs, and frequencies) and statistical differences between study instruments will be examined. We will use convolutional neural networks and machine learning to inform model development, testing, and validation. We will report model performance statistics, including accuracy, precision, recall, and the F1-score. Results: We are currently in the recruitment and data collection phase. As of March 2025, we screened 3980 patients (32% eligible, n=1307), approached 665 (50%), and enrolled 150 participants (23% enrollment rate). Among the 150 patients, the median age was 67 (IQR 61-74) years, 62% (93/150) were male, and 91% (136/150) were White. Conclusions: The PDM-Del could provide real-time, actionable feedback to direct care clinicians on the brain health of patients in the ICU. Early mitigation of delirium severity may decrease the risk of mortality, future Alzheimer disease and related dementia, and length of hospital stay. Trial Registration: ClinicalTrials.gov NCT06172491; https://clinicaltrials.gov/study/NCT06172491 International Registered Report Identifier (IRRID): DERR1-10.2196/62912

Multi-object tracking (MOT) is the “killer app” of edge video analytics. Deploying MOT pipelines for live video analytics poses a significant system challenge due to their computation-intensive nature. In this paper, we propose FastTuner, a model-agnostic framework that aims to accelerate MOT pipelines by adapting frame resolutions and backbone models. Unlike prior works that utilize a separate and time-consuming online profiling procedure to identify the optimal configuration, FastTuner incorporates multi-task learning to perform configuration selection and object tracking through a shared model. Multi-resolution training is employed to further improve the tracking accuracy across different resolutions. Furthermore, two workload placement schemes are designed for the practical deployment of FastTuner in edge video analytics systems. Extensive experiments demonstrate that FastTuner can achieve 1.1%–9.2% higher tracking accuracy and 2.5%–25.5% higher speed compared to the state-of-the-art methods, and accelerate end-to-end processing by 1.7%–22.5% in a real-world testbed consisting of an embedded device and an edge server.