16 November 2022 in IEEE Journal on Selected Areas in Communications
Edge-cloud collaborative video analytics is transforming the way data is being handled, processed, and transmitted from the ever-growing number of surveillance cameras around the world. To avoid wasting limited bandwidth on unrelated content transmission, existing video analytics solutions usually perform temporal or spatial filtering to realize aggressive compression of irrelevant pixels. However, most of them work in a context-agnostic way while being oblivious to the circumstances where the video content is happening and the context-dependent characteristics under the hood. In this work, we propose VaBUS, a real-time video analytics system that leverages the rich contextual information of surveillance cameras to reduce bandwidth consumption for semantic compression. As a task-oriented communication system, VaBUS dynamically maintains the background image of the video on the edge with minimal system overhead and sends only highly confident Region of Interests (RoIs) to the cloud through adaptive weighting and encoding. With a lightweight experience-driven learning module, VaBUS is able to achieve high offline inference accuracy even when network congestion occurs. Experimental results show that VaBUS reduces bandwidth consumption by 25.0%-76.9% while achieving 90.7% accuracy for both the object detection and human keypoint detection tasks.
1 June 2025 in IEEE Transactions on Broadcasting
20 September 2024 in IEEE Transactions on Mobile Computing
Deploying deep neural network (DNN) models on resource-constrained edge devices for real-time video analytics poses significant challenges due to the high resource demands of these models. Current edge-based video analytics approaches often overlook optimizing deep learning models and GPU resource allocations in multi-tenant scenarios. In this paper, we present JSAS-MTMGS, a collaborative video analytics system employing three innovative design strategies. First, we propose a novel video configuration optimization space based on a joint DNN model sharing and splitting scheme to balance computational loads for collaborative processing. This approach reduces network transmission data volume and alleviates resource contention. Second, we design a GPU resource allocation scheme that combines GPU batching with spatial sharing to optimize GPU utilization and increase system throughput, all without relying on costly offline latency collection. Finally, we define the configuration optimization problem alongside GPU allocation as a convex problem and apply convex optimization to make scheduling decisions dynamically. Our experiments demonstrate that JSAS-MTMGS has the best service quality among all compared algorithms.
5 August 2024 in IEEE Transactions on Mobile Computing
Real-time video analytics on edge devices has gained increasing attention across a wide range of business areas. However, edge devices usually have limited computing resources. Consequently, conventional approaches to video analytics either deploy simplified models on the edge (resulting in low accuracy) or transmit video content to the cloud (resulting in high latency and network overheads) to enable deep learning inference (e.g., object detection). In this paper, we introduce ParaLoupe, a novel real-time video analytics system that parallelizes deep learning inference in the edge cluster with task-oriented mini models. These mini models do not attain State-of-the-Art accuracy individually, but collectively can achieve much better accuracy-latency tradeoff than State-of-the-Art models. To achieve this, ParaLoupe crops multiple single-object patches from a given video frame. These single-object patches are then sent to multiple edge devices for parallel inference with specifically designed mini models. A patch-based task scheduling algorithm is further proposed to leverage the computing resources of the edge cluster to meet the service-level objectives. Our experimental results on real-world datasets show that ParaLoupe significantly outperforms baseline methods, achieving up to 14.1× inference speedup with accuracy on par with state-of-the-art models, or improving accuracy up to 45.1% under the same latency constraints.
24 October 2025 in IEEE Transactions on Networking
Traditional Adaptive Bitrate (ABR) schemes assume that users watch videos sequentially, focusing solely on the sequential downloading of video chunks. However, these schemes often result in significant degradation of Quality of Experience (QoE) when users skip directly to their preferred segments. To address this issue, we propose JumpDASH, which leverages Large Language Model (LLM)-based content perception to enhance mobile adaptive video streaming. First, JumpDASH incorporates a low-cost video text summarization module based on large language models, enabling users to identify and navigate to the most relevant sections of videos. Second, we introduce a dynamic partitioned buffer and a Proximal Policy Optimization (PPO)-based ABR algorithm to facilitate prefetching video chunks corresponding to the perceived points of interest, along with differentiated encoding techniques to further minimize rebuffering. Extensive experiments conducted using real trace datasets under actual network conditions show that JumpDASH improves QoE by 13.82% to 262.94% compared to existing ABR technologies.