Kaijie Xiao

Kaijie Xiao

Auto-generated profile, based on publicly available information

Kaijie Xiao

State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou City, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
22 August 2024 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
Yi GaoYi Gao
Yi Gao
KXKaijie Xiao
Kaijie Xiao
FLFu Li
Fu Li
WXWeifeng Xu
Weifeng Xu
JHJiaming Huang
Jiaming Huang
Wei DongWei Dong
Wei Dong
Trigger-Action Program (TAP) is a simple but powerful format to realize intelligent IoT applications, especially in home automation scenarios. Existing trace-driven approaches and in-situ programming approaches depend on either customized interaction commands or well-labeled datasets, resulting in limited applicable scenarios. In this paper, we propose ChatIoT, a zero-code TAP generation system based on large language models (LLMs). With a novel context-aware compressive prompting scheme, ChatIoT is able to automatically generate TAPs from user requests in a token-efficient manner and deploy them to the TAP runtime. Further, for those TAP requests including unknown sensing abilities, ChatIoT can also generate new AI models with knowledge distillation by multimodal LLMs, with a novel model customization method based on deep reinforcement learning. We implemented ChatIoT and evaluated its performance extensively. Results show that ChatIoT can reduce token consumption by 26.1-84.9% and improve TAP generation accuracy by 4.2-65.5% compared to state-of-the-art approaches in multiple settings. We also conducted a real user study, and ChatIoT can achieve 91.57% TAP generation accuracy.

21 November 2024 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
KXKaijie Xiao
Kaijie Xiao
Yi GaoYi Gao
Yi Gao
FLFu Li
Fu Li
WXWeifeng Xu
Weifeng Xu
PCPengzhi Chen
Pengzhi Chen
Wei DongWei Dong
Wei Dong
Cameras are ubiquitous in society, with users increasingly looking to extract insights about the physical world. Current human-to-camera interaction methods, while advanced, still need to support an intuitive, conversational interaction as one would expect in human-to-human communication. To achieve a more natural interaction between humans and cameras, we proposed a novel contextual chatting-to-camera paradigm. This paradigm allows users to interact with the camera using natural language including raising interests and questions. In response, the camera can customize specific tasks tailored to these interests and attempt to provide answers to the questions asked. We designed ChatCam, embracing LLMs for contextual chatting-to-camera with interest-oriented video summarization. With a novel prompt with the actor-critic LLMs approach, ChatCam can understand users' interests and translate them into some tasks and objects. ChatCam can also customize relevant models with the help of the multi-modal large language model and deep reinforcement learning on the resource-constrained edge and maintain high accuracy. Results show that ChatCam achieves an improvement up to 43.9% in understanding user interests and 21.1% in model accuracy compared to state-of-the-art methods in multiple settings. Various examples and the user study also prove the effectiveness of ChatCam in practice.

3 November 2025
Verified
Conference Paper
Open access
YZYang Zhang
Yang Zhang
KXKaijie Xiao
Kaijie Xiao
Yi GaoYi Gao
Yi Gao
Wei DongWei Dong
Wei Dong

30 September 2026 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
KXKaijie Xiao
Kaijie Xiao
Yi GaoYi Gao
Yi Gao
Wei DongWei Dong
Wei Dong
Video analytics is ubiquitous in modern society, and the emergence of Multimodal Large Language Models (MLLMs) has made its application even more extensive. A common way to support MLLM-based video analytics is to continuously stream video to the cloud and then extracts visual features and samples frames for analysis. However, this cloud-centric architecture can introduce high transmission overhead. Simultaneously, the fixed-rate or uniform sampling approaches employed in these systems will miss critical visual evidence relevant to the query, resulting in reduced accuracy.; AB@In this paper, we propose UniOVA , an edge-cloud collaborative framework for universal on-demand video analytics. First, query-aware feature retrieval improves accuracy and reduces transmission overhead by retrieving and transmitting only relevant ViT features from the edge. Second, Interest-Aware ViT reduces edge overhead through hierarchical token merging which compresses irrelevant visual data based on user interests. Our evaluation shows that UniOVA achieves comparable or higher accuracy than LongVA while reducing transmission overhead by 37.1%-95.8%. Compared with ChatCam, UniOVA also improves accuracy by 63.2%-193.8%. In addition, on Jetson Xavier NX, UniOVA achieves 7.14-11.43 FPS for feature extraction, and its end-to-end query latency ranges from 3.67-8.18s under different bandwidth settings. A real-world user study further demonstrates its potential for efficient and precise video analytics tailored to individual user needs.