15 June 2026 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Smart homes are receiving growing interest in global markets. However, current smart home assistant systems either control appliances solely through users' explicit commands or rely on cloud-based large language models (LLMs) for vague command understanding, which brings drawbacks such as high latency, privacy concern, and high cost. In this work, we propose VCU-LLM, the first system to deploy LLMs on edge devices for local vague command understanding and smart device control plan generation. VCU-LLM introduces a novel vague command knowledge retrieval algorithm that refines device-related information in the input prompt, thereby accelerating the LLM's on-device inference and reducing task complexity. We further construct a dataset for LLM fine-tuning to simulate the use of smart home assistants in controlling devices across different households. During inference, a customized KV-cache technique is applied for further inference acceleration. Our evaluations with both human-based and LLM-based scoring demonstrate that VCU-LLM improves the quality of generated control plans by an average of 43.3% compared with SOTA baselines, while reducing time overhead by an average of 8.44x compared with other on-device baselines. We also implement VCU-LLM through a case study in a real home environment, demonstrating its feasibility in real-world application.
10 March 2026 in IEEE Transactions on Mobile Computing
Multi-modal human activity recognition (HAR) is a key technology for a wide range of applications and has received widespread attention in recent years. However, the difficulty of achieving generalizability in multi-modal sensing models, combined with heterogeneous and unlabeled downstream data, significantly hinders their broader adoption. In this work, we propose PFHAR, a unified framework for practically adopting multi-modal foundation HAR model to target user groups. PFHAR uses a novel dynamic masked contrastive learning method to pre-train a foundation model on various heterogeneous public HAR datasets, ensuring strong generalizability across different modal combinations. It then adopts semi-supervised edge-cloud collaborative learning to fine-tune the pre-trained model with heterogeneous and unlabeled local data, adapting it for the target user group. Our evaluations on public and self-collected datasets demonstrate that PFHAR significantly outperforms SOTA baselines in both the pre-training and edge-cloud collaborative fine-tuning stages.