22 August 2024 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Trigger-Action Program (TAP) is a simple but powerful format to realize intelligent IoT applications, especially in home automation scenarios. Existing trace-driven approaches and in-situ programming approaches depend on either customized interaction commands or well-labeled datasets, resulting in limited applicable scenarios. In this paper, we propose ChatIoT, a zero-code TAP generation system based on large language models (LLMs). With a novel context-aware compressive prompting scheme, ChatIoT is able to automatically generate TAPs from user requests in a token-efficient manner and deploy them to the TAP runtime. Further, for those TAP requests including unknown sensing abilities, ChatIoT can also generate new AI models with knowledge distillation by multimodal LLMs, with a novel model customization method based on deep reinforcement learning. We implemented ChatIoT and evaluated its performance extensively. Results show that ChatIoT can reduce token consumption by 26.1-84.9% and improve TAP generation accuracy by 4.2-65.5% compared to state-of-the-art approaches in multiple settings. We also conducted a real user study, and ChatIoT can achieve 91.57% TAP generation accuracy.
21 November 2024 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Cameras are ubiquitous in society, with users increasingly looking to extract insights about the physical world. Current human-to-camera interaction methods, while advanced, still need to support an intuitive, conversational interaction as one would expect in human-to-human communication. To achieve a more natural interaction between humans and cameras, we proposed a novel contextual chatting-to-camera paradigm. This paradigm allows users to interact with the camera using natural language including raising interests and questions. In response, the camera can customize specific tasks tailored to these interests and attempt to provide answers to the questions asked. We designed ChatCam, embracing LLMs for contextual chatting-to-camera with interest-oriented video summarization. With a novel prompt with the actor-critic LLMs approach, ChatCam can understand users' interests and translate them into some tasks and objects. ChatCam can also customize relevant models with the help of the multi-modal large language model and deep reinforcement learning on the resource-constrained edge and maintain high accuracy. Results show that ChatCam achieves an improvement up to 43.9% in understanding user interests and 21.1% in model accuracy compared to state-of-the-art methods in multiple settings. Various examples and the user study also prove the effectiveness of ChatCam in practice.
19 March 2024 in IEEE Transactions on Mobile Computing
The high energy consumption of 5G New Radio (NR) poses a major challenge to user experience. A major source of energy consumption in User Equipments (UE) is the radio tail, during which the UE remains in a high-power state to release the radio sources. Existing energy optimization approaches cut radio tails by forcing the UE to enter a low-power state. However, these approaches introduce extra promotion delays and energy consumption with soon-coming data transmissions. In this paper, we first conduct an empirical study to reveal that the 5G radio tail introduces significant energy waste on UEs. Then we propose 5GSaver, a two-phase energy-saving approach that utilizes the inactive state of New Radio to better eliminate the tail phenomenon in 5G cellular networks. 5GSaver identifies the end of App communication events in the first phase and predicts the next packet arrival time in the second phase. With the learning results, 5GSaver can automatically help the UE determine which radio resource control state to enter for saving energy. We evaluate 5GSaver using 15 mobile Apps on commercial smartphones. Evaluation results show that 5GSaver can reduce radio energy consumption by 9.5% and communication delay by 12.4% on average compared to the state-of-the-art approach.
1 August 2022 in Academy of Management Proceedings
Building on the resource-based theory, we investigate the impact of external environmental uncertainty on firm technological innovation taking into consideration the mediating role of key resource ...