Indoor occupant sensing enables many smart home applications, and various sensing systems have been explored. Based on their installation requirements, we consider two categories of sensors – on- and off-body – and we look into the combination of them for occupant sensing due to their spatial and temporal complementarity. We focus on an example modality pair of wearable IMU and structural vibration that demonstrate modality complementarity in prior work. However, current efforts are built upon the assumption that the knowledge of the signal segments from two modalities are known, which is challenged in a multiple occupants co-living scenario. Therefore, establishing accurate cross-modal signal segment associations is essential to ensure that a correct complementary relationship is learned. We present CMA, a cross-modal signal segment association scheme between structural vibration and wearable sensors. We propose AD-TCN, a framework built upon a temporal convolutional network that calculates the amount of shared context between an structural vibration sensor and associated wearable sensor candidates from the parameters of the trained model. We evaluate CMA via a public multimodal dataset for systematic evaluation, and we collect a continuous uncontrolled dataset for robustness evaluation. CMA achieves up to AUC value, F1 score, and accuracy improvement compared to baselines.
The correlation analysis between socioeconomic factors and environmental impact is essential for policy making to ensure sustainability and economic development simultaneously. With the development of Internet of Things (IoT), citizen science IoT monitoring provides valuable environmental measurements, such as PM 2.5 for air quality monitoring. However, socioeconomic factors are usually interconnected and confound each other, making accurate correlation analysis challenging. To isolate this information on an individual socioeconomic factor, we need to mitigate the confounding effect (e.g., propensity score matching) of other factors on the environmental sensing data. Large language models (LLMs) have shown remarkable capabilities in data reasoning, making us wonder if they can conduct causal reasoning and answer questions like "What is the most important socioeconomic factor that impacts regional air quality?" In this paper, we present a new evaluation framework named "Order-of-Thought" based on Bloom's Taxonomy pedagogical framework to quantify the LLMs' ability for causal reasoning. We apply this evaluation framework with both natural language-based and program-based prompting strategies. Our evaluation uncovers the exceptional potentials of LLMs in causal reasoning for sensing data analysis, offering valuable insights regarding their capabilities and limitations, and providing useful directions to further achieve a higher-order thought.
6 July 2021 in CCF Transactions on Pervasive Computing and Interaction
In this paper, we introduce AutoQual, a mobile-based assessment scheme for infrastructure sensing task performance prediction under new deployment environments. With the growth of the Internet-of-Things (IoT), many non-intrusive sensing systems have been explored for various indoor applications, such as structural vibration sensing. This indirect sensing approach’s learning performance is prone to deployment variance when signals propagate through the environment. As a result, current systems heavily rely on expert knowledge and manual assessment to achieve effective deployments and high sensing task performance. In order to mitigate this expert effort, we propose to systematically study factors that reflect deployment environment characteristics and methods to measure them autonomously. We present AutoQual that measures a series of assessment factors (AFs) reflecting how the deployment environment impacts the system performance. AutoQual outputs a task-oriented sensing quality (TSQ) score by integrating measured AFs trained from known deployments as a prediction of untested system’s performance. In addition, AutoQual achieves this assessment without manual effort by leveraging co-located mobile sensing context to extract structural vibration signal for processing automatically. We evaluate AutoQual by using it to predict untested systems’ performance over multiple sensing tasks. We conduct real-world experiments and investigate 48 deployments in 11 environments. AutoQual achieves less than 0.10 average absolute error when auto-assessing multiple tasks at untested deployments, which shows a $$\le 0.018$$ absolute error difference compared to the manual assessment approach.
6 March 2024 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
While occlusal diseases - the main cause of tooth loss -- significantly impact patients' teeth and well-being, they are the most underdiagnosed dental diseases nowadays. Experiencing occlusal diseases could result in difficulties in eating, speaking, and chronicle headaches, ultimately impacting patients' quality of life. Although attempts have been made to develop sensing systems for teeth activity monitoring, solutions that support sufficient sensing resolution for occlusal monitoring are missing. To fill that gap, this paper presents IOTeeth, a cost-effective and automated intra-oral sensing system for continuous and fine-grained monitoring of occlusal diseases. The IOTeeth system includes an intra-oral piezoelectric-based sensing array integrated into a dental retainer platform to support reliable occlusal disease recognition. IOTeeth focuses on biting and grinding activities from the canines and front teeth, which contain essential information of occlusion. IOTeeth's intra-oral wearable collects signals from the sensors and fetches them into a lightweight and robust deep learning model called Physioaware Attention Network (PAN Net) for occlusal disease recognition. We evaluate IOTeeth with 12 articulator teeth models from dental clinic patients. Evaluation results show an F1 score of 0.97 for activity recognition with leave-one-out validation and an average F1 score of 0.92 for dental disease recognition for different activities with leave-one-out validation.
Internet of Things (IoT) provides a wide range of services to improve convenience and comfort in our daily lives. However, various sensors equipped on IoT devices often raise privacy concerns. Prior works on privacy focus on passive protection from the data and device perspective, such as data encryption and communication protocol design. In this work, we introduce PrivaSee, an augmented reality (AR)-enabled privacy visualization platform to empower users with proactive privacy protection by enhancing their understanding of privacy perception for multimodal sensors.