Person identification is crucial in various smart building applications, including customer behavior analysis, patient monitoring, etc. Prior works on person identification mainly focused on access control related applications. They achieve identification by sensing certain biometrics with specific sensors. However, these methods and apparatuses can be intrusive and not scalable because of instrumentation and sensing limitations. In this paper, we introduce our indoor person identification system that utilizes footstep induced structural vibration. Because structural vibration can be measured without interrupting human activities, our system is suitable for many ubiquitous sensing applications. Our system senses floor vibration and detects the signal induced by footsteps. Then the system extracts features from the signals that represent characteristics of each person's gait pattern. With the extracted features, the system conducts hierarchical classification at an individual step level and then at a trace (i.e., collection of consecutive steps) level. Our system achieves over 83% identification accuracy on average. Furthermore, when the application requires different levels of accuracy, our system can adjust confidence level threshold to discard uncertain traces. For example, at a threshold that allows only most certain 50% traces for classification, the identification accuracy increases to 96.5%.

Context-based pairing solutions increase the usability of IoT device pairing by eliminating any human involvement in the pairing process. This is possible by utilizing on-board sensors (with same sensing modalities) to capture a common physical context (e.g., ambient sound via each device's microphone). However, in a smart home scenario, it is impractical to assume that all devices will share a common sensing modality. For example, a motion detector is only equipped with an infrared sensor while Amazon Echo only has microphones. In this paper, we develop a new context-based pairing mechanism called Perceptio that uses time as the common factor across differing sensor types. By focusing on the event timing, rather than the specific event sensor data, Perceptio creates event fingerprints that can be matched across a variety of IoT devices. We propose Perceptio based on the idea that devices co-located within a physically secure boundary (e.g., single family house) can observe more events in common over time, as opposed to devices outside. Devices make use of the observed contextual information to provide entropy for Perceptio's pairing protocol. We design and implement Perceptio, and evaluate its effectiveness as an autonomous secure pairing solution. Our implementation demonstrates the ability to sufficiently distinguish between legitimate devices (placed within the boundary) and attacker devices (placed outside) by imposing a threshold on fingerprint similarity. Perceptio demonstrates an average fingerprint similarity of 94.9% between legitimate devices while even a hypothetical impossibly well-performing attacker yields only 68.9% between itself and a valid device.

Individual perspiration level indicates a person’s physical status as well as their comfort level. Therefore, continuous perspiration level measurement enables people to monitor these conditions for applications including fitness assessment, athlete physical status monitoring, and patient/elderly care. Prior work on perspiration (sweat) sensing required the user either to be static or to wear the adhesive sensor directly on the skin, which limits users’ mobility and comfort. In this paper, we present a novel conductive thread-based textile sensor that measures an individual’s on-cloth sweat quantity. The sensor consists of three conductive threads. Each conductive thread is surrounded by a braided cotton cover. An additional braided cotton cover is placed outside the three conductive threads, holding them in a position that is stable for measurement. the sensor can be embedded at various locations on a person’s clothing. When the person sweats, the cotton braids absorb the sweat and change the conductivity (resistance) between conductive threads. We used a voltage dividing circuit to measure this resistance as the sensor output (DC). We then conducted a sensor calibration to map this measured voltage to the quantity of electrolyte solution (with the same density as sweat) applied to the sensor. We used this sensor to measure individuals’ perspiration quantity and infer their perceived perspiration levels. The system is able to limit the average prediction error to 0.4 levels when compared to five pre-defined perceived perspiration levels.

In this paper, we present MuteIt, an ear-worn system for recognizing unvoiced human commands. MuteIt presents an intuitive alternative to voice-based interactions that can be unreliable in noisy environments, disruptive to those around us, and compromise our privacy. We propose a twin-IMU set up to track the user's jaw motion and cancel motion artifacts caused by head and body movements. MuteIt processes jaw motion during word articulation to break each word signal into its constituent syllables, and further each syllable into phonemes (vowels, visemes, and plosives). Recognizing unvoiced commands by only tracking jaw motion is challenging. As a secondary articulator, jaw motion is not distinctive enough for unvoiced speech recognition. MuteIt combines IMU data with the anatomy of jaw movement as well as principles from linguistics, to model the task of word recognition as an estimation problem. Rather than employing machine learning to train a word classifier, we reconstruct each word as a sequence of phonemes using a bi-directional particle filter, enabling the system to be easily scaled to a large set of words. We validate MuteIt for 20 subjects with diverse speech accents to recognize 100 common command words. MuteIt achieves a mean word recognition accuracy of 94.8% in noise-free conditions. When compared with common voice assistants, MuteIt outperforms them in noisy acoustic environments, achieving higher than 90% recognition accuracy. Even in the presence of motion artifacts, such as head movement, walking, and riding in a moving vehicle, MuteIt achieves mean word recognition accuracy of 91% over all scenarios.

This paper presents a floor-vibration-based step-level occupant-detection approach that enables detection across different structures through model transfer. Detecting the occupants through detecting their footsteps (i.e., step-level occupant detection) is useful in various smart building applications such as senior/healthcare and energy management. Current sensing approaches (e.g., vision-based, pressure-based, radio frequency–based, and mobile-based) for step-level occupant detection are limited due to installation and maintenance requirements such as dense deployment and requiring the occupants to carry a device. To overcome these requirements, previous research used ambient structural vibration sensing for footstep modeling and step-level occupant detection together with supervised learning to train a footstep model to distinguish footsteps from nonfootsteps using a set of labeled data. However, floor-vibration-based footstep models are influenced by the structural properties, which may vary from structure to structure. Consequently, a footstep model in one structure does not accurately capture the responses in another structure, which leads to high detection errors and the costly need for acquiring labeled data in every structure. To address this challenge, the effect of the structure on the footstep-induced floor vibration responses is here characterized to develop a physics-driven model transfer approach that enables step-level occupant detection across structures. Specifically, the proposed model transfer approach projects the data into a feature space in which the structural effects are minimized. By minimizing the structure effect in this projected feature space, the footstep models mainly represent the differences in the excitation types and therefore are transferable across structures. To this end, it is analytically shown that the structural effects are correlated to the maximum-mean-discrepancy (MMD) distance between the source and target marginal data distributions. Therefore, to reduce the structural effect, the MMD between the distributions in the source and target structures is minimized. The robustness of the proposed approach was evaluated through field experiments in three types of structures. The evaluation consists of training a footstep model in a set of structures and testing it in a different structure. Across the three structures, the evaluation results show footstep detection F1 score of up to 99% for the proposed approach, corresponding to a 29-fold improvement compared to the baseline approach, which do not transfer the model.