Wentao Xie

Wentao Xie

Auto-generated profile, based on publicly available information

Wentao Xie

Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong, Hong Kong

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
22 August 2024 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
TSTao Sun
Tao Sun
YZYankai Zhao
Yankai Zhao
WXWentao Xie
Wentao Xie
JLJiao Li
Jiao Li
YMYongyu Ma
Yongyu Ma
AZAndrew Zhang
Andrew Zhang
The smart glasses market has witnessed significant growth in recent years. The interaction of commercial smart glasses mostly relies on the hand, which is unsuitable for scenarios where both hands are occupied. In this paper, we propose EyeGesener, an eye gesture listener for smart glasses interaction using acoustic sensing. To mitigate the Midas touch problem, we meticulously design eye gestures for interaction as two intentional consecutive saccades in a specific direction without visual dwell. The proposed system is a glass-mounted acoustic sensing system with two pairs of commercial speakers and microphones to sense eye gestures. To capture the subtle movements of the eyelid and surrounding skin induced by eye gestures, we design an Orthogonal Frequency Division Multiplexing (OFDM)-based channel impulse response (CIR) estimation schema that allows two speakers to transmit at the same time and in the same frequency band without collision. We implement eye gesture filtering and adversarial-based eye gesture recognition to identify eye gestures for interaction, filtering out daily eye movements. To address the differences in eye size and facial structure among different users, we employ adversarial training to achieve user-independent eye gesture recognition. We evaluate the performance of our system through experiments on data collected from 16 subjects. The experimental result shows that our system can recognize eight eye gestures with an average F1-score of 0.93, and the false alarm rate of our system is 0.03. We develop an interactive real-time audio-video player based on EyeGesener and then conduct a user study. The result demonstrates the high usability of the proposed system.

4 September 2020 in IEEE Internet of Things Journal
Verified
Journal Article
Closed access
WXWentao Xie
Wentao Xie
RTRunxin Tian
Runxin Tian
JZJin Zhang
Jin Zhang
QZQian Zhang
Qian Zhang
Recent works have shown that acoustic signals can be leveraged to perform respiration monitoring with high accuracy and low energy consumption. Since smartphones, smart speakers, and many other IoT devices are already equipped with microphones and speakers, it is convenient to implement the acoustic sensing solutions on those devices. However, the existing technologies require the speaker to transmit certain ultrasonic signals to detect respiration. Although these signals are inaudible to adults, they are audible to children and pets and they may even have negative impacts on plants 1 2 3. In this paper, instead of using ultrasonic signals, we are trying to leverage audible signals in daily lives, e.g., music or broadcasting audios, to detect human respiration. We design a respiration detection system which derives the respiration rate by continuously estimates the channel impulse response (CIR) using music and broadcast signals. We study the inter-symbol-interference (ISI) brought by the randomness of music and broadcast signal and give our strategy to minimize the interference. We also propose several techniques to resolve some practical issues such as the multi-path effect and sampling frequency offset between the speaker and the microphone. Extensive experiments are conducted to demonstrate the feasibility of our system. The result shows that our system can achieve high respiration detection accuracy with the mean error of less than 0.5 BPM when different audio signals are used.

22 August 2024 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
WXWentao Xie
Wentao Xie
CXChi Xu
Chi Xu
YGYanbin Gong
Yanbin Gong
YWYu Wang
Yu Wang
YLYuxin Liu
Yuxin Liu
SYShifang Yang
Shifang Yang
Practicing breathing exercises is crucial for patients with chronic obstructive pulmonary disease (COPD) to enhance lung function. Breathing mode (chest or belly breathing) and lung volume are two important metrics for supervising breathing exercises. Previous works propose that these metrics can be sensed separately in a contactless way, but they are impractical with unrealistic assumptions such as distinguishable chest and belly breathing patterns, the requirement of calibration, and the absence of body motions. In response, this research proposes DeepBreath, a novel depth camera-based breathing exercise assessment system, to overcome the limitations of the existing methods. DeepBreath, for the first time, considers breathing mode and lung volume as two correlated measurements and estimates them cooperatively with a multitask learning framework. This design boosts the performance of breathing mode classification. To achieve calibration-free lung volume measurement, DeepBreath uses a data-driven approach with a novel UNet-based deep-learning model to achieve one-model-fit-all lung volume estimation, and it is designed with a lightweight silhouette segmentation model with knowledge transferred from a state-of-the-art large segmentation model that enhances the estimation performance. In addition, DeepBreath is designed to be resilient to involuntary motion artifacts with a temporal-aware body motion compensation algorithm. We collaborate with a clinical center and conduct experiments with 22 healthy subjects and 14 COPD patients to evaluate DeepBreath. The experimental result shows that DeepBreath can achieve high breathing metrics estimation accuracy but with a much more realistic setup compared with previous works.

21 December 2022 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
WXWentao Xie
Wentao Xie
QHQingyong Hu
Qingyong Hu
AZAndrew Zhang
Andrew Zhang
QZQian Zhang
Qian Zhang
Spirometry is the gold standard for evaluating lung functions. Recent research has proposed that mobile devices can measure lung function indices cost-efficiently. However, these designs fall short in two aspects. First, they cannot provide the flow-volume (F-V) curve, which is more informative than lung function indices. Secondly, these solutions lack inspiratory measurement, which is sensitive to lung diseases such as variable extrathoracic obstruction. In this paper, we present EarSpiro, an earphone-based solution that interprets the recorded airflow sound during a spirometry test into an F-V curve, including both the expiratory and inspiratory measurements. EarSpiro leverages a convolutional neural network (CNN) and a recurrent neural network (RNN) to capture the complex correlation between airflow sound and airflow speed. Meanwhile, EarSpiro adopts a clustering-based segmentation algorithm to track the weak inspiratory signals from the raw audio recording to enable inspiratory measurement. We also enable EarSpiro with daily mouthpiece-like objects such as a funnel using transfer learning and a decoder network with the help of only a few true lung function indices from the user. Extensive experiments with 60 subjects show that EarSpiro achieves mean errors of 0.20L/s and 0.42L/s for expiratory and inspiratory flow rate estimation, and 0.61L/s and 0.83L/s for expiratory and inspiratory F-V curve estimation. The mean correlation coefficient between the estimated F-V curve and the true one is 0.94. The mean estimation error for four common lung function indices is 7.3%.

3 March 2025 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Verified
Journal Article
Open access
MXMeng Xue
Meng Xue
YZYinan Zhu
Yinan Zhu
WXWentao Xie
Wentao Xie
ZWZhixian Wang
Zhixian Wang
YCYanjiao Chen
Yanjiao Chen
QZQian Zhang
Qian Zhang
Human Activity Recognition (HAR) faces significant challenges when deployed in real-world scenarios due to non-independent and identically distributed (non-IID) data distributions. While existing domain adaptation (DA) approaches attempt to address this issue, they either require access to source data or struggle with large domain shifts. This paper presents a novel source-free domain adaptation framework for HAR that effectively handles substantial domain discrepancies across different datasets. Our approach introduces two key innovations: (1) a Discriminative Information Gramian (DIG) method that quantifies the relationship between target-domain samples and the source domain without requiring access to source data, and (2) an unsupervised domain generalization technique that ensures consistent feature extraction across augmented data samples, enhancing the model's effectiveness in the target domain. We evaluate our framework on five diverse HAR datasets comprising 87 users with varying demographics, devices, and environmental conditions. In single-source scenarios, our method achieves 76.77% accuracy and 67.03% F1-score, surpassing state-of-the-art solutions by 9.77% and 17.43%, respectively. For multi-source scenarios, we attain 85.33% accuracy and 79.55% F1-score, exceeding existing methods by at least 8.8% and 14.8%, respectively. This work represents the first successful attempt at dataset-level HAR domain adaptation without access to source data, marking a significant advancement in practical HAR applications.

12 November 2023
Verified
Conference Paper
Open access
BYBaichen Yang
Baichen Yang
QHQingyong Hu
Qingyong Hu
WXWentao Xie
Wentao Xie
XWXinchen Wang
Xinchen Wang
WLWei Luo
Wei Luo
QZQian Zhang
Qian Zhang
In-time disease assessment is essential to better customize the medication scheme and improve the quality of life for chronic diseases like Parkinson's disease (PD). Toward the inconvenience problem in current clinical assessment practice, mobile sensing solutions based on detecting Parkinson's vocal changes are proposed. However, current solutions either can only achieve binary disease detection task or require patients to perform specific speaking tasks, which is not effective and practical for disease stage assessment in daily scenario. Moreover, most of existing solutions do not take speech privacy into consideration. In this work, we present PDAssess, a free speech-based daily assessment system that can perform 4-stage Parkinson's disease assessment in a privacy-preserving manner. We observe that current solutions did not fully leverage the rich information embedded in free speech due to the linguistic content variations, and therefore leverage a pre-trained automatic speech recognition (ASR) model to achieve a content variation-aware feature-extraction. In order to distinguish subtle stage-wise differences, we design a novel attention-based neural network architecture with a customized loss function for disease assessment task. Towards the potential privacy leakage problem, we design a Split Learning-based framework with pseudo-labeling and local domain adversarial training to better preserve speech content privacy. We collaborate with a medical center and evaluate the performance of PDAssess on real-world speech data collected from 50 PD subjects and 50 healthy subjects. The evaluation result shows that PDAssess can perform 4-stage PD assessment with an average person-wise F1 score of 89.1% and voice sample-wise F1 score of 75.1%.