Millimeter wave (mmWave) sensing promises to enable contactless and high-precision “in-air” gesture-based human–computer interaction (HCI). While previous works have demonstrated its feasibility, they require tedious gesture collecting for person-independent recognition and they operate in an off-line mode without considering practical issues, such as segmenting gesture and recognition latency. In this work, we propose M-Gesture , a person-independent real-time mmWave gesture recognition solution. We first build a compact gesture model with a custom-designed neural network to distill the unique features underlying each gesture, while suppressing personalized discrepancy across different users without extra collection and retraining. Furthermore, we design a system status transition (SST) to decide when a gesture begins and ends, which enables automatic gesture segmentation and hence real-time recognition. We prototype M-Gesture on a commodity mmWave sensor and demonstrate its advantages using two practical applications: 1) a contactless music player and 2) camera. Extensive experiments and user studies show that M-Gesture has an accuracy of 99% and a short response latency within 25 ms. Moreover, we also collect and release a comprehensive mmWave gesture data set consisting of 54 620 instances from 144 persons, which may have an independent value of facilitating future research.

Blood pressure (BP), an important vital sign to assess human health, is expected to be monitored conveniently. The existing BP monitoring methods, either traditional cuff based or newly emerging wearable based, all require skin contact, which may cause unpleasant user experience and is even injurious to certain users. In this article, we explore contactless BP monitoring and propose airBP, which emits millimeter-wave signals toward a user’s wrist, and captures the reflected signal bounded off from the pulsating artery underlying the wrist. By analyzing the reflected signal strength of the signal, airBP generates the arterial pulse and further estimates BP by exploiting the relationship between the arterial pulse and BP. To realize airBP, we design a new beam-forming method to keep focusing on the tiny and hidden wrist artery, by leveraging the inherent periodicity of the arterial pulse. Moreover, we custom design a pre-training and neural network architecture, to combat the challenges from the arterial pulse sparsity and ambiguity, so as to estimate BP accurately. We prototype airBP using a coin-size commercial off-the-shelf millimeter-wave radar and perform extensive experiments on 41 subjects. The results demonstrate that airBP accurately estimates systolic and diastolic BP, with a mean error of –0.30 mmHg and –0.23 mmHg, as well as a standard deviation error of 4.80 mmHg and 3.79 mmHg (within the acceptable range regulated by the FDA’s AAMI protocol), respectively, at a distance up to 26 cm.

Millimeter wave radar is a promising sensing modality for enabling pervasive and privacy-preserving human sensing. However, the lack of large-scale radar datasets limits the potential of training deep learning models to achieve generalization and robustness. To close this gap, we resort to designing a software pipeline that leverages wealthy video repositories to generate synthetic radar data, but it confronts key challenges including i) multipath reflection and attenuation of radar signals among multiple humans, ii) unconvertible generated data leading to poor generality for various applications, and iii) the class-imbalance issue of videos leading to low model stability. To this end, we design Midas to generate realistic, convertible radar data from videos via two components: (i) a data generation network (DG-Net) combines several key modules, depth prediction, human mesh fitting and multi-human reflection model, to simulate the multipath reflection and attenuation of radar signals to output convertible coarse radar data, followed by a Transformer model to generate realistic radar data; (ii) a variant Siamese network (VS-Net) selects key video clips to eliminate data redundancy for addressing the class-imbalance issue. We implement and evaluate Midas with video data from various external data sources and real-world radar data, demonstrating its great advantages over the state-of-the-art approach for both activity recognition and object detection tasks.

The electrocardiogram (ECG) has always served as a crucial biomedical examination for cardiac diseases monitoring and diagnosing. Typical ECG measurement requires attaching electrodes to the body, which is inconvenient for long-term monitoring. Recent wireless sensing maps wireless signals reflected from human chest into electrical activities of heart so as to reconstruct ECG contactlessly. While making great progress, we find existing works are effective only for healthy populations with normal ECG, but fall short when confronted with the most desired usage scenario: reconstructing ECG accurately for people with cardiac diseases such as atrial fibrillation, premature ventricular beat. To bridge the gap, we propose AirECG, which moves forward to reconstruct ECG for both healthy people and even cardiac patients with morbid ECG, i.e., irregular rhythm and anomalous ECG waveform, via contactless millimeter-wave sensing. To realize AirECG, we first custom-design a cross-domain diffusion model that can perform multiple iteration denoising inference, in contrast with the single-step generative models widely used in previous works. In this way, AirECG is able to identify and eliminate the distortion due to the unstable and irregular cardiac activities, so as to synthesize ECG even during abnormal beats. Furthermore, we enhance the determinacy of AirECG, i.e., to generate high-fidelity ECG, by designing a calibration guidance mechanism to combat the inherent randomness issue of the probabilistic diffusion model. Empirical evaluation demonstrates AirECG's ability of ECG synthesis with Pearson correlation coefficient (PCC) of 0.955 for normal beats. Especially for abnormal beats, the PCC still exhibits a strong correlation of 0.860, with 15.0%~21.1% improvement compared with state-of-the-art approaches.

Arrhythmia is a common problem of irregular heartbeats, which may lead to serious complications such as stroke and even mortality. Due to the paroxysmal nature of arrhythmia, its long-term monitoring and early detection in daily household scenarios, instead of depending on ECG examination only available during clinical visits, are of critical importance. While ambulatory ECG Holter and wearables like smartwatches have been used, they are still inconvenient and interfere with users' daily activities. In this paper, we bridge the gap by proposing mmArrhythmia, which employs low-cost mmWave radar to passively sense cardiac motions and detect arrhythmia, in an unobtrusive contact-less way. Different from previous mmWave cardiac sensing works focusing on healthy people, mmArrhythmia needs to distinguish the minute and transient abnormal cardiac activities of arrhythmia patients. To overcome the challenge, we custom-design an encoder-decoder model that can perform arrhythmia feature encoding, sampling and fusion over raw IQ sensing data directly, so as to discriminate normal heartbeat and arrhythmia. Furthermore, we enhance the robustness of mmArrhythmia by designing multichannel ensemble learning to solve the model bias problem caused by unbalanced arrhythmia data distribution. Empirical evaluation over 79,910 heartbeats demonstrates mmArrhythmia's ability of robust arrhythmia detection, with 97.32% accuracy, 98.63% specificity, and 92.30% sensitivity.

Emotion recognition, particularly contactless recognition via wireless sensing, has shown its promise in diverse applications. However, the previous works only focus on emotions rather than the person, i.e., the premise is already knowing who the subject is, without considering the issue of identifying subjects. We envision that user identification and emotion recognition together will bring more adaptive and personalized Internet of Things applications, e.g., a smart home system can react to specific emotions of a specific user, independently. In this work, we move forward to investigate the problem of simultaneous user identification, using only physiological indicators embedded in wireless signals reflected off from targets. Toward the objective, in this article, we first carry out a comprehensive measurement study, which validates the feasibility of simultaneous user identification and emotion recognition. Moreover, the measurement also discovers that the key challenge lies in the limitation of artificial features and the substantial emotion feature deviation across different days, which hinders accurate and robust sensing. To resolve the challenge, we design two multiscale neural networks, incorporated with a custom-built feature attention mechanism, so as to obtain rich feature expression and, thus, enhance the important features for accurate recognition. We prototype mmEMO using a commercial off-the-shelf millimeter-wave radar and experimental evaluation shows that mmEMO can achieve 87.68% user identification accuracy and 80.59% emotion recognition accuracy, respectively.