With the rapid development of the Internet of Things (IoT) and the rise of 5G communication networks and automatic driving, millimeter wave (mmWave) sensing is emerging and starts impacting our life and workspace. mmWave sensing can sense humans and objects in a contactless way, providing fine-grained sensing ability. In the past few years, many mmWave sensing techniques have been proposed and applied in various human sensing applications (e.g., human localization, gesture recognition, and vital monitoring). We discover the need of a comprehensive survey to summarize the technology, platforms and applications of mmWave-based human sensing. In this survey, we first present the mmWave hardware platforms and some key techniques of mmWave sensing. We then provide a comprehensive review of existing mmWave-based human sensing works. Specifically, we divide existing works into four categories according to the sensing granularity: human tracking and localization, motion recognition, biometric measurement and human imaging. Finally, we discuss the potential research challenges and present future directions in this area.

Deep Learning plays an increasingly important role in device-free WiFi Sensing for human activity recognition (HAR). Despite its strong potential, significant challenges exist and are associated with the fact that one may require a large amount of samples for training, and the trained network cannot be easily adapted to a new environment. To address these challenges, we develop a novel scheme using Matching Network with enhanced channel state information (MatNet-eCSI) to facilitate one-shot learning HAR. We propose a CSI Correlation Feature Extraction (CCFE) method to improve and condense the activity-related information in input signals. It can also significantly reduce the computational complexity by decreasing the dimensions of input signals. We also propose novel training strategy which effectively utilizes the data set from the previously seen environments (PSE). In the least, the strategy can effectively realize human activity recognition using only one sample for each activity from the testing environment and the data set from one PSE. Numerous experiments are conducted and the results demonstrate that our proposed scheme significantly outperforms state-of-the-art HAR methods, achieving higher recognition accuracy and less training time.

Deep learning has demonstrated its great potential in channel state information (CSI)-based human activity recognition (HAR), and hence has attracted increasing attention in both the industry and academic communities. While promising, most existing high-accuracy methodologies require to retrain their models when applying the previous-trained ones to a new/unseen environment. This issue has limited their practical usabilities. In order to overcome this challenge, this article proposes an innovative scheme, which combines an activity-related feature extraction and enhancement (AFEE) method and matching network (AFEE-MatNet). The proposed scheme is “one-fits-all,” meaning that the trained model can be directly applied in new/unseen environments without any retraining. We introduce the AFEE method to enhance CSI quality by eliminating noise. Specifically, the approach mitigates environmental noises unrelated to activity while better compressing and preserving the behavior-related information. Moreover, the size of feature signals generated by AFEE are reduced, which in turn significantly shortens the training time. For effective feature extraction, we propose to use the MatNet architecture to learn transferable features shared among source environments. To further improve the recognition performance, we introduce a prediction checking and correction scheme to rectify some classification errors that do not abide by the state transition of human behaviors. Extensive experimental results demonstrate that our proposed AFEE-MatNet significantly outperforms existing state-of-the-art HAR methods, in terms of both recognition accuracy and training time.

Spectrum sensing plays a critical role in dynamic spectrum sharing, a promising technology to address the radio spectrum shortage. In particular, sensing of orthogonal frequency division multiplexing (OFDM) signals, a widely accepted multi-carrier transmission paradigm, has received paramount interest. Despite various efforts, noise uncertainty, timing delay and carrier frequency offset (CFO) still remain as challenging problems, significantly degrading the sensing performance. In this work, we develop two novel OFDM sensing frameworks utilizing the properties of deep learning networks. Specifically, we first propose a stacked autoencoder based spectrum sensing method (SAE-SS), in which a stacked autoencoder network is designed to extract the hidden features of OFDM signals for classifying the users activities. Compared to the conventional OFDM sensing methods, SAE-SS is significantly superior in the robustness to noise uncertainty, timing delay, and CFO. Moreover, SAE-SS requires neither any prior information of signals (e.g., signal structure, pilot tones, cyclic prefix) nor explicit feature extraction algorithms which however are essential for the conventional OFDM sensing methods. To further improve the sensing accuracy of SAE-SS, especially under low SNR conditions, we propose a stacked autoencoder based spectrum sensing method using time-frequency domain signals (SAE-TF). SAE-TF achieves higher sensing accuracy than SAE-SS using the features extracted from both time and frequency domains, at the cost of higher computational complexity. Through extensive simulation results, both SAE-SS and SAE-TF are shown to achieve notably higher sensing accuracy than that of state of the art approaches.

Blood pressure (BP) measurement is an indispensable tool in diagnosing and treating many diseases such as cardiovascular failure and stroke. Traditional direct measurement can be invasive, and wearable-based methods may have limitations of discomfort and inconvenience. Contact-free BP measurement has been recently advocated as a promising alternative. In particular, Millimetre-wave (mmWave) sensing has demonstrated its promising potential, however it is confronted with several challenges including noise and vulnerability to human's tiny motions which may occur intentionally and inevitably. In this paper, we propose mmBP, a contact-free mmWave-based BP measurement system with high accuracy and motion robustness. Due to the high frequency and short wavelength, mmWave signals received in the time domain are dramatically susceptible to ambient noise, and deteriorating signal quality. To reduce noise, we propose a novel delay-Doppler domain feature transformation method to exploit mmWave signal's characteristics and features in the delay-Doppler domain to significantly improve signal quality for pulse waveform construction. We also propose a temporal referential functional link adaptive filter leveraging on the periodic and correlation characteristics of pulse waveform signals to alleviate the impact of human's tiny motions. Extensive experiment results achieved by the leave-one-out cross-validation (LOOCV) method demonstrate that mmBP achieves the mean errors of 0.87mmHg and 1.55mmHg for systolic blood pressure (SBP) and diastolic blood pressure (DBP), respectively; and the standard deviation errors of 5.01mmHg and 5.27mmHg for SBP and DBP, respectively.

Facial expression recognition plays a vital role to enable emotional awareness in multimedia Internet of Things applications. Traditional camera or wearable sensor based approaches may compromise user privacy or cause discomfort. Recent device-free approaches open a promising direction by exploring Wi-Fi or ultrasound signals reflected from facial muscle movements, but limitations exist such as poor performance in presence of body motions and not being able to detect multiple targets. To bridge the gap, we propose mmFER, a novel millimeter wave (mmWave) radar based system that extracts facial muscle movements associated with mmWave signals to recognize facial expressions. We propose a novel dual-locating approach based on MIMO that explores spatial information from raw mmWave signals for face localization in space, eliminating ambient noise. In addition, collecting mmWave training data can be very costly in practice, and insufficient training dataset may lead to low accuracy. To overcome, we design a cross-domain transfer pipeline to enable effective and safe model knowledge transformation from image to mmWave. Extensive evaluations demonstrate that mmFER achieves an accuracy of 80.57% on average within a detection range between 0.3m and 2.5m, and it is robust to various real-world settings.