Radar, as one of the sensors for human activity recognition (HAR), has unique characteristics such as privacy protection and contactless sensing. Radar-based HAR has been applied in many fields such as human–computer interaction, smart surveillance and health assessment. Conventional machine learning approaches rely on heuristic hand-crafted feature extraction, and their generalization capability is limited. Additionally, extracting features manually is time–consuming and inefficient. Deep learning acts as a hierarchical approach to learn high-level features automatically and has achieved superior performance for HAR. This paper surveys deep learning based HAR in radar from three aspects: deep learning techniques, radar systems, and deep learning for radar-based HAR. Especially, we elaborate deep learning approaches designed for activity recognition in radar according to the dimension of radar returns (i.e., 1D, 2D and 3D echoes). Due to the difference of echo forms, corresponding deep learning approaches are different to fully exploit motion information. Experimental results have demonstrated the feasibility of applying deep learning for radar-based HAR in 1D, 2D and 3D echoes. Finally, we address some current research considerations and future opportunities.

Human activity recognition (HAR) plays a vital role in many applications, such as surveillance, in-home monitoring, and health care. Portable radar sensor has been increasingly used in HAR systems in combination with deep learning (DL). However, it is both difficult and time-consuming to obtain a large-scale radar dataset with reliable labels. Insufficient labeled data often limit the generalization of DL models. As a result, the performance of DL models will drop when being applied to a new scenario. In this sense, only labeling a small portion of data in the large-scale radar dataset is more feasible. In this article, we propose a semisupervised transfer learning (TL) algorithm, “ joint domain and semantic transfer learning ( JDS-TL ),” for radar-based HAR, which is composed of two modules: unsupervised domain adaptation (DA) and supervised semantic transfer. By employing a sparsely labeled dataset to train the HAR model, the proposed method alleviates the need of labeling a significantly large number of radar signals. We adopt a public radar micro-Doppler spectrogram dataset including six human activities to evaluate JDS-TL . Experiments show that the proposed JDS-TL is able to recognize the six activities with an average accuracy of 87.6% when there are only 10% instances labeled in the training dataset. Ablation analysis also demonstrates the efficiency of the DA and the semantic transfer modules.

Estimating the Doppler frequency is an important part in sensing moving targets in integrated sensing and communications (ISAC) systems, such as human tracking and activity recognition. However, it can be highly challenging when there is clock asynchronism between the transmitter and receiver, in bistatic setups that are common nowadays. In this paper, we propose three algorithms for Doppler frequency estimation based on the ratio of channel state information (CSI). These algorithms explore different properties of the CSI ratio, including the circle-preserving property of the Mobius transform, the periodicity of CSI ratio, and the difference (or correlation) between segments of CSI-ratio signals. Experimental results demonstrate that the proposed algorithms can estimate Doppler frequency accurately, outperforming the commonly used approach based on cross-antenna cross correlation.

The performance of deep learning (DL) algorithms for radar-based human motion recognition (HMR) is hindered by the diversity and volume of the available training data. In this article, to tackle the issue of insufficient training data for HMR, we propose an instance-based transfer learning (ITL) method with limited radar micro-Doppler (MD) signatures, alleviating the burden of collecting and annotating a large number of radar samples. ITL is a unique algorithm that consists of three interconnected parts, including DL model pretraining, correlated source data selection, and adaptive collaborative fine-tuning (FT). Any of the three components cannot be excluded; otherwise, the performance of the entire algorithm decreases. The experiments with a radar data set of six human motions show that ITL achieves state-of-the-art performance for HMR with limited training samples, outperforming several existing transfer learning approaches. Especially, when there are only 100 samples per person per class, ITL yields an F1 score of 96.7%. Last but not least, ITL is more generalized to human motion differences. Though adapted to recognize the persons' motions in a small-scale target data set, ITL can also classify the persons' motion data used for pretraining, achieving up to 11.0% F1 score enhancement over the conventional FT method.

Short-range radar has become one of the latest sensor technologies for the Internet of Things (IoT), and it plays an increasingly vital role in IoT applications. As the essential task for various smart-sensing applications, radar-based human activity recognition and person identification have received more attention due to radar’s robustness to the environment and low power consumption. Activity recognition and person identification are generally treated as separate problems. However, designing different networks for these two tasks brings a high computational complexity and wastes of resources to some extent. Furthermore, there are some correlations in activity recognition and person identification tasks. In this work, we propose a multiscale residual attention network (MRA-Net) for joint activity recognition and person identification with radar micro-Doppler signatures. A fine-grained loss weight learning (FLWL) mechanism is presented for elaborating a multitask loss to optimize MRA-Net. In addition, we construct a new radar micro-Doppler dataset with dual labels of activity and identity. With the proposed model trained on this dataset, we demonstrate that our method achieves the state-of-the-art performance in both radar-based activity recognition and person identification tasks. The impact of the FLWL mechanism was further investigated, and ablation studies of the efficacy of each component in MRA-Net were also conducted.

With the application of deep learning (DL) techniques, radar-based human activity recognition (HAR) attracts increasing attention thanks to its high accuracy and good privacy. However, training a DL model requires a large volume of data, and generally the trained model cannot be adapted to a new scenario. In this paper, we propose a supervised few-shot adversarial domain adaptation ( FS-ADA ) method for HAR, where only limited radar training data is collected from a new application scenario. We adopt the domain adaptation method to learn a common feature space between a pre-existing radar dataset and the newly acquired training data. We also design a multi-class discriminator network, which integrates the category classifier and the binary domain discriminator, to employ the supervised label information in the limited radar data for model training. Then, a multitask generative adversarial training mechanism is proposed to optimize FS-ADA . In this way, both domain-invariant and category-discriminative features can be extracted for HAR in a new scenario. Experimental results for two few-shot radar-based HAR tasks show that the proposed FS-ADA method is effective and outperforms state-of-the-art methods.