Deep learning (DL) has shown tremendous promise in radar applications that involve target classification and imaging. In the field of indoor monitoring, researchers have shown an interest in DL for classifying daily human activities, detecting falls, and monitoring gait abnormalities. Driving this interest are emerging applications related to smart and secure homes, assisted living, and medical diagnosis. The success of DL in providing an accurate real-time accounting of observed humanmotion articulations fundamentally depends on the neural network structure, input data representation, and proper training. This article puts DL in the context of data-driven approaches for motion classification and compares its performance with other approaches employing handcrafted features. We discuss recent proposed enhancements of DL classification performance and report on important challenges and possible future research to realize its full potential.

Radar-based activity recognition is a problem that has been of great interest due to applications such as border control and security, pedestrian identification for automotive safety, and remote health monitoring. This work seeks to show the efficacy of micro-Doppler analysis to distinguish even those gaits whose micro-Doppler signatures are not visually distinguishable. Moreover, a 3-layer, deep convolutional autoencoder (CAE) is proposed, which utilizes unsupervised pre-training to initialize the weights in the subsequent convolutional layers. This architecture is shown to be more effective than other deep learning architectures, such as convolutional neural networks (CNN) and autoencoders (AE), as well as conventional classifiers employing pre-defined features, such as support vector machines (SVM), random forest (RF) and extreme gradient boosting (Xgboost). Results show the performance of the proposed deep CAE yields a correct classification rate of 94.2% for micro-Doppler signatures of 12 different human activities measured indoors using a 4 GHz continuous wave radar - 17.3% improvement over SVM.

Automotive perception involves understanding the external driving environment as well as the internal state of the vehicle cabin and occupants using sensor data. It is critical to achieving high levels of safety and autonomy in driving. This paper provides an overview of different sensor modalities like cameras, radars, and LiDARs used commonly for perception, along with the associated data processing techniques. Critical aspects in perception are considered, like architectures for processing data from single or multiple sensor modalities, sensor data processing algorithms and the role of machine learning techniques, methodologies for validating the performance of perception systems, and safety. The technical challenges for each aspect are analyzed, emphasizing machine learning approaches given their potential impact on improving perception. Finally, future research opportunities in automotive perception for their wider deployment are outlined.

Deep neural networks (DNNs) require large-scale labeled data sets to prevent overfitting while having good generalization. In radar applications, however, acquiring a measured data set of the order of thousands is challenging due to constraints on manpower, cost, and other resources. In this letter, the efficacy of two neural network initialization techniques—unsupervised pretraining and transfer learning—for dealing with training DNNs on small data sets is compared. Unsupervised pretraining is implemented through the design of a convolutional autoencoder (CAE), while transfer learning from two popular convolutional neural network architectures (VGGNet and GoogleNet) is used to augment measured RF data for training. A 12-class problem for discrimination of micro-Doppler signatures for indoor human activities is utilized to analyze activation maps, bottleneck features, class model, and classification accuracy with respect to training sample size. Results show that on meager data sets, transfer learning outperforms unsupervised pretraining and random initialization by 10% and 25%, respectively, but that when the sample size exceeds 650, unsupervised pretraining surpasses transfer learning and random initialization by 5% and 10%, respectively. Visualization of activation layers and learned models reveals how the CAE succeeds in representing the micro-Doppler signature.

Deep neural networks (DNNs) have recently received vast attention in applications requiring classification of radar returns, including radar-based human activity recognition for security, smart homes, assisted living, and biomedicine. However, acquiring a sufficiently large training dataset remains a daunting task due to the high human costs and resources required for radar data collection. In this paper, an extended approach to adversarial learning is proposed for generation of synthetic radar micro-Doppler signatures that are well-adapted to different environments. The synthetic data is evaluated using visual interpretation, analysis of kinematic consistency, data diversity, dimensions of the latent space, and saliency maps. A principlecomponent analysis (PCA) based kinematic-sifting algorithm is introduced to ensure that synthetic signatures are consistent with physically possible human motions. The synthetic dataset is used to train a 19-layer deep convolutional neural network (DCNN) to classify micro-Doppler signatures acquired from an environment different from that of the dataset supplied to the adversarial network. An overall accuracy 93% is achieved on a dataset that contains multiple aspect angles (0°, 30°, and 45° as well as 60°), with 9% improvement as a result of kinematic sifting.