In the present paper, an attempt was made to achieve high-performance continuous human activity recognition (CHAR) using deep neural networks. The present study focuses on recognizing different activities in a continuous stream, which means ‘back-to-back’ consecutive set of activities, from only inertial measurement unit (IMU) sensors mounted on smartwatches. For that purpose, a new dataset called ‘CHAR-SW’, which includes numerous streams of daily activities, was collected using smartwatches, and feature representations and network architectures were designed. Experimental comparisons using our own dataset and public datasets (Aruba and Tulum) have been performed. They demonstrated that cascading bidirectional long short-term memory (Bi-LSTM) with featured data performed well in offline mode from the viewpoints of accuracy, computational time, and storage space required. The input to the Bi-LSTM is a descriptor which composed of a stream of the following features: autocorrelation, median, entropy, and instantaneous frequency. Additionally, a novel technique to operate the CHAR system online was introduced and shown to be effective. Experimental results can be summarized as: the offline CHARM-Deep enhanced the accuracy compared with using raw data or the existing approaches, and it reduced the processing time by 86% at least relative to the time consumed in executing the Bi- LSTM classifier directly on the raw data. It also reduced storage space by approximately 97:77% compared with using raw data. The online evaluation shows that it can recognize activities in real-time with an accuracy of 91%.

Counting humans is an essential part of many people-centric applications. In this paper, we propose CrossCount: an accurate deep-learning-based human count estimator that uses a single WiFi link to estimate the human count in an area of interest. The main idea is to depend on the temporal link-blockage pattern as a discriminant feature that is more robust to wireless channel noise than the signal strength, hence delivering a ubiquitous and accurate human counting system. As part of its design, CrossCount addresses a number of deep learning challenges such as class imbalance and training data augmentation for enhancing the model generalizability. Implementation and evaluation of CrossCount in multiple testbeds show that it can achieve a human counting accuracy to within a maximum of 2 persons 100% of the time. This highlights the promise of CrossCount as a ubiquitous crowd estimator with nonlabor-intensive data collection from off-the-shelf devices.

Human activity recognition (HAR) using inertial motion data has gained a lot of momentum in recent years both in research and industrial applications. From the abstract perspective, this has been driven by the rapid dynamics for building intelligent, smart environments, and ubiquitous systems that cover all aspects of human life including healthcare, sports, manufacturing, commerce, etc., which necessitate and subsume activity recognition aiming at recognizing the actions, characteristics, and goals of one or more agent(s) from a temporal series of observations streamed from one or more sensors. From a more concrete and seemingly orthogonal perspective, such momentum has been driven by the ubiquity of inertial motion sensors on-board mobile and wearable devices including smartphones, smartwatches, etc. In this paper we give an introductory and a comprehensive survey to the subject from a given perspective. We focus on a subset of topics, that we think are major, that will have significant and influential impacts on the future research and industrial-scale deployment of HAR systems. These include: (1) a comprehensive and detailed description of the inertial motion benchmark datasets that are publicly available and/or accessible, (2) feature selection and extraction techniques and the corresponding learning methods used to build workable HAR systems; we survey classical handcrafted datasets as well as data-oriented automatic representation learning approach to the subject, (3) transfer learning as a way to overcome many hurdles in actual deployments of HAR systems on a large scale, (4) embedded implementations of HAR systems on mobile and/or wearable devices, and finally (5) we touch on adversarial attacks, a topic that is essentially related to the security and privacy of HAR systems. As the field is very huge and diverse, this article is by no means comprehensive; it is though meant to provide a logically and conceptually rather complete picture to advanced practitioners, as well as to present a readable guided introduction to newcomers. Our logical and conceptual perspectives mimic the typical data science pipeline for state-of-the-art AI-based systems.