With the development of the Internet of things (IoT) and wearable devices, the sensor-based human activity recognition (HAR) has attracted more and more attentions from researchers due to its outstanding characteristics of convenience and privacy. Meanwhile, deep learning algorithms can extract high-dimensional features automatically, which makes it possible to achieve the end-to-end learning. Especially the convolutional neural network (CNN) has been widely used in the field of computer vision, while the influence of environmental background, camera shielding, and other factors are the biggest challenges to it. However, the sensor-based HAR can circumvent these problems well. Two improved HAR methods based on Gramian angular field (GAF) and deep CNN are proposed in this paper. Firstly, the GAF algorithm is used to transform the one-dimensional sensor data into the two-dimensional images. Then, through the multi-dilated kernel residual (Mdk-Res) module, a new improved deep CNN network Mdk-ResNet is proposed, which extracts the features among sampling points with different intervals. Furthermore, the Fusion-Mdk-ResNet is adopted to process and fuse data collected by different sensors automatically. The comparative experiments are conducted on three public activity datasets, which are WISDM, UCI HAR and OPPORTUNITY. The optimal results are obtained by using the indexes such as accuracy, precision, recall and F-measure, which verifies the effectiveness of the proposed methods.

The advances in deep learning with the ability to automatically extract advanced features have achieved a bright prospect for human activity recognition (HAR). However, the traditional HAR methods still have the deficiencies of incomplete feature extraction, which may lead to incorrect recognition results. To resolve the above problem, a novel framework for spatiotemporal multi-feature extraction on HAR called CapsGaNet is propounded, which is based on capsule and gated recurrent units (GRU) with attention mechanisms. The proposed framework involves a spatial feature extraction layer consisting of capsule blocks, a temporal feature extraction layer consisting of GRU with attention mechanisms, and an output layer. At the same time, considering the actual demands for recognizing aggressive activities in some specific scenarios like smart prison, we constructed a daily and aggressive activity dataset (DAAD). Moreover, based on the acceleration characteristics of aggressive activity, a threshold-based approach for aggressive activity detection is propounded to meet the needs of high real-time and low computational complexity in prison scenarios. The experiments are performed on the wireless sensor data mining (WISDM) dataset and the DAAD dataset, and the results verify that the propounded CapsGaNet could effectually improve the recognition accuracy. The proposed threshold-based approach for aggressive activity detection provides a more effective HAR way by using smart sensor devices in smart prison scenarios.