7 February 2023 in IEEE Internet of Things Journal
In recent years, sensor-based human activity recognition (HAR) technology has been the focus of extensive research and has been successfully applied to many aspects of people’s lives, but there are still some deficiencies. Most studies only distinguish daily activities and have low accuracy for easy confusing activities. In addition, many deep learning models only consider closed set HAR, but the real world contains unknown class (UC) activities that cannot be foreseen, which makes it challenging to apply these models to practice. In view of the above problems, this article proposes a multiresolution fusion convolution network (MRFC-Net) to cover the shortcoming that confusing activities are difficult to correctly identify, thus improving the accuracy of recognition. Furthermore, a multiresolution fusion convolution variational auto-encoder network (MRFC-VAE-Net) for open set HAR is proposed. According to the reconstruction loss of the network, the corresponding threshold is set to effectively classify the known and UC activities in the open set. At the same time, a rich data set named daily-abnormal activity of special group (DAASG) is constructed, which can be applied to the daily monitoring of special groups, such as prisoners and the elderly. Experiments and analyses are carried out on the wireless sensor data mining (WISDM), physical activity monitoring for aging people (PAMAP2) and DAASG data sets, to prove the effectiveness and superiority of the proposed networks.
11 April 2023 in IEEE Internet of Things Journal
Human activity recognition (HAR) technology based on wearables has received increasing attention in recent years. The traditional methods have used hand-crafted features to recognize human activities, resulting in shallow feature extraction. With the development of deep learning, an increasing number of researchers have focused on studying deep learning methods. To achieve higher recognition accuracy, the majority of the current HAR research involves multisource and multimodal sensors (MMSs) data. However, due to the limitations in the receptive fields of single-dimensional convolutional kernels, these networks are still infeasible for extracting spatiotemporal features. In this study, a multidimensional parallel convolutional connected (MPCC) deep learning network based on MMS data for HAR is proposed that fully utilizes the advantages of multidimensional convolutional kernels. Moreover, multiscale residual convolutional squeeze-and-excitation (MRCSE) modules are proposed to enrich the diversity of feature information by combining squeeze-and-excitation (SE) blocks. A daily home activity (DHA) data set is constructed based on the requirements for HAR in certain scenarios, such as smart home, and we conduct experiments on the optimal combination of sensor locations on the DHA data set according to a weighted $\text{F}1~({\mathrm{ F}}_{\mathrm{ W}})$ -score. Both tenfold and leave-one-subject-out (LOSO) cross-validations (CVs) are used to evaluate the performance of the proposed network. The MPCC-MRCSE network achieves ${\mathrm{ F}}_{\mathrm{ W}}$ -scores of 98.33% and 95.42% on the physical activity monitoring for aging people (PAMAP2) and OPPORTUNITY data sets using tenfold CVs, respectively, and achieves ${\mathrm{ F}}_{\mathrm{ W}}$ -scores of 81.47% on the PAMAP2 when applying an LOSO CV.