Human action recognition has been widely used in many fields of life, and many human action datasets have been published at the same time. However, most of the multi-modal databases have some shortcomings in the layout and number of sensors, which cannot fully represent the action features. Regarding the problems, this paper proposes a freely available dataset, named CZU-MHAD (Changzhou University: a comprehensive multi-modal human action dataset). It consists of 22 actions and three modals temporal synchronized data. These modals include depth videos and skeleton positions from a kinect v2 camera, and inertial signals from 10 wearable sensors. Compared with single modal sensors, multi-modal sensors can collect different modal data, so the use of multi-modal sensors can describe actions more accurately. Moreover, CZU-MHAD obtains the 3-axis acceleration and 3-axis angular velocity of 10 main motion joints by binding inertial sensors to them, and these data were captured at the same time. Experimental results are provided to show that this dataset can be used to study structural relationships between different parts of the human body when performing actions and fusion approaches that involve multi-modal sensor data.

In contemporary research on human action recognition, most methods separately consider the movement features of each joint. However, they ignore that human action is a result of integrally cooperative movement of each joint. Regarding the problem, this paper proposes an action feature representation, called Motion Collaborative Spatio-Temporal Vector (MCSTV) and Motion Spatio-Temporal Map (MSTM). MCSTV comprehensively considers the integral and cooperative between the motion joints. MCSTV weighted accumulates limbs’ motion vector to form a new vector to account for the movement features of human action. To describe the action more comprehensively and accurately, we extract key motion energy by key information extraction based on inter-frame energy fluctuation, project the energy to three orthogonal axes and stitch them in temporal series to construct the MSTM. To combine the advantages of MSTM and MCSTV, we propose Multi-Target Subspace Learning (MTSL). MTSL projects MSTM and MCSTV into a common subspace and makes them complement each other. The results on MSR-Action3D and UTD-MHAD show that our method has higher recognition accuracy than most existing human action recognition algorithms.

To tackle two prevalent challenges in video anomaly detection, misclassification of abnormal frames as normal and the prolonged runtime of existing methods, this paper proposes a time-efficient anomaly detection method utilizing twice-reconstruction and parallel computing. The proposed method integrates three key innovations: a twice-reconstruction model to amplify the reconstruction error for abnormal frames, a memory module to better capture and utilize normal distribution patterns, and a parallel computing strategy to significantly accelerate anomaly detection. By reconstructing frames twice, our method enhances the model’s anomaly detection capabilities. Meanwhile a memory module is incorporated to retain normal distribution patterns more effectively, reducing the likelihood of misclassifying abnormal frames. Furthermore, parallel computing is employed to minimize runtime and boost detection efficiency. Unlike existing methods, TPR-VAD achieves both high detection accuracy and time efficiency, making it well-suited for real-world applications such as intelligent surveillance systems. Experimental results on the UCSD Ped2, CUHK Avenue, and ShanghaiTech datasets show that our method achieves superior detection accuracy, reaching 99.01%, 91.21%, and 82.77%, respectively, while significantly reduces runtime.

Nowadays, human action recognition based on depth video sequences has become a popular research topic. However, some existing feature map representations suffer from the issue of missing temporal information and cannot completely represent the spatio-temporal information. Moreover, the redundant information is not effectively removed, and the key information cannot be highlighted. To address these issues, we propose an effective method called Key Spatio-temporal Energy Information Mapping (KSTEIM). First, we construct a human motion energy field to strengthen the distribution of motion energy. Next, we present a new key frame algorithm that removes the redundant frames based on redundancy values calculated from image differences to obtain keyframe sequences. Then, each key frame is projected onto three orthogonal axes to obtain three 1D motion energy lists. The energy lists are concatenated in temporal sequence under each projection axis to form KSTEIM. Then, we extract Histogram of Oriented Gradients (HOG) from KSTEIM and make KSTEIM-HOG sparser through Multi-kernel Subspace Representation Enhancement (MKSRE). With only positive sequence actions, our method achieves recognition rate of 93.41% on MSR-Action3D, and 90.93% on UTD-MHAD. It indicates that the performance of our method is no weaker than most existing methods. Codes are available at: https://github.com/chaoxin118/KSTEIM.