With the rapid development of the computer and sensor field, inertial sensor data have been widely used in human activity recognition. At present, most relevant studies divide human activities into basic actions and transitional actions, in which basic actions are classified by unified features, while transitional actions usually use context information to determine the category. For the existing single method that cannot well realize human activity recognition, this paper proposes a human activity classification and recognition model based on smartphone inertial sensor data. The model fully considers the feature differences of different properties of actions, uses a fixed sliding window to segment the human activity data of inertial sensors with different attributes and, finally, extracts the features and recognizes them on different classifiers. The experimental results show that dynamic and transitional actions could obtain the best recognition performance on support vector machines, while static actions could obtain better classification effects on ensemble classifiers; as for feature selection, the frequency-domain feature used in dynamic action had a high recognition rate, up to 99.35%. When time-domain features were used for static and transitional actions, higher recognition rates were obtained, 98.40% and 91.98%, respectively.

Human action recognition plays a key role in human-computer interaction in complex environments. However, similar actions will lead to poor feature sequence extraction and result in a reduction in recognition accuracy. This paper proposes a method (Action-Fusion: Multi-label subspace Learning (MLSL)) from depth maps called Depth Sequential Information Entropy Maps (DSIEM) and skeleton data for human action recognition in multiple modal features. The DSIEM describe the spatial information of human motion with information entropy, and describe the temporal information through stitching. DSIEM can reduce the redundancy of depth sequences and effectively capture spatial motion states. MLSL studies the relationship between different modalities and the inherent connection between different labels. The method is evaluated on three public datasets: Microsoft action 3D dataset (MSR Action3D), University of Texas at Dallas-multimodal human action dataset (UTD-MHAD), UTD MHAD-Kinect Version-2 (UTD-MHAD-Kinect V2). Experimental results show that the proposed MLSL model obtains new state-of-the-art results, including achieving the average rate of the MSR Action3D to 93.55%, the average rate of the UTD-MHAD to 88.37% and the average rate of the UTD-MHAD-Kinect V2 to 90.66%.

Human action recognition(HAR) has become a hot topic in the field of computer vision and pattern recognition due to its wide range of application prospects. In most deep learning and multi-sensor data-based action recognition works, sequences from sensors are first stacked into two-dimensional images and then fed into convolutional neural network models to extract features. This approach based on data-level fusion learns different sensor data features but loses the uniqueness of different sensor data. In this paper, we propose a new deep learning model called DSFNet (Distributed Sensors Fusion Network). For the property that acceleration sequences can reflect motion trends, we first learn the features of different sensors and then use the Transformer Encoder module to model the dependencies of multi-sensor actions and extract features. For the property that angular velocity can reflect the direction and velocity of the local pose, we first learn the features of a single sensor in different motion directions in time sequence, and then learn the output feature maps of multi-sensor features in time sequence and extract the features. Finally, the outputs of different data modalities are used for decision-level fusion, which significantly improves the performance of the model. We evaluate the performance of the proposed model on the self-built dataset CZU-MHAD, and the experimental results show that the DSFNet model outperforms the existing methods.

Defect detection on the surface of the steel strip is essential for the quality assurance of the steel strip. Precise localization and classification, the two significant tasks of defect detection, still need to be completed due to the diversity of defect scales. In this paper, a residual atrous spatial pyramid pooling (RASPP) module is first designed to enrich the multi-scale information of the feature maps and increase the receptive field of the feature maps. Secondly, a double pyramid network (DPN) that combines RASPP and feature pyramid is proposed to fuse multi-scale features further so that similar semantic features are shared among the features of each layer. Finally, DPN-Detector, an automatic surface defects detection network, is proposed, which embeds the DPN module into Faster R-CNN and replaces the original detection head with a designed double head. Experiments are carried out on the steel strip surface defect dataset (NEU-DET), and the results show that the mAP of DPN-Detector is as high as 80.93%, which is 3.52% higher than that of the baseline network Faster R-CNN. The classification accuracy is 74.64%, and the detection speed reaches 18.62 FPS. The proposed method performs better robustness, classification and regression capability than other steel strip defect detection methods.