Human activity recognition using sensor data has become a research hotspot in the field of ubiquitous computing and has a wide range of application scenarios in real life. Effectively extracting and fusing the rich spatiotemporal information contained in the sensor data from various sensors and multiple wearable devices is a challenging task. Moreover, for practical deployment applications, the limited computational and storage resources of edge devices also impose requirements on the model size and inference speed. Hence, we propose STFNet, an enhanced and lightweight spatiotemporal fusion network for wearable human activity recognition. In which, we innovatively divide the extraction and fusion process of spatiotemporal features into three stages, including sensing feature extraction, temporal feature extraction, and channel fusion, to learn incrementally and reduce information loss. We first extract shallow intra-channel features and learn local inter-channel information by using a lightweight local attention mechanism. Then, the learned features are fed into a hierarchical convolution structure with residual connections to extract multi-scale and comprehensive temporal features. Finally, we use graph convolutions to further fuse the deep inter-channel information. By replacing the conventional global attention mechanism with a local one and using hierarchical convolution to replace dilated or multi-kernel convolution, STFNet achieves the purpose of lightweight design. In the experiments, STFNet performs better on public datasets than the selected WHAR models published in recent years and demonstrates its lightweight characteristics on edge device deployment.

Using wearable sensors to identify human activities has elicited significant interest within the discipline of ubiquitous computing for everyday facilitation. Recent research has employed hybrid models to better leverage the modal information of sensors and temporal information, enabling improved performance for wearable human activity recognition. Nevertheless, the lack of effective exploitation of human structural information and limited capacity for cross-channel fusion remains a major challenge. This study proposes a generic design, called GT-WHAR, to accommodate the varying application scenarios and datasets while performing effective feature extraction and fusion. Firstly, a novel and unified representation paradigm, namely Body-Sensing Graph Representation , has been proposed to represent body movement by a graph set, which incorporates structural information by considering the intrinsic connectivity of the skeletal structure. Secondly, the newly designed Body-Node Attention Graph Network employs graph neural networks to extract and fuse the cross-channel information within the graph set. Eventually, the graph network has been embedded in the proposed Bidirectional Temporal Learning Network , facilitating the extraction of temporal information in conjunction with the learned structural features. GT-WHAR outperformed the state-of-the-art methods in extensive experiments conducted on benchmark datasets, proving its validity and efficacy. Besides, we have demonstrated the generality of the framework through multiple research questions and provided an in-depth investigation of various influential factors.

Wearable sensor-based Human Activity Recognition (WHAR) is an important application field that uses sensors for measurement, especially the broad application scenarios in human-computer interaction. Despite the strides made by deep learning-based approaches, practical issues like deployment on edge devices are often overlooked due to their dependence on massive network layers that require extensive computational and storage resources. This paper proposes a novel approach named Eff-WHAR, which aims to improve recognition accuracy and reduce computational burden in the context of WHAR, especially for practical applications on edge devices. The overall approach consists of two parts: Multiscale Temporal Embedding Network and Channel Mixing Network . The first part involves utilizing convolution operations with varying receptive fields in the temporal dimension to extract and merge features from the input data. It facilitates a more comprehensive understanding of different behaviors, resulting in enhanced accuracy. In the second part, features are extracted and fused in the sensing channel dimension through a combination of intra-channel and inter-channel convolutions. It is better to reduce computational cost and model size by replacing the traditional 2D convolution and attention mechanisms, which are commonly used but require a lot of computing power and storage, making them unsuitable for devices with limited resources. We validated Eff-WHAR on eight commonly used WHAR datasets, obtaining outstanding performance compared with SOTA approaches. In addition, we conducted experiments in actual edge devices, verifying the practicality and efficiency of our approach.

The identification of slag inclusion defects in welds is of the utmost importance in guaranteeing the integrity, safety, and prolonged service life of welded structures. Most research focuses on different kinds of weld defects, but branch research on categories of slag inclusion material is limited and critical for safeguarding the quality of engineering and the well-being of personnel. To address this issue, we design a simulated method using ultrasonic testing to identify the inclusion of material categories in austenitic stainless steel. It is based on a simulated experiment in a water environment, and six categories of cubic specimens, including four metallic and two non-metallic materials, are selected to simulate the slag materials of the inclusion defects. Variational mode decomposition optimized by particle swarm optimization is employed for ultrasonic signals denoising. Moreover, the phase spectrum of the denoised signal is utilized to extract the phase characteristic of the echo signal from the water–slag specimen interface. The experimental results show that our method has the characteristics of appropriate decomposition and good denoising performance. Compared with famous signal denoising algorithms, the proposed method extracted the lowest number of intrinsic mode functions from the echo signal with the highest signal-to-noise ratio and lowest normalized cross-correlation among all of the comparative algorithms in signal denoising of weld slag inclusion defects. Finally, the phase spectrum can ascertain whether the slag inclusion is a thicker or thinner medium compared with the weld base material based on the half-wave loss existing or not in the echo signal phase.