Learning underlying patterns from sensory data is crucial in the Human Activity Recognition (HAR) task to avoid poor generalization when coping with unseen data. A key solution to such an issue is representation learning, which becomes essential when input signals contain activities with similar patterns or when patterns generated by different subjects for the same activity vary. To address these issues, we seek a solution to increase generalization by learning the underlying factors of each sensor signal. We develop a novel multi-channel asymmetric auto-encoder to recreate input signals precisely and extract indicative unsupervised futures. Further, we investigate the role of various activation functions in signal reconstruction to ensure the model preserves the patterns of each activity in the output. Our main contribution is that we propose a multi-task learning model to enhance representation learning through shared layers between signal reconstruction and the HAR task to improve the robustness of the model in coping with users not included in the training phase. The proposed model learns shared features between different tasks that are indeed the underlying factors of each input signal. We validate our multi-task learning model using several publicly available HAR datasets, UCI-HAR, MHealth, PAMAP2, and USC-HAD, and an in-house alpine skiing dataset collected in the wild, where our model achieved 99%, 99%, 95%, 88%, and 92% accuracy. Our proposed method shows consistent performance and good generalization on all the datasets compared to the state of the art.

Globalization in the field of industry is fostering the need for cognitive production systems. To implement modern concepts that enable tools and systems for such a cognitive production system, several challenges on the shop floor level must first be resolved. This paper discusses the implementation of selected cognitive technologies on a real industrial case-study of a construction machine manufacturer. The partner company works on the concept of mass customization but utilizes manual labour for the high-variety assembly stations or lines. Sensing and guidance devices are used to provide information to the worker and also retrieve and monitor the working, with respecting data privacy policies. Next, a specified process of data contextualization, visual analytics, and causal discovery is used to extract useful information from the retrieved data via sensors. Communications and safety systems are explained further to complete the loop of implementation of cognitive entities on a manual assembly line. This deepened involvement of cognitive technologies are human-centered, rather than automated systems. The explained cognitive technologies enhance human interaction with the processes and ease the production methods. These concepts form a quintessential vision for an effective assembly line. This paper revolutionizes the existing industry 4.0 with an even-intensified human–machine interaction and moving towards cognitivity.

This paper presents a novel approach for counting hand-performed activities using deep learning and inertial measurement units (IMUs). The particular challenge in this task is finding the correct window size for capturing activities with different durations. Traditionally, fixed window sizes have been used, which occasionally result in incorrectly represented activities. To address this limitation, we propose segmenting the time series data into variable-length sequences using ragged tensors to store and process the data. Additionally, our approach utilizes weakly labeled data to simplify the annotation process and reduce the time to prepare annotated data for machine learning algorithms. Thus, the model receives only partial information about the performed activity. Therefore, we propose an LSTM-based architecture, which takes into account both the ragged tensors and the weak labels. To the best of our knowledge, no prior studies attempted counting utilizing variable-size IMU acceleration data with relatively low computational requirements using the number of completed repetitions of hand-performed activities as a label. Hence, we present the data segmentation method we employed and the model architecture that we implemented to show the effectiveness of our approach. Our results are evaluated using the Skoda public dataset for Human activity recognition (HAR) and demonstrate a repetition error of $\pm 1$ even in the most challenging cases. The findings of this study have applications and can be beneficial for various fields, including healthcare, sports and fitness, human–computer interaction, robotics, and the manufacturing industry.

Transformers are a very important asset in the electrical transmission grid, and they can suffer from destructive events—e.g., rare transformer fires. Unfortunately, destructive events often lead to a lack of data available for investigators during post-event forensics and failure analysis. This fact has motivated our design and implementation of a robotic multi-sensor platform and cloud backend solution for the lifecycle monitoring, inspection, diagnostics, and condition assessment of transformers. The robotic platform collects data from specific viewpoints around the transformer during operation and at specific relevant lifecycle milestones of the transformer (e.g., at the factory acceptance test) in an automated, repetitive, precise, and reliable manner. The acquired data are stored in the cloud backend, which also provides computing resources and data access to relevant in- and off-premises services (e.g., respectively, SCADA systems, and weather reports). In this paper, we present the results of our first measurement campaign to showcase the value of our solution for transformer lifecycle monitoring, for anomaly detection, and as a crucial tool for post-event forensics in the case of destructive events.

Manual metal arc welding is a common and important part of the metal manufacturing industry. Although the capabilities of welding robots improved in the past century special welding tasks can only be performed by human workers. Thus, research about analyzing the welding quality automatically to support workers in real time gained importance during the last years. The welding quality depends on the skills of the worker. Recent studies corroborated the correlation between the performed movements with the arc welder (and the corresponding torch manipulation) and the welder’s skill level. However, related studies reveal a research gap in developing skill level assessment for real-world welding manufacturing processes. An adapted experimental design in this study involving realistic welding tasks addresses this gap. A specialized Weld Monitoring System was used to record the three dimensional movement of the arc welder through an embedded IMU and its current and voltage from the power source synchronously. State-of-the-art deep learning was applied on this data to generate an online skill level assessment of the welder. The classifier’s performance was analyzed on each welding technique separately to optimize the network structure accordingly, based on the revealed differences in the reached accuracies. Finally, using a subset of the data excluding recorded data of certain welding techniques increased the overall performance of the classifier.

Activity recognition combined with artificial intelligence is a vital area of research, ranging across diverse domains, from sports and healthcare to smart homes. In the industrial domain, and the manual assembly lines, the emphasis shifts to human–machine interaction and thus to human activity recognition (HAR) within complex operational environments. Developing models and methods that can reliably and efficiently identify human activities, traditionally just categorized as either simple or complex activities, remains a key challenge in the field. Limitations of the existing methods and approaches include their inability to consider the contextual complexities associated with the performed activities. Our approach to address this challenge is to create different levels of activity abstractions, which allow for a more nuanced comprehension of activities and define their underlying patterns. Specifically, we propose a new hierarchical taxonomy for human activity abstraction levels based on the context of the performed activities that can be used in HAR. The proposed hierarchy consists of five levels, namely atomic, micro, meso, macro, and mega. We compare this taxonomy with other approaches that divide activities into simple and complex categories as well as other similar classification schemes and provide real-world examples in different applications to demonstrate its efficacy. Regarding advanced technologies like artificial intelligence, our study aims to guide and optimize industrial assembly procedures, particularly in uncontrolled non-laboratory environments, by shaping workflows to enable structured data analysis and highlighting correlations across various levels throughout the assembly progression. In addition, it establishes effective communication and shared understanding between researchers and industry professionals while also providing them with the essential resources to facilitate the development of systems, sensors, and algorithms for custom industrial use cases that adapt to the level of abstraction.