Human–robot teaming (HrT) is being adopted in an increasing range of industries and work environments. Effective HrT relies on the success of complex and dynamic human–robot interaction. Although it may be optimal for robots to possess all the social and emotional skills to function as productive team members, certain cognitive capabilities can enable them to develop attitude-based competencies for optimizing teams. Despite the extensive research into the human–human team structure, the domain of HrT research remains relatively limited. In this sense, incorporating established human–human teaming (HhT) elements may prove practical. One key element is mutual performance monitoring (MPM), which involves the reciprocal observation and active anticipation of team members’ actions within the team setting, fostering enhanced team coordination and communication. By adopting this concept, this study uses ML-based visual action recognition as a potential tool for developing an effective way to monitor the human component in HrT. This study utilizes a data modeling approach on an existing dataset, the “Industrial Human Action Recognition Dataset” (InHARD), curated specifically for human action recognition assembly tasks in industrial environments involving human–robot collaborations. This paper presents the results of this modeling approach in analyzing the dataset to implement a theoretical concept that can be a first step toward enabling the system to adapt dynamically. The outcomes emphasize the significance of implementing state-of-the-art team concepts by integrating modern technologies and assessing the possibility of advancing HrT in this direction.

Collaborative intelligence (CI) involves human-machine interactions and is deemed safety-critical because their reliable interactions are crucial in preventing severe injuries and environmental damage. As these applications become increasingly data-driven, the reliability of CI applications depends on the quality of data, shaping the system’s ability to interpret and respond in diverse and often unpredictable environments. In this regard, it is important to adhere to data quality standards and guidelines, thus facilitating the advancement of these collaborative systems in industry. This study presents the challenges of data quality in CI applications within industrial environments, with two use cases that focus on the collection of data in Human-Robot Interaction (HRI). The first use case involves a framework for quantifying human and robot performance within the context of naturalistic robot learning, wherein humans teach robots using intuitive programming methods within the domain of HRI. The second use case presents real-time user state monitoring for adaptive multi-modal teleoperation, that allows for a dynamic adaptation of the system’s interface, interaction modality and automation level based on user needs. The article proposes a hybrid standardization derived from established data quality-related ISO standards and addresses the unique challenges associated with multi-modal HRI data acquisition. The use cases presented in this study were carried out as part of an EU-funded project, Collaborative Intelligence for Safety-Critical Systems (CISC).

Human-Robot Interaction (HRI) systems are becoming increasingly integral to collaborative industrial and service environments. However, understanding human performance within such settings, particularly in Programming by Demonstration (PbD) frameworks, remains a challenge due to limited availability of comprehensive datasets. This study presents a multimodal dataset designed to assess human performance in HRI applications. The dataset includes both objective and subjective measures collected using various tools within a PbD framework. The data is collected to assess the overall system performance, with a focus on the effectiveness of PbD methods in improving robot learning process. The performance attributes encompass programming efficiency, cognitive workload, usability, and ergonomic assessment of human posture in physical HRI settings. Using tools such as a robot manipulator (UR10e), a vision -based motion tracking system, eye- tracking glasses, NASA Task Load Index (NASA-TLX) and system usability scale (SUS) questionnaire, we recorded interactions between human and robot comprising-robot trajectories, human motion trajectories, participant eye tracking metrices, and participant subjective responses. The data was collected in an experimental setting involving two kinaesthetic robot teaching tasks, under four feedback conditions that provide performance related insights and actionable instructions to improve participant performance. A total of N=28 participants performed three trials of each task using developed Human-Machine Interface (HMI). This dataset is valuable for advancing human-robot teaching interaction and is applicable for developing adaptive HRI systems. The dataset can serve as a significant resource for researchers and machine learning engineers aiming to develop advance HRI systems, improve PbD algorithms, train HRI based machine learning models and facilitate further studies in safety and ergonomic assessment applications in robot learning and collaboration.

Action recognition has become a prerequisite approach to fluent Human-Robot Interaction (HRI) due to a high degree of movement flexibility. With the improvements in machine learning algorithms, robots are gradually transitioning into more human-populated areas. However, HRI systems demand the need for robots to possess enough cognition. The action recognition algorithms require massive training datasets, structural information of objects in the environment, and less expensive models in terms of computational complexity. In addition, many such algorithms are trained on datasets derived from daily activities. The algorithms trained on non-industrial datasets may have an unfavorable impact on implementing models and validating actions in an industrial context. This study proposed a lightweight deep learning model for classifying low-level actions in an assembly setting. The model is based on optical flow feature elicitation and mobilenetV2-SSD action classification and is trained and assessed on an actual industrial activities’ dataset. The experimental outcomes show that the presented method is futuristic and does not require extensive preprocessing; therefore, it can be promising in terms of the feasibility of action recognition for mutual performance monitoring in real-world HRI applications. The test result shows 80% accuracy for low-level RGB action classes. The study's primary objective is to generate experimental results that may be used as a reference for future HRI algorithms based on the InHard dataset.