Laparoscopic surgery can be exhausting and frustrating, and the cognitive load experienced by surgeons may have a major impact on patient safety as well as healthcare economics. As cognitive load decreases with increasing proficiency, its robust assessment through physiological data can help to develop more effective training and certification procedures in this area. We measured data from 31 novices during laparoscopic exercises to extract features based on cardiac and ocular variables. These were compared with traditional behavioural and subjective measures in a dual-task setting. We found significant correlations between the features and the traditional measures. The subjective task difficulty, reaction time, and completion time were well predicted by the physiology features. Reaction times to randomly timed auditory stimuli were correlated with the mean of the heart rate ( $$r = - 0.29$$r = - 0.29 ) and heart rate variability ( $$r = 0.4$$r = 0.4 ). Completion times were correlated with the physiologically predicted values with a correlation coefficient of 0.84. We found that the multi-modal set of physiology features was a better predictor than any individual feature and artificial neural networks performed better than linear regression. The physiological correlates studied in this paper, translated into technological products, could help develop standardised and more easily regulated frameworks for training and certification.

Collaborative robots (cobots) have largely replaced conventional industrial robots in today’s workplaces, particularly in manufacturing setups, due to their improved performance and intelligent design. In the framework of Industry 5.0, humans are working alongside cobots to accomplish the required level of automation. However, human–robot interaction has brought up concerns regarding human factors (HF) and ergonomics. A human worker may experience cognitive stress as a result of cobots’ irresponsive nature in unpredictably occurring situations, which adversely affects productivity. Therefore, there is a necessity to measure stress to enhance a human worker’s performance in a human–robot collaborative environment. In this study, factory workers’ mental workload was assessed using physiological, behavioural, and subjective measures. Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) signals were collected to acquire brain signals and track hemodynamic activity, respectively. The effect of task complexity, cobot movement speed, and cobot payload capacity on the mental stress of a human worker were observed for a task designed in the context of a smart factory. Task complexity and cobot speed proved to be more impactful. As physiological measures are unbiased and more authentic means to estimate stress, eventually they may replace the other conventional measures if they prove to correlate with the results of traditional ones. Here, regression and artificial neural networks (ANN) were utilised to determine the correlation between physiological data and subjective and behavioural measures. Regression performed better for most of the targets and the best correlation (rsq-adj = 0.654146) was achieved for predicting missed beeps, a behavioural measure, using a combination of multiple EEG and fNIRS predictors. The k-nearest neighbours (KNN) algorithm was used to evaluate the accuracy of correlation between traditional measures and physiological variables, with the highest accuracy of 77.8% achieved for missed beeps as the target. Results show that physiological measures can be more insightful and have the tendency to replace other biased parameters.

Human–robot collaboration (HRC) has become increasingly prevalent due to innovative advancements in the automation industry, especially in manufacturing setups. Although HRC increases productivity and efficacy, it exposes human workers to psychological stress while interfacing with collaborative robotic systems as robots may not provide visual or auditory cues. It is crucial to comprehend how HRC impacts mental stress in order to enhance occupational safety and well-being. Though academics and industrial interest in HRC is expanding, safety and mental stress problems are still not adequately studied. In particular, human coworkers’ cognitive strain during HRC has not been explored well, although being fundamental to sustaining a secure and constructive workplace environment. This study, therefore, aims to monitor the mental stress of factory workers during HRC using behavioural, physiological and subjective measures. Physiological measures, being objective and more authentic, have the potential to replace conventional measures i.e., behavioural and subjective measures, if they demonstrate a good correlation with traditional measures. Two neuroimaging modalities including electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) have been used as physiological measures to track neuronal and hemodynamic activity of the brain, respectively. Here, the correlation between physiological data and behavioural and subjective measurements has been ascertained through the implementation of seven different machine learning algorithms. The results imply that the EEG and fNIRS features combined produced the best results for most of the targets. For subjective measures being the target, linear regression has outperformed all other models, whereas tree and ensemble performed the best for predicting the behavioural measures. The outcomes indicate that physiological measures have the potential to be more informative and often substitute other skewed metrics.

Collaborative robots (cobots) have largely replaced conventional industrial robots in today’s workplaces, particularly in manufacturing setups due to their improved performance and intelligent design. In the framework of Industry 5.0, humans are working alongside cobots to accomplish the required level of automation. However, human-robot interaction has brought up concerns regarding human factors (HF) and ergonomics. A human worker may experience cognitive stress as a result of cobots' irresponsive nature in unpredictably occurring situations, which adversely affects productivity. Therefore, there is a necessity to measure stress to enhance a human worker’s performance in a human-robot collaborative environment. In this study, factory workers’ mental workload has been assessed using physiological, behavioural, and subjective measures. Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) signals have been collected to acquire brain signals and track hemodynamic activity, respectively. The effect of task complexity, cobot’s movement speed, and cobot’s payload capacity on the mental stress of a human worker have been observed, for a task designed in the context of a smart factory. Task complexity and cobot’s speed have proved to be more impactful. As physiological measures are unbiased and more authentic means to estimate stress, eventually, they may replace the other conventional measures if they prove to correlate with the results of traditional ones. Here, regression and artificial neural networks (ANN) have been utilised to determine the correlation between physiological data and subjective and behavioural measures. Regression has performed better for most of the targets and the best correlation (rsq-adj = 0.654146) has been achieved for predicting missed beeps, behavioural measure, using a combination of multiple EEG and fNIRS predictors. The k-nearest neighbours (KNN) algorithm has been used to evaluate the accuracy of correlation between traditional measures and physiological variables, with highest accuracy of 77.8% achieved for missed beeps as the target. Results show that physiological measures can be more insightful and have the tendency to replace other biased parameters.

Mental stress is a common issue in demanding occupational setups, such as smart industrial settings, particularly from working with robots, being one of the primary reasons for decreased performance and productivity. Quantifying and evaluating stress are critical for worker safety, performance, and overall well-being. A novel integrated framework is proposed in this research for digitising and assessing cognitive stress that combines neuroimaging (EEG and fNIRS), gaze tracking and machine learning. A factory workers’ stress assessment experiment is designed and implemented, which employs physiological, behavioural and subjective measures to assess mental stress from different perspectives. Physiological features extracted from multimodal data are used for training supervised classification and regression models. To further optimise the pipeline, multiple feature selection algorithms are tested, followed by ensemble learning approaches, and the best one is chosen for stress prediction. After implementing the novel stress quantification framework for the factory workers' stress assessment experiment, the ensemble learning approach produced the best results for both regression (RMSE: 10.86) and classification (accuracy: 84.1%) techniques using the STAI score as the target. The behavioural and subjective measures demonstrate the effect of varying process variables (light, noise, task speed, and complexity) during the experiment. Multimodal data, machine learning, and other computational approaches are integrated in this study to objectively quantify cognitive stress, utilising the novel stress quantification framework presented in this research, thereby bridging the gap between research and practical application. This study proposes a multi-domain framework for measuring stress, providing a promising solution for worker well-being in occupational setups.