In the dynamic and complex environment of industrial control rooms, operators are often inundated with numerous tasks and alerts, leading to a state known as task overload. This condition can result in decision fatigue and increased reliance on cognitive biases, which may compromise the decision-making process. To mitigate these risks, the implementation of decision support systems (DSSs) is essential. These systems are designed to aid operators in making swift, well-informed decisions, especially when their judgment may be faltering. Our research presents an artificial intelligence (AI)-based framework utilizing dynamic influence diagrams and reinforcement learning to develop a powerful decision support system. The foundation of this AI framework is the creation of a robust, interpretable, and effective DSS that aids control room operators during critical process disturbances. By incorporating expert knowledge, the dynamic influence diagram provides a comprehensive model that captures the uncertainties inherent in complex industrial processes. It excels in anomaly detection and recommending optimal actions. Furthermore, this model is improved through a strategic collaboration with reinforcement learning, which refines the recommendations to be more context-specific and accurate. The primary goal of this AI framework is to equip operators with a live, reliable DSS that significantly enhances their response during process upsets. This paper describes the development of the AI framework and its implementation in a simulated control room environment. Our results show that the DSS can improve operator performance and reduce cognitive workload. However, it also uncovers a trade-off with situation awareness, which may decrease as operators become overly dependent on the system’s guidance. Our study highlights the necessity of balancing the advantages of decision support with the need to maintain operator engagement and understanding during process operations.

These datasets contain measures from multi-modal data sources. They include objective and subjective measures commonly used to determine cognitive states of workload, situational awareness, stress, and fatigue using data collection tools such as NASA-TLX, SART, eye tracking, EEG, Health Monitoring Watch, a survey to assess training, and a think-aloud situational awareness assessment following the SPAM methodology. Also, data from a simulation formaldehyde production plant based on the interaction of the participants in a controlled control room experimental setting is included. The interaction with the plant is based on a human-in-the-loop alarm handling and process control task flow, which includes Monitoring, Alarm Handling, Recovery planning, and intervention (Troubleshooting, Control and Evaluation). Data was collected from 92 participants, split into four groups while they underwent the described task flow. Each participant tested three scenarios lasting 15–18 min with a –10-min survey completion and break period in between using different combinations of decision support tools. The decision support tools tested and varied for each group include alarm prioritisation vs. none, paper-based vs. Digitised screen-based procedures, and an AI recommendation system. This is relevant to compare current practices in the industry and the impact on operators’ performance and safety. It is also applicable to validate proposed solutions for the industry. A statistical analysis was performed on the dataset to compare the outcomes of the different groups. Decision-makers can use these datasets for control room design and optimisation, process safety engineers, system engineers, human factors engineers, all in process industries, and researchers in similar or close domains.

Prosody is an integral part of communication, but remains an open problem in state-of-the-art speech synthesis. There are two major issues faced when modelling prosody: (1) prosody varies at a slower rate compared with other content in the acoustic signal (e.g. segmental information and background noise); (2) determining appropriate prosody without sufficient context is an ill-posed problem. In this paper, we propose solutions to both these issues. To mitigate the challenge of modelling a slow-varying signal, we learn to disentangle prosodic information using a word level representation. To alleviate the ill-posed nature of prosody modelling, we use syntactic and semantic information derived from text to learn a context-dependent prior over our prosodic space. Our context-aware model of prosody (CAMP) outperforms the state-of-the-art technique, closing the gap with natural speech by 26%. We also find that replacing attention with a jointly-trained duration model improves prosody significantly.