The overall aim of this article is to contribute to the further development of the area of benchmarking in risk management. The article introduces a two-step benchmarking model to assess the efficacy of ISO risk management systems. It furthermore aims at verifying its usefulness in terms of finding hidden risk issues and improvement opportunities. The existence of all key elements of an ISO 31000-based risk management system is examined at the beginning of this study. Then, the quality in terms of efficacy of important aspects of the risk management system is examined in more detail with special benchmarks. The application of the model to six ISO-certified organizations follows and reinforces the novelty of this study, which is to combine risk science knowledge with benchmarking theory in the application of ISO risk management standards in organizations. The results show that the benchmarking model developed in this study provides rigor when assessing and evaluating the efficacy of an ISO risk management system. By applying the model, risk issues and risk factors can be found that had not previously been identified. The findings are of importance for risk management, the benchmarking science, and for the development of ISO risk management standards.

Automation surprises in aviation continue to be a significant safety concern and the community’s search for effective strategies to mitigate them are ongoing. The literature has offered two fundamentally divergent directions, based on different ideas about the nature of cognition and collaboration with automation. In this paper, we report the results of a field study that empirically compared and contrasted two models of automation surprises: a normative individual-cognition model and a sensemaking model based on distributed cognition. Our data prove a good fit for the sense-making model. This finding is relevant for aviation safety, since our understanding of the cognitive processes that govern human interaction with automation drive what we need to do to reduce the frequency of automation-induced events.

Crew resource management (CRM) training for flight crews is widespread and has been credited with improving aviation safety. As other industries have adopted CRM, they have interpreted CRM in different ways. We sought to understand how industries have adopted CRM, regarding its conceptualisation and evaluation. For this, we conducted a systematic review of CRM studies in the Maritime, Nuclear Power, Oil and Gas, and Air Traffic Control industries. We searched three electronic databases (Web of Science, Science Direct, Scopus) and CRM reviews for papers. We analysed these papers on their goals, scope, levers of change, and evaluation. To synthesise, we compared the analysis results across industries. We found that most CRM programs have the broad goals of improving safety and efficiency. However, there are differences in the scope and levers of change between programs, both within and between industries. Most evaluative studies suffer from methodological weaknesses, and the evaluation does not align with how studies conceptualise CRM. These results challenge the assumption that there is a clear link between CRM training and enhanced safety in the analysed industries. Future CRM research needs to provide a clear conceptualisation—how CRM is expected to improve safety—and select evaluation measures consistent with this.

Automation surprise (AS) has often been associated with aviation safety incidents. Although numerous laboratory studies have been conducted, few data are available from routine flight operations. A survey among a representative sample of 200 Dutch airline pilots was used to determine the prevalence of AS and the severity of its consequences, and to test some of the factors leading to AS. Results show that AS is a relatively widespread phenomenon that occurs three times per year per pilot on average but rarely has serious consequences. In less than 10% of the AS cases that were reviewed, an undesired aircraft state was induced. Reportable occurrences are estimated to occur only once every 1–3 years per pilot. Factors leading to a higher prevalence of AS include less flying experience, increasing complexity of the flight control mode, and flight duty periods of over 8 hr. It is concluded that AS is a manifestation of system and interface complexity rather than cognitive errors.

In this study we identify the differences in goal realisation when applying two conflicting paradigms regarding rule perception and management. We gathered more than 30 scenarios where goal conflicts were apparent in a military operational unit. We found that operators repetitively utilized certain routines in executing their tasks in an effort to realize several conflicting goals. These routines were not originally intended nor designed into the rules and not explicitly included in documentation. They were not necessarily at odds with the literal wording and/or the intent of rules and regulations, although we did find examples of this. Our data showed that local ingenuity was created innovatively within the frame of existing rules or kept invisible to those outside the unit. The routines were introduced and passed on informally, and we found no evidence of testing for the introduction of new risks, no migration into the knowledge base of the organisation, and no dissemination as new best practices. An explanation for this phenomenon was found in the fact that the military organisation was applying a top-down, classical, rational approach to rules. In contrast, the routines were generated by adopting a constructivist view of rules as dynamic, local, situated constructions with operators as experts. The results of this study suggest that organisations are more effective in solving goal conflicts and creating transparency on local ingenuity if they adopt a constructivist paradigm instead of, or together with, a classical paradigm.

Although rules support people while executing tasks, they are not the same as work-as-done. It can be impossible to follow the rules and finish the job at the same time. In this study, the objective is to better understand the stakes and interests that lie behind retaining gaps between work-as-prescribed and work-as-done, mapping the benefits and fears of noncompliance. The study was conducted along the vertical hierarchy of an operational flight squadron of the Royal Netherlands Air Force. We applied a qualitative survey research methodology using semi-structured interviews, complemented by an investigation of relevant documents. We found a public and political commitment to compliance made by the Dutch Department of Defence, which reinforces a cycle of issuing promises followed by pressure to keep the promise. This contradicts the found need for adaptation and freedom to use expertise. The official safety narrative seems to convey a hidden message—bad things happen to bad people, reminiscent of a bogeyman. One opportunity to resolve the situation is a doctrine change, changing prescriptive rules to guidelines.