Background. The use of simulation games for complex systems analysis and design has been acknowledged about 50 years ago. However, articles do not combine all salient factors for successful simulation games, and often stem from a clear view of one particular field of science only. With combining multiple disciplines, connect analysis and design as well as research and practice, we provide deep insights in design and use of simulation games. Aim. This article analyzes the design and evaluation process of a variety of game-based projects and activities, using existing scientific concepts and approaches, in order to establish games as a valid research tool. Our focus lies on the approach towards the use of games as design instrument; using them as an intervention in a larger, complex context, in order to design this context. With our contribution, we aim at providing insights and recommendations on the design and use of games as valid research tools, the limitations of this use, possible pitfalls, but also best practices. Method. We carried out a literature review of related work to identify the most important scientific concepts related to our approach of game design. Further use of combined quantitative and qualitative case study analyses highlights the design process and results of our own game studies. Results. The analyses yielded a consolidated conceptualization of simulation games as research instruments in complex systems analysis and design. The results also include methods for the evaluation of simulation games, additional evaluation methods, and limitations to use simulation games as research instruments. Conclusions. We propose guidelines for using simulation games as research instruments that may be of value to practitioners and scientists alike. Recommendation. We recommend practitioners and scientists to apply the guidelines presented here in their efforts to analyze and design complex systems.

The authors define the requirements and a conceptual model for comparative evaluation research of simulation games and serious games (SGs) in a learning context. A first operationalisation of the model was used to comparatively evaluate a suite of 14 SGs on varying topics played between 2004 and 2009 in 13 institutes of higher education in the Netherlands. The questions in this research were: what is the perceived learning effectiveness of the games and what factors explain it? How can we comparatively evaluate games for learning? Data were gathered through pre- and post-game questionnaires among 1000 students, leading to 500 useful datasets and 230 complete datasets for analysis (factor analysis, scaling, t-test and correlation analysis) to give an explorative, structural model. The findings are discussed and a number of propositions for further research are formulated. The conclusion of the analysis is that the students’ motivation and attitudes towards game-based learning before the game, their actual enjoyment, their efforts during the game and the quality of the facilitator/teacher are most strongly correlated with their learning satisfaction. The degree to which the experiences during the game were translated back into the underlying theories significantly determines the students’ learning satisfaction. The quality of the virtual game environment did not matter so much. The authors reflect upon the general methodology used and offer suggestions for further research and development.

From the observations of successful entertainment games, it is hypothesized that implementing a single-player option may require a different approach from that in a multiplayer option, in terms of game design. To find out whether this could be true and to understand what the specific approaches could entail for educational games, three separate yet related investigations were conducted to examine single- versus multiplayer games: an investigation of two educational games designed and evaluated by the authors, a theoretical investigation from a game and learning perspective, and an empirical investigation of 23 case studies. From these three investigations, it turned out that a “single-player approach” is data intensive, has formal rules, and uses direct transfer and individual learning. On the other hand, a “multiplayer approach” is less straightforward. From a game perspective, it can be characterized as process intensive and having social rules. When related to learning, however, it could be positioned on any dimension. This exploration shows that the approaches differ to a large extent and that designers have to find a fit between what option they choose and approach they take.

Background. Our world today is made up of complex, socio-technical systems like large infrastructure systems, global logistic chains and information networks. Simulation games represent one way to enable participation and governance of such systems. Aim. The current study explores the development of mental models and shared experiences of professionals and students through a simulation game. Method. The simulation game SIMPORT-MV2 focused on the development of a new port area in the Port of Rotterdam, The Netherlands. Insights in the perceived development of mental models were measured through questionnaires and through observations in the debriefing. Likewise, observations in the debriefing were obtained for measurements of shared experiences. Results. Results showed that participants perceived positive learning on their declarative, procedural and strategic knowledge (mental model) about the port project. A significant difference was found for both the overall perceived knowledge and the strategic knowledge between students and professionals. Furthermore, qualitative findings showed that players are enabled to probe designs of, to gain insights into, and to develop an understanding of complex systems in a safe environment through shared experiences. Conclusions. This study shows that it is possible to design a simulation game that represents such a complex socio-technical system based on a layered approach of agent, network and system behavior.