There is a consensus that serious games have a significant potential as a tool for instruction. However, their effectiveness in terms of learning outcomes is still understudied mainly due to the complexity involved in assessing intangible measures. A systematic approach—based on established principles and guidelines—is necessary to enhance the design of serious games, and many studies lack a rigorous assessment. An important aspect in the evaluation of serious games, like other educational tools, is user performance assessment. This is an important area of exploration because serious games are intended to evaluate the learning progress as well as the outcomes. This also emphasizes the importance of providing appropriate feedback to the player. Moreover, performance assessment enables adaptivity and personalization to meet individual needs in various aspects, such as learning styles, information provision rates, feedback, and so forth. This paper first reviews related literature regarding the educational effectiveness of serious games. It then discusses how to assess the learning impact of serious games and methods for competence and skill assessment. Finally, it suggests two major directions for future research: characterization of the player’s activity and better integration of assessment in games.

Serious Games represent an acknowledged potential for instruction, because they are able to strongly motivate learners. They can also provide immersive environments where advanced users can practice knowledge and skills, also exploiting multimodal interaction. They can combine the effectiveness of computer processing and data storage, with high levels of attractiveness. Our work has investigated the state of the art research on SGs, starting from the cognitive aspects, that are necessary in order to root technological development and applications in sound theoretical foundations. The paper discusses some key aspects about SG design and exploitation: choice of components-off-the-shelf or from-scratch design, tools and methodologies for development or adaptation, intelligent tutoring, virtual coaches and affective learning, living worlds, game mechanics, Human-Computer Interaction. While several SGs have been developed, still the literature stresses a lack of significant, extensive user tests. Further research is necessary to investigate in greater detail the real effectiveness of the various types of SGs. The paper proposes several research questions - that range from requirements elicitation to design and from deployment to use and evaluation - to be answered in order to avoid technology pushing and drive technological research according to the requirements of the end-users and stakeholders. We believe that deepening the analysis about these issues is key to strengthen the foundations of SG research, for which we identify four major directions: definition of metrics and learning progress evaluation tools; methodologies and tools for designing games from various topics and for various users; computing and communication architectures; technologies that can enhance the overall system performance.

Passive brain-computer interaction (BCI) can provide useful information to understand a user's state and anticipate intentions, which is needed to support adaptivity and personalization. Given the huge variety of audiences, a game's capability of adapting to different user profiles-in particular to keep the player in flow-is crucial to make it ever more enjoyable and satisfying. We have performed a user experiment exploiting a four-electrode electroencephalogram (EEG) tool similar to the ones that are soon likely to appear in the market for game control. We have performed a spectral characterization of the video-gaming experience, also in comparison with other tasks. Results show that the most informative frequency bands for discriminating among gaming conditions are around low beta. Simple signals from the peripheral nervous system add marginal information. Classification of three levels of user states is possible, with good accuracy, using a support vector machine (SVM) classifier. A user-independent classification performs worse than a user-dependent approach (50.1% versus 66.4% rate). Personalized SVM training and validation time is reasonable (7-8 min). Thus, we argue that a personalized system could be implemented in a consumer context and research should aim at improving classifiers that can be trained online by end users.

Serious games present a promising opportunity for learning, but the genre still lacks methodologies and tools for efficient and low-cost production, particularly for teacher and domain experts. This article gives an authoring framework that aims to provide structured support, from content design to final implementation. In particular, we have abstracted a conceptual model—the SandBox Serious Game - which relies on a generalization of task-based learning theory. The model invites players to perform cognitive tasks contextually while exploring information-rich virtual environments. We consider it particularly suited for cultural heritage entertainment applications. The model defines games that are set in realistic virtual worlds enriched with embedded educational tasks, which we have implemented as minigames. This approach simplifies the authoring work, which can easily be supported by visual authoring tools for ontology-based urban 3D modeling and implementation tasks, thus allowing an approach similar to the mind-maps concept. We propose a top-down methodology for content preparation, starting from a city-level analysis down to the single points of interest and associated tasks, which are instances of simple predefined minigame/quiz typologies. We provide examples and discuss criteria for selecting task typologies according to the authors' cognitive targets. Finally, we discuss the results of a user test, which took place in a lab, aimed at verifying the acquisition of cultural heritage knowledge in a pleasant and engaging way. Games appear particularly suited for supporting the study of images, especially of iconography. Compared to reading text, a game forces the player to focus more strongly on problems, which favors knowledge acquisition and retention. Learning complex concepts requires an investigative attitude, which can be spurred by well-designed games. Good design involves usability, graphic appeal, appropriate content, and the presence of connections which a player must discover in the content. Players should be asked to pay attention to and reason about their whole game activity - including the relationships between the game content, the brief introduction, and concluding texts. More comprehensive tests are needed to better investigate the educational effectiveness—however, the first results are promising, especially in terms of user motivation and creation of new opportunities for learning about CH.