In this paper, we present a Tabletop Role-Playing game (TTRPG) driven by ChatGPT. Prompts are employed to instruct ChatGPT to act as Game Masters (GMs). In crafting each prompt to integrate a distinctive role, three roles denoted as Role 1, Role 2, and Role 3, are established. Subsequently, we perform pre-game and post-game emotional assessments employing the Positive and Negative Affect Schedule (PANAS) questionnaire to scrutinize players’ emotional dynamics throughout the gaming experience. Upon analyzing the collected data, we observe that Role 1 and Role 2 affect players’ positive emotions. Notably, Role 2 exhibits the most pronounced influence on players’ positive emotions. Our findings demonstrate that a TTRPG GM powered by ChatGPT can significantly enhance players’ positive emotions. This leads us to recognize that TTRPG GM powered by ChatGPT plays a positive role in enhancing the mental well-being of specific populations.
This paper presents BenchING, a new benchmark for evaluating large language models (LLMs) on their ability to follow structured output format instructions in text-based procedural content generation (PCG) tasks. The ability to condition LLMs to output in specified formats proves useful, as downstream components in LLM-integrated games often require structured outputs for exchanging information. However, there is a gap in evaluating this aspect of LLMs, especially in narrative PCG tasks, making it difficult to select LLMs and design games or applications integrating these LLMs. To demonstrate the potential of our benchmark, we evaluate nine LLMs for their ability to generate parseable formatted outputs using five selected text-based PCG tasks. We report on the performance of these LLMs on these tasks. Additionally, we categorize more detailed error types and propose solutions by utilizing LLMs to fix these errors. We also conduct a scaling study, investigating an emergent point of LLMs for their ability to fix malformed formatted content using eight quantized LLMs with varying original sizes from 0.62B to 72.3B. Furthermore, we perform a qualitative study to assess the quality of the generated content. We make our source code and raw data available for future research.
This paper analyzes the experience of first-person shooter (FPS) players when game difficulty is adjusted by personalizing their audio cue settings, considering the balance between player performance, modeled using Gaussian process regression (GPR), and prior data serving as designer preference. In addition, we investigate why player experience changes according to in-game audio cues. Previous studies have proposed various dynamic difficulty adjustment (DDA) methods for FPS games. However, few studies have considered the role of audio cues in the player experience. This paper compares the player experience of personalized enemy audio cue volume settings (GPR-DDA) with that of predetermined settings in an FPS game. Two comprehensive experimental phases, involving 80 participants, are conducted to assess the efficacy of GPR-DDA. The experience of our players is measured using questions taken from the Game User Experience Satisfaction Scale (GUESS) questionnaire and a final survey asking for their open-ended feedback. A large language model (LLM) is used to analyze the natural language expressions of the players according to their native languages. To ensure the LLM effectively assists a limited number of qualified human evaluators in classifying player responses, we have developed an original procedure for this task. The GUESS results show that GPR-DDA can improve the player experience. In addition, the high consistency in the classification results over multiple runs of the selected LLM, as well as the similarity between its results and those of our human evaluators, reflects the reliability of the proposed LLM-assisted procedure.
This paper analyzes the experience of first-person shooter (FPS) players when the game difficulty is adjusted by personalizing their audio cue settings, considering the balance between player performance, modeled using Gaussian Process Regression (GPR), and prior data serving as designer preference. In addition, we investigate the reason why player experience changes according to in-game audio cues. Previous studies have proposed various dynamic difficulty adjustment (DDA) methods for FPS games. However, few studies have considered the role of audio cues in the player experience. This paper compares the player experience of personalized enemy audio cue volume setting (GPR-DDA) and that of predetermined setting in an FPS game. Two comprehensive experimental phases, involving 80 participants, are conducted to assess the efficacy of GPR-DDA. The experience of our players is measured using questions taken from the Game User Experience Satisfaction Scale (GUESS) questionnaire and a final survey asking their open-ended feedback. A large language model (LLM) is used to analyze the natural language expressions of the players according to their native languages. To effectively have an LLM assist a limited number of qualified human evaluators in the classification of player responses, we develop an original procedure for this task. The GUESS results show that GPR-DDA can improve the player experience, and, in addition, the high consistency in the classification results over multiple runs of the selected LLM, as well as the similarity between its results and those of our human evaluators, reflect the reliability of the proposed LLM-assisted procedure.