23 March 2024 in Creativity Research Journal
15 May 2024 in The Journal of Creative Behavior
Automated scoring is a current hot topic in creativity research. However, most research has focused on the English language and popular verbal creative thinking tasks, such as the alternate uses task. Therefore, in this study, we present a large language model approach for automated scoring of a scientific creative thinking task that assesses divergent ideation in experimental tasks in the German language. Participants are required to generate alternative explanations for an empirical observation. This work analyzed a total of 13,423 unique responses. To predict human ratings of originality, we used XLM‐RoBERTa (Cross‐lingual Language Model‐RoBERTa), a large, multilingual model. The prediction model was trained on 9,400 responses. Results showed a strong correlation between model predictions and human ratings in a held‐out test set (n = 2,682; r = 0.80; CI‐95% [0.79, 0.81]). These promising findings underscore the potential of large language models for automated scoring of scientific creative thinking in the German language. We encourage researchers to further investigate automated scoring of other domain‐specific creative thinking tasks.
7 October 2023 in PsyArXiv
Creative problem-solving is a naturalistic form of creative thinking involving the generation of solutions that are not only original but also of high quality (i.e., plausible and effective). Naturalistic tasks that evaluate both originality and quality are vital for the promotion of creativity in real-world settings—yet scoring such tasks remains challenging, due to costly human labor required to manually rate task responses. Past work has shown that large language models (LLMs) can be trained to predict human originality ratings of responses to tests of divergent thinking. In the present research, we extend this work to creative problem-solving, examining whether both originality and quality can be automatically scored for a naturalistic creativity task. We gathered data from 10 studies, amounting to 3,235 participants who completed a creative problem-solving task (CPST). We then fine-tuned two open-source LLMs, RoBERTa and GPT-2, to predict human ratings of originality and quality on the CPST, and compared their performance to two other scoring methods: elaboration (i.e., word count) and semantic distance. We found that RoBERTa and GPT-2 models predict solution quality (RoBERTa, r = .83; GPT-2, r = .83) better than solution originality (RoBERTa, r = .79; GPT-2, r = .80). Moreover, we found that both models outperformed elaboration and semantic distance methods and generalized to new CPST items not present in their training set. We therefore show for the first time that naturalistic creativity tasks can be automatically scored for both originality and quality. Open access is provided to the models and training data.
2 January 2025 in Creativity Research Journal
Successful problem-solving and enhanced creative ability may improve physical health, cognitive well-being, and overall independence of older adults. In general, older adults who are more creative, may be better able to cope with cognitive decline and navigate everyday tasks. While previous research on creative performance in older adulthood showed age-related stability, open questions remain regarding the specific underlying cognitive basis for this invariability across the lifespan. Mediation analyses showed that intelligence measures served as underlying cognitive mechanisms for the stability of creative thinking in older age. The broader implications of these findings provide insight into the complex relationships supporting age-related preservation in creativity.
8 December 2024 in Thinking Skills and Creativity
10 March 2025 in PsyArXiv
As generative artificial intelligence (genAI) becomes increasingly integrated into education and work, understanding who benefits most from human-AI collaboration is crucial. This study examines how domain expertise and individual differences—creative self-efficacy and baseline creative ability—influence human-AI co-creativity in an engineering design task. Using pre-generated ideas from GPT-3.5-turbo, engineering (N = 99) and psychology students (N = 212) generated an initial solution, evaluated AI-generated ideas, and revised their idea. Linear mixed-effects models demonstrated expertise and generation ability predicted solution quality. Engineering students produced more original and effective solutions, yet both groups improved comparably supporting the “rising tides lift all boats” hypothesis. A novel categorization scheme revealed group differences in idea inspiration: engineers generated more novel solutions, while psychology students tended to adopt existing ideas. These findings highlight the role of domain knowledge and individual differences in pre-existing creativity in maximizing human-AI co-creativity, emphasizing the need to develop these human abilities alongside genAI.