Fabian Kieser

Fabian Kieser

Auto-generated profile, based on publicly available information

Fabian Kieser

Department of Physics Education Research, Free University of Berlin, Berlin, Germany

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
25 October 2023 in Physical Review Physics Education Research
Verified
Journal Article
Open access
PDF file
FKFabian Kieser
Fabian Kieser
PWPeter Wulff
Peter Wulff
Jochen KuhnJochen Kuhn
Jochen Kuhn
Stefan KüchemannStefan Küchemann
Stefan Küchemann
Generative AI technologies such as large language models show novel potential to enhance educational research. For example, generative large language models were shown to be capable of solving quantitative reasoning tasks in physics and concept tests such as the Force Concept Inventory (FCI). Given the importance of such concept inventories for physics education research, and the challenges in developing them such as field testing with representative populations, this study seeks to examine to what extent a generative large language model could be utilized to generate a synthetic dataset for the FCI that exhibits content-related variability in responses. We use the recently introduced ChatGPT based on the GPT 4 generative large language model and investigate to what extent ChatGPT could solve the FCI accurately (RQ1) and could be prompted to solve the FCI as if it were a student belonging to a different cohort (RQ2). Furthermore, we study, to what extent ChatGPT could be prompted to solve the FCI as if it were a student having a different force- and mechanics-related preconception (RQ3). In alignment with other research, we found that ChatGPT could accurately solve the FCI. We furthermore found that prompting ChatGPT to respond to the inventory as if it belonged to a different cohort yielded no variance in responses, however, responding as if it had a certain preconception introduced much variance in responses that approximate real human responses on the FCI in some regards.

13 August 2025 in Physical Review Physics Education Research
Verified
Journal Article
Open access
PTPaul Tschisgale
Paul Tschisgale
HMHolger Maus
Holger Maus
FKFabian Kieser
Fabian Kieser
BKBen Kroehs
Ben Kroehs
SPStefan Petersen
Stefan Petersen
PWPeter Wulff
Peter Wulff
Large language models (LLMs) are now widely accessible, reaching learners across all educational levels. This development has raised concerns that their use may circumvent essential learning processes and compromise the integrity of established assessment formats. In physics education, where problem solving plays a central role in both instruction and assessment, it is therefore essential to understand the physics-specific problem-solving capabilities of LLMs. Such understanding is key to informing responsible and pedagogically sound approaches to integrating LLMs into instruction and assessment. This study therefore compares the problem-solving performance of a general-purpose LLM ( G P T − 4 o , using varying prompting techniques) and a reasoning-optimized model ( o 1 ) with that of participants in the German Physics Olympiad, based on a set of well-defined Olympiad problems. In addition to evaluating the correctness of the generated solutions, the study analyzes the characteristic strengths and limitations of LLM-generated solutions. The results of this study indicate that both tested LLMs ( G P T − 4 o and o 1 ) demonstrate advanced problem-solving capabilities on Olympiad-type physics problems, on average outperforming the human participants. Prompting techniques had little effect on G P T − 4 o ’s performance, and o 1 almost consistently outperformed both G P T − 4 o and the human benchmark. The main implications of these findings are twofold: LLMs pose a challenge for summative assessment in unsupervised settings, as they can solve advanced physics problems at a level that exceeds top-performing students, making it difficult to ensure the authenticity of student work. At the same time, their problem-solving capabilities offer potential for formative assessment, where LLMs can support students in evaluating their own solutions to problems.

1 January 2026 in The Physics Educator
Verified
Journal Article
Closed access
HMHolger Maus
Holger Maus
FKFabian Kieser
Fabian Kieser
SPStefan Petersen
Stefan Petersen
PWPeter Wulff
Peter Wulff
PTPaul Tschisgale
Paul Tschisgale