21 January 2025 in Physical Review Physics Education Research
In physics education research, instructors and researchers often use research-based assessments (RBAs) to assess students’ skills and knowledge. In this paper, we support the development of a mechanics cognitive diagnostic to test and implement effective and equitable pedagogies for physics instruction. Adaptive assessments using cognitive diagnostic models provide significant advantages over fixed-length RBAs commonly used in physics education research. As part of a broader project to develop a cognitive diagnostic assessment for introductory mechanics within an evidence-centered design framework, we identified and tested the student models of four skills that cross content areas in introductory physics: apply vectors, conceptual relationships, algebra, and visualizations. We developed the student models in three steps. First, we based the model on learning objectives from instructors. Second, we coded the items on RBAs using the student models. Finally, we then tested and refined this coding using a common cognitive diagnostic model, the deterministic inputs, noisy “and” gate model. The data included 19 889 students who completed either the Force Concept Inventory, Force and Motion Conceptual Evaluation, or Energy and Momentum Conceptual Survey on the LASSO platform. The results indicated a good to adequate fit for the student models with high accuracies for classifying students with many of the skills. The items from these three RBAs do not cover all of the skills in enough detail, however, they will form a useful initial item bank for the development of the mechanics cognitive diagnostic. Published by the American Physical Society 2025
21 November 2025 in Gifted Child Quarterly
Despite the rich insights they offer, open-ended survey responses remain underutilized in educational research, often due to a lack of awareness of emergent methods that can efficiently analyze such data. This methodological brief addresses this gap by introducing the topic modeling technique, particularly the latent Dirichlet allocation, as a tool for analyzing open-ended responses and facilitating the investigation of relationships between textual responses and measurable outcomes. This brief outlines the rationale behind topic modeling and details the implementation process through the analysis of open-ended responses on students’ career interests as an example. In addition, we discuss the advantages and limitations of this technique and provide practical considerations for its application in educational research.