The ability to assess learning outcomes is vital to effective teaching. Without understanding what students have learned, it is impossible to tailor information, tasks or feedback adequately to their individual needs. Thus, assessment literacy has been increasingly recognized as a core teacher competency in educational research, with many empirical studies investigating teachers’ abilities, knowledge and subjective views in relation to classroom assessment. In contrast, relatively few studies have focused on students’ perspectives of assessment. This is surprising, since gathering students’ feedback on their teachers’ assessment practices seems a logical step toward improving those practices. To help fill this gap, we present an explorative study using the recently developed Fairness Barometer as a tool to help identify specific strengths and weaknesses in individual teachers’ assessment methods. Viewing assessment through the lens of classroom justice theory, the Fairness Barometer asks students and teachers to rate aspects of procedural and informational justice in their own (teachers’) assessment practices. We examined the resulting fairness discrepancy profiles for 10 Austrian secondary school classes (177 students). Results showed wide variation in profile pattern, evidence that both students and teachers can differentiate between different aspects of assessment fairness. Further exploration of the resulting discrepancy-profiles revealed certain problem types, with some teachers differing from their students’ perception in almost every rated aspect, some showing specific assessment-related behaviors that require improvement (e.g., explaining grading criteria of oral exams), and others demonstrating almost identical responses as their students to the addressed fairness aspects. Results clearly indicate the potential of the Fairness Barometer to be used for teacher training and teacher self-development within the domain of teacher assessment literacy.

Background: The increasing capabilities of generative artificial intelligence (AI), exemplified by OpenAI’s transformer-based language model GPT-4 (ChatGPT), have drawn attention to its application in educational contexts. This study evaluates the potential of such models in generating German reading comprehension texts for educational large-scale assessments, within the multilingual context of Luxembourg. Addressing the challenges faced by item developers in sourcing or manually developing numerous suitable texts, the study aims to determine if ChatGPT can assist text creation while maintaining high-quality standards. Methods: The study employed a mixed-methods approach. In a qualitative focus group discussion, experts identified the strengths, weaknesses, opportunities and threats (SWOT) of using GPT-4 for text generation. These insights informed the construction of a Text Analysis Cognitive Model (TACM), which served as theoretical foundation. Narrative and informative reading comprehension texts were then generated using two distinct prompt engineering techniques, derived from original passages and TACM specifications. In a blinded online review, N = 89 participants evaluated human-written and AI-generated texts with regard to their readability, correctness, coherence, engagement and adequacy for reading assessment. Results: All administered texts were of similarly high quality, with reviewers being unable to consistently identify authorship origins. Quantitative evaluations indicated that one-shot prompts are effective for creating high-quality informative texts, whereas human-written texts remain superior for narratives. Zero-shot prompts offer considerable flexibility and creativity, but still require human refinement. Conclusion: These findings offer promising first insights into GPT-4’s capacity to emulate human-written texts which can be used in the large-scale assessment context. The considerable potential of using generative AI-models as a flexible and efficacious assistant in the creation of reading comprehension texts is highlighted. Still, the necessity of human oversight is emphasized through an augmented intelligence-driven perspective. Given the jurisdictional framework of the European Union, an effective implementation of ChatGPT in the test development process remains hypothetical at this time but is likely to change.

This paper gives an example of how computer-based problem-solving scenarios can be embedded in a course on sustainability, in order to illustrate the highly versatile way in which such scenarios can be used to structure and evaluate learning on complexity on an individual level, as well as learning in diverse groups. After defining criteria, a computer-based problem-solving scenario has to meet in order to be useful for training competencies associated with confronting complexity, the application of one specific scenario, the Genetics Lab, is empirically evaluated on base of three student cohorts. Results suggest that existing approaches to sustainability education can be substantially complemented by computer-based problem-solving scenarios, offering genuine learning opportunities and deepening and personalizing the comprehension of known phenomena in complex problem-solving. The paper closes by offering lessons learned from the presented approach and gives advice and outlook on future applications of such scenarios in sustainability education.

Teaser Webbasierten Selbsttest zur Studienfachwahl. Zusammenfassung Bachelor- und Masterstudiengänge erfordern Neuerungen in Beratung von Studieninteressierten bei der Studien(fach)wahl. Immer mehr Hochschulen optimieren die Passung zwischen Studier...

In this rejoinder, we respond to two commentaries on the study by Greiff, S.; Stadler, M.; Sonnleitner, P.; Wolff, C.; Martin, R. Sometimes less is more: Comparing the validity of complex problem solving measures. Intelligence 2015, 50, 100–113. The study was the first to address the important comparison between a classical measure of complex problem solving (CPS) and the more recent multiple complex systems (MCS) approach regarding their validity. In the study, we investigated the relations between one classical microworld as the initially developed method (here, the Tailorshop) with three more recently developed multiple complex systems (MCS; here, MicroDYN, Genetics Lab, and MicroFIN) tests. We found that the MCS tests showed higher levels of convergent validity with each other than with the Tailorshop even after reasoning was controlled for, thus empirically distinguishing between the two approaches. The commentary by Kretzschmar and the commentary by Funke, Fischer, and Holt expressed several concerns with how our study was conducted, our data was analyzed, and our results were interpreted. Whereas we acknowledge and agree with some of the more general statements made in these commentaries, we respectfully disagree with others, or we consider them to be at least partially in contrast with the existing literature and the currently available empirical evidence.

Diagnostic classification models (DCMs) assess students’ mastery of cognitive attributes to provide personalized ability profiles. Retrofitting DCMs to large‐scale mathematics assessments usually relies on inferred Q‐matrices, which can reduce accuracy and diagnostic value. This study evaluated whether constructing items from cognitive models—yielding Q‐matrices directly—and incorporating hierarchical relationships among attributes improve diagnostic outcomes. Responses from 5,336 third‐grade students to a Luxembourgish image‐based, large‐scale standardized mathematics exam were analyzed using multiple DCMs and their hierarchical extensions. Items were constructed based on a Q‐matrix, derived from the curriculum and cognitive models. The hierarchical A‐CDM outperformed other models, classifying students into 60 latent classes with acceptable attribute‐ and test‐level accuracy and more interpretable results than the G‐DINA model. Using cognitive model‐based item generation and Q‐matrices as well as specifying attribute hierarchies enhance the accuracy and interpretability of DCM‐based diagnostics in large‐scale assessments, complementing traditional psychometric approaches by discerning meaningful within‐score differences.