In line with the positive effects of personalized learning, personalized assessments are expected to maximize learner motivation and engagement, allowing learners to show what they truly know and can do. Considering the advances in Generative Artificial Intelligence (GenAI), in this perspective article, we elaborate on the opportunities of integrating GenAI into personalized educational assessments to maximize learner engagement, performance, and access. We also draw attention to the challenges of integrating GenAI into personalized educational assessments regarding its potential risks to the assessment’s core values of validity, reliability, and fairness. Finally, we discuss possible solutions and future directions.
12 March 2020 in Educational Measurement: Issues and Practice
Students exhibit many behaviors when responding to items on a computer-based test, but only some of these behaviors are relevant to estimating their proficiencies. In this study, we analyzed data from computer-based math achievement tests administered to elementary school students in grades 3 (ages 8–9) and 4 (ages 9–10). We investigated students’ response process data, including the total amount of time they spent on an item, the amount of time they took to first respond to an item, the number of times they “visited” an item and the number of times they changed their responses to items, in order to explore whether these behaviors were related to overall proficiency and whether they differed across item formats and grades. The results indicated a non-linear relationship between the mean number of actions and proficiency, as well as some notable interactions between correctly answering an item, item format, response time, and response time latency. Implications for test construction and future analyses in this area are discussed.
21 January 2022 in Educational Measurement: Issues and Practice
In educational settings, students rely on metacognitive processes to determine whether or not to exert effort. We investigated ways to minimize cognitively disengaged responses (i.e., not-fully-effortful responses) during a low-stakes mathematics assessment. Initially, we established theory-driven time thresholds for each item to detect such responses. We then administered the test to 800 eighth-graders across three conditions: (a) control (n = 271); (b) instruction (n = 267); and (c) nudge (n = 262). In the instruction condition, students were told to exert their best effort before starting the assessment. In the nudge condition, students were prompted to give their best effort following each first-attempt response that was both incorrect and not-fully-effortful. Therefore, students had multiple opportunities to adjust their level of effort. Nudges, but not effort instruction, significantly reduced students’ not-fully-effortful responses. Neither the nudges nor the effort instruction significantly impacted performance. In a post-test survey, most students reported that they received nudges whenever they did not know the answer (55%). Overall, these findings suggest that while nudges reduce cognitively disengaged responses, most students appear to strategically modulate their level of effort based on self-monitoring their knowledge and response effort.