Researchers have increasingly investigated sources of variance in reading comprehension test scores, particularly with students with reading difficulties (RD). The purpose of this meta-analysis was to determine if the achievement gap between students with RD and typically developing (TD) students varies as a function of different reading comprehension response formats (e.g., multiple choice, cloze). A systematic literature review identified 82 eligible studies. All studies administered reading comprehension assessments to students with RD and TD students in Grades K–12. Hedge’s g standardized mean difference effect sizes were calculated, and random effects robust variance estimation techniques were used to aggregate average weighted effect sizes for each response format. Results indicated that the achievement gap between students with RD and TD students was larger for some response formats (e.g., picture selection ES g = −1.80) than others (e.g., retell ES g = −0.60). Moreover, for multiple-choice, cloze, and open-ended question response formats, single-predictor metaregression models explored potential moderators of heterogeneity in effect sizes. No clear patterns, however, emerged in regard to moderators of heterogeneity in effect sizes across response formats. Findings suggest that the use of different response formats may lead to variability in the achievement gap between students with RD and TD students.
Educators use reading comprehension assessments to summarize academic achievement, make decisions in diagnostic evaluations, and identify intervention needs. A challenge, however, with using different assessments in practice is that student performance may change depending on which assessment is administered. This article guides educators in evaluating student performance across multiple assessments, specifically when making decisions for students with learning disabilities (LD) in reading. First, this article provides educators with guidance in establishing a student’s foundational skills, specifically those that may contribute to low performance on reading comprehension assessments. Next, the article presents steps for examining the texts and assessment methods commonly used in measures of reading comprehension. The article concludes with recommendations for evaluating student performance when considering a student’s foundational skills and characteristics of reading comprehension assessments.