The Likert item response format for items is almost ubiquitous in the social sciences and has particular virtues regarding the relative simplicity of item-generation and the efficiency for coding responses. However, in this article, we critique this very common item format, focusing on its affordance for interpretation in terms of internal structure validity evidence. We suggest an alternative, the Guttman response format, which we see as providing a better approach for gathering and interpreting internal structure validity evidence. Using a specific survey-based example, we illustrate how items in this alternative format can be developed, exemplify how such items operate, and explore some comparisons between the results from using the two formats. In conclusion, we recommend usage of the Guttman response format for improving the interpretability of the resulting outcomes. Finally, we also note how this approach may be used in tandem with items that use the Likert response format to help balance efficiency with interpretability. Translational Abstract The Likert item response format for items, which features a stem statement, and a series of simple alternatives for the respondent (usually Strongly Agree to Strongly Disagree) is almost ubiquitous in the social sciences and has particular virtues regarding the relative simplicity of item-generation and the efficiency for coding responses. However, in this article, we critique this very common item format, focusing on its affordance for interpretation in terms of internal structure validity evidence, that is, whether there is evidence in the response data that supports the underlying structure of the latent variable being measured. We suggest an alternative, the Guttman response format, which we see as providing a better approach for gathering and interpreting this internal structure validity evidence. Using a specific survey-based example, we illustrate how items in this alternative format can be developed, exemplify how such items operate, and explore some comparisons between the results from using the two formats. In conclusion, we recommend usage of the Guttman response format for improving the interpretability of the resulting outcomes. Finally, we also note how this approach may be used in tandem with items that use the Likert response format to help balance efficiency with interpretability.

This paper describes a hypothesized learning progression for how secondary students understand interdependent relationships in ecosystems, a key concept in the field of ecology and for public understanding of science. In this study, a hypothetical learning progression was developed and empirically investigated using Rasch modeling of data from 1366 students in a large, diverse, urban school district. We found that the empirical evidence supported the general structure of the hypothesized learning progression for relationships in ecosystems. There were notable exceptions, and we describe the ways in which we altered the items and the learning progression to address empirical inconsistencies with our a priori conceptions. The assessment items developed through this study are immediately available online for formative assessment purposes, and the learning progression can support teachers’ thinking about students’ understanding of ecosystems. In particular, the upper reach of the learning progression offers a more complete description of the ways in which students might develop their understanding of complex interactions in ecosystems, beyond what is currently offered in the literature and standards documents about students’ understanding.

The authentic research experience, which provides students with meaningful collaborative research opportunities designed to promote discovery and innovation under the guidance of mentors, is increasing as a way to attract and engage students in STEM fields. However, despite the increase in authentic research experiences offered to students, there has been little research, particularly at the high school level, investigating students’ attitudes about themselves as researchers. To address this need, we developed a theory (or construct) for how high school age students self-identify as researchers and a companion survey to measure their identity. After three iterative development cycles, 823 high school students from diverse backgrounds were administered the 12-item survey, the Researcher Identity Survey—Form G (RISG). The partial credit Rasch model (1960/1980) was used to analyze the survey data. The results indicate that the survey identifies and locates high school age students as researchers validly and reliably along an easy to use and interpret scale. The survey holds promise as an important element for use in programs designed to broaden the entryway for students into the STEM disciplines.

Learning progressions built and based on empirical evidence provide a foundation for advancing the state of the art of teaching. This paper contributes to the advancement of the idea that assessment can provide empirical evidence to examine the underlying theory of cognition and provides a method for developing and verifying the theory. Using a construct within the science domain of Structure of Matter as a basis for discussion, issues of how to address theoretical ambiguities and practical dilemmas are explored.