This article provides an introduction to structural equation modelling (SEM) for language assessment research in five sections. In Section I, the general objectives of SEM applications relevant to language assessment are presented. In Section II, a brief overview of SEM that considers the methodology and the statistical assumptions about data that have to be met. In Section III, the commonly used steps and concepts in SEM are presented. In Section IV, SEM application matters with example models are discussed. In Section V, recent critical discussions and some directions for future SEM applications in language assessment research are addressed.

This study investigated the application of WriteToLearn on Chinese undergraduate English majors’ essays in terms of its scoring ability and the accuracy of its error feedback. Participants were 163 second-year English majors from a university located in Sichuan province who wrote 326 essays from two writing prompts. Each paper was marked by four human raters as well as WriteToLearn. Many-facet Rasch measurement (MFRM) was conducted to calibrate WriteToLearn’s rating performance in scoring the whole set of essays against those of four trained human raters. The accuracy of WriteToLearn’s feedback on 60 randomly selected essays was compared with the feedback provided by human raters. The two main findings related to scoring were that WriteToLearn was more consistent but highly stringent relative to the four trained human raters in scoring essays and that it failed to score 7 essays. In terms of error feedback, WriteToLearn had an overall precision and recall of 49% and 18.7% respectively. These figures did not meet the minimum threshold of 90% precision for it to be a reliable error detecting tool set by Burstein, Chodorow, and Leacock (2003). Furthermore, it had difficulty in identifying the errors made by Chinese undergraduate English majors in the use of articles, prepositions, word choice and expression.

Although the United States has had a language requirement for citizenship (through the naturalization process) since the first decade of the 20th century, very little research and discussion has taken place in academic circles about this requirement, which has been enforced for the past two decades through a test and testing practice to which millions of citizenship applicants have been subjected. Overall, the requirement (and the test) is said to promote “civic integration,” “political allegiance,” “social cohesion,” and/or “social harmony.” The main question addressed in this article is whether the requirement (and the test), its purpose, content, administration, and consequence provide the impetus for citizenship applicants to integrate or develop a civic nationalism. I will apply the Test Context Framework ( Kunnan, 2008 Kunnan, A. J. Towards a model of test evaluation: Using the Test Fairness and Wider Context frameworks. Papers from the ALTE Conference. Berlin. Multilingualism and Assessment: Achieving transparency, assuring quality, sustaining diversity, Edited by: Taylor, L. and Weir, C. pp.229–251. Cambridge, , U.K: Cambridge University Press. [Google Scholar] ) to the U.S. Naturalization Test and testing practice. This framework argues for the examination of the wider context of testing including the political, economic, and legal contexts in which a test is deployed for a fuller understanding of the testing practice.

Computer technology made its way into writing instruction and assessment with spelling and grammar checkers decades ago, but more recently it has done so with automated essay evaluation (AEE) and diagnostic feedback. And although many programs and tools have been developed in the last decade, not enough research has been conducted to support or evaluate the claims of the developers. This study examined the effectiveness of automated writing instructional programs in consistent scoring of essays and appropriate feedback to student writers. It examined the scoring and instructional program called MY Access! Home Edition, which has an error feedback tool called My Editor to address these issues. The study combined a quantitative study of agreement and correlational analyses with an analysis of content and topic. Participants included 114 English language learners who wrote 147 essays to three writing prompts, which were graded by trained EFL raters and MY Access. From the sample, 15 randomly selected essays were also used for an error analysis comparing My Editor with human annotations to examine My Editor’s accuracy. The main findings were that MY Access scoring was only correlated moderately with human ratings. Furthermore, because MY Access scoring is limited to the recognition of content words, not how these words are organized at the discourse level, it did not detect slightly off-topic essays and plagiarism. Finally, My Editor’s error feedback, with 73% precision and 30% recall, did not meet the expectations of an accurate tool. In conclusion, the home edition of MY Access was not found to be useful as an independent instructional tool. These findings give us pause regarding the effectiveness of MY Access.

Evaluating Language Assessments offers a comprehensive overview of the theoretical bases and research methodologies for the evaluation of language assessments and demonstrates the importance of a fuller understanding of this widely used evaluative tool. The volume explores language assessment evaluation in its wider political, economic, social, legal, and ethical contexts while also illustrating quantitative and qualitative methods through discussions of key research studies. Suitable for students in applied linguistics, second language acquisition and language assessment and education, this book makes the case for a clear and rigorous understanding of the theoretical and methodological underpinnings of language assessment evaluation in order to achieve fair assessments and just institutions.

This study investigates the relationships between test takers’ metacognitive and cognitive strategy use through a questionnaire and their test performance on an English as a Foreign Language reading test. A total of 593 Chinese college test takers responded to a 38-item metacognitive and cognitive strategy questionnaire and a 50-item reading test. The data were randomly split into two samples (N = 296 and N = 297). Based on relevant literature, three models (i.e., unitary, higher order, and correlated) of strategy use and test performance were hypothesized and tested to identify the baseline model. Further, cross-validation analyses were conducted. The results supported the invariance of factor loadings, measurement error variances, structural regression coefficients, and factor variances for the unitary model. It was found that college test takers’ strategy use affected their lexico-grammatical reading ability significantly. Findings from this study provide empirical and validating evidence for Bachman and Palmer's (2010) model of strategic competence.