The present study extended research on the effectiveness of automated writing evaluation (AWE) systems. Sixth graders were randomly assigned by classroom to an AWE condition that used Project Essay Grade Writing (n = 56) or a word-processing condition that used Google Docs (n = 58). Effectiveness was evaluated using multiple metrics: writing self-efficacy, holistic writing quality, performance on a state English language arts test, and teachers' perceptions of AWE's social validity. Path analyses showed that after controlling for pretest measures, composing condition had no effect on holistic writing quality, but students in the AWE condition had more positive writing self-efficacy and better performance on the state English language arts test. Posttest writing self-efficacy partially mediated the effect of composing condition on state test performance. Teachers reported positive perceptions of AWE's social validity. Results emphasize the importance of using multiple metrics and considering both contextual factors and AWE implementation methods when evaluating AWE effectiveness.

This study examined a naturalistic, districtwide implementation of an automated writing evaluation (AWE) software program called MI Write in elementary schools. We specifically examined the degree to which aspects of MI Write were implemented, teacher and student attitudes towards MI Write, and whether MI Write usage along with other predictors like demographics and writing self-efficacy explained variability in students’ performance on a proximal and distal measure of writing performance. The participants included 1935 students in Grades 3–5 and 135 writing teachers from 14 elementary schools in a mid-Atlantic school district. Findings indicated that though MI Write was somewhat under-utilized, teachers and students held positive attitudes towards the AWE system. Usage of MI Write had a mixed and limited predictive effect on outcomes: The number of essays written had a small predictive effect on state test performance for Grades 3 and 5; gain on revision had a moderate predictive effect on posttest writing quality and a small predictive effect for Grade 5 state test performance. Students’ average AWE scores showed consistently moderate to large predictive effects for all outcomes. Interpreted in light of the underlying architecture of MI Write, findings have implications for other school districts considering implementing AWE as well as the design of AWE systems intended to support the teaching and learning of writing.

This systematic literature review examines how artificial intelligence (AI)-integrated tools are being used to scaffold writing instruction for K-12 English as a Foreign Language (EFL) students, with a particular focus on preserving student agency and creativity. Drawing from sociocultural and cognitive theories of writing, this study synthesizes findings from 14 peer-reviewed empirical studies published between 2018 and 2024. Studies were analyzed using a thematic synthesis approach to identify how AI tools function as cognitive supports, creative collaborators, and language enhancement systems. The results reveal that AI tools, when implemented thoughtfully, can reduce cognitive load, foster ideation, and support self-regulated learning without undermining student autonomy. Three major categories of AI scaffolding emerged, i.e., cognitive support tools (e.g., grammar correction, idea organization), creative support systems (e.g., story generation, narrative feedback), and language enhancement tools (e.g., vocabulary expansion, stylistic refinement). Key themes include the importance of balancing AI support with student decision making, aligning tool use with theoretical models, and ensuring equitable access. Limitations in the current literature include short-term study durations, a lack of standardized metrics for creativity and agency, and the predominance of research in EFL contexts. Future research should expand on ESL settings, adopt rigorous methodologies, and explore long-term impacts on writing development.

We used multivariate generalizability theory to examine the reliability of hand-scoring and automated essay scoring (AES) and to identify how these scoring methods could be used in conjunction to optimize writing assessment. Students ( n = 113) included subsamples of struggling writers and non-struggling writers in Grades 3–5 drawn from a larger study. Students wrote six essays across three genres. All essays were hand-scored by four raters and an AES system called Project Essay Grade (PEG). Both scoring methods were highly reliable, but PEG was more reliable for non-struggling students, while hand-scoring was more reliable for struggling students. We provide recommendations regarding ways of optimizing writing assessment and blending hand-scoring with AES.