In writing assessment, expert human evaluators ideally judge individual essays with attention to variance among writers’ syntactic patterns. There are many ways to compose text successfully or less successfully. For automated writing evaluation (AWE) systems to provide accurate assessment and relevant feedback, they must be able to consider similar kinds of variance. The current study employed natural language processing (NLP) to explore variance in syntactic complexity and sophistication across clusters characterized in a large corpus (n = 36,207) of middle school and high school argumentative essays. Using NLP tools, k-means clustering, and discriminant function analysis (DFA), we observed that student writers employed four distinct syntactic patterns: (1) familiar and descriptive language, (2) consistently simple noun phrases, (3) variably complex noun phrases, and (4) moderate complexity with less familiar language. Importantly, each pattern spanned the full range of writing quality; there were no syntactic patterns consistently evaluated as “good” or “bad”. These findings support the need for nuanced approaches in automated writing assessment while informing ways that AWE can participate in that process. Future AWE research can and should explore similar variability across other detectable elements of writing (e.g., vocabulary, cohesion, discursive cues, and sentiment) via diverse modeling methods.

Writing assessment is a ubiquitous component of education, job seeking and promotion, everyday communication, and more. To serve these crucial activities, automated writing evaluation (AWE) technologies have been developed to facilitate and streamline assessment and feedback by providing faster, more consistent, and ostensibly less biased judgments than human evaluators. This chapter synthesizes insights from computer science, sociolinguistics, social psychology, and ethics to (a) highlight the potential for bias inherent to the building and implementation of AWE along with barriers to resolving those issues, and (b) discuss research on bias mitigation strategies. This chapter is intended to propel future research toward more equitable development and use of AWE. <!----> <!----> <!----> <!---->

Introduction: Student retention is a critical issue in higher education. Universities have responded by implementing supports like early alert systems. Objective: We investigated students’ knowledge of and experiences with an early alert system designed to enhance academic persistence. Method: We surveyed (N = 356) undergraduates at a large public university in the U.S. The survey was researcher-created and administered online. Participants self-selected into the study and provided informed consent prior to beginning the study. Data were coded by experts, who achieved excellent IRR. The analyses were frequency-based to understand diverse student knowledge, experiences, and responses with early alert systems. Results: Students commonly reported experiencing negative emotions after receiving an alert, but also reported that receiving an alert motivated them to increase their course attendance, improve their study habits, and participate more in class. Finally, students indicated that receiving an early alert facilitated supportive interactions with instructors, though student communication with academic advisors was less frequent. Student recommendations for system improvement included using positive language in alerts and providing actionable guidance. Conclusions: These results provide new insight into student views of early alert systems and suggest that these systems can positively impact students in need of support.