Student motivation to write is a pivotal factor influencing their writing achievement. However, individual motivation to write is not independent of the learning environment. It also is crucial for teachers to develop their own efficacy, knowledge, and ability in writing and writing instruction to help them utilize effective instructional methods that stimulate students' motivation to write and further promote their writing achievement. Given these considerations, we utilized a two-level hierarchical linear model to examine the relationships among student motivation, teacher personal and professional traits, teacher writing instruction, and writing achievement at student and teacher levels. Our analysis of the dataset, which included 346 fourth and fifth graders nested within 41 classrooms, found that motivation had a positive predictive effect on writing ability at both student and teacher levels. Moreover, female students, fifth graders, and typically achieving students demonstrated higher writing achievement than their counterparts. While there were no significant effects of teacher efficacy, knowledge, ability, or professional development on student writing achievement, we observed that higher frequency of classroom management practices during writing instruction had a significant negative effect on student writing achievement. Our full model revealed that the relationship between student motivation and achievement was negatively moderated by teachers' increased use of instructional practices related to process features and using writing instruction materials, but positively moderated by increased use of varied teaching tactics. Overall, our findings emphasize the importance of contextual factors in understanding the complexity of student writing achievement and draw attention to the need for effective instructional practices to support students' writing development.
The primary purpose of this study is to investigate the degree to which register knowledge, register-specific motivation, and diverse linguistic features are predictive of human judgment of writing quality in three registers—narrative, informative, and opinion. The secondary purpose is to compare the evaluation metrics of register-partitioned automated writing evaluation models in three conditions: (1) register-related factors alone, (2) linguistic features alone, and (3) the combination of these two. A total of 1006 essays ( n = 327, 342, and 337 for informative, narrative, and opinion, respectively) written by 92 fourth- and fifth-graders were examined. A series of hierarchical linear regression analyses controlling for the effects of demographics were conducted to select the most useful features to capture text quality, scored by humans, in the three registers. These features were in turn entered into automated writing evaluation predictive models with tuning of the parameters in a tenfold cross-validation procedure. The average validity coefficients (i.e., quadratic-weighed kappa, Pearson correlation r, standardized mean score difference, score deviation analysis) were computed. The results demonstrate that (1) diverse feature sets are utilized to predict quality in the three registers, and (2) the combination of register-related factors and linguistic features increases the accuracy and validity of all human and automated scoring models, especially for the registers of informative and opinion writing. The findings from this study suggest that students’ register knowledge and register-specific motivation add additional predictive information when evaluating writing quality across registers beyond that afforded by linguistic features of the paper itself, whether using human scoring or automated evaluation. These findings have practical implications for educational practitioners and scholars in that they can help strengthen consideration of register-specific writing skills and cognitive and motivational forces that are essential components of effective writing instruction and assessment.
3 November 2024 in British Journal of Educational Psychology
Background: Learning to write the complex academic language (AL) associated with a discipline (like science) is a critical task in education, with middle school being a key developmental period. However, we need more research to guide how we assess students' learning to write AL, especially if we want to create assessment that guides more effective instruction. Aims: We evaluated middle school students' informational writing for six different measures of AL to determine which ones were most strongly related to writing quality and were most indicative of the unique features of informational writing. We also examined which metrics were sensitive to growth across middle school. Sample: Our sample consists of informational compositions from 285 students in Grade 5 ( n = 175) and Grade 8 ( n = 110) in a Midwestern state in the United States. Methods: Path modelling was used to determine the degree to which the AL metrics are associated with writing quality and narrativity in each grade. Results: Overall, the six measures of AL explained 70% of the variance in students' writing quality. We found that a new measure, number of long words, outperformed other more established measures at the word level and should be used in assessment of informational writing quality. We also found that automated scores at the sentence level better detected development across middle school grade levels than typical rubrics of writing quality. Conclusion: Results provide promising avenues for the assessment of malleable aspects of AL in middle school informational writing.
Background: Learning to write the complex academic language associated with a discipline (like science) is a critical task in education, with middle school being a key developmental period. However, we need more research to guide how we assess students’ learning to write academic language, especially if we want to create assessment that guides more effective instruction.Aims: We evaluated middle school students’ informational writing for six different measures of academic language to determine which ones were most strongly related to writing quality and were most indicative of the unique features of informational writing. We also examined which metrics were sensitive to growth across middle school.Sample: Our sample consists of informational compositions from 285 students in Grade 5 (n = 175) and Grade 8 (n = 110) in a Midwestern state in the United States. Methods: Path modeling was used to determine the degree to which the academic language metrics are associated with writing quality and narrativity in each grade. Results: Overall, the six measures of academic language explained 70% of the variance in students’ writing quality. We found that a new measure, number of long words, outperformed other more established measures at the word level and should be used in assessment of informational writing quality. We also found that automated scores at the sentence level better detected development across middle school grade level than typical rubrics of writing quality. Conclusion: Results provide promising avenues for the assessment of malleable aspects of academic language in middle school informational writing.
24 November 2025 in Disciplinary and Interdisciplinary Science Education Research
The complex and interdisciplinary nature of scientific concepts presents formidable challenges for students in developing their knowledge-in-use skills. The utilization of computerized analysis for evaluating students’ contextualized constructed responses offers a potential avenue for educators to develop personalized and scalable interventions, thus supporting the current teaching and learning of science. While prior research in artificial intelligence has demonstrated the effectiveness of algorithms, including Bidirectional Encoder Representations from Transformers (BERT), in tasks like automated classifications of constructed responses, these efforts have predominantly leaned towards text-level features, often overlooking the exploration of conceptual ideas embedded in students’ responses from a cognitive perspective. Despite BERT’s performance in downstream tasks, challenges may arise in domain-specific tasks, particularly in establishing knowledge connections between specialized and open domains. These challenges become pronounced in small-scale and imbalanced educational datasets, where the available information for fine-tuning is frequently inadequate to capture task-specific nuances and contextual details. The primary objective of the present study is to investigate the effectiveness of a pretrained language model, when integrated with an ontological framework aligned with a contextualized science assessment, in classifying students’ expertise levels in scientific explanation. Our findings indicate that while pretrained language models, such as BERT, contribute to enhanced performance in language-related tasks within educational contexts, the incorporation of identifying domain-specific terms and extracting and substituting with their associated sibling terms in sentences through ontology-based systems can significantly improve classification model performance. Further, we qualitatively examined student responses and found that, as expected, the ontology framework identified and substituted key domain-specific terms in student responses that led to more accurate predictive scores. The study explores the practical implementation of ontology in assessment evaluation to facilitate formative assessment and formulate instructional strategies.