This study proposes a framework that leverages natural language processing and unsupervised machine learning techniques to measure, identify, and classify examinees’ writing strategies. The framework integrates three categories of writing strategies (text complexity, evidence use, and argument structure) to identify the characteristics of examinees’ writing. Additionally, a measurement model is used to calibrate examinees’ writing proficiency. An empirical example is presented to demonstrate the performance of the framework. The data comprise 430 Grade 8 examinees’ responses to English Language Arts (ELA) assessments in the United States. Using K-means clustering, distinct patterns were identified in each category. The one-parameter logistic measurement model was applied to estimate examinees’ writing proficiency. Analyses revealed significant effects of text complexity and evidence use on writing proficiency, while argument structure was not significant. This study has implications for writing instruction and assessment design that highlight the point that effective writing is not simply a matter of isolated skill acquisition, but rather the coordinated implementation of complementary strategies, a finding that supports cognitive developmental theories of writing.
7 August 2025 in Journal of Educational Measurement
Constructed responses (CRs) within testlets are widely used to assess complex skills but can pose calibration challenges due to local item dependence. A few current testlet models incorporate testlet‐specific effects to address local dependence but struggle with interpreting these effects and may not fully capture the complexities of CR items because they rely only on response or score patterns. A Topic Testlet Model (TTM) integrates topic modeling within a psychometric framework was proposed. It uses latent topics from student written responses to adjust for local dependence, enable simultaneous calibration, and provide insights into evaluating student reasoning and writing in testlet CR items. Using empirical data from both English Language and Arts as well as Science assessments for grades 3‐12, we compare the TTM with existing models in terms of ability estimates, item parameter estimates, and overall model fit. Simulation studies further demonstrate parameter recovery under various testing scenarios. Results show that the TTM effectively accounts for local dependence, improves testlet effect interpretability, and demonstrates a better fit than the existing models. TTM advances CR testlet calibration, leveraging additional information from student written responses to improve the precision of the assessment systems and validity of the use of test scores.
Constructed-response items offer rich evidence of writing proficiency, but the linguistic signals they contain vary with grade level. This study presents a cross-sectional analysis of 5638 English Language Arts essays from Grades 6–12 to identify which linguistic features predict proficiency and to characterize how their importance shifts across grade levels. We extracted a suite of lexical, syntactic, and semantic-cohesion features, and evaluated their predictive power using an interpretive dual-model framework combining LASSO and XGBoost algorithms. Feature importance was assessed through LASSO coefficients, XGBoost Gain scores, and SHAP values, and interpreted by isolating both consensus and divergences of the three metrics. Results show moderate, generalizable predictive signals in Grades 6–8, but no generalizable predictive power was found in the Grades 9–12 cohort. Across the middle grades, three findings achieved strong consensus. Essay length, syntactic density, and global semantic organization served as strong predictors of writing proficiency. Lexical diversity emerged as a key divergent feature, it was a top predictor for XGBoost but ignored by LASSO, suggesting its contribution depends on interactions with other features. These findings inform actionable, grade-sensitive feedback, highlighting stable, diagnostic targets for middle school while cautioning that discourse-level features are necessary to model high-school writing.
Urban logistics systems face growing delivery demand and complex traffic and operational constraints, which make unmanned delivery carriers, including unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs), a promising solution. Existing studies typically focus on a single delivery carrier type and rely on idealized assumptions, overlooking heterogeneous cooperation under multiple stations, multiple time windows, and real-world transport conditions. To address these gaps, we propose the Multi-Station UAV–UGV Cooperative Delivery Scheduling Problem with Temporally Discontinuous Service Availability (MSUUCDSP) to minimize the total travel and waiting time of UAVs and UGVs. To solve the problem, we propose a mixed-integer linear programming (MILP) model with a novel mathematical approach and a Hybrid Large Neighborhood Search (HLNS) algorithm. Additionally, we adopt a Hidden Markov Model (HMM)-based map-matching method and big data techniques to capture realistic operational characteristics. Computational experiments are conducted on various realistic instances under four diverse scenarios. Results show that UAV–UGV cooperation significantly improves efficiency, reducing total time cost by 17.12% compared with single-mode delivery, and they reveal substantial discrepancies between idealized assumptions and realistic scenarios. We further develop an ArcGIS-based simulation to support practical implementation. The findings provide valuable insights for decision-making and engineering applications for logistics operators.
Mixed-format assessments, which include both multiple-choice (MC) and constructed-response (CR) items, often produce separate scoring scales, with MC items scored dichotomously and CR items scored polytomously. Conventional methods for estimating composite ability scores, such as weighting or summing, rely on subject matter expertise but overlook the information embedded in MC item scores. While recent progress takes advantage of empirical Bayes analysis for estimating composite ability scores, it may also introduce biases because it relies solely on point estimates without accounting for the variability in unknown ability distributions and other parameters. To address these gaps, this study introduces a practical and easily implementable method, empirical fully Bayesian, that leverages MC item scores to derive empirical priors, leading to more accurate composite score estimates. It has been found that MC scores have effectively captured students’ achievement in the assessment domain, and they could provide valuable information for final scoring. Through empirical analyses of students in Grades 3 to 10 and two additional simulation studies based on real-world data, we demonstrate that this approach enhances composite ability score reliability, reduces reporting biases, and provides a valuable empirical evaluation tool for mixed-format assessments.
The accurate measurement of individual cognitive abilities in educational settings increasingly relies on personalized assessment systems that adapt to the learner’s current standing on the measured ability. These systems generate and use assessment items continuously and in real-time, bypassing the field-test cycle that traditional calibration requires. Item banks therefore contain a growing share of items whose parameters must be predicted first, and each test form must be assembled in real time to match the learner’s current ability level and content needs for effective personalized instruction. This study proposes an Ising framework that addresses both requirements by predicting item parameters from text with calibrated uncertainty estimates and assembling personalized test forms within the time budget of the assessment loop. This framework is validated through a factorial simulation and an empirical study on state-level English Language Arts items across Grades 3 through 12. In the simulation, the proposed uncertainty-aware method achieves a mean test information function (TIF) gap of 0.226, compared with 1.171 for the conventional baseline that ignores prediction uncertainty. The proposed method also produces feasible forms across the full operational region, whereas the chance-constrained and robust baselines become infeasible at high prediction error combined with a high share of predicted items. In the empirical study, predictive intervals achieve coverage within ten percentage points of the nominal level in the pooled condition, and the uncertainty-aware assembly reduces the TIF gap relative to the baseline, with the largest gains at the extremes of the ability distribution where accurate measurement matters most for instructional placement decisions. By separating prediction uncertainty from true individual differences in ability, this framework preserves the interpretability of cognitive ability estimates and provides a foundation for personalized assessment systems that support differentiated instruction under continuous item bank growth.