Background: Courses in engineering often use peer evaluation to monitor teamwork behaviors and team dynamics. The qualitative peer comments written for peer evaluations hold potential as a valuable source of formative feedback for students, yet little is known about their content and quality.Purpose: This study uses a large language model (LLM) to apply a previously tested feedback quality rubric to peer feedback comments. Our research questions interrogate the reliability of LLMs for qualitative analysis with a rubric and use Bandura's self‐regulated learning theory to assess peer feedback quality of first‐year engineering students' comments.Method: An open‐source, local LLM was used to score each comment according to four rubric criteria. Inter‐rater reliability (IRR) with human raters using Cohen's quadratic weighted kappa was the primary metric of reliability. Our assessment of peer feedback quality utilized descriptive statistics.Results: The LLM achieved lower IRR than human raters, but the model's challenges mimic those of human raters. The model did achieve an excellent quadratic weighted kappa of 0.80 for one rubric criterion, which shows promise for LLM capability. For feedback quality, students generally wrote low‐ to medium‐quality comments that were infrequently grounded in specific teamwork behaviors. We identified five types of peer feedback that inform how students perceive the feedback process.Conclusions: Our implementation of GAI suggests that LLMs can be helpful for rapid iteration of research designs, but consistent and reliable analysis with generative artificial intelligence (GAI) requires significant effort and testing. To develop feedback literacy, students must understand how to provide high‐quality feedback.
Educational institutions are grappling with declining enrollments and low mathematical achievements. This study investigates how a math placement exam (ALEKS) influences enrollment in Calculus 1 and student persistence, taking into account academic preparation and demographic factors. It also evaluates the effects of remedial math courses for students near the placement cutoff. Using Astin’s input–environment–outcome model, this study analyzed data from 3380 students employing a Kitagawa-Oaxaca-Blinder decomposition and fuzzy regression discontinuity. These methods were used to identify unexplained differences across demographic groups and capture outcomes near the math placement cutoff. Based on the findings, a cutoff of 80% for the ALEKS exam is appropriate. This study underscores the role of math placement exams in shaping engineering enrollment and student success. These findings prompt reevaluating placement strategies and support mechanisms, particularly for URM, first-generation, and female students, to enhance equity and retention in engineering.
16 November 2026 in Journal of Engineering Education
Background: The sequential structure of the engineering curriculum, along with the central role of math, creates an interconnected system of prerequisite and corequisite requirements and course sequencing, resulting in curricula with high structural complexity in which one course can influence student pathways. Considering complexity and math requirements side‐by‐side can illuminate levers for curricular change. Purpose: This paper aims to quantify and compare structural complexity and math cruciality (proportion of structural complexity due to math requirements) across engineering disciplines and institutions, illuminate the role of mathematics in the curriculum, and examine how institutional characteristics (e.g., size, acceptance rate) and program characteristics (e.g., ranking, graduates) shape these patterns. Method: We collected curriculum plans and calculated structural complexity scores and math cruciality for three engineering majors (civil, mechanical, and electrical) across 48 institutions. Results: Our results showed that Mechanical Engineering had a higher structural complexity score compared to Civil Engineering. Despite disciplinary differences, the math cruciality was ~30% across disciplines, and beyond the core calculus sequence, institutions and programs vary in their math requirements. Additionally, regression and cluster analysis revealed that lower‐ranked institutions often exhibit higher structural complexity scores. Conclusions: Despite differences in structural complexity scores and required math courses, math cruciality varied little across institutions in the study. Programs can rethink course sequencing, prerequisite chains, and math requirements to reduce unnecessary structural barriers and create more flexible pathways in engineering.