As a branch of artificial intelligence, automated speech recognition (ASR) technology is increasingly used to detect speech, process it to text, and derive the meaning of natural language for various learning and assessment purposes. ASR inaccuracy may pose serious threats to valid score interpretations and fair score use for all when it is exacerbated by test takers’ characteristics, such as language background and accent, and assessment task type. The present study investigated the extent to which speech-to-text accuracy rates of three major ASR systems vary across different oral tasks and students’ language background variables. Results indicate that task types and students’ language backgrounds have statistically significant main and interaction effects on ASR accuracy. The paper discusses the implications of the study results for applying ASR to computerized assessment design and automated scoring.
Lifestyle choices and consumption play a large role in contributing to per capita greenhouse gas emissions. Certain activities, like fossil fuel ground transportation, long-haul flights, diets with animal products and residential heating and cooling contribute significantly to per capita emissions. There is uncertainty around whether literacy about these actions encourages individuals to act pro-environmentally to reduce personal carbon footprints or to prioritize the most effective actions. This study investigated the relationship between carbon literacy and pro-environmental actions performed to reduce greenhouse gas emissions among undergraduate engineering students at the University of Toronto. The pro-environmental actions by the participants produced an average carbon footprint of 4.8 tCO2 (within the subset of actions included in the survey) which was lower than the average for residents each of Toronto, Ontario, and Canada overall but still higher than the global target of ∼2.8 tCO2e. The carbon literacy by participants was best for high impact actions like ground transportation and dietary choices but less so for air travel and there was mixed awareness for the moderate and low impact actions. For high impact actions and many moderate and low impact actions, participants who thought the action was high impact (even if incorrect) had lower carbon footprints related to the associated activity than those who thought the action was moderate or low impact. The overall relationship between pro-environmental action and carbon literacy was weak. It showed that for high impact actions, there is a slight negative correlation between carbon literacy and personal carbon footprint whereas for moderate and low impact actions, there is a positive correlation.
While the automated assessment of oral reading fluency (ORF) using accuracy and speech rate has proliferated, expressiveness of speech, as measured by prosodic features, has been neglected due to its inherent complexity and lack of technological resources. Despite the potential benefits of burgeoning technology for assessing hard-to-measure constructs such as ORF, insensitivity to linguistic diversities threatens valid score interpretations and fair use for all learners. The present study investigated the potential benefits of developing an automated, prosody-inclusive ORF assessment in a post-secondary education setting involving many English language learners (ELLs). The analysis focused on three ways the inclusion of prosody may improve automated ORF assessments: by reducing bias against ELLs, improved prediction of reading comprehension, and improved diagnostic information. Data were analyzed by comparing two different scoring outcomes, the traditional ORF measure and a new prosody-inclusive outcome score, comparing these measures across language background. Results showed that the inclusion of prosody improves automated ORF assessment by reducing discrepancies between ELLs and English first language students caused by automated speech recognition inaccuracies, leads to better prediction of reading comprehension with ELLs, and provides meaningful diagnostic information. Detailed descriptions of the models, their relevance, and implications for the language testing community are discussed.
As international students increasingly pursue higher education in English‐dominant countries, developing their academic writing skills is crucial. However, limited access to individualized feedback remains a challenge. AI‐driven tools and self‐assessment offer promising solutions, making feedback more accessible. This study involved 50 international graduate students who spoke English as an additional language, randomly assigned to two groups: one received BERT‐generated automated diagnostic scores and feedback, while the other engaged in self‐assessment. Using a sequential explanatory mixed‐methods design, this study investigated the effects of automated diagnostic feedback and self‐assessment on students' academic writing performance and self‐perceived writing abilities. It also explored how cognitive, metacognitive, behavioral, and affective engagement with feedback varied across the two groups. Results indicated that the machine feedback group significantly outperformed the self‐assessment group in task fulfillment, organization, and total writing scores, while no significant differences were observed for vocabulary and grammar. Additionally, the machine feedback group demonstrated deeper metacognitive, behavioral, and affective feedback engagement. However, they also reported a decline in academic writing self‐confidence. While automated diagnostic feedback proved more effective than self‐assessment in enhancing academic writing skills, its potential negative impact on students' confidence highlighted the need for future research to balance precision with emotional support.