Introduction: Despite the necessity for adults with lower literacy skills to undergo and succeed in high-stakes computer-administered assessments (e.g., GED, HiSET), there remains a gap in understanding their engagement with digital literacy assessments.Methods: This study analyzed process data, specifically time allocation data, from the Program for the International Assessment of Adult Competencies (PIAAC), to investigate adult respondents’ patterns of engagement across all proficiency levels on nine digital literacy items. We used cluster analysis to identify distinct groups with similar time allocation patterns among adults scoring lower on the digital literacy assessment. Finally, we employed logistic regression to examine whether the groups varied by demographic factors, in particular individual (e.g., race/ethnicity, age) and contextual factors (e.g., skills-use at home).Results: Adults with lower literacy skills spent significantly less time on many of the items than adults with higher literacy skills. Among adults with lower literacy skills, two groups of time allocation patterns emerged: one group (Cluster 1) exhibited significantly longer engagement times, whereas the other group (Cluster 2) demonstrated comparatively shorter durations. Finally, we found that adults who had a higher probability of Cluster 1 membership (spending more time) exhibited relatively higher literacy scores, higher self-reported engagement in writing skills at home, were older, unemployed, and self-identified as Black.Discussion: These findings emphasize differences in digital literacy engagement among adults with varying proficiency levels. Additionally, this study provides insights for the development of targeted interventions aimed at improving digital literacy assessment outcomes for adults with lower literacy skills.
Purpose: College placement assessments often overlook multilingual learners’ full linguistic abilities and literacy engagement, as standardized tests primarily assess English proficiency rather than how students interact with academic texts. Directed Self-Placement (DSP) offers an alternative approach through self-assessment, with some models incorporating post-task self-ratings of students’ competence beliefs. However, this approach does not fully capture how motivation (i.e., competence beliefs and task value) interacts with varying literacy skills within a task. This exploratory study applies Explanatory Item Response Models (EIRMs) to examine how self-rated motivation relates to vocabulary performance on higher-and lower-frequency words, offering insights to refine DSP frameworks for multilingual learners.Method: A total of 39 multilingual learners and 249 monolingual English-speaking college students completed a vocabulary assessment and responded to self-report motivation scale questions assessing their reading motivation. Item-level analyses were implemented to examine the interaction between motivation and multilingual learner status on higher-and lower-frequency words.Results: Lower-frequency words posed greater challenges across all participants but disproportionately undermined the performance of multilingual learners with lower reading competence beliefs. However, multilingual learners with higher reading competence performed comparably to monolingual English speakers on both higher and lower frequency words.Conclusion: EIRM-based analyses offer novel insights into the ways that motivation interacts with different dimensions of literacy performance at the item level. Future research should develop validated self-assessment tools that incorporate additional aspects of motivation and multilingual learners’ linguistic strategies, which could inform more equitable placement practices.
1 August 2024 in Zeitschrift für Weiterbildungsforschung
This study uses Program for the International Assessment of Adult Competencies (PIAAC) data to examine latent profiles of engagement in reading, numeracy, writing, and computer skills-use for U.S. adults with low literacy skills. Additionally, we examine how these profiles relate to overall literacy performance and demographic heterogeneity. Results indicate five engagement profiles: High (8%), Average-High (44%), Average-Low (40%), and two generally Low profiles that are most differentiated on numeracy skills-use engagement (5.5%, 2%). High and Average-High engagement profiles had higher probabilities of adults scoring at a higher literacy level than the Average-Low profile and to a greater degree than the lowest engagement profile. Covariates of learning disability status, English speaker status, race, ethnicity, age, and educational attainment were also related to differences in literacy performance and skills-use engagement. Employment status was unrelated to differences in literacy performance and skills-use engagement. Generally, native English speakers, non-learning disabled adults, younger adults, those identifying as non-Hispanic and White, and adults with higher educational attainment (high school/some college and college degree or higher) had higher probabilities of higher literacy performance, particularly in the High and Average-High engagement profiles. Findings have implications for increasing skills-use engagement in daily life to increase literacy performance and suggest differences by demographic characteristics that may be of interest to adult foundational education programs.
Technological advances have transformed the landscape of educational assessments, particularly in how we collect and analyze data on individuals’ cognitive processes and interactions with assessment items. The rich data recorded in log files during human–machine interactions are often referred to as process data. This study uses sequential process data from the 2012 Program for the International Assessment of Adult Competencies (PIAAC), focusing specifically on how respondents navigate an interactive numeracy item (“Map”) that involves spatial cognition. The objectives of this study are trifold: (1) to explore factors that contribute to success or failure on a spatial numeracy item, (2) to identify spatial cognitive process features across high and low numeracy performance levels, and (3) to introduce a novel time-embedded n-grams model to incorporate elapsed time with sequential actions in process data analysis. Using a sample of 596 U.S. adult respondents, we employed the time-embedded n-grams model and two machine learning methods, random forest and XGBoost, to predict respondents’ numeracy performance and to identify robust classifiers. Results indicate that time-related features and understanding directions in the spatial dimensions are the most predictive of adults’ numeracy skills. The findings highlight the potential of process data to support analyses of latent numeracy skills and cognitive processes in educational contexts.