Knowledge tracing (KT) involves utilizing historical data from students’ learning interactions to model their mastery of knowledge over time, with the aim of predicting their future performance in interactions. Recently, significant advancements have been achieved through the application of various deep learning methodologies to address the KT challenge. However, a considerable proportion of deep learning-based knowledge tracing (DLKT) approaches exhibit striking similarities in their methodologies, and model designs, and even the outcomes demonstrate minimal divergence. In addition, the evaluation procedures employed in current DLKT studies are not standardized, resulting in substantial inconsistencies in the reported area under the curve (AUC) outcomes, despite analyzing the same model on identical datasets. To address the two aforementioned problems, this paper proposes a generalized DLKT framework and represents the existing DLKT models with five components, i.e., multimodal data encoder, student knowledge memory, auxiliary knowledge base, learning outcome objective, and computational efficiency and scalability. Furthermore, we develop and open source a standardized DLKT benchmark platform named pyKT,1 that consists of a standardized set of integrated data preprocessing procedures on 9 popular datasets across different domains, and 21 frequently compared DLKT model implementations. With pyKT, we conduct empirical and reproducible research to assess the performance of prevalent DLKT algorithms in an unbiased and clear setting over multiple data sources. Finally, we discuss the applications of KT techniques in the educational sector and their future development directions.
Measuring digital literacy (DL) across ages and tracking its growth over time have remained challenging in the area of digital literacy assessment. The current analysis reports on the psychometric properties of a performance-based Digital Literacy Assessment (DLA) instrument grounded in the DigComp 2.1 framework. Utilising a longitudinal cohort study design, the DLA was administered to Hong Kong students across three age cohorts, from lower primary to upper secondary, over a two-year period. Data were collected in 2019, before the COVID-19 pandemic, and in 2021, during the pandemic. The analysis provides validity and reliability evidence for using the DLA in longitudinal studies to assess DL from late childhood through late adolescence. The results further suggest that students’ DL improved with grade level, with secondary students outperforming primary students but also displaying greater variability in scores. Over the two years, students generally demonstrated improvement in DL while inter-individual differences in DL growth rates widened. These findings indicate the widening of digital divides and highlight the need to investigate factors that contribute to diversity in DL development. In conclusion, our study provides evidence for the robustness of the DLA as an instrument to assess DL growth across ages and over time. Further, the DLA allowed us to uncover the substantial overlap in DL ability across different age groups and the widening second-level digital divide as children move into higher grades, and that the digital divide aggravated during the COVID-19 pandemic. Implications and challenges to the learning and assessment of DL are discussed.
25 April 2023 in Journal of Educational and Behavioral Statistics
To expand the use of cognitive diagnosis models (CDMs) to longitudinal assessments, this study proposes a bias-corrected three-step estimation approach for latent transition CDMs with covariates by integrating a general CDM and a latent transition model. The proposed method can be used to assess changes in attribute mastery status and attribute profiles and to evaluate the covariate effects on both the initial state and transition probabilities over time using latent (multinomial) logistic regression. Because stepwise approaches generally yield biased estimates, correction for classification error probabilities is considered in this study. The results of the simulation study showed that the proposed method yielded more accurate parameter estimates than the uncorrected approach. The use of the proposed method is also illustrated using a set of real data.
Background: With increasing digital engagement, children and adolescents face both opportunities and risks online. While exposure to online risks does not always lead to harm, digital resilience (DR)—the ability to cope with and recover from negative online experiences—mitigates potential adverse effects. Despite its importance, validated tools to assess DR in young populations are scarce. We developed and validated a multidimensional Digital Resilience Scale that assesses coping strategies, bouncing back, and personal growth following online risk exposure. Methods: Guided by the UK Council for Internet Safety (UKCIS) Digital Resilience Framework (DRF), we validated the scale with 2,014 primary and 6,014 secondary school students in Hong Kong. The instrument comprises two subscales: (1) Coping Strategies (productive coping, reference to others, non-productive coping) across three common online risks (unwanted messages, personal-information leakage, cyberbullying); and (2) Recovery (bouncing back, growth). We conducted confirmatory factor analyses (CFAs), tested measurement invariance across gender and school level, and examined external validity with digital literacy (DL) and well-being. Results: CFAs supported a five-factor structure (three coping factors, bouncing back, growth) with good fit (CFI = 0.967–1; RMSEA = 0.00 – 0.046). Subscales showed high reliability (ω = 0.764 – 0.910). Measurement invariance across gender and school level was supported. DR correlated positively with DL ( r = 0.036 – 0.254) and well-being ( r = 0.042 – 0.659), supporting external validity. Conclusions: The Digital Resilience Scale provides a reliable, multidimensional measure of students’ capacity to navigate and recover from online risks. It offers a practical tool for educators, researchers, and policymakers to assess DR and inform targeted interventions. Future research should evaluate cross-cultural generalizability and longitudinal sensitivity.