Although the attainment gap between black and minority ethnic (BME) students and white students has persisted for decades, the potential causes of these disparities are highly debated. The emergence of learning analytics allows researchers to understand how students engage in learning activities based on their digital traces in a naturalistic setting. This study investigates the attainment gap by analysing the differences in behavioural engagement between different ethnic groups. Using multilevel models of academic performance, demographics, and online traces of 149,672 students enrolled in 401 modules in a distance learning setting, we confirmed the existing attainment gap. After controlling for other demographics, module characteristics and engagement, BME students were between 19% and 79% less likely to complete, pass or achieve an excellent grade compared to white students. Given the same academic performance, BME students spent 4-12% more time on studying than white students. While the attainment gap remained persistent after controlling for academic engagement, our study further highlighted the inequality of attainment between BME and white students.
Extensive research in learning science has established the importance of time management in online learning. Recently, learning analytics (LA) has shed further lights on the temporal characteristics of learning by allowing researchers to capture authentic digital footprints of student learning behaviours. Nonetheless, students’ timing of engagement and its relation to learning design (LD) and academic performance have received limited attention. This study investigates to what extent students’ timing of engagement aligned with instructor learning design, and how engagement varied across different levels of performance. Our findings revealed a mismatch between how instructors designed for learning and how students study. In most weeks, students spent less time studying the assigned materials on the virtual learning environment (VLE) compared to the number of hours recommended by instructors. The timing of engagement also varied, from in advance to catching up patterns. High-performing students spent more time studying in advance, while low-performing students spent a higher proportion of their time on catching-up activities. By incorporating the pedagogical context into learning analytics, not only we can understand what, why, and when students engage, but also how their behaviours are influenced by the way instructors design for learning.
Modern machine learning increasingly supports paradigms that are multi-institutional (using data from multiple institutions during training) or cross-institutional (using models from multiple institutions for inference), but the empirical effects of these paradigms are not well understood. This study investigates cross-institutional learning via an empirical case study in higher education. We propose a framework and metrics for assessing the utility and fairness of student dropout prediction models that are transferred across institutions. We examine the feasibility of cross-institutional transfer under real-world data- and model-sharing constraints, quantifying model biases for intersectional student identities, characterizing potential disparate impact due to these biases, and investigating the impact of various cross-institutional ensembling approaches on fairness and overall model performance. We perform this analysis on data representing over 200,000 enrolled students annually from four universities without sharing training data between institutions. We find that a simple zero-shot cross-institutional transfer procedure can achieve similar performance to locally-trained models for all institutions in our study, without sacrificing model fairness. We also find that stacked ensembling provides no additional benefits to overall performance or fairness compared to either a local model or the zero-shot transfer procedure we tested. We find no evidence of a fairness-accuracy tradeoff across dozens of models and transfer schemes evaluated. Our auditing procedure also highlights the importance of intersectional fairness analysis, revealing performance disparities at the intersection of sensitive identity groups that are concealed under one-dimensional analysis.1