In this study, we present a case study involving two self-service dashboards providing feedback on learning and study skills and on academic achievement. These dashboards were offered to first-year university students in several study programmes in Flanders, Belgium. Data for this study were collected using usage tracking (N = 2875) and a survey taken at the beginning of the second year before (N = 484) and after (N = 538) the introduction of the dashboards. We found that early dashboard usage is related to academic achievement later in the academic year and that students’ review of the feedback received in the first year improved. Although these results are modest in comparison to how high the bar is sometimes set for learning analytics applications, we argue that low-cost deployments of self-service dashboards are an interesting approach to start building experience with similar tools and to start paving the way for future developments.
To start medical or dentistry studies in Flanders, prospective students need to pass a central admission test. A blended program with four Small Private Online Courses (SPOCs) was designed to support those students. The logs from the platform provide an opportunity to delve into the learners’ interactions and to develop predictive models to forecast success in the test. Moreover, the use of different courses allows analyzing how models can generalize across courses. This article has the following objectives: (1) to develop and analyze predictive models to forecast who will pass the admission test, (2) to discover which variables have more effect on success in different courses, (3) to analyze to what extent models can be generalized to other courses and subsequent cohorts, and (4) to discuss the conditions to achieve generalizability. The results show that the average grade in SPOC exercises using only first attempts is the best predictor and that it is possible to transfer predictive models with enough reliability when some context-related conditions are met. The best performance is achieved when transferring within the same cohort to other SPOCs in a similar context. The performance is still acceptable in a consecutive edition of a course. These findings support the sustainability of predictive models.
3 July 2017 in Journal of Research in Innovative Teaching & Learning
Purpose: The purpose of this paper is to draw attention to the potential of “small data” to complement research in learning analytics (LA) and to share some of the insights learned from this approach. Design/methodology/approach: This study demonstrates an approach inspired by design science research, making a dashboard available to n=1,905 students in 11 study programs (used by n=887) to learn how it is being used and to gather student feedback. Findings: Students react positively to the LA dashboard, but usage and feedback differ depending on study success. Research limitations/implications: More research is needed to explore the expectations of a high-performing student with regards to LA dashboards. Originality/value: This publication demonstrates how a small data approach to LA contributes to building a better understanding.