1 March 2008 in Acta Psychologica
1 January 2012 in Journal of Psychosomatic Research
25 September 2016 in Assessment & Evaluation in Higher Education
MSMaarten J. van der Smagt
Prediction accuracy of academic achievement for admission purposes requires adequate sensitivity and specificity of admission tools, yet the available information on the validity and predictive power of admission tools is largely based on studies using correlational and regression statistics. The goal of this study was to explore signal detection theory as a tool to extend the available information; signal detection theory allows for comparisons of selection outcomes on both group and individual levels and the development of tailor-made criteria for specific programmes and admission goals. We investigated who would or would not have been admitted applying specific criteria for each of three common admission tools, how many admitted students would fail and how many applicants who would have been successful would be rejected. Both comparisons at an individual level and the receiver operating characteristic curves at a group level revealed that scores obtained in a programme-specific matching programme and non-cognitive factors appear more valuable than regression statistics suggest when it comes to admitting applicants who will become successful students. Signal detection theory allows not only for admission-goal-specific and programme-specific fine-tuning of the content of admission tools, it also informs about the effects of criteria and thus allows the setting of criteria.
1 May 2018 in Frontiers in Education
MSMaarten J. van der Smagt
Using multiple admission tools in university admission procedures is common practice. This is particularly useful if different admission tools uniquely select different subgroups of students who will be successful in university programs. A signal-detection approach was used to investigate the accuracy of Secondary School grade point average (SSGPA), an admission test score (ACS), and a non-cognitive score (NCS) in uniquely selecting successful students. This was done for three consecutive first year cohorts of a broad psychology program. Each applicant's score on SSGPA, ACS, or NCS alone—and on seven combinations of these scores, all considered separate “admission tools”—was compared at two different (medium and high) cut-off scores (criterion levels). Each of the tools selected successful students who were not selected by any of the other tools. Both sensitivity and specificity were enhanced by implementing multiple tools. The signal-detection approach distinctively provided useful information for decisions on admission instruments and cut-off scores.