Purpose: The purpose of this paper is to give an empirically‐based review of the strengths and weaknesses of situational judgment tests (SJTs).Design/methodology/approach: The features, history, and development of SJTs are discussed. Next, a computerized search (from 1990‐2007) is conducted to retrieve empirical studies related to SJTs. The effectiveness of SJTs is discussed in terms of reliability, criterion‐related validity, incremental validity, construct‐related validity, utility, adverse impact, applicant perceptions, fakability, and susceptibility to practice and coaching effects.Findings: Strengths of SJTs are that they show criterion‐related validity and incremental validity above cognitive ability and personality tests. SJTs have also less adverse impact towards minorities (especially if the cognitive loading of the SJT is low). Furthermore, applicant reactions towards SJTs are positive and SJTs enable to test large applicant groups at once (through the Internet). In terms of weaknesses, SJTs might be prone to faking, practice, and coaching effects. There is also debate about what constructs are measured by SJTs.Research limitations/implications: Five avenues for future research are discussed: construct‐related validity of SJTs, utility of SJTs vis‐à‐vis other predictors, impact of SJT features on validity and adverse impact, examination of alternative stimulus and response formats, and cross‐cultural transportability of SJTs.Practical implications: Practitioners receive evidence‐based information about the features, development, and strengths and weaknesses of SJTs.Originality/value: Apart from the USA, SJTs have not made strong inroads in selection practice in Europe and other parts of the world. This evidence‐based paper might highlight the value of SJTs.
1 September 2009 in Organizational Behavior and Human Decision Processes
This article contributes to the understanding of why the use of a frame-of-reference leads to increased criterion-related validity of personality inventories. Two competing explanations are described and tested. A between-subjects (N = 337) and a within-subject (N = 105) study are conducted to test the hypothesized effects of use of a frame of reference on reliability and validity. Regarding the effects on reliability, use of a frame of reference reduces within-person inconsistency (instead of between-person variability) in responding to generic items. Use of a frame of reference further leads to higher validity as a result of the reduction of between-person variability and within-person inconsistency. Yet, reducing these inconsistencies is not enough. It is also important to use a frame of reference that is conceptually relevant to the criterion. Besides implications for contextualized personality inventories, these results provide an explanation for the moderate validities of generic personality inventories.
8 September 2021 in Corporate Social Responsibility and Environmental Management