Organizations are increasingly adopting automated video interviews (AVIs) to screen job applicants despite a paucity of research on their reliability, validity, and generalizability. In this study, we address this gap by developing AVIs that use verbal, paraverbal, and nonverbal behaviors extracted from video interviews to assess Big Five personality traits. We developed and validated machine learning models within (using nested cross-validation) and across three separate samples of mock video interviews (total N = 1,073). Also, we examined their test-retest reliability in a fourth sample (N = 99). In general, we found that the AVI personality assessments exhibited stronger evidence of validity when they were trained on interviewer-reports rather than self-reports. When cross-validated in the other samples, AVI personality assessments trained on interviewer-reports had mixed evidence of reliability, exhibited consistent convergent and discriminant relations, used predictors that appear to be conceptually relevant to the focal traits, and predicted academic outcomes. On the other hand, there was little evidence of reliability or validity for the AVIs trained on self-reports. We discuss the implications for future work on AVIs and personality theory, and provide practical recommendations for the vendors marketing such approaches and organizations considering adopting them. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
23 January 2020 in Perspectives on Psychological Science
A seemingly universal lesson is that anything taken to its extreme is detrimental. Indeed, there has been growing interest in testing this idea within psychology. These studies have often been framed in terms of Aristotle’s doctrine of the golden mean or the idea that virtue lies between the vices of deficiency and excess. Recent explicit reviews of this hypothesis in the psychological literature have led to the paradoxical conclusion that one can have too much virtue (i.e., the too-much-of-a-good-thing effect) despite virtue being identified by the golden mean. We argue in this article that this conclusion is due to a reductionist account of virtues in psychology and the resultant measurement of virtues as general dispositional tendencies in behavior. We review philosophical theory on the golden mean to show that the relationship between virtue and relevant behavior is fundamentally about situation-specific optimality . Using schematic models, we contrast the former measurement approach against the latter to explain the too-much-of-a-good-thing effect and further demonstrate why virtues cannot be properly measured as general tendencies in behavior. We conclude with methodological implications of our theory-informed approach to virtue measurement for research design, evaluation, and conceptualization.
The factor structure of the Values in Action Inventory of Strengths (VIA-IS; Peterson & Seligman, 2004) has not been well established as a result of methodological challenges primarily attributable to a global positivity factor, item cross-loading across character strengths, and questions concerning the unidimensionality of the scales assessing character strengths. We sought to overcome these methodological challenges by applying exploratory structural equation modeling (ESEM) at the item level using a bifactor analytic approach to a large sample of 447,573 participants who completed the VIA-IS with all 240 character strengths items and a reduced set of 107 unidimensional character strength items. It was found that a 6-factor bifactor structure generally held for the reduced set of unidimensional character strength items; these dimensions were justice, temperance, courage, wisdom, transcendence, humanity, and an overarching general factor that is best described as dispositional positivity. (PsycINFO Database Record
There has been reemerging interest within psychology in the construct of character, yet assessing it can be difficult due to social desirability of character traits. Forced-choice formats offer one way to address response bias, but traditional scoring methods (i.e., ipsative) associated with this format makes comparing scores between people problematic. Nevertheless, recent advances in modeling item responding (Thurstonian IRT) enable scoring that recovers absolute standing on latent traits and allows for score comparisons between people. Based on recent work in character measurement (CIVIC), we developed a multidimensional forced-choice measure of character (CIVIC-MFC) and scored it using Thurstonian IRT. Initial validation using a sample of 798 participants demonstrated good support for factorial, convergent, and concurrent validity for scores on the CIVIC-MFC, although they did not demonstrate more faking resistance than scores on a Likert-type format version. Potential explanations are discussed.
13 February 2017 in The Journal of Positive Psychology
The present research study sought to develop and validate a character scale – the Comprehensive Inventory of Virtuous Instantiations of Character using a total sample size of 3679 across five studies. In Study 1, character trait items were generated using an integrative classification system. In Study 2, character trait scales were further refined and their factor structure examined, revealing eight higher-order character dimensions or character cores: appreciation, intellectual engagement, fortitude, interpersonal consideration, sincerity, temperance, transcendence, and empathy. Study 3 established convergent validity of character traits with extant measures and discriminability from personality facets, social desirability, and moral cognitive development. Study 4 revealed that character cores were more strongly related to evaluative constructs than personality dimensions. Study 5 demonstrated that character cores predicted performance and psychological well-being outcomes above and beyond personality. The implications of our findings for the assessment and taxonomy of character are discussed.
Visualizations in organizational research have primarily been used in the context of traditional survey data, where individual data points (e.g., responses) can typically be plotted, and qualitative (e.g., language data) and quantitative (e.g., frequency data) information are not typically combined. Moreover, visualizations are typically used in a hypothetico-deductive fashion to showcase significant hypothesized results. With the advent of big data, which has been characterized as being particularly high in volume, variety, and velocity of collection, visualizations need to more explicitly and formally consider the issues of (a) identification (isolating or highlighting relevant data pertaining to the phenomena of interest), (b) integration (combining different modes of data to reveal insights about a phenomenon of interest), (c) immediacy (examining real-time data in a time-sensitive manner), and (d) interactivity (inductively uncovering and identifying new patterns). We discuss basic ideas for addressing these issues and provide illustrative examples of visualizations that incorporate and highlight ways of addressing these issues. Examples in our article include visualizing multiple performance criteria for police officers, publication network of organizational researchers, and social media language of Fortune 500 companies.