Bifactor confirmatory factor analysis models and statistical indices computed from them have previously been used to provide evidence for the appropriateness of utilizing a unidimensional interpretation of multidimensional data. However, the ability of bifactor indices to aid in the assessment of subscore strength has not been investigated. A simulation study was conducted to relate bifactor indices to the strength of subscores corresponding to specific factors. The bifactor indices OmegaHS and ECVSS were found to be strongly predictive of subscore strength conditional upon OmegaS. The number of factors was also found to play a minor role in this relationship. Cutoffs for assessing the appropriateness of interpreting subscores were constructed based on OmegaHS or ECVSS conditional upon OmegaS and the number of factors. For low subscore reliability (OmegaS = .60), OmegaHS = .25 or ECVSS = .45 is sufficient that the subscore has a good chance of having added value (VAR > 1.1) above and beyond the total score. For moderate reliability (OmegaS = .80), OmegaHS = .20 or ECVSS = .30 is sufficient, and the role of OmegaHS or ECVSS diminishes as OmegaS increases further. A subscore having added value does not necessitate its interpretation. Instead, when subscores are desired to be interpreted, high OmegaHS or ECVSS can be considered as evidence that such an interpretation is statistically appropriate. Finally, we illustrate the use of these cutoffs with an empirical data set. When combined with prior bifactor research, this work extends a framework of using confirmatory bifactor models for dimensionality assessment. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
To investigate the effect of using negatively oriented items, we wrote semantic reversals of the items in the Rosenberg Self Esteem Scale, UCLA Loneliness Scale, and the General Belongingness Scale and used them to create four experimental conditions. Participants (N = 2,019) were recruited through Amazon’s Mechanical Turk. Data were assessed for dimensionality, item functioning, instrument properties, and associations with other variables. Regarding dimensionality, although a two-factor model (positively vs. negatively oriented factors) exhibits better fit than a unidimensional model across all conditions, bifactor indices were used to argue that a unidimensional interpretation of the data can be employed. With respect to item functioning, factor loadings were found to be nearly invariant across conditions, but thresholds were not. Concerning instrument properties, inclusion of negatively oriented items results in lower mean scores and higher score variances. Instruments with both positively and negatively oriented items demonstrated lower reliability estimates than those with only one orientation. For associations with other variables, path coefficients in a model where loneliness mediates the effects of belongingness on life satisfaction and self-esteem were found to vary across conditions. Findings suggest that negatively oriented items have minor impact on instrument quality, but influence measurement model and path coefficients.
1 March 2020 in Psychology of Sexual Orientation and Gender Diversity
One of the most cited methodological issues is with the response format, which is traditionally a single-response Likert response format. Therefore, our study aims to elucidate and illustrate an alternative response format and analytic technique, Thurstonian item response theory (IRT), for analyzing data from surveys using an alternate response format, the forced-choice format. Specifically, we strove to give a thorough introduction of Thurstonian IRT at a more elementary level than previous publications in order to widen the possible audience. This article presents analyses and comparison of two versions of a self-report scale, one version using a single-response format and the other using a forced-choice format. Drawing from lessons learned from our study and literature, we present a number of recommendations for conducting research using the forced-choice format and Thurstonian IRT, as well as suggested avenues for future research.