Goodness-of-fit (GOF) indexes provide "rules of thumb"—recommended cutoff values for assessing fit in structural equation modeling. Hu and Bentler (1999) proposed a more rigorous approach to evaluating decision rules based on GOF indexes and, on this basis, proposed new and more stringent cutoff values for many indexes. This article discusses potential problems underlying the hypothesis-testing rationale of their research, which is more appropriate to testing statistical significance than evaluating GOF. Many of their misspecified models resulted in a fit that should have been deemed acceptable according to even their new, more demanding criteria. Hence, rejection of these acceptable-misspecified models should have constituted a Type 1 error (incorrect rejection of an "acceptable" model), leading to the seemingly paradoxical results whereby the probability of correctly rejecting misspecified models decreased substantially with increasing N. In contrast to the application of cutoff values to evaluate each ...
Interactions between (multiple indicator) latent variables are rarely used because of implementation complexity and competing strategies. Based on 4 simulation studies, the traditional constrained approach performed more poorly than did 3 new approaches--unconstrained, generalized appended product indicator, and quasi-maximum-likelihood (QML). The authors' new unconstrained approach was easiest to apply. All 4 approaches were relatively unbiased for normally distributed indicators, but the constrained and QML approaches were more biased for nonnormal data; the size and direction of the bias varied with the distribution but not with the sample size. QML had more power, but this advantage was qualified by consistently higher Type I error rates. The authors also compared general strategies for defining product indicators to represent the latent interaction factor.
We evaluated whether "more is ever too much" for the number of indicators (p) per factor (p/f) in confirmatory factor analysis by varying sample size (N = 50-1000) and p/f (2-12 items per factor) in 35,000 Monte Carlo solutions. For all N's, solution behavior steadily improved (more proper solutions, more accurate parameter estimates, greater reliability) with increasing p/f. There was a compensatory relation between N and p/f: large p/f compensated for small N and large N compensated for small p/f, but large-N and large-p/f was best. A bias in the behavior of the χ2 was also demonstrated where apparent goodness of fit declined with increasing p/f ratios even though approximating models were "true". Fit was similar for proper and improper solutions, as were parameter estimates form improper solutions not involving offending estimates. We also used the 12-p/f data to construct 2, 3, 4, or 6 parcels of items (e.g., two parcels of 6 items per factor, three parcels of 4 items per factor, etc.), but the 12-ind...
Academically selective schools are intended to affect academic self-concept positively, but theoretical and empirical research demonstrates that the effects are negative. The big-fish-little-pond effect (BFLPE), an application of social comparison theory to educational settings, posits that a student will have a lower academic self-concept in an academically selective school than in a nonselective school. This study, the largest cross-cultural study of the BFLPE ever undertaken, tested theoretical predictions for nationally representative samples of approximately 4,000 15-year-olds from each of 26 countries (N = 103,558) who completed the same self-concept instrument and achievement tests. Consistent with the BFLPE, the effects of school-average achievement were negative in all 26 countries (M beta = -.20, SD = .08), demonstrating the BFLPE's cross-cultural generalizability.
1 November 2004 in British Journal of Mathematical and Statistical Psychology
The big-fish–little-pond effect (BFLPE) predicts that equally able students have lower academic self-concepts (ASCs) when attending schools where the average ability levels of classmates is high, and higher ASCs when attending schools where the school-average ability is low. BFLPE findings are remarkably robust, generalizing over a wide variety of different individual student and contextual level characteristics, settings, countries, long-term follow-ups, and research designs. Because of the importance of ASC in predicting future achievement, coursework selection, and educational attainment, the results have important implications for the way in which schools are organized (e.g., tracking, ability grouping, academically selective schools, and gifted education programs). In response to Dai and Rinn (Educ. Psychol. Rev., 2008 ), we summarize the theoretical model underlying the BFLPE, minimal conditions for testing the BFLPE, support for its robust generalizability, its relation to social comparison theory, and recent research extending previous implications, demonstrating that the BFLPE stands up to scrutiny.