This Monte Carlo study examines the performance of fit indices commonly used by applied researchers interested in finite mixture modeling for the purposes of classification. Conditions for the simulation study were selected to reflect conditions found in applied educational and psychological research. The factors included in the investigation were metric level of indicators, sample size, and class prevalence. All models contained a combination of categorical and continuous indicators. All categorical indicators were dichotomous, and continuous indicators were normally distributed. The fit indices examined were Akaike’s information criterion, Bayesian information criterion (BIC), sample size-adjusted Bayesian information criterion (SSBIC), integrated classification likelihood criterion with Bayesian-type approximation, and Lo–Mendell–Rubin likelihood ratio test. Overall, SSBIC tended to identify the correct solution with higher frequency than other indices. BIC tended to identify the correct solution with higher frequency than the other indices in models with more continuous than categorical indicators, or when rare classes were absent.
Bi-factor confirmatory factor models have been influential in research on cognitive abilities because they often better fit the data than correlated factors and higher-order models. They also instantiate a perspective that differs from that offered by other models. Motivated by previous work that hypothesized an inherent statistical bias of fit indices favoring the bi-factor model, we compared the fit of correlated factors, higher-order, and bi-factor models via Monte Carlo methods. When data were sampled from a true bi-factor structure, each of the approximate fit indices was more likely than not to identify the bi-factor solution as the best fitting. When samples were selected from a true multiple correlated factors structure, approximate fit indices were more likely overall to identify the correlated factors solution as the best fitting. In contrast, when samples were generated from a true higher-order structure, approximate fit indices tended to identify the bi-factor solution as best fitting. There was extensive overlap of fit values across the models regardless of true structure. Although one model may fit a given dataset best relative to the other models, each of the models tended to fit the data well in absolute terms. Given this variability, models must also be judged on substantive and conceptual grounds.
8 September 2020 in Structural Equation Modeling: A Multidisciplinary Journal
When questionnaires include Likert scales, items endorsed by relatively few respondents may result from characteristics of examinees or the constructs under study. Researchers may collapse categories to increase cell sample size; however, effects of this practice have not been systematically investigated. A five-point ordinal scale was simulated where data included few responses in extreme categories. Different estimators were applied to sparsely distributed and collapsed category data; characteristics of sample size, number of categories, number of items including sparse data, and percentage of sparse data were manipulated. Collapsing categories were advantageous for ULSMV and WLSMV, yielding higher convergence rates, more accurate estimation of parameters and standard errors, and chi-square test rejection rates close to the nominal level. With many response categories (e.g., ≥5), treating sparse data as continuous and using MLMV may serve as an alternative, especially when a small percentage of total items contain low cell frequencies.
Whereas general sample size guidelines have been suggested when estimating multilevel models, they are only generalizable to a relatively limited number of data conditions and model structures, both of which are not very feasible for the applied researcher. In an effort to expand our understanding of two-level multilevel models under less than ideal conditions, Monte Carlo methods, through SAS/IML, were used to examine model convergence rates, parameter point estimates (statistical bias), parameter interval estimates (confidence interval accuracy and precision), and both Type I error control and statistical power of tests associated with the fixed effects from linear two-level models estimated with PROC MIXED. These outcomes were analyzed as a function of: (a) level-1 sample size, (b) level-2 sample size, (c) intercept variance, (d) slope variance, (e) collinearity, and (f) model complexity. Bias was minimal across nearly all conditions simulated. The 95% confidence interval coverage and Type I error rate tended to be slightly conservative. The degree of statistical power was related to sample sizes and level of fixed effects; higher power was observed with larger sample sizes and level-1 fixed effects.
9 August 2013 in Community College Journal of Research and Practice
In the knowledge economy, now more than ever, students are encouraged to attend an institution of higher education. Students actively search for resources to assist them in their progress toward a college degree, even before high school graduation. Dual enrollment is an opportunity for students to complete college courses while still in high school, and it encourages stronger collaborations between K–12 and higher education. The purpose of this study was to explore the relationship between demographic and policy variables in South Carolina technical college dual enrollment programs and first-to-second-year persistence of dually enrolled students once entering college. All participants in this study were high school students who continued at a technical college after graduation. Outcomes were analyzed using logistic regression to determine persistence, defined as first-to-second-year retention. Variables included dual enrollment course type, course setting, ethnicity, gender, and county of residence. Results show that course type and course setting are significant in predicting college student persistence. Recommendations and implications for future research are discussed.