1 May 2010 in Journal of Educational Psychology
8 August 2014 in The Journal of Experimental Education
Multilevel modeling has grown in use over the years as a way to deal with the nonindependent nature of observations found in clustered data. However, other alternatives to multilevel modeling are available that can account for observations nested within clusters, including the use of Taylor series linearization for variance estimation, the design effect adjusted standard errors approach, and fixed effects modeling. Using 1,000 replications of 12 conditions with varied Level 1 and Level 2 sample sizes, the author compared parameter estimates, standard errors, and statistical significance using various alternative procedures. Results indicate that several acceptable procedures can be used in lieu of or together with multilevel modeling, depending on the type of research question asked and the number of clusters under investigation. Guidelines for applied researchers are discussed.
1 February 2013 in Journal of Educational Psychology
1 September 2018 in School Psychology Quarterly
The use of multilevel modeling (MLM) to analyze nested data has grown in popularity over the years in the study of school psychology. However, with the increase in use, several statistical misconceptions about the technique have also proliferated. We discuss some commonly cited myths and golden rules related to the use of MLM, explain their origin, and suggest approaches to dealing with certain issues. Misunderstandings related to the use of the intraclass correlation, design effects, minimum sample size, multilevel factor structures, model R², and the misestimation of standard errors are reviewed. Many of the cited myths have much truth in them-though at times, researchers may not be aware of the exceptions to the rules that prevent their overall generalization. Although nesting should be accounted for, researchers should realize that MLM, which is a powerful and flexible technique, is not the only method that can be used to account for the clustering effect. (PsycINFO Database Record
19 January 2016 in Journal of Youth and Adolescence
24 November 2016 in Educational and Psychological Measurement
Cluster randomized trials involving participants nested within intact treatment and control groups are commonly performed in various educational, psychological, and biomedical studies. However, recruiting and retaining intact groups present various practical, financial, and logistical challenges to evaluators and often, cluster randomized trials are performed with a low number of clusters (~20 groups). Although multilevel models are often used to analyze nested data, researchers may be concerned of potentially biased results due to having only a few groups under study. Cluster bootstrapping has been suggested as an alternative procedure when analyzing clustered data though it has seen very little use in educational and psychological studies. Using a Monte Carlo simulation that varied the number of clusters, average cluster size, and intraclass correlations, we compared standard errors using cluster bootstrapping with those derived using ordinary least squares regression and multilevel models. Results indicate that cluster bootstrapping, though more computationally demanding, can be used as an alternative procedure for the analysis of clustered data when treatment effects at the group level are of primary interest. Supplementary material showing how to perform cluster bootstrapped regressions using R is also provided.