1 January 1991 in Vision Research
1 December 2010 in Psychological Methods
A meta-analysis of the reliability of the scores from a specific test, also called reliability generalization, allows the quantitative synthesis of its properties from a set of studies. It is usually assumed that part of the variation in the reliability coefficients is due to some unknown and implicit mechanism that restricts and biases the selection of participants in the studies' samples. Sometimes this variation has been reduced by adjusting the coefficients by a formula associated with range restrictions. We propose a framework in which that variation is included (instead of adjusted) in the models intended to explain the variability and in which parallel analyses of the studies' means and variances are performed. Furthermore, the analysis of the residuals enables inferences to be made about the nature of the variability accounted for by moderator variables. The meta-analysis of the 3 studies' statistics-reliability coefficient, mean, and variance--allows psychometric inferences about the test scores. A numerical example illustrates the proposed framework.
14 February 2017 in Frontiers in Psychology
The Debate on the Ego-Depletion Effect: Evidence from Meta-Analysis with the p-Uniform Method
1 February 2006 in Behavior Research Methods
1 January 2001 in Journal of Experimental Psychology: Human Perception and Performance
The authors present a model to account for the miscombination of features when stimuli are presented using the rapid serial visual presentation (RSVP) technique (illusory conjunctions in the time domain). It explains the distributions of responses through a mixture of trial outcomes. In some trials, attention is successfully focused on the target, whereas in others, the responses are based on partial information. Two experiments are presented that manipulated the mean processing time of the target-defining dimension and of the to-be-reported dimension, respectively. As predicted, the average origin of the responses is delayed when lengthening the target-defining dimension, whereas it is earlier when lengthening the to-be-reported dimension; in the first case the number of correct responses is dramatically reduced, whereas in the second it does not change. The results, a review of other research, and simulations carried out with a formal version of the model are all in close accordance with the predictions.
1 August 2023 in Psychological Methods
One of the most widely used effect size indices for meta-analysis in psychology is the standardized mean difference (SMD). The most common way to synthesize a set of estimates of the SMD is to weight them by the inverse of their variances. For this, it is necessary to estimate the corresponding sampling variances. Meta-analysts have a formula for obtaining unbiased estimates of sampling variances, but they often use a variety of alternative, simpler methods. The bias and efficiency of five different methods that have been proposed and that are implemented in different computerized calculation tools are compared and assessed. The data from a set of published meta-analyses are also reanalyzed, calculating the combined estimates and their confidence intervals, as well as estimates of the specific, between-studies variance, using the five estimation methods. This test of sensitivity shows that the results of a meta-analysis can change noticeably depending on the method used to estimate the sampling variance of SMD values, especially under a random-effects model. Some practical recommendations are made about how to choose and implement the methods in calculation resources. (PsycInfo Database Record (c) 2021 APA, all rights reserved).