Traditional indices of effect size are designed to answer questions about average group differences, associations between variables, and relative risk. For many researchers, an additional, important question is, “How many people in my study behaved or responded in a manner consistent with theoretical expectation?” We show how the answer to this question can be computed and reported as a straightforward percentage for a wide variety of study designs. This percentage essentially treats persons as an effect size, and it can easily be understood by scientists, professionals, and laypersons alike. For instance, imagine that in addition to d or η 2 , a researcher reports that 80% of participants matched theoretical expectation. No statistical training is required to understand the basic meaning of this percentage. By analyzing recently published studies, we show how computing this percentage can reveal novel patterns within data that provide insights for extending and developing the theory under investigation.
Serious criticisms of psychology’s research practices and data analysis methods date back to at least the mid-1900s after the Galtonian school of thought had thoroughly triumphed over the Wundtian school. In the wake of Bem’s (2011) recent, highly publicized study on psi phenomena in a prestigious journal, psychologists are again raising serious questions about their dominant research script. These concerns are echoed in the current paper, and Observation Oriented Modeling (OOM) is presented as an alternative approach toward data conceptualization and analysis for the social and life sciences. This approach is rooted in philosophical realism and an attitude toward data analysis centered around causality and common sense. Three example studies and accompanying data analyses are presented and discussed to demonstrate a number of OOM’s advantages over current researcher practices.
A novel approach for conceptualizing and analyzing data from psychological studies is presented and discussed. This approach is centered on model building in an effort to explicate the structures and processes believed to generate a set of observations. These models therefore go beyond the variable-based, path models in use today which are limiting with regard to the types of inferences psychologists can draw from their research. In terms of analysis, the newer approach replaces traditional aggregate statistics such as means, variances, and covariances with methods of pattern detection and analysis. While these methods are person-centered and do not require parametric assumptions, they are both demanding and rigorous. They also provide psychologists with the information needed to draw the primary inference they often wish to make from their research; namely, the inference to best explanation.
Social scientists are often interested in computing the proportion of overlap and nonoverlap between two normal distributions that are separated by some magnitude. In his popular book, Statistical Power Analysis for the Behavioral Sciences (1988, 2nd ed.), Jacob Cohen provided a table (Table 2.2.1) for determining such proportions from common values of separation. Unfortunately, Cohen's proportions are inconsistent with his explication of the popular index of effect size, d; and his proportions are underestimates of distributional overlap and overestimates of nonoverlap. The authors explain how Cohen derived his values and then provide a revised, corrected table of proportions that also match values presented elsewhere.
4 July 2015 in Basic and Applied Social Psychology
The hope of mediation modeling is that psychologists can go beyond tests of association to truly uncover mechanisms of change. We argue this hope can be realized only if psychologists make important distinctions regarding causality and inference. From the perspective of Aristotelian philosophy, mediation models are sequences of efficient causes, and psychologists should therefore seek to identify those persons who can be traced through the entire sequence successfully. By reanalyzing data from two mediation studies we demonstrate that contemporary, aggregate methods of analysis are not suitable for this task because they are instead focused on making inferences about population parameters. In both studies alternative, person-centered methods revealed that majorities of participants were not traceable through the statistically significant mediation models.