Moderator variables are widely hypothesized and studied in the organizational sciences, but the empirical track record of moderator variable studies is very discouraging. These studies often lack sufficient statistical power and the type of designs and measures common in organizational research virtually guarantee that the moderator effects that are found are usually extremely small. We recommend that future attempts to identify and estimate moderator effects should be limited to situations where better measures, stronger research designs and a realistic cost-benefit assessment are available. Researchers should avoid moderator hypotheses in contexts where the measures and research designs employed do not allow them to be tested in a meaningful way, and should be cautious about interpreting the very small effects they are likely to find.
14 December 2021 in Industrial and Organizational Psychology
As data analytic methods in the managerial sciences become more sophisticated, the gap between the descriptive data typically presented in Table 1 and the analyses used to test the principal hypotheses advanced has become increasingly large. This contributes to several problems including: (1) the increasing likelihood that analyses presented in published research will be performed and/or interpreted incorrectly, (2) an increasing reliance on statistical significance as the principal criterion for evaluating results, and (3) the increasing difficulty of describing our research and explaining our findings to non-specialists. A set of simple methods for assessing whether hypotheses about interventions, moderator relationships and mediation, are plausible that are based on the simplest possible examination of descriptive statistics are proposed.
Most management theories include hypotheses about interaction effects (i.e., the relation between two variables depends on values of another), but it is common for articles to present results that make it difficult to evaluate the nature, strength, and importance of the effect. We offer recommendations for improving the reporting of interaction effects by focusing on (a) visualizations, (b) effect size estimates, and (c) assessments of the nature, meaning, and importance of interactions for theory and practice.