A comprehensive power comparison study of existing tests for normality is proposed. Given the importance of this subject and the widespread development of normality tests, comprehensive descriptions and power comparisons of such tests are of considerable interest. Since recent comparison studies do not include several interesting and more recently developed tests, a further comparison of normality tests is considered to be of foremost interest. The study addresses the performance of 33 normality tests, for various sample sizes, considering several significance levels and for a number of symmetric, asymmetric and modified normal distributions. General recommendations for normality testing resulting from the study are defined according to the nature of the non-normality.

We argue that making accept/reject decisions on scientific hypotheses, including a recent call for changing the canonical alpha level from p = 0.05 to p = 0.005, is deleterious for the finding of new discoveries and the progress of science. Given that blanket and variable alpha levels both are problematic, it is sensible to dispense with significance testing altogether. There are alternatives that address study design and sample size much more directly than significance testing does; but none of the statistical tools should be taken as the new magic method giving clear-cut mechanical answers. Inference should not be based on single studies at all, but on cumulative evidence from multiple independent studies. When evaluating the strength of the evidence, we should consider, for example, auxiliary assumptions, the strength of the experimental design, and implications for applications. To boil all this down to a binary decision based on a p-value threshold of 0.05, 0.01, 0.005, or anything else, is not acceptable.