Although variation in effect sizes and predicted values among studies of similar phenomena is inevitable, such variation far exceeds what might be produced by sampling error alone. One possible explanation for variation among results is differences among researchers in the decisions they make regarding statistical analyses. A growing array of studies has explored this analytical variability in different fields and has found substantial variability among results despite analysts having the same data and research question. Many of these studies have been in the social sciences, but one small “many analyst” study found similar variability in ecology. We expanded the scope of this prior work by implementing a large-scale empirical exploration of the variation in effect sizes and model predictions generated by the analytical decisions of different researchers in ecology and evolutionary biology. We used two unpublished datasets, one from evolutionary ecology (blue tit, Cyanistes caeruleus, to compare sibling number and nestling growth) and one from conservation ecology (Eucalyptus, to compare grass cover and tree seedling recruitment). The project leaders recruited 174 analyst teams, comprising 246 analysts, to investigate the answers to prespecified research questions. Analyses conducted by these teams yielded 141 usable effects (compatible with our meta-analyses and with all necessary information provided) for the blue tit dataset, and 85 usable effects for the Eucalyptus dataset. We found substantial heterogeneity among results for both datasets, although the patterns of variation differed between them. For the blue tit analyses, the average effect was convincingly negative, with less growth for nestlings living with more siblings, but there was near continuous variation in effect size from large negative effects to effects near zero, and even effects crossing the traditional threshold of statistical significance in the opposite direction. In contrast, the average relationship between grass cover and Eucalyptus seedling number was only slightly negative and not convincingly different from zero, and most effects ranged from weakly negative to weakly positive, with about a third of effects crossing the traditional threshold of significance in one direction or the other. However, there were also several striking outliers in the Eucalyptus dataset, with effects far from zero. For both datasets, we found substantial variation in the variable selection and random effects structures among analyses, as well as in the ratings of the analytical methods by peer reviewers, but we found no strong relationship between any of these and deviation from the meta-analytic mean. In other words, analyses with results that were far from the mean were no more or less likely to have dissimilar variable sets, use random effects in their models, or receive poor peer reviews than those analyses that found results that were close to the mean. The existence of substantial variability among analysis outcomes raises important questions about how ecologists and evolutionary biologists should interpret published results, and how they should conduct analyses in the future.

Increasing attention has been drawn to the misuse of statistical methods over recent years, with particular concern about the prevalence of practices such as poor experimental design, cherry picking and inadequate reporting. These failures are largely unintentional and no more common in ecology than in other scientific disciplines, with many of them easily remedied given the right guidance. Originating from a discussion at the 2020 International Statistical Ecology Conference, we show how ecologists can build their research following four guiding principles for impactful statistical research practices: (1) define a focussed research question, then plan sampling and analysis to answer it; (2) develop a model that accounts for the distribution and dependence of your data; (3) emphasise effect sizes to replace statistical significance with ecological relevance; and (4) report your methods and findings in sufficient detail so that your research is valid and reproducible. These principles provide a framework for experimental design and reporting that guards against unsound practices. Starting with a well‐defined research question allows researchers to create an efficient study to answer it, and guards against poor research practices that lead to poor estimation of the direction, magnitude, and uncertainty of ecological relationships, and to poor replicability. Correct and appropriate statistical models give sound conclusions. Good reporting practices and a focus on ecological relevance make results impactful and replicable. Illustrated with two examples—an experiment to study the impact of disturbance on upland wetlands, and an observational study on blue tit colouring—this paper explains the rationale for the selection and use of effective statistical practices and provides practical guidance for ecologists seeking to improve their use of statistical methods.

The use of chemical cues and signals is essential for communication in insects. Wasps of the genus Nasonia (Hymenoptera, Pteromalidae) are gregarious parasitoids that lay their eggs into puparia of cyclorrhaphous flies. During their life cycle, various kinds of semiochemicals are used: (1) a male abdominal sex pheromone that attracts females and induces site fidelity in males, (2) a female-derived contact sex pheromone eliciting courtship behavior in males, (3) an oral male aphrodisiac eliciting receptivity signaling in females and causing a switch in the females' olfactory preferences, (4) chemicals derived from host habitat and host puparia used in olfactory host finding by female wasps, and (5) chemicals used by females to assess the quality and parasitization status of potential hosts. We review the literature on the chemical ecology of Nasonia spp. following the wasps' life cycle from emergence to oviposition. We depict biosynthetic pathways where available, discuss ecological implications, highlight differences among Nasonia species, summarize insights into their olfactory perception and associative learning abilities, and point out gaps in our understanding of the chemical ecology of these parasitoids to be addressed in future studies.

Current regulatory guidelines for pesticide risk assessment recommend that nonsignificant results should be complemented by the minimum detectable difference (MDD), a statistical indicator that is used to decide whether the experiment could have detected biologically relevant effects. We review the statistical theory of the MDD and perform simulations to understand its properties and error rates. Most importantly, we compare the skill of the MDD in distinguishing between true and false negatives (i.e., type II errors) with 2 alternatives: the minimum detectable effect (MDE), an indicator based on a post hoc power analysis common in medical studies; and confidence intervals (CIs). Our results demonstrate that MDD and MDE only differ in that the power of the MDD depends on the sample size. Moreover, although both MDD and MDE have some skill in distinguishing between false negatives and true absence of an effect, they do not perform as well as using CI upper bounds to establish trust in a nonsignificant result. The reason is that, unlike the CI, neither MDD nor MDE consider the estimated effect size in their calculation. We also show that MDD and MDE are no better than CIs in identifying larger effects among the false negatives. We conclude that, although MDDs are useful, CIs are preferable for deciding whether to treat a nonsignificant test result as a true negative, or for determining an upper bound for an unknown true effect. Environ Toxicol Chem 2020;39:2109–2123. © 2020 The Authors. Environmental Toxicology and Chemistry published by Wiley Periodicals LLC on behalf of SETAC. Abstract In ecotoxicological risk assessment, the minimum detectable difference (MDD) is used as a secondary filter to check whether nonsignificant test results are based on sufficient power and should thus be trusted. In our review, we show that the MDD has some skill in distinguishing between true absence of effects and false negatives (type II errors), but that confidence intervals (CIs) clearly outperform MDDs in this task, and should thus be preferred. MDE = minimum detectable effect; pCI = proportional upper bound of the confidence interval; pMDD = proportional MDD; pMDE = proportional MDE.

Terrestrial ecosystems are exposed to many anthropogenic pollutants. Non-target effects of pesticides and fertilizers have put agricultural intensification in the focus as a driver for biodiversity loss. However, other pollutants, such as heavy metals, particulate matter, or microplastic also enter the environment, e.g. via traffic and industrial activities in urban areas. As soil acts as a potential sink for such pollutants, soil invertebrates like earthworms may be particularly affected by them. Under natural conditions soil invertebrates will likely be exposed to combinations of pollutants simultaneously, which may result in stronger negative effects if pollutants act synergistically. Within this work we study how multiple pollutants affect the soil-dwelling, substrate feeding earthworm Eisenia fetida. We compared the effects of the single stressors, polystyrene microplastic fragments, polystyrene fibers, brake dust and carbon black, with the combined effect of these pollutants when applied as a mixture. Endpoints measured were survival, increase in body weight, reproductive fitness, and changes in three oxidative stress markers (glutathione S-transferase, catalase and malondialdehyde). We found that among single pollutant treatments, brake dust imposed the strongest negative effects on earthworms in all measured endpoints including increased mortality rates. Sub-lethal effects were found for all pollutants. Exposing earthworms to all four pollutants simultaneously led to effects on mortality and oxidative stress markers that were smaller than expected by the respective null models. These antagonistic effects are likely a result of the adsorption of toxic substances found in brake dust to the other pollutants. With this study we show that effects of combinations of pollutants cannot necessarily be predicted from their individual effects and that combined effects will likely depend on identity and concentration of the pollutants.

In recent years, the number of publications on nano- and microplastic particles (NMPs) effects on freshwater organisms has increased rapidly. Freshwater crustaceans of the genus Daphnia are widely used in ecotoxicological research as model organisms for assessing the impact of NMPs. However, the diversity of experimental designs in these studies makes conclusions about the general impact of NMPs on Daphnia challenging. To approach this, we systematically reviewed the literature on NMP effects on Daphnia and summarized the diversity of test organisms, experimental conditions, NMP properties and measured endpoints to identify gaps in our knowledge of NMP effects on Daphnia. We use a meta-analysis on mortality and immobilization rates extracted from the compiled literature to illustrate how NMP properties, study parameters and the biology of Daphnia can impact outcomes in toxicity bioassays. In addition, we investigate the extent to which the available data can be used to predict the toxicity of untested NMPs based on the extracted parameters. Based on our results, we argue that focusing on a more diverse set of NMP properties combined with a more detailed characterization of the particles in future studies will help to fill current research gaps, improve predictive models and allow the identification of NMP properties linked to toxicity.