Systematic Reviews (SRs) of experimental animal studies are not yet common practice, but awareness of the merits of conducting such SRs is steadily increasing. As animal intervention studies differ from randomized clinical trials (RCT) in many aspects, the methodology for SRs of clinical trials needs to be adapted and optimized for animal intervention studies. The Cochrane Collaboration developed a Risk of Bias (RoB) tool to establish consistency and avoid discrepancies in assessing the methodological quality of RCTs. A similar initiative is warranted in the field of animal experimentation.

Objectives: Evaluating the variation in the strength of the effect across studies is a key feature of meta-analyses. This variability is reflected by measures like τ ² or I ² , but their clinical interpretation is not straightforward. A prediction interval is less complicated: it presents the expected range of true effects in similar studies. We aimed to show the advantages of having the prediction interval routinely reported in meta-analyses. Design: We show how the prediction interval can help understand the uncertainty about whether an intervention works or not. To evaluate the implications of using this interval to interpret the results, we selected the first meta-analysis per intervention review of the Cochrane Database of Systematic Reviews Issues 2009–2013 with a dichotomous (n=2009) or continuous (n=1254) outcome, and generated 95% prediction intervals for them. Results: In 72.4% of 479 statistically significant (random-effects p0), the 95% prediction interval suggested that the intervention effect could be null or even be in the opposite direction. In 20.3% of those 479 meta-analyses, the prediction interval showed that the effect could be completely opposite to the point estimate of the meta-analysis. We demonstrate also how the prediction interval can be used to calculate the probability that a new trial will show a negative effect and to improve the calculations of the power of a new trial. Conclusions: The prediction interval reflects the variation in treatment effects over different settings, including what effect is to be expected in future patients, such as the patients that a clinician is interested to treat. Prediction intervals should be routinely reported to allow more informative inferences in meta-analyses.

Systematic reviews and meta-analyses of individual participant data (IPD) aim to collect, check, and reanalyze individual-level data from all studies addressing a particular research question and are therefore considered a gold standard approach to evidence synthesis. They are likely to be used with increasing frequency as current initiatives to share clinical trial data gain momentum and may be particularly important in reviewing controversial therapeutic areas. To develop PRISMA-IPD as a stand-alone extension to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) Statement, tailored to the specific requirements of reporting systematic reviews and meta-analyses of IPD. Although developed primarily for reviews of randomized trials, many items will apply in other contexts, including reviews of diagnosis and prognosis. Development of PRISMA-IPD followed the EQUATOR Network framework guidance and used the existing standard PRISMA Statement as a starting point to draft additional relevant material. A web-based survey informed discussion at an international workshop that included researchers, clinicians, methodologists experienced in conducting systematic reviews and meta-analyses of IPD, and journal editors. The statement was drafted and iterative refinements were made by the project, advisory, and development groups. The PRISMA-IPD Development Group reached agreement on the PRISMA-IPD checklist and flow diagram by consensus. Compared with standard PRISMA, the PRISMA-IPD checklist includes 3 new items that address (1) methods of checking the integrity of the IPD (such as pattern of randomization, data consistency, baseline imbalance, and missing data), (2) reporting any important issues that emerge, and (3) exploring variation (such as whether certain types of individual benefit more from the intervention than others). A further additional item was created by reorganization of standard PRISMA items relating to interpreting results. Wording was modified in 23 items to reflect the IPD approach. PRISMA-IPD provides guidelines for reporting systematic reviews and meta-analyses of IPD.

Variance between studies in a meta-analysis will exist. This heterogeneity may be of clinical, methodological or statistical origin. The last of these is quantified by the I(2) -statistic. We investigated, using simulated studies, the accuracy of I(2) in the assessment of heterogeneity and the effects of heterogeneity on the predictive value of meta-analyses. The relevance of quantifying I(2) was determined according to the likely presence of heterogeneity between studies (low, high, or unknown) and the calculated I(2) (low or high). The findings were illustrated by published meta-analyses of selective digestive decontamination and weaning protocols. As expected, I(2) increases and the likelihood of drawing correct inferences from a meta-analysis decreases with increasing heterogeneity. With low levels of heterogeneity, I(2) does not appear to be predictive of the accuracy of the meta-analysis result. With high levels of heterogeneity, even meta-analyses with low I(2) -values have low predictive values. Most commonly, the level of heterogeneity in a meta-analysis will be unknown. In these scenarios, I(2) determination may help to identify estimates with low predictive values (high I(2) ). In this situation, the results of a meta-analysis will be unreliable. With low I(2) -values and unknown levels of heterogeneity, predictive values of pooled estimates may range extensively, and findings should be interpreted with caution. In conclusion, quantifying statistical heterogeneity through I(2) -statistics is only helpful when the amount of clinical heterogeneity is unknown and I(2) is high. Objective methods to quantify the levels of clinical and methodological heterogeneity are urgently needed to allow reliable determination of the accuracy of meta-analyses.

Measures of interaction on an additive scale (relative excess risk due to interaction [RERI], attributable proportion [AP], synergy index [S]), were developed for risk factors rather than preventive factors. It has been suggested that preventive factors should be recoded to risk factors before calculating these measures. We aimed to show that these measures are problematic with preventive factors prior to recoding, and to clarify the recoding method to be used to circumvent these problems. Recoding of preventive factors should be done such that the stratum with the lowest risk becomes the reference category when both factors are considered jointly (rather than one at a time). We used data from a case-control study on the interaction between ACE inhibitors and the ACE gene on incident diabetes. Use of ACE inhibitors was a preventive factor and DD ACE genotype was a risk factor. Before recoding, the RERI, AP and S showed inconsistent results (RERI = 0.26 [95%CI: −0.30; 0.82], AP = 0.30 [95%CI: −0.28; 0.88], S = 0.35 [95%CI: 0.02; 7.38]), with the first two measures suggesting positive interaction and the third negative interaction. After recoding the use of ACE inhibitors, they showed consistent results (RERI = −0.37 [95%CI: −1.23; 0.49], AP = −0.29 [95%CI: −0.98; 0.40], S = 0.43 [95%CI: 0.07; 2.60]), all indicating negative interaction. Preventive factors should not be used to calculate measures of interaction on an additive scale without recoding.