Objective To examine whether the association of inadequate or unclear allocation concealment and lack of blinding with biased estimates of intervention effects varies with the nature of the intervention or outcome. Design Combined analysis of data from three meta-epidemiological studies based on collections of meta-analyses. Data sources 146 meta-analyses including 1346 trials examining a wide range of interventions and outcomes. Main outcome measures Ratios of odds ratios quantifying the degree of bias associated with inadequate or unclear allocation concealment, and lack of blinding, for trials with different types of intervention and outcome. A ratio of odds ratios <1 implies that inadequately concealed or non-blinded trials exaggerate intervention effect estimates. Results In trials with subjective outcomes effect estimates were exaggerated when there was inadequate or unclear allocation concealment (ratio of odds ratios 0.69 (95% CI 0.59 to 0.82)) or lack of blinding (0.75 (0.61 to 0.93)). In contrast, there was little evidence of bias in trials with objective outcomes: ratios of odds ratios 0.91 (0.80 to 1.03) for inadequate or unclear allocation concealment and 1.01 (0.92 to 1.10) for lack of blinding. There was little evidence for a difference between trials of drug and non-drug interventions. Except for trials with all cause mortality as the outcome, the magnitude of bias varied between meta-analyses. Conclusions The average bias associated with defects in the conduct of randomised trials varies with the type of outcome. Systematic reviewers should routinely assess the risk of bias in the results of trials, and should report meta-analyses restricted to trials at low risk of bias either as the primary analysis or in conjunction with less restrictive analyses.

To explore whether reported methodologic quality affects estimated intervention effects in randomized trials and contributes to discrepancies between the results of large randomized trials and small randomized trials in meta-analyses. Meta-analyses of randomized trials that included at least one large trial (≥ 1000 participants) were included, regardless of the therapeutic area. Eligible meta-analyses were identified through electronic searches and bibliographies of relevant articles. Full-length randomized trials. Methodologic quality was assessed according to reported randomization, double blinding, and follow-up as separate components and by using the Jadad composite scale. Fourteen meta-analyses involving 190 randomized trials from eight therapeutic areas were included. Compared with large trials, intervention effects were exaggerated in small trials with inadequate allocation sequence generation (ratio of odds ratios, 0.46 [95% CI, 0.25 to 0.83]; P = 0.011), inadequate allocation concealment (ratio of odds ratios, 0.49 [CI, 0.27 to 0.86]; P = 0.014), and no double blinding (ratio of odds ratios, 0.52 [CI, 0.28 to 0.96]; P = 0.01). Large trials did not differ significantly from small trials with adequate generation of the allocation sequence, adequate allocation concealment, or adequate double blinding. No association was seen between reported follow-up and intervention effects. The Jadad scale provided no additional information because the scale and the quality components overlapped substantially. Inadequate generation of the allocation sequence, allocation concealment, and double blinding lead to exaggerated estimates of intervention benefit and may contribute to discrepancies between the results of large randomized trials and small randomized trials in meta-analyses.

Review from JAMA — Mortality in Randomized Trials of Antioxidant Supplements for Primary and Secondary Prevention — Systematic Review and Meta-analysis — ContextAntioxidant supplements are used for prevention of several diseases.ObjectiveTo assess the effect of antioxidant supplements on mortality in randomized primary and secondary prevention trials.Data Sources and Trial SelectionWe searched electronic databases and bibliographies published by October 2005. All randomized trials involving adults comparing beta carotene, vitamin A, vitamin C (ascorbic acid), vitamin E, and selenium either singly or combined vs placebo or vs no intervention were included in our analysis. Randomization, blinding, and follow-up were considered markers of bias in the included trials. The effect of antioxidant supplements on all-cause mortality was analyzed with random-effects meta-analyses and reported as relative risk (RR) with 95% confidence intervals (CIs). Meta-regression was used to assess the effect of covariates across the trials.Data ExtractionWe included 68 randomized trials with 232 606 participants (385 publications).Data SynthesisWhen all low- and high-bias risk trials of antioxidant supplements were pooled together there was no significant effect on mortality (RR, 1.02; 95% CI, 0.98-1.06). Multivariate meta-regression analyses showed that low-bias risk trials (RR, 1.16; 95% CI, 1.05-1.29) and selenium (RR, 0.998; 95% CI, 0.997-0.9995) were significantly associated with mortality. In 47 low-bias trials with 180 938 participants, the antioxidant supplements significantly increased mortality (RR, 1.05; 95% CI, 1.02-1.08). In low-bias risk trials, after exclusion of selenium trials, beta carotene (RR, 1.07; 95% CI, 1.02-1.11), vitamin A (RR, 1.16; 95% CI, 1.10-1.24), and vitamin E (RR, 1.04; 95% CI, 1.01-1.07), singly or combined, significantly increased mortality. Vitamin C and selenium had no significant effect on mortality.ConclusionsTreatment with beta carotene, vitamin A, and vitamin E may increase mortality. The potential roles of vitamin C and selenium on mortality need further study.

Published evidence suggests that aspects of trial design lead to biased intervention effect estimates, but findings from different studies are inconsistent. This study combined data from 7 meta-epidemiologic studies and removed overlaps to derive a final data set of 234 unique meta-analyses containing 1973 trials. Outcome measures were classified as “mortality,” “other objective,” “or subjective,” and Bayesian hierarchical models were used to estimate associations of trial characteristics with average bias and between-trial heterogeneity. Intervention effect estimates seemed to be exaggerated in trials with inadequate or unclear (vs. adequate) random-sequence generation (ratio of odds ratios, 0.89 [95% credible interval {CrI}, 0.82 to 0.96]) and with inadequate or unclear (vs. adequate) allocation concealment (ratio of odds ratios, 0.93 [CrI, 0.87 to 0.99]). Lack of or unclear double-blinding (vs. double-blinding) was associated with an average of 13% exaggeration of intervention effects (ratio of odds ratios, 0.87 [CrI, 0.79 to 0.96]), and between-trial heterogeneity was increased for such studies (SD increase in heterogeneity, 0.14 [CrI, 0.02 to 0.30]). For each characteristic, average bias and increases in between-trial heterogeneity were driven primarily by trials with subjective outcomes, with little evidence of bias in trials with objective and mortality outcomes. This study is limited by incomplete trial reporting, and findings may be confounded by other study design characteristics. Bias associated with study design characteristics may lead to exaggeration of intervention effect estimates and increases in between-trial heterogeneity in trials reporting subjectively assessed outcomes.

Objective To determine whether treatment with agonists of glucagon-like peptide-1 receptor (GLP-1R) result in weight loss in overweight or obese patients with or without type 2 diabetes mellitus.Design Systematic review with meta-analyses.Data sources Electronic searches (Cochrane Library, Medline, Embase, and Web of Science) and manual searches (up to May 2011).Review methods Randomised controlled trials of adult participants with a body mass index of 25 or higher; with or without type 2 diabetes mellitus; and who received exenatide twice daily, exenatide once weekly, or liraglutide once daily at clinically relevant doses for at least 20 weeks. Control interventions assessed were placebo, oral antidiabetic drugs, or insulin.Data extraction Three authors independently extracted data. We used random effects models for the primary meta-analyses. We also did subgroup, sensitivity, regression, and sequential analyses to evaluate sources of intertrial heterogeneity, bias, and the robustness of results after adjusting for multiple testing and random errors.Results 25 trials were included in the analysis. GLP-1R agonist groups achieved a greater weight loss than control groups (weighted mean difference −2.9 kg, 95% confidence interval –3.6 to –2.2; 21 trials, 6411 participants). We found evidence of intertrial heterogeneity, but no evidence of bias or small study effects in regression analyses. The results were confirmed in sequential analyses. We recorded weight loss in the GLP-1R agonist groups for patients without diabetes (–3.2 kg, –4.3 to –2.1; three trials) as well as patients with diabetes (–2.8 kg, –3.4 to –2.3; 18 trials). In the overall analysis, GLP-1R agonists had beneficial effects on systolic and diastolic blood pressure, plasma concentrations of cholesterol, and glycaemic control, but did not have a significant effect on plasma concentrations of liver enzymes. GLP-1R agonists were associated with nausea, diarrhoea, and vomiting, but not with hypoglycaemia.Conclusions The present review provides evidence that treatment with GLP-1R agonists leads to weight loss in overweight or obese patients with or without type 2 diabetes mellitus.

Background Results from apparently conclusive meta-analyses may be false. A limited number of events from a few small trials and the associated random error may be under-recognized sources of spurious findings. The information size (IS, i.e. number of participants) required for a reliable and conclusive meta-analysis should be no less rigorous than the sample size of a single, optimally powered randomized clinical trial. If a meta-analysis is conducted before a sufficient IS is reached, it should be evaluated in a manner that accounts for the increased risk that the result might represent a chance finding (i.e. applying trial sequential monitoring boundaries).