7 November 2025 in medRxiv
Background: Reproducing published findings from clinical trials is a critical component of scientific transparency, yet it remains a challenging and under-practiced task. Despite increasing emphasis on reproducibility and data reuse in research policies, few real-world examples exist where independent teams have reproduced complex analyses using clinical trial data. In this case study, the aim was to independently reproduce the key findings of a high-impact clinical trial on rectal cancer treatment using shared trial data. Method: We organized a multi-team datathon, where each team was provided with the same dataset and supporting material, and was tasked to reproduce the results of the CAO/ARO/AIO-04 trial, with optional additional analysis. We contacted the original investigators for data access and reuse, and consulted them to understand the study, clinically and scientifically. Results: Five teams used R or Python to reproduce the statistical results, and the corresponding scripts can be found on Gitlab. All teams reproduced the analyses for primary outcome disease-free survival (DFS). The key findings on DFS were consistently reproduced, reinforcing confidence in the trial main conclusions. Result robustness was investigated using a different analytical software or statistical models. Nevertheless, challenges were encountered when the supplementary materials were not easily identified. Minor reporting issues were noticed in the reproduced paper. Conclusion: Reproduction of a major oncology clinical trial confirmed the reliability of its main conclusions. Divergences highlighted reporting gaps, such as incomplete protocols and broken links, that future trials should address. This case study demonstrates the value of systematic reproducibility checks for clinical research transparency and challenges in data sharing for reproducibility.
13 November 2026 in Statistics in Medicine
Hybrid randomized controlled trials (hybrid RCTs) integrate external control data, such as historical or concurrent data, with data from randomized trials. While numerous frequentist and Bayesian methods, such as the test‐then‐pool and the meta‐analytic‐predictive prior, have been developed to account for potential disagreement between the external control and randomized data, they cannot ensure strict Type I error rate control. However, these methods can reduce biases stemming from systematic differences between external controls and trial data. A critical yet underexplored issue in hybrid RCTs is the prespecification of external data to be used in the analysis. The validity of statistical conclusions in hybrid RCTs depends on the assumption that external control selection is independent of historical trial outcomes. In practice, historical data may be accessible during the planning stage, potentially influencing important decisions, such as which historical datasets to include or the sample size of the prospective part of the hybrid trial, thus introducing bias. Such data‐driven design choices can be an additional source of bias, which can occur even when historical and prospective controls are exchangeable. Through a simulation study, we quantify the bias introduced by outcome‐dependent selection of historical controls in hybrid RCTs using both Bayesian and frequentist approaches, and discuss potential strategies to mitigate this bias. Our scenarios consider variability and time trends in the historical studies, distributional shifts between historical and prospective control groups, sample sizes and allocation ratios, as well as the number of studies included. The impact of different hypothetical selection mechanisms for selecting external controls is demonstrated using a clinical trial example.