Background: Regulators are evaluating the use of non-interventional real-world evidence (RWE) studies to assess the effectiveness of medical products. The RCT-DUPLICATE initiative uses a structured process to design RWE studies emulating randomized controlled trials (RCTs) and compare results. Here, we report findings of the first 10 trial emulations, evaluating cardiovascular outcomes of antidiabetic or antiplatelet medications. Methods: We selected 3 active-controlled and 7 placebo-controlled RCTs for replication. Using patient-level claims data from US commercial and Medicare payers, we implemented inclusion/exclusion criteria, selected primary endpoints, and comparator populations to emulate those of each corresponding RCT. Within the trial-mimicking populations, we conducted propensity score matching to control for >120 pre-exposure confounders. All study parameters were prospectively defined and protocols registered before hazard ratios (HRs) and 95% confidence intervals (CIs) were computed. Success criteria for the primary analysis were pre-specified for each replication. Results: Despite attempts to emulate RCT design as closely as possible, differences between the RCT and corresponding RWE study populations remained. The regulatory conclusions were equivalent in 6 of 10. The RWE emulations achieved a HR estimate that was within the 95% CI from the corresponding RCT in 8 of 10 studies. In 9 of 10, either the regulatory or estimate agreement success criteria were fulfilled. The largest differences in effect estimates were found for RCTs where second-generation sulfonylureas were used as a proxy for placebo regarding cardiovascular effects. Nine of 10 replications had a standardized difference between effect estimates of <2, which suggests differences within expected random variation. Conclusions: Agreement between RCT and RWE findings varies depending on which agreement metric is used. Interim findings indicate that selection of active comparator therapies with similar indications and use patterns enhances the validity of RWE. Even in the context of active comparators, concordance between RCT and RWE findings is not guaranteed, partially because trials are not emulated exactly. More trial emulations are needed to understand how often and in what contexts RWE findings match RCTs. Clinical Trial Registration: URL: https://clinicaltrials.gov Unique Identifiers: NCT03936049, NCT04215523, NCT04215536, NCT03936010, NCT03936036, NCT03936062, NCT03936023, NCT03648424, NCT04237935, NCT04237922

The US Food and Drug Administration (FDA) is open to accepting real‐world evidence (RWE) to support its assessment of medical products. However, RWE stakeholders lack a shared understanding of FDA’s evidentiary expectations for the use of RWE in applications for new drugs and biologics. We conducted a systematic review of publicly available FDA approval documents from January 2019 to June 2021. We sought to quantify, by year, how many approvals incorporated RWE in any form, and the intended use of RWE in those applications. Among approvals with RWE intended to support safety and/or effectiveness, we classified whether and how those studies impacted FDA’s benefit‐risk considerations, whether those studies were incorporated into the product label, and the therapeutic area of the medical product. Finally, we qualified FDA’s documented feedback where available. We found that 116 approvals incorporated RWE in any form, with the proportion of approvals incorporating RWE increasing each year. Of these approvals, 88 included an RWE study intended to provide evidence of safety or effectiveness. Among these 88 approvals, 65 of the studies influenced FDA’s final decision and 38 were included in product labels. The 88 approvals spanned 18 therapeutic areas. FDA’s feedback on RWE study quality included methodological issues, sample size concerns, omission of patient level data, and other limitations. Based on these findings, we would anticipate that future guidance on FDA’s evidentiary expectations of RWE use will incorporate fit‐for‐purpose real‐world data selection and careful attention to study design and analysis.

The mpox 2022 outbreak was declared a public health emergency in July 2022. In August 2022, the MVA–BN vaccine received emergency use authorization in the United States (US) to target at-risk groups. This study (EUPAS104386) used HealthVerity’s administrative US healthcare data to generate real-world evidence for MVA–BN vaccine effectiveness and safety to prevent mpox disease in men who have sex with men (MSM) and transgender women, the most affected population during the 2022 mpox outbreak. Fully vaccinated subjects (two doses ≥ 28 days apart) were initially matched with five unvaccinated subjects on calendar date, age, US region, and insurance type. Subjects were followed from index date (14 days after the second dose) until death or data end to ascertain mpox occurrence. After propensity score adjustment, the MVA–BN vaccine effectiveness against mpox disease was 89% (95% CI: 12%, 99%) among those fully vaccinated; attenuated to 64% (95% CI: 40%, 78%) among those with any dose and 70% (95% CI: 44%, 84%) for those with only a single dose. One pericarditis adverse event of special interest was observed when the risk window was extended to 28 days. These results contribute to the totality of evidence supporting the favorable benefit/risk profile of the MVA–BN vaccine.

To describe the creation of prevalent new user (PNU) cohorts and compare the relative bias and computational efficiency of several alternative analytic and matching approaches in PNU studies. In a simulated cohort, we estimated the effect of a treatment of interest vs a comparator among those who switched to the treatment of interest using the originally proposed time-conditional propensity score (TCPS) matching, standardized morbidity ratio weighting (SMRW), disease risk scores (DRS), and several alternative propensity score matching approaches. For each analytic method, we compared the average RR (across 2000 replicates) to the known risk ratio (RR) of 1.00. SMRW and DRS yielded unbiased results (RR = 0.998 and 0.997, respectively). TCPS matching with replacement was also unbiased (RR = 0.999). TCPS matching without replacement was unbiased when matches were identified starting with patients with the shortest treatment history as initially proposed (RR = 0.999), but it resulted in very slight bias (RR = 0.983) when starting with patients with the longest treatment history. Similarly, creating a match pool without replacement starting with patients with the shortest treatment history yielded an unbiased estimate (RR = 0.997), but matching with the longest treatment history first resulted in substantial bias (RR = 0.903). The most biased strategy was matching after selecting one random comparator observation per individual that continued on the comparator (RR = 0.802). Multiple analytic methods can estimate treatment effects without bias in a PNU cohort. Still, researchers should be wary of introducing bias when selecting controls for complex matching strategies beyond the initially proposed TCPS.