5 January 2023 in Health Services and Outcomes Research Methodology
2 October 2020 in SSRN Electronic Journal
Researchers regularly use synthetic control methods for estimating causal effects when a sub-set of units receive a single persistent treatment, and the rest are unaffected by the change. In many applications, however, units not assigned to treatment are nevertheless impacted by the intervention because of cross-unit interactions. This paper extends the synthetic control methods to accommodate partial interference, allowing interactions within predefined groups, but not between them. Focusing on a class of causal estimands that capture the effect both on the treated and control units, we develop a multivariate Bayesian structural time series model for generating synthetic controls that would have occurred in the absence of an intervention enabling us to estimate our novel effects. In a simulation study, we explore our Bayesian procedure’s empirical properties and show that it achieves good frequentists coverage even when the model is misspecified. Our work is motivated by an analysis of a marketing campaign’s effectiveness by an Italian supermarket chain that permanently reduced the price of hundreds of store-brand products. We use our new methodology to make causal statements about the impact on sales of the affected store-brands and their direct competitors. Our proposed approach is implemented in the CausalMBSTS R package.
10 February 2026 in Technometrics
1 January 2024 in SSRN Electronic Journal
Researchers and practitioners have embraced factorial experiments to simultaneously test multiple treatments, each with different levels. With the rise of technologies like Generative AI, factorial experimentation has become even more accessible: it is easier than ever to generate different versions of potential treatments. Typically, in large-scale factorial experiments, the primary objective is to identify the treatment with the largest causal effect. This is especially true for experiments that suffer from measurement error, attrition, non-compliance, and censoring: point estimates are unreliable, but—as we show—the asymmetry in the largest treatment effect makes it possible to identify the most impactful treatment even when point estimates are biased. To exploit this asymmetry, we propose using a Fisher randomization test as a general non-parametric approach for inference, which we apply to an existing field experiment that measured intern performance at a large financial firm. We show that the earliest possible intervention has an immediate and enduring impact: performance improves in the week of the intervention and in future weeks, sometimes even to a greater extent than interventions in those future weeks. The takeaway—intervene early—has important consequences across the many contexts of workplace programs.