MUSE: Multi-Treatment Experiment Design for Winner Selection and Effect Estimation

Abstract

We study the design of experiments with multiple treatment levels, a setting common in clinical trials and online A/B/n testing. Unlike single-treatment studies, practical analyses of multi-treatment experiments typically first select a winning treatment, and then only estimate the effect therein. Motivated by this analysis paradigm, we propose a design for MUlti-treatment experiments that jointly maximizes the accuracy of winner Selection and effect Estimation (MUSE). Explicitly, we introduce a single objective that balances selection and estimation, and determine the unit allocation to treatments and control by optimizing this objective. Theoretically, we establish finite-sample guarantees and asymptotic equivalence between our proposal and the Neyman allocation for the true optimal treatment and control. Across simulations and a real data application, our method performs favorably in both selection and estimation compared to various standard alternatives.