Statistics Seminar - Design-based Inference for Bandit Algorithms

Dates

Tuesday, October 19, 2021 - 14:00 to 15:00

Speaker:   Sofía S. Villar (MRC Biostatistics Unit, University of Cambridge)

Abstract: 

The use of bandit algorithms to conduct response-adaptive randomised experiments may improve performance in terms of regret (or rewards) but it poses major problems for traditional statistical inference (e.g., biased estimators, inflated type-I error and reduced power). Recent work to address these problems typically work by imposing restrictions on the "exploitative" nature of the bandit algorithm, thus trading-off regret and inference goals. Additionally, these methods also require large sample sizes to ensure asymptotic guarantees. However, in many contexts bandit algorithms could be best used in exploratory experiments that are also tightly constrained in its size or duration (e.g. pilot studies for clinical trials).

Increasing power in such (small) experiments, without having to limit the adaptive nature of the algorithm, could allow for more promising interventions to reach a larger confirmatory experimental phase.

In this talk, I will present a novel hypothesis test approach particularly tailored for highly "exploitative" adaptive bandit algorithms. Such design-based tests are centred on the allocation probabilities of the underlying bandit algorithm, and do not require constraining its adaptive nature or a minimum experimental size. I will illustrate the finite-sample performance of the test for two different bandit algorithms, the CARA Forward Looking Gittins Index  (introduced by Villar & Rosenberger, 2017) and Thompson Sampling ( first introduced by Thompson, 1933), using both extensive simulations and a real-world field experiment.

Sofia is a MRC Investigator in the clinical trials methodology, working as part of the Design and Analysis of Randomised Trials (DART) theme at the Biostatistics Unit (University of Cambridge). She leads the statistics team at Papworth Hospital Trials Collaboration Unit (now based in Cambridge). She was awarded the first Biometrika post-doctoral fellowship to work her research project entitled "Bandit models for the optimal design of Clinical Trials"