Optimal experimental design for a class of bandit problems

Bandit problems are a class of sequential decision-making problems that are useful for studying human decision-making, especially in the context of understanding how people balance exploration with exploitation. A major goal of measuring people’s behavior using bandit problems is to distinguish between competing models of their decision-making. This raises a question of experimental design: How should a set of bandit problems be designed to maximize the ability to discriminate between models? We apply a previously developed design optimization framework to the problem of finding good bandit problem experiments, and develop computational sampling schemes for implementing the approach. We demonstrate the approach in a number of simple cases, varying the priors on parameters for some standard models. We also demonstrate the approach using empirical priors, inferred by hierarchical Bayesian analysis from human data, and show that optimally designed bandit problems significantly enhance the ability to discriminate between competing models.