Decision-making under uncertainty: biases and Bayesians

Animals (including humans) often face circumstances in which the best choice of action is not certain. Environmental cues may be ambiguous, and choices may be risky. This paper reviews the theoretical side of decision-making under uncertainty, particularly with regard to unknown risk (ambiguity). We use simple models to show that, irrespective of pay-offs, whether it is optimal to bias probability estimates depends upon how those estimates have been generated. In particular, if estimates have been calculated in a Bayesian framework with a sensible prior, it is best to use unbiased estimates. We review the extent of evidence for and against viewing animals (including humans) as Bayesian decision-makers. We pay particular attention to the Ellsberg Paradox, a classic result from experimental economics, in which human subjects appear to deviate from optimal decision-making by demonstrating an apparent aversion to ambiguity in a choice between two options with equal expected rewards. The paradox initially seems to be an example where decision-making estimates are biased relative to the Bayesian optimum. We discuss the extent to which the Bayesian paradigm might be applied to the evolution of decision-makers and how the Ellsberg Paradox may, with a deeper understanding, be resolved.     

The ecological rationality of state-dependent valuation

Laboratory studies on a range of animals have identified a bias that seems to violate basic principles of rational behavior: a preference is shown for feeding options that previously provided food when reserves were low, even though another option had been found to give the same reward with less delay. The bias presents a challenge to normative models of decision making (which only take account of expected rewards and the state of the animal at the decision time). To understand the behavior, we take a broad ecological perspective and consider how valuation mechanisms evolve when the best action depends upon the environment being faced. We show that in a changing and uncertain environment, state-dependent valuation can be favored by natural selection: Individuals should allow their hunger to affect learning for future decisions. The valuation mechanism that typically evolves produces the kind of behavior seen in standard laboratory tests. By providing an insight into why learning should be affected by the state of an individual, we provide a basis for understanding psychological principles in terms of an animal's ecology.