Analytic Confidence and Political Decision‐Making: Theoretical Principles and Experimental Evidence From National Security Professionals

When making decisions under uncertainty, it is important to distinguish between the probability that a judgment is true and the confidence analysts possess in drawing their conclusions. Yet analysts and decision‐makers often struggle to define “confidence” in this context, and many ways that scholars use this term do not necessarily facilitate decision‐making under uncertainty. To help resolve this confusion, we argue for disaggregating analytic confidence along three dimensions: reliability of available evidence, range of reasonable opinion, and responsiveness to new information. After explaining how these attributes hold different implications for decision‐making in principle, we present survey experiments examining how analysts and decision‐makers employ these ideas in practice. Our first experiment found that each conception of confidence distinctively influenced national security professionals' evaluations of high‐stakes decisions. Our second experiment showed that inexperienced assessors of uncertainty could consistently discriminate among our conceptions of confidence when making political forecasts. We focus on national security, where debates about defining “confidence levels” have clear practical implications. But our theoretical framework generalizes to nearly any area of political decision‐making, and our empirical results provide encouraging evidence that analysts and decision‐makers can grasp these abstract elements of uncertainty.