Laypeople do use sample variance: The effect of embedding data in a variance-implying story

When using sample data to decide whether two populations differ, laypeople attend to the difference between group means, but largely overlook within-group variability (Obrecht, Chapman, & Gelman, 2007 Obrecht, N. A., Chapman, G. B. and Gelman, R. 2007. Intuitive t-tests: Lay use of statistical information. Psychonomic Bulletin & Review, 14: 1147–1152.  Google Scholar]). We show, first, that laypeople know about and use story-implied variability when making pairwise comparisons. Then we demonstrate that participants’ sensitivity to variance in a dataset is boosted when presented in a context that implies consistent variance information. Statistical data were couched in stories about electrical conductivity measurements obtained from element samples (low-variability category) or body weight measurements from samples of peoples (high-variability category). We manipulated, between participants, whether the data variance matched or mismatched the story-implied variability. Participants who received data in a matching context showed high sensitivity to variance, while those in the mismatching condition did not. Laypeople use statistical data to make reasonable inferences when those data are provided in a context that makes sense.