A picture is worth a thousandp values: On the irrelevance of hypothesis testing in the microcomputer age

Hypothesis testing, while by far the most common statistical technique for generating conclusions from data, is nonetheless not very informative. It emphasizes a banal and confusing question (“Is it true that some set of population means are not all identical to one another?”) whose answer is, in a mathematical sense, almost inevitably known (“No”). Hypothesis testing, as it is customarily implemented, ignores two issues that are generally much more interesting, important, and relevant: What is thepattern of population means over conditions, and what are the magnitudes of various variability measures (e.g., standard errors of the mean, estimates of population standard deviations)? The simple expedient of plotting relevant sample statistics with associated variability bars is a substantially better way of conveying the results of an experiment. In today’s microcomputer environment, there are many relatively cheap and easily available applications that allow one to do this. I make some brief, informal comments about some of these applications.