Normative and Substantive Expertise in Multiple Hypotheses Evaluation

We distinguish two criteria for evaluating the judgments of trained professionals. One criterion is conformance with a theoretical model and the other is conformance with known external criteria. Further, we label judgments that depart from a theoretical model as errors and those that depart from known external criteria as mistakes. Following this distinction, we hypothesize that auditors′ multiple hypotheses judgments will be characterized by errors but not mistakes. This hypothesis was tested by asking professional auditors to evaluate multiple hypotheses. The results confirm our expectations. Auditors′ judgments reflected ecological base rate information, and they appropriately ignored nondiagnostic evidence. Moreover, auditors did not exhibit a perseverance bias and 84% of them identified the correct hypothesis. The absence of mistakes reflect substantive expertise. Conversely, auditors′ probabilities were not additive, and, when a hypothesis was eliminated, they did not adjust beliefs for the remaining hypotheses. These errors reflect a lack of normative expertise. A second study employed inexperienced subjects to establish whether the strong substantive performance of the professional auditors was attributed to their expertise or was an artifact of the task. The results from the second study were strongly supportive of the substantive expertise explanation. Taken together, these results suggest that substantive expertise can help contain mistakes but it is not sufficient to mitigate errors. Further, lack of normative expertise can lead to errors but these errors do not translate into mistakes. The paper concludes that the distinction between normative and substantive expertise on one hand and errors and mistakes on the other is crucial to understanding when basic findings will generalize to professional settings.