Calibration of probability assessments by professional blackjack dealers, statistical experts, and lay people

The question addressed in this study is whether experts are better calibrated than lay people. We investigated how well people are calibrated when they assess the probabilities of card combinations in the game of blackjack. Three groups of subjects were used: professional dealers, statistical experts, and control subjects. The results showed that experience and statistical expertise do not make people better calibrated in this task. It is argued that the concept of calibration is not wholly appropriate to describe the obtained deviations from the normatively correct responses. This is illustrated by a discriminant analysis performed on the signed deviation scores, which resulted in an almost perfect separation of the three groups, although they were overlapping with respect to calibration.