Detection of errors in statistical problem solving

Ten subjects were asked to think aloud while solving two statistical problems. The subjects were instructed after each substep of his/her problem solving, to check in various ways the solution of the previous substep. The subjects detected 25 out of a total of 56 errors when they solved the problems. About half of the detected errors were computational errors. Nine errors were eliminated in response to the checking instructions. The think aloud data indicated that subjects' most common way of detecting their own errors was by noting that computations resulted in extreme values. Subjects also detected errors by (a) "spontaneous discovery"; (b) discontent with other aspects of a solution than the numerical value of the answer; (c) repeating a solution. The last mentioned type of error detection only occurred when subjects responded to the checking instructions. Finally it was found that subjects had a strong tendency to respond to the checking instructions either in a routinized or in a non-elaborated way. It was discussed how the formulation of checking instructions can be improved in order to avoid this effect.