| Abstract: | Uncertain knowledge about continuous quantities is usually formalized through subjective probability distributions (SPD′s). However, results from past experimental research have often pointed to the rather poor quality of SPD′s. Moreover, previous attempts to improve SPD quality frequently appeared to be only partly successful; overconfidence especially proved to be an exceptionally persistent phenomenon. In the present study, method-induced biases in particular are considered to be responsible for the poor quality of SPD′s, and an alternative means for eliciting uncertain knowledge was designed to meet fundamental quality criteria to a more satisfactory degree. The direct realization of a proper scoring rule in a graphically oriented interactive computer program is one of the central features of this new elicitation technique, ELI. An experiment with 48 subjects was carried out to evaluate ELI performance and to compare it with the performance of (a) an existing elicitation technique and (b) a simple technique that only asks for a best guess and a lower and upper bound. The three techniques were evaluated and compared with respect to their contribution to reliable and valid SPD′s and with respect to their practical usefulness. The results indicated that for ELI the fundamental requirements were realized to a great extent. Furthermore, compared with the two other techniques, ELI performance appeared to be superior. In particular, only with the support of ELI did overconfidence appear to be almost absent. |
