ELI: An Interactive Elicitation Technique for Subjective Probability Distributions

Uncertain knowledge about continuous quantities is usually formalized through subjective probability distributions (SPD′s). However, results from past experimental research have often pointed to the rather poor quality of SPD′s. Moreover, previous attempts to improve SPD quality frequently appeared to be only partly successful; overconfidence especially proved to be an exceptionally persistent phenomenon. In the present study, method-induced biases in particular are considered to be responsible for the poor quality of SPD′s, and an alternative means for eliciting uncertain knowledge was designed to meet fundamental quality criteria to a more satisfactory degree. The direct realization of a proper scoring rule in a graphically oriented interactive computer program is one of the central features of this new elicitation technique, ELI. An experiment with 48 subjects was carried out to evaluate ELI performance and to compare it with the performance of (a) an existing elicitation technique and (b) a simple technique that only asks for a best guess and a lower and upper bound. The three techniques were evaluated and compared with respect to their contribution to reliable and valid SPD′s and with respect to their practical usefulness. The results indicated that for ELI the fundamental requirements were realized to a great extent. Furthermore, compared with the two other techniques, ELI performance appeared to be superior. In particular, only with the support of ELI did overconfidence appear to be almost absent.     

Order effects in belief updating with consistent and inconsistent evidence

The current study tests for the presence of differential order effects in evaluation tasks with consistent and inconsistent evidence as predicted by the Hogarth and Einhorn (1992) belief-adjustment model. The results, based on both between-subjects and within-subjects experiments, demonstrate that there were significant recency effects with inconsistent evidence as predicted, larger recency effects when the inconsistent evidence was farther apart in subjective value as predicted, and significant recency effects even when subjects were given training designed to both help them understand the task as completely as possible and to be better able to assess the pieces of evidence. By including a within-subjects design, we were able to demonstrate that the difference in subjective value between two pieces of evidence is the primary factor influencing the magnitude of the recency effect, regardless of whether the evidence is consistent or inconsistent. This latter finding is unique and contrary to previous research and theory.     

Procedural fairness and self-esteem

We argue that people's self-esteem is affected by the fairness of procedures to which they are subjected; unfair treatment will lower self-esteem. Moreover, since this influence on self-esteem is presumably due to the implicit evaluation expressed by the choice of procedure and hence by the evaluation expressed by the person implementing the procedure, people's concern with the fairness of treatment will be focused on the interactional aspects of the procedure. In two experiments designed to test these hypotheses subjects received either a high or a low grade on an ability test on the basis of either fair or unfair grading procedures. The results of Experiment 1 indicated that subjects' self-esteem was lower after unfair treatment, and this influence was only apparent when subjects received high test feedback. Additionally, ratings of the fairness of the interaction were lower following unfair grading procedures. Experiment 2 also manipulated level of involvement with the test. Self-esteem was affected by procedural fairness and procedural fairness influenced perceived fairness of the interaction only in the high involvement condition.     

Racial bias and the MCMI

We studied the scores obtained on the Millon Clinical Multiaxial Inventory (MCMI) by Black and White male psychiatric inpatients to determine the presence or absence of racial bias. In predicting psychopathology for the two races, comparisons of MCMI performance indicated significant differences for all diagnoses except the personality disorders. The subjects were then matched into two groups of 209 patients each, according to DSM-III psychiatric diagnoses. The data were analyzed at the item, scale, and structural levels. At the item level, application of the Mantel-Haenszel Procedure revealed that 45 of the 175 items of the inventory were answered significantly different by the two racial groups. Because this number was higher than what could be expected by chance, the finding suggested possible deficiencies in terms of the culture-fairness of the items used in the test. At the scale level, an analysis of variance (ANOVA) demonstrated that the scores obtained by the Black and White groups were significantly different in 9 of the 20 scales (Histrionic, Narcissistic, Antisocial, Paraphrenia, Hypomania, Dysthymia, Alcohol Abuse, Drug Abuse, and Psychotic Delusion). With the exception of the Dysthymic scale, all of the differences were in the direction of the Blacks obtaining a higher score than the Whites. At the structural level, however, a principal components factor analysis performed on each group resulted in factor structures that looked identical.     

An on-line assessment of causal reasoning during comprehension

Fletcher and Bloom (1988) have argued that as readers read narratives, clause by clause, they repeatedly focus their attention on the last preceding clause that contains antecedents but no consequences in the text. This strategy allows them to discover a causal path linking the text's opening to its final outcome while minimizing the number of times long-term memory must be searched for missing antecedents or consequences. In order to test this hypothesis, we examined the reading times of 25 subjects for each clause of eight simple narrative texts. The results show that: (1) causal links between clauses that co-occur in short-term memory (as predicted by the strategy) increase the time required to read the second clause; (2) potential causal links between clauses that never co-occur in short-term memory (again as predicted by the strategy) have no effect on reading time; and (3) reinstatement searches are initiated at the end of sentences that are causally unrelated to the contents of short-term memory or that contain clauses that satisfy goals no longer in short-term memory. These results support the claim that subjects engage in a form of causal reasoning when they read simple narrative texts.