EQUATING INVERSE PROBABILITIES IN IMPLICIT PERSONALITY JUDGMENTS

Abstract— Meehl and Rosen (1955) observed that their colleagues tended to equate the sensitivity of a sign (its probability given a diagnostic category) with its predictive accuracy (the probability of the diagnostic category given the sign). Such equating of inverse probabilities implies ignoring base rates. Later, many researchers demonstrated such neglect in a number of experimental contexts involving hypothetical judgments; others have found some use of base rates—but only when base rates were supplied by the experimenter, in which case even irrelevant base rates influenced judgment. The present study tested for equating conditional probabilities in a context similar to the contexts involved in the initial observations—that is, subjects strove for accuracy rather than attempted to make good judgments about hypothetical problems, and the base rates were generated by the subjects themselves. Neglect of base rates was revealed by within-individual comparisons showing subjects equated inverse probabilities without equating the corresponding unconditional ones.     

Learning from experience in nonlinear environments: Evidence from a competition scenario

We test people’s ability to learn to estimate a criterion (probability of success in a competition scenario) that requires aggregating information in a nonlinear manner. The learning environments faced by experimental participants are kind in that they are characterized by immediate, accurate feedback involving either naturalistic outcomes (information on winning and/or ranking) or the normatively correct probabilities. We find no evidence of learning from the former and modest learning from the latter, except that a group of participants endowed with a memory aid performed substantially better. However, when the task is restructured such that information should be aggregated in a linear fashion, participants learn to make more accurate assessments. Our experiments highlight the important role played by prior beliefs in learning tasks, the default status of linear aggregation in many inferential judgments, and the difficulty of learning in nonlinear environments even in the presence of veridical feedback.     

Diagnostic causal reasoning with verbal information

In diagnostic causal reasoning, the goal is to infer the probability of causes from one or multiple observed effects. Typically, studies investigating such tasks provide subjects with precise quantitative information regarding the strength of the relations between causes and effects or sample data from which the relevant quantities can be learned. By contrast, we sought to examine people’s inferences when causal information is communicated through qualitative, rather vague verbal expressions (e.g., “X occasionally causes A”). We conducted three experiments using a sequential diagnostic inference task, where multiple pieces of evidence were obtained one after the other. Quantitative predictions of different probabilistic models were derived using the numerical equivalents of the verbal terms, taken from an unrelated study with different subjects. We present a novel Bayesian model that allows for incorporating the temporal weighting of information in sequential diagnostic reasoning, which can be used to model both primacy and recency effects. On the basis of 19,848 judgments from 292 subjects, we found a remarkably close correspondence between the diagnostic inferences made by subjects who received only verbal information and those of a matched control group to whom information was presented numerically. Whether information was conveyed through verbal terms or numerical estimates, diagnostic judgments closely resembled the posterior probabilities entailed by the causes’ prior probabilities and the effects’ likelihoods. We observed interindividual differences regarding the temporal weighting of evidence in sequential diagnostic reasoning. Our work provides pathways for investigating judgment and decision making with verbal information within a computational modeling framework.     

Asymmetries in predictive and diagnostic reasoning

In this article, we address the apparent discrepancy between causal Bayes net theories of cognition, which posit that judgments of uncertainty are generated from causal beliefs in a way that respects the norms of probability, and evidence that probability judgments based on causal beliefs are systematically in error. One purported source of bias is the ease of reasoning forward from cause to effect (predictive reasoning) versus backward from effect to cause (diagnostic reasoning). Using causal Bayes nets, we developed a normative formulation of how predictive and diagnostic probability judgments should vary with the strength of alternative causes, causal power, and prior probability. This model was tested through two experiments that elicited predictive and diagnostic judgments as well as judgments of the causal parameters for a variety of scenarios that were designed to differ in strength of alternatives. Model predictions fit the diagnostic judgments closely, but predictive judgments displayed systematic neglect of alternative causes, yielding a relatively poor fit. Three additional experiments provided more evidence of the neglect of alternative causes in predictive reasoning and ruled out pragmatic explanations. We conclude that people use causal structure to generate probability judgments in a sophisticated but not entirely veridical way.     

Integration of base rates and new information in an abstract hypothesis‐testing task

In two studies, we investigated how people use base rates and the presence versus the absence of new information to judge which of two hypotheses is more likely. Participants were given problems based on two decks of cards printed with 0–4 letters. A table showed the relative frequencies of the letters on the cards within each deck. Participants were told the letters that were printed on or absent from a card the experimenter had drawn. Base rates were conveyed by telling participants that the experimenter had chosen the deck by drawing from an urn containing, in different proportions, tickets marked either ‘deck 1’ or ‘deck 2’. The task was to judge from which of the two decks the card was most likely drawn. Prior probabilities and the evidential strength of the subset of present clues (computed as ‘weight of evidence’) were the only significant predictors of participants’ dichotomous (both studies) and continuous (Study 2) judgments. The evidential strength of all clues was not a significant predictor of participants’ judgments in either study, and no significant interactions emerged. We discuss the results as evidence for additive integration of base rates and the new present information in hypothesis testing.     

Schematic knowledge changes what judgments of learning predict in a source memory task

Source monitoring can be influenced by information that is external to the study context, such as beliefs and general knowledge (Johnson, Hashtroudi, & Lindsay, 1993). We investigated the extent to which metamnemonic judgments predict memory for items and sources when schematic information about the sources is or is not provided at encoding. Participants made judgments of learning (JOLs) to statements presented by two speakers and were informed of the occupation of each speaker either before or after the encoding session. Replicating earlier work, prior knowledge decreased participants’ tendency to erroneously attribute statements to schematically consistent but episodically incorrect speakers. The origin of this effect can be understood by examining the relationship between JOLs and performance: JOLs were equally predictive of item and source memory in the absence of prior knowledge, but were exclusively predictive of source memory when participants knew of the relationship between speakers and statements during study. Background knowledge determines the information that people solicit in service of metamnemonic judgments, suggesting that these judgments reflect control processes during encoding that reduce schematic errors.     

When learning order affects sensitivity to base rates: challenges for theories of causal learning

In three experiments we investigated whether two procedures of acquiring knowledge about the same causal structure, predictive learning (from causes to effects) versus diagnostic learning (from effects to causes), would lead to different base-rate use in diagnostic judgments. Results showed that learners are capable of incorporating base-rate information in their judgments regardless of the direction in which the causal structure is learned. However, this only holds true for relatively simple scenarios. When complexity was increased, base rates were only used after diagnostic learning, but were largely neglected after predictive learning. It could be shown that this asymmetry is not due to a failure of encoding base rates in predictive learning because participants in all conditions were fairly good at reporting them. The findings present challenges for all theories of causal learning.     

Social stereotypes and judgments of individuals: An instance of the base-rate fallacy

Social stereotypes may be defined as beliefs that various traits or acts are characteristic of particular social groups. As such, stereotypic beliefs represent subjective estimates of the frequencies of attributes within social groups, and so should be expected to “behave like” base-rate information within the context of judgments of individuals: specifically, individuating target case information should induce subjects to disregard their own stereotypic beliefs. Although the results of previous research are consisten with this prediction, no studies have permitted normative evaluation of stereotypic judgments. Because the hypothesis equates base rates and stereotypes, normative evaluation is essential for demonstrating equivalence between the base-rate fallacy and neglect of stereotypes in the presence of individuating case information. Two experiments were conducted, allowing for normative evaluation of effects of stereotypes on judgments of individuals. The results confirmed the hypothesis and established the generalizability of the effect across controversial and uncontroversial, socially desirable and socially underirable stereotypic beliefs. More generally, an examination of the differences between intuitive and normative statistical models of the judgment task suggest that the base-rate fallacy is but one instance of a general characteristic of intuitive judgment processes: namely, the failure to appropriately adjust evaluations of any one cue in the light of concurrent evaluations of other cues.     

The impact of domain-specific beliefs on decisions and causal judgments

Extensive evidence suggests that people often rely on their causal beliefs in their decisions and causal judgments. To date, however, there is a dearth of research comparing the impact of causal beliefs in different domains. We conducted two experiments to map the influence of domain-specific causal beliefs on the evaluation of empirical evidence when making decisions and subsequent causal judgments. Participants made 120 decisions in a two-alternative forced-choice task, framed in either a medical or a financial domain. Before each decision, participants could actively search for information about the outcome (“occurrence of a disease” or “decrease in a company's share price”) on the basis of four cues. To analyze the strength of causal beliefs, we set two cues to have a generative relation to the outcome and two to have a preventive relation to the outcome. To examine the influence of empirical evidence, we manipulated the predictive power (i.e., cue validities) of the cues. Both experiments included a validity switch, where the four selectable cues switched from high to low validity or vice versa. Participants had to make a causal judgment about each cue before and after the validity switch. In the medical domain, participants stuck to the causal information in causal judgments, even when evidence was contradictory, while decisions showed an effect of both empirical and causal information. In contrast, in the financial domain, participants mainly adapted their decisions and judgments to the cue validities. We conclude that the strength of causal beliefs (1) is shaped by the domain, and (2) has a differential influence on the degree to which empirical evidence is taken into account in causal judgments and decision making.     

The Effect of Inference Order and Experience-Related Knowledge on Diagnostic Conjunction Probabilities

Ideally, a decision maker′s diagnostic probability judgments should not be affected by making predictive judgments before making diagnostic inferences. The purpose of this study is to investigate how experience-related knowledge and the inference presentation order affect a decision maker′s diagnostic conjunction probability judgments. Specifically, when decision makers are asked to make diagnoses in different judgment domains with which they have different levels of experience, we examine how making predictions first affects their subsequent diagnostic judgments in a standard conjunction paradigm. Professional auditors with experience in the auditing domain and MBA students with little or no auditing experience participated in the experiment. The results indicate that when the task involves a domain with which people have experience, making predictions prior to diagnoses has a significant influence on their subsequent diagnostic conjunction probabilities. When auditors made diagnoses in a familiar audit task situation, they were strongly influenced by whether or not they were asked to make predictions in advance. However, there was no influence of inference order on auditors′ diagnoses in a medical task, with which they do not have experience-related knowledge. Similarly, MBA students, having no experience-related knowledge in either audit or medical domains, were not affected by the inference order in making diagnoses. In the discussion of these exploratory results, we suggest that this inference order effect may be due to subjects′ anchoring on the predictive probability and insufficiently adjusting it to yield the diagnostic probability judgment.     

Sample representativeness affects whether judgments are influenced by base rate or sample size

We investigated how people use base rates and sample size information when combining data to make overall probability judgments. Participants considered two samples from an animal population in order to estimate the probability of that animal being aggressive. Participants' judgments were influenced by subpopulation base rates when they were provided and linked to specific samples. When samples were not identified as coming from different subpopulations, judgments typically reflected sample size information. We conclude that 1) People can use base rates when combining samples to make an inference; 2) People can correctly use sampling information to determine when to use base rates, and 3) People are able to consider base rate and sample size information at the same time. Additionally, we found that individuals' numeracy correlates with the extent to which base rate and sample size information is used.     

The power of causal beliefs and conflicting evidence on causal judgments and decision making

In two experiments, we investigated the relative impact of causal beliefs and empirical evidence on both decision making and causal judgments, and whether this relative impact could be altered by previous experience. Participants had to decide which of two alternatives would attain a higher outcome on the basis of four cues. After completing the decision task, they were asked to estimate to what extent each cue was a reliable cause of the outcome. Participants were provided with instructions that causally related two of the cues to the outcome, whereas they received neutral information about the other two cues. Two of the four cues—a causal and a neutral cue—had high validity and were both generative. The remaining two cues had low validity, and were generative in Experiment 1, but almost not related to the outcome in Experiment 2. Selected groups of participants in both experiments received pre-training with either causal or neutral cues, or no pre-training was provided. Results revealed that the impact of causal beliefs and empirical evidence depends on both the experienced pre-training and cue validity. When all cues were generative and participants received pre-training with causal cues, they mostly relied on their causal beliefs, whereas they relied on empirical evidence when they received pre-training with neutral cues. In contrast, when some of the cues were almost not related to the outcome, participants’ responses were primarily influenced by validity and—to a lesser extent—by causal beliefs. In either case, however, the influence of causal beliefs was higher in causal judgments than in decision making. While current theoretical approaches in causal learning focus either on the effect of causal beliefs or empirical evidence, the present research shows that both factors are required to explain the flexibility involved in human inferences.     

Strategy selection in causal reasoning: when beliefs and covariation collide.

The present study investigated how people combine covariation information (Cheng & Novick, 1990, 1992) with pre-existing beliefs (White, 1989) when evaluating causal hypotheses. Three experiments, using both within- and between-subjects designs, found that the use of covariation information and beliefs interacted, such that the effects of covariation were larger when people assessed hypotheses about believable than about unbelievable causal candidates. In Experiment 2, this interaction was observed when participants made judgments in stages (e.g., first evaluating covariation information about a causal candidate and then evaluating the believability of a candidate), as well as when the information was presented simultaneously. Experiment 3 demonstrated that this pattern was also reflected in participants' metacognitive judgments: Participants indicated that they weighed covariation information more heavily for believable than unbelievable candidates. Finally, Experiments 1 and 2 demonstrated the presence of individual differences in the use of covariation- and belief-based cues. That is, individuals who tended to base their causality judgments primarily on belief were less likely to make use of covariation information and vice versa. The findings were most consistent with White's (1989) causal power theory, which suggests that covariation information is more likely to be considered relevant to believable than unbelievable causes.     

The effects of prior expectations and outcome knowledge on polygraph examiners' decisions

The present study deals with the question of whether judgments made by experts working in familiar contexts are affected by prior expectations and beliefs. Two experiments in which prior expectations were manipulated were designed to determine whether and to what extent polygraph examiners are affected by their prior expectations when analyzing and interpreting polygraph charts. Prior expectations affected the examiners' judgments when the polygraph charts did not include clear indications of guilt or innocence, but when the objective physiological evidence included strong indications which clearly contradicted the examiner's expectations, judgments were not affected by these expectations. Theoretical and practical implications of these results are discussed.     

Dyadic judgments based on conflicting samples: The failure to ignore invalid input

The present research demonstrates a so far unrecognized impediment of group performance, metacognitive myopia (Fiedler, 2012). Judges and decision-makers follow the given samples of information uncritically and neglect the metacognitive assessment of the samples' validity. Applying this notion to dyadic judgments, we instructed dyads to jointly estimate conditional probabilities p (Win|A) and p (Win|B) of Lotteries A and B. One person per dyad experienced a valid sample (winning rates conditional on lotteries). The other person experienced an invalid, reverse sample (lotteries conditional on winning). Whereas valid samples provide unbiased estimates of lotteries' winning probabilities, invalid samples can greatly misrepresent the association of winning and lotteries (depending on lottery base rates). Across three experiments, metacognitive myopia—both at the individual and at the dyadic group level—prevented participants from discriminating valid and invalid samples. Group judgments were biased toward erroneous implications of invalid samples, reflecting an equality bias among unequal group members.     

A generalized signal detection model to predict rational variation in base rate use

The predominant models of rational behavior currently used to analyze a large class of experiments imply that subjects neglect or place insufficient weight on base rates when making probabilistic judgments. We argue that the evidence is inadequate for this conclusion because the models make needlessly restrictive assumptions about how base rates should be used. The restrictive assumptions stem from a misuse of Bayes' rule that ignores specific aspects of how the proportions arose. We develop a model of rational behavior that generalizes signal detection theory to reflect the environment subjects routinely face and we reexamine the relevant experimental literature. Variation observed in subjects' responses to base rate information is explained by the present rational model more fully than by extant models.     

Normative and Substantive Expertise in Multiple Hypotheses Evaluation

We distinguish two criteria for evaluating the judgments of trained professionals. One criterion is conformance with a theoretical model and the other is conformance with known external criteria. Further, we label judgments that depart from a theoretical model as errors and those that depart from known external criteria as mistakes. Following this distinction, we hypothesize that auditors′ multiple hypotheses judgments will be characterized by errors but not mistakes. This hypothesis was tested by asking professional auditors to evaluate multiple hypotheses. The results confirm our expectations. Auditors′ judgments reflected ecological base rate information, and they appropriately ignored nondiagnostic evidence. Moreover, auditors did not exhibit a perseverance bias and 84% of them identified the correct hypothesis. The absence of mistakes reflect substantive expertise. Conversely, auditors′ probabilities were not additive, and, when a hypothesis was eliminated, they did not adjust beliefs for the remaining hypotheses. These errors reflect a lack of normative expertise. A second study employed inexperienced subjects to establish whether the strong substantive performance of the professional auditors was attributed to their expertise or was an artifact of the task. The results from the second study were strongly supportive of the substantive expertise explanation. Taken together, these results suggest that substantive expertise can help contain mistakes but it is not sufficient to mitigate errors. Further, lack of normative expertise can lead to errors but these errors do not translate into mistakes. The paper concludes that the distinction between normative and substantive expertise on one hand and errors and mistakes on the other is crucial to understanding when basic findings will generalize to professional settings.     

Feeling of knowing and duration of unsuccessful memory search

Five experiments examined the impact of feeling of knowing on decisions to continue or to terminate the search of memory in question answering. First, two pairs of experiments respectively scrutinized knowledge about (1) ordinary facts and (2) national capitals. The first experiment of each pair extracted normative data: The participants indicated whether they had probably once known the answer to a question (once-knew-it scale), supplied the answer if they knew it, and either judged the likelihood of their recognizing the answer or made other pertinent metacognitive judgments. In these norming experiments, recognition ratings were highly correlated with once-knew-it responses, and both measures were highly predictive of performance. Thisindicated that both measures reflect feeling of knowing judgments. In the second experiment of each pair, different participants were timed as they indicated whether they knew the answer to the same questions. Responselatencies for responding "don't know" were strongly positively correlated with the once-knew-it judgments made by the norming participants. This relationship was corroborated by Experiment 5, which compared the crucial measures within participants. These outcomes suggest that, in this context, feeling of knowing judgments are predictive of how long people will search memory for requested information.