Selective hypothesis testing

A diverse set of biases that have been found to characterize judgment may be similarly mediated by a process of selective hypothesis testing. Our paper begins with a definition of selective hypothesis testing and an explanation of how and when this process leads to error. We then review a diverse and often disconnected set of findings in the person perception, judgment, cognition, attitudes, attribution, and rule discovery literatures that can be explained by this process. Finally, we examine the question of why the selective testing of hypotheses occurs. Although the psychological literature suggests that selective hypothesis testing contributes to a variety of errors, in many contexts it may be a useful and efficient strategy that leads to satisfactory judgment.     

Is a Single‐Bladed Knife Enough to Dissect Human Cognition? Commentary on Griffiths et al.

Griffiths, Christian, and Kalish (this issue) present an iterative-learning paradigm applying a Bayesian model to understand inductive biases in categorization. The authors argue that the paradigm is useful as an exploratory tool to understand inductive biases in situations where little is known about the task. It is argued that a theory developed only at the computational level is much like a single-bladed knife that is only useful in highly idealized situations. To be useful as a general tool that cuts through the complex fabric of cognition, we need at least two-bladed scissors that combine both computational and psychological constraints to characterize human behavior. To temper its sometimes expansive claims, it is time to show what a Bayesian model cannot explain. Insight as to how human reality may differ from the Bayesian predictions may shed more light on human cognition than the simpler focus on what the Bayesian approach can explain. There remains much to be done in terms of integrating Bayesian approaches and other approaches in modeling human cognition.     

Aging and conditional probability judgments: a global matching approach

Age differences in bias in conditional probability judgments were investigated based on predictions derived from the Minerva-Decision Making model (M. R. P. Dougherty, C. F. Gettys, & E. E. Ogden, 1999), a global matching model of likelihood judgment. In this study, 248 younger and older adults completed frequency judgment and conditional probability judgment tasks. Age differences in the frequency judgment task are interpreted as an age-related deficit in memory encoding. Older adults' stronger biases in the probability judgment task point to age differences in criterion setting. Age-related biases were eliminated when age groups were equated on memory encoding by means of study time manipulation. The authors conclude that older adults' stronger judgment biases are a function of memory impairment.     

Towards a balanced social psychology: causes, consequences, and cures for the problem-seeking approach to social behavior and cognition

Mainstream social psychology focuses on how people characteristically violate norms of action through social misbehaviors such as conformity with false majority judgments, destructive obedience, and failures to help those in need. Likewise, they are seen to violate norms of reasoning through cognitive errors such as misuse of social information, self-enhancement, and an over-readiness to attribute dispositional characteristics. The causes of this negative research emphasis include the apparent informativeness of norm violation, the status of good behavior and judgment as unconfirmable null hypotheses, and the allure of counter-intuitive findings. The shortcomings of this orientation include frequently erroneous imputations of error, findings of mutually contradictory errors, incoherent interpretations of error, an inability to explain the sources of behavioral or cognitive achievement, and the inhibition of generalized theory. Possible remedies include increased attention to the complete range of behavior and judgmental accomplishment, analytic reforms emphasizing effect sizes and Bayesian inference, and a theoretical paradigm able to account for both the sources of accomplishment and of error. A more balanced social psychology would yield not only a more positive view of human nature, but also an improved understanding of the bases of good behavior and accurate judgment, coherent explanations of occasional lapses, and theoretically grounded suggestions for improvement.     

The ideologically objectionable premise model: Predicting biased political judgments on the left and right

This paper introduces the ideologically objectionable premise model (IOPM), which predicts that biased political judgments will emerge on both the political left and right, but only when the premise of a judgment is not ideologically objectionable to the perceiver. The IOPM generates three hypothesized patterns of bias: biases among both those on the left and right, bias only among those on the right, and bias only among those on the left. These hypotheses were tested within the context of the dual process motivational model of ideological attitudes (DPM; Duckitt, 2001), which posits that right-wing authoritarianism (RWA) and social dominance orientation (SDO) are related but distinct ideological attitudes. Across two studies, all three IOPM hypotheses were tested and supported on the RWA ideological attitude dimension, and two of the three IOPM hypotheses were tested and supported on the SDO dimension. These findings indicate that the context of the judgment is an important determinant of whether biases emerge in political judgment.     

The truth and bias model of judgment

We present a new model for the general study of how the truth and biases affect human judgment. In the truth and bias model, judgments about the world are pulled by 2 primary forces, the truth force and the bias force, and these 2 forces are interrelated. The truth and bias model differentiates force and value, where the force is the strength of the attraction and the value is the location toward which the judgment is attracted. The model also makes a formal theoretical distinction between bias and moderator variables. Two major classes of biases are discussed: biases that are measured with variables (e.g., assumed similarity) and directional bias, which refers to the extent to which judgments are pulled toward 1 end of the judgment continuum. Moderator variables are conceptualized as variables that affect the accuracy and bias forces but that do not affect judgments directly. We illustrate the model with 4 examples. We discuss the theoretical, empirical, methodological, measurement, and design implications of the model.     

Error management theory: a new perspective on biases in cross-sex mind reading

A new theory of cognitive biases, called error management theory (EMT), proposes that psychological mechanisms are designed to be predictably biased when the costs of false-positive and false-negative errors were asymmetrical over evolutionary history. This theory explains known phenomena such as men's overperception of women's sexual intent, and it predicts new biases in social inference such as women's underestimation of men's commitment. In Study 1 (N = 217), the authors documented the commitment underperception effect predicted by EMT. In Study 2 (N = 289), the authors replicated the commitment bias and documented a condition in which men's sexual overperception bias is corrected. Discussion contrasts EMT with the heuristics and biases approach and suggests additional testable hypotheses based on EMT.     

Generation of hypotheses in Wason's 2-4-6 task: an information theory approach.

We explored the "context of discovery" in Wason's 2-4-6 task, focusing on how the first hypothesis is generated. According to Oaksford and Chater (1994a) people generate hypotheses extracting "common features", or regularities, from the available triples, but their model does not explain why some regularities contribute to the hypothesis more than do other regularities. Our conjecture is that some regularities contribute to the hypothesis more than do other regularities because people estimate the amount of information in the perceived regularities and try to preserve as much information as possible in their initial hypotheses. Experiment 1, which used two initial triples, showed that the presence of high-information relational regularities in the initial triples affected the information in the initial hypotheses more than did the presence of low-information object regularities. Experiment 2 extended the results to the classic situation in which only one initial triple is given. It also suggested that amount of information is the only aspect of the structure of the triple that affects hypotheses generation. Experiment 3 confirmed the latter finding: Although relations are commonly distinguished between first-order and higher order relations, the latter being most important for generating hypotheses (Gentner, 1983), higher order relations do have an effect on Wason's 2-4-6 task only if their presence incre ases information. In the conclusion we discuss the statistical soundness of human hypotheses generation processes, and we ask an unanswered question: Amount of information explains why some regularities are preferred to others, but only within a set of "nonarbitrary" regularities; there are object regularities that are rich in information content, but are considered "arbitrary", and are not used in generating hypotheses. Which formal property can distinguish between these two sets of regularities?     

The amendment of large-magnitude aiming-movement errors

Summary The amendment of large-magnitude errors is discussed with particular reference to the correction of directional errors. It is argued that the correction of a large-magnitude error may be understood as a double-step tracking situation in which the second step reflects an artificially induced error signal. At issue is how the motor system may respond to such stimuli in rapid succession. The temporal integration of error stimuli predicts an initial response which reflects the weighted average of the step positions. This approach, however, does not explain the marked increase in peak velocity for the corrective response, compared to an equivalent single-step response. Alternatively, it has been argued that the initial response is initiated as if it were a single-step response to the initial step position and is subsequently amended. The superposition hypothesis argued that the two responses are planned in parallel and overlap in time, to be superimposed one on the other. This hypothesis does not explain changes in the direction of a double-step response at its initiation. The braking hypothesis argues that the initial response is halted and a corrective response initiated as rapidly as possible. This approach cannot explain changes in the slope of an ATF as a function of the second target-step amplitude. A model of double-step tracking is proposed which integrates the temporal integration and braking hypotheses. Since the braking of the initial response would involve the application of large forces, it is argued that braking is facilitated by the temporal integration of step stimuli. The corrective response is then implemented as rapidly as possible. The implications of these findings to the understanding of directional errors is discussed.     

Probabilistic judgment on a coarser scale

We hypothesize that when assessing the likelihood of uncertain events, statistically unsophisticated people utilize a coarse internal scale that only has a limited number of categories. The present paper reports two experiments on probabilistic judgment to test this hypothesis. In Experiment 1, participants estimated event probabilities freely and we found that their responses were highly clustered despite the even and nearly continuous distribution of the target probabilities. In Experiment 2, participants made forced comparisons using two different external response scales (coarser versus finer). We found that their performance did not measure up to the requirement of the finer scale. These findings indicate that besides the systematic biases, a certain portion of human errors in probabilistic judgment may be due to the low resolution of the internal representations.     

A dynamic context model of interactive behavior

A dynamic context model of interactive behavior was developed to explain results from two experiments that tested the effects of interaction costs on encoding strategies, cognitive representations, and response selection processes in a decision-making and a judgment task. The model assumes that the dynamic context defined by the mixes of internal and external representations and processes are sensitive to the interaction cost imposed by the task environment. The model predicts that changes in the dynamic context may lead to systematic biases in cognitive representations and processes that eventually influence decision-making and judgment outcomes. Consistent with the predictions by the model, results from the experiments showed that as interaction costs increased, encoding strategies and cognitive representations shifted from perception-based to memory-based. Memory-based comparisons of the stimuli enhanced the similarity and dominance effects, and led to stronger systematic biases in response outcomes in a choice task. However, in a judgment task, memory-based representations enhanced only the dominance effects. Results suggested that systematic response biases in the dominance context were caused by biases in the cognitive representations of the stimuli, but response biases in the similarity context were caused by biases in the comparison process induced by the choice task. Results suggest that changes in interaction costs not only change whether information was assessed from the external world or from memory but also introduce systematic biases in the cognitive representation of the information, which act as biased inputs to the subsequent decision-making and judgment processes. Results are consistent with the idea of interactive cognition, which proposes that representations and processes are contingent on the dynamic context defined by the information flow between the external task environment and internal cognition.     

The Eye of the Beholder: Romantic Goals and Impression Biases

Four experiments investigated the effects of romantic goals on task and social impression formation. The clouded judgment hypothesis holds that romantic goals encourage positive biases on less relationship-relevant dimensions (e.g., task competence). The default positivity hypothesis holds that romantic goals encourage positivity biases on relationship-relevant dimensions in the absence of relevant information. The selective accuracy hypothesis posits that romantic goals encourage accuracy regarding relationship-relevant attributes (e.g., social competence) when relevant information is available. In four studies, male and female perceivers evaluated opposite-sex targets whom they expected to date (romantic goals) or to meet for nonromantic interaction (baseline). Videotaped targets displayed competence or incompetence on task (Experiments 1 and 2) or social (Experiments 3 and 4) dimensions. All three hypotheses were supported.     

Strategic control in word reading: evidence from speeded responding in the tempo-naming task

To investigate strategic control over response initiation in word reading, the authors introduce the tempo-naming task. Relative to baseline performance in the standard-naming task, participants were induced to respond with faster latencies, shorter durations, and lower levels of accuracy by instructing them to time response initiation with an experimentally controlled tempo. The tempo response cue attenuated stimulus effects, and as faster tempos reduced latencies, the number of spelling-sound errors remained constant, whereas the number of word, nonword, and articulatory errors increased. To explain these results, the authors propose input gain as a mechanism of control over processing speed. The experimenters sketch how input gain could account for the current results as well as for the results from stimulus-blocking experiments testing the route emphasis and time criterion hypotheses of strategic control.     

What is the reference in reference repulsion?

Reference repulsion is a mechanism posited to explain systematic biases of direction judgment of single drifting dot displays (Rauber and Treue, 1998 Perception 27 393-402). Rauber and Treue obtained systematic but, surprisingly, very different effects depending upon whether standard and comparison stimuli were presented simultaneously or successively. Successive effects were described as exhibiting repulsion from both vertical and horizontal cardinal axes, whereas simultaneous effects showed repulsion from horizontal only. We contend that the proposed mechanism makes no testable predictions because the so-called reference can only be specified a posteriori, a fact acknowledged by Rauber and Treue. We attempted to replicate Rauber and Treue's experiments, but we obtained no systematic biases of direction judgment. Comparisons across several studies suggest that errors in direction judgments of single drifting dot patterns vary widely in magnitude and direction, as might be expected with what are essentially baseline or pretest measures. In our view, reference repulsion describes neither a real perceptual mechanism nor a predictable pattern of direction misjudgments.     

道德相对主义和厌恶情绪对道德直觉判断的影响

社会直觉模型认为有意识的道德推理过程发生在道德直觉判断之后。那么, 道德直觉判断又是怎么形成的, 是否受认知推理和情绪的影响?实验1首先验证道德直觉判断的存在; 实验2考察了道德相对主义对道德直觉判断的影响; 实验3考察了厌恶情绪对道德直觉判断的影响。结果发现: (1)道德绝对主义比道德相对主义条件下, 个体更倾向于做出道德直觉判断, 说明道德直觉判断受认知推理影响。(2)厌恶情绪比中立情绪启动条件下, 个体更倾向于做出道德直觉判断, 说明道德直觉判断受情绪影响。因此, 道德直觉判断会受认知推理和情绪的影响。     

Dual Space Search During Scientific Reasoning

The purpose of the two studies reported here was to develop an integrated model of the scientific reasoning process. Subjects were placed in a simulated scientific discovery context by first teaching them how to use an electronic device and then asking them to discover how a hitherto unencountered function worked. To do this task, subjects had to formulate hypotheses based on their prior knowledge, conduct experiments, and evaluate the results of their experiments. In the first study, using 20 adult subjects, we identified two main strategies that subjects used to generate new hypotheses. One strategy was to search memory and the other was to generalize from the results of previous experiments. We described the former group as searching an hypothesis space, and the latter as searching an experiment space. In a second study, with 10 adults, we investigated how subjects search the hypothesis space by instructing them to state all the hypotheses that they could think of prior to conducting any experiments. Following this phase, subjects were then allowed to conduct experiments. Subjects who could not think of the correct rule in the hypothesis generation phase discovered the correct rule only by generalizing from the results of experiments in the experimental phase.
Both studies provide support for the view that scientific reasoning can be characterized as search in two problem spaces. By extending Simon and Lea's (1974) Generalized Rule Inducer, we present a general model of Scientific Discovery as Dual Search (SDDS) that shows how search in two problem spaces (an hypothesis space and an experiment space) shapes hypothesis generation, experimental design, and the evaluation of hypotheses. The model also shows how these processes interact with each other. Finally, we interpret earlier findings about the psychology of scientific reasoning in terms of the SDDS model.     

Using category structures to test iterated learning as a method for identifying inductive biases

Many of the problems studied in cognitive science are inductive problems, requiring people to evaluate hypotheses in the light of data. The key to solving these problems successfully is having the right inductive biases—assumptions about the world that make it possible to choose between hypotheses that are equally consistent with the observed data. This article explores a novel experimental method for identifying the biases that guide human inductive inferences. The idea behind this method is simple: This article uses the responses produced by a participant on one trial to generate the stimuli that either they or another participant will see on the next. A formal analysis of this "iterated learning" procedure, based on the assumption that the learners are Bayesian agents, predicts that it should reveal the inductive biases of these learners, as expressed in a prior probability distribution over hypotheses. This article presents a series of experiments using stimuli based on a well-studied set of category structures, demonstrating that iterated learning can be used to reveal the inductive biases of human learners.     

Rational learning and information sampling: on the "naivety" assumption in sampling explanations of judgment biases

Recent research has argued that several well-known judgment biases may be due to biases in the available information sample rather than to biased information processing. Most of these sample-based explanations assume that decision makers are "naive": They are not aware of the biases in the available information sample and do not correct for them. Here, we show that this "naivety" assumption is not necessary. Systematically biased judgments can emerge even when decision makers process available information perfectly and are also aware of how the information sample has been generated. Specifically, we develop a rational analysis of Denrell's (2005) experience sampling model, and we prove that when information search is interested rather than disinterested, even rational information sampling and processing can give rise to systematic patterns of errors in judgments. Our results illustrate that a tendency to favor alternatives for which outcome information is more accessible can be consistent with rational behavior. The model offers a rational explanation for behaviors that had previously been attributed to cognitive and motivational biases, such as the in-group bias or the tendency to prefer popular alternatives.