Seeking Confirmation Is Rational for Deterministic Hypotheses

The tendency to test outcomes that are predicted by our current theory (the confirmation bias) is one of the best‐known biases of human decision making. We prove that the confirmation bias is an optimal strategy for testing hypotheses when those hypotheses are deterministic, each making a single prediction about the next event in a sequence. Our proof applies for two normative standards commonly used for evaluating hypothesis testing: maximizing expected information gain and maximizing the probability of falsifying the current hypothesis. This analysis rests on two assumptions: (a) that people predict the next event in a sequence in a way that is consistent with Bayesian inference; and (b) when testing hypotheses, people test the hypothesis to which they assign highest posterior probability. We present four behavioral experiments that support these assumptions, showing that a simple Bayesian model can capture people's predictions about numerical sequences (Experiments 1 and 2), and that we can alter the hypotheses that people choose to test by manipulating the prior probability of those hypotheses (Experiments 3 and 4).     

Utility maximization and melioration: Internalities in individual choice

How do people go about choosing between alternatives in relatively simple settings? This study explores some of the variables that past work suggests may be relevant. Volunteer subjects worked for money in six procedures in which the probability of a payment from either of two alternatives was 1.0, but the rate of pay (i.e. the speed with which a payment was delivered or the size of the payment) interacted with the subjects recent allocation of choices, which we define as the ‘internalities’. Because of the internalities, choosing the currently more profitable alternative did not maximize total earnings. Subjects were more likely to fail to maximize when the interaction between present pay and past choices was spread over longer sequences of choices, or when the reward variable was the speed, rather than the value, of each payment. Subjects often disregarded the internalities and were instead guided by the current yields of the two alternatives, which is a frequently observed tendency, called ‘melioration’, in experiments on choices by animals. The tendency toward melioration was only partially counteracted by explicit instructions on how to maximize earnings. We discuss a theoretical framework for melioration that postulates both motivational and cognitive sources.     

Causal diversity effects in information seeking

When trying to determine the root cause of an observed effect, people may seek out information with which to test a candidate hypothesis. In two studies, we investigated how knowledge of causal structure influences this information-seeking process. Specifically, we asked whether people would choose to test for pieces of evidence that were far apart or close together in the learned causal structure of a disease category. In parallel with findings showing people’s tendency to select diverse evidence in argument testing (López, 1995), our participants tested for evidence distantly located within the causal structure. Simultaneously, they rated the probability of occurrence of such diverse evidence as comparatively low. These findings suggest that rather than seeking out information most likely to confirm the hypothesis, people seek out evidence that they believe will most strongly support the hypothesis if present but that they also believe is relatively unlikely to be present (that is, might disconfirm the hypothesis).     

Scientific reasoning in a real‐world context: The effect of prior belief and outcome on children's hypothesis‐testing strategies

The impact of event outcome and prior belief on scientific reasoning was investigated within a real‐world oral health context. Participants (N= 144; ranging from 3 to 11 years) were given hypothesis‐testing tasks and asked to explain their answers. Participants were presented with information that was either consistent or inconsistent with their own beliefs. Each task consisted of scenarios in which the outcome was either good or bad oral health. When the information was belief consistent and the outcome was good, or when the information was belief inconsistent and the outcome was bad, children were more likely to choose scientifically appropriate tests of the stated hypothesis (i.e. manipulate only one variable). Evidence‐based explanations were associated with scientifically appropriate choices in the good‐outcome, belief‐inconsistent scenario and the belief‐consistent, bad‐outcome scenario. Participants' performance on these tasks is explained by considering the plausibility of causal variables. A control of variables strategy was used to test hypotheses in cases in which the evidence was consistent with participants' beliefs and knowledge of causal mechanisms. In contrast, when the evidence was inconsistent with participants' beliefs, children chose to manipulate behaviours likely to lead to a positive health outcome. These findings demonstrate that context and prior knowledge interact to play an important role in children's scientific reasoning.     

Confirming expectations in asymmetric and symmetric social hypothesis testing

This article examines individuals' expectations in a social hypothesis testing task. Participants selected questions from a list to investigate the presence of personality traits in a target individual. They also identified the responses that they expected to receive and the likelihood of the expected responses. The results of two studies indicated that when people asked questions inquiring about the hypothesized traits that did not entail strong a priori beliefs, they expected to find evidence confirming the hypothesis under investigation. These confirming expectations were more pronounced for symmetric questions, in which the diagnosticity and frequency of the expected evidence did not conflict. When the search for information was asymmetric, confirming expectations were diminished, likely as a consequence of either the rareness or low diagnosticity of the hypothesis-confirming outcome. We also discuss the implications of these findings for confirmation bias.     

Dynamic testing with tangible electronics: Measuring children's change in strategy use with a series completion task

Aims. This study sought to explore the use of a novel approach that incorporates dynamic testing and tangible electronics in the assessment of children's learning potential and strategy use. Sample. A total of 77 children with a mean age 8.9 years participated in the study; half of them were dynamically tested using graduate prompt techniques; the others served as a control group. Method. Children in the experimental group received a series of inputs consisting of a pre‐test, two training sessions, and a post‐test all involving a number of series completion tasks; the controls received only pre‐ and post‐tests. All test sessions were undertaken individually using a specially designed programme incorporating an electronic console and tangible materials equipped internally with sensors. Results. As a consequence of training, children significantly outperformed controls on a number of series completion tasks. Significant individual differences were noted in terms of the children's response to assistance. The study's hypothesis that dynamic testing would increase analytical, and reduce trial‐and‐error, behaviour was supported. While a significant proportion of the children employed strategies that had earlier been identified as optimal, a sizeable minority demonstrated rather more idiosyncratic approaches. Conclusions. Findings from the study demonstrate the potential value of electronic dynamic testing using graduated prompts. However, a number of further refinements to improve the procedure are suggested.     

Paradoxes of Falsification

This paper deals with the concept of falsification in hypothesis testing research. A theoretical analysis of assumptions about falsifying behaviour and hypothesis-falsifying observations is presented, with two experimental studies. Both the theoretical analysis and the experimental results point to a number of paradoxes underlying the normative principle of falsification in cognitive psychology. First, subjects experience the falsificatory testing strategy as an impossible strategy to conduct. Obtaining falsifying results is a consequence of the quality of the hypothesis rather than of specific testing behaviour (Experiment 1 and Experiment 2). Second, under some conditions falsifying results impede rather than facilitate discovery (Experiment 2). Confirmatory testing and falsificatory testing, which have been the crucial concepts in the study of hypothesis-testing behaviour, may actually be questionable approaches to testing behaviour. The theoretical analysis is related to the standard analyses of Popper (1963) and Klayman and Ha (1987). The empirical results are discussed in relation to previous studies on falsificatory testing behaviour.     

Information-gathering processes: Diagnosticity,hypothesis-confirmatory strategies,and perceived hypothesis confirmation

Recent research has investigated the information-gathering strategies that people employ as they attempt to test hypotheses. Three such strategies of information seeking were examined. Two kinds of hypothesis-confirmation strategies were considered. The first of these concerned evidence being sought to the extent that it is more likely under the hypothesis being tested than under the alternative. The second kind of hypothesis-confirmation strategy refers to the tendency to ask questions that will have the effect of making the hypothesis under test appear to be true. In addition, a third kind of strategy is a diagnosing strategy under which people prefer evidence that is most differentially probable under the hypothesis and the alternative. Important changes in methodology from past work were made, and the data supported a predominant diagnosing strategy and a less significant, but nonetheless strong and consistent, tendency to ask hypothesis-confirming questions. In addition, subjects' choice of questions made it likely that they would perceive as confirmed the specific hypotheses they were testing. This occurred even though the questions employed were not constraining. Discussion involves the strategies of information gathering and the reasons underlying them as well as the implications of these strategies for the inferences people make about their predictive abilities.     

Strategy Selection in Risky Choice: The Impact of Numeracy,Affect, and Cross‐Cultural Differences

Real‐world decisions often involve options with outcomes that are uncertain and trigger strong affect (e.g., side effects of a drug). Previous work suggests that when choosing among affect‐rich risky prospects, people are rather insensitive to probability information, potentially compromising decision quality. We modeled the strategies of less and more numerate participants in the United States and in Germany when choosing between affect‐rich prospects and between monetarily equivalent affect‐poor prospects. Using large probabilistic national samples (n = 1047 from the United States and Germany), Study 1 showed that compared with more numerate participants, less numerate participants chose the normatively better option (i.e., the one with the higher expected value) less often, guessed more often, and relied more on a simple risk‐minimizing strategy. U.S. participants—although less numerate—selected the normatively better option more frequently and were more consistent across affect‐rich and affect‐poor problems than the German participants. Using a targeted quota sample (n = 118 from Germany), Study 2 indicated that although both more and less numerate participants paid less attention to probability information in affect‐rich than in affect‐poor problems, the two numeracy groups relied on different outcome‐based heuristics: More numerate participants often followed the minimax heuristic, and less numerate participants the affect heuristic. The observed strategy differences suggest that attempts to improve decision‐making need to take into account individual differences in numeracy as well as cultural‐specific experiences in making trade‐offs. Copyright © 2012 John Wiley & Sons, Ltd.     

Influencing Risk Preference in Decision Making: The Effects of Framing and Salience

Prospect theory predicts that people tend to prefer the sure option when choosing between two alternative courses of action framed in terms of gains and prefer the risky option when choosing between two alternatives framed as losses. Related research investigated the impact of emphasizing the probability of the positive outcome of a risky option versus emphasizing the probability of the negative outcome on preference. Most of these studies on the effects of "outcome salience" related their findings to prospect theory′s framing effect. It will be argued that most of these studies inaccurately applied prospect theory to explain the obtained effects and that these might be better understood in terms of salience. In four experiments we test the predictions that (1) choosing between two options in a gain problem will lead to decreased risk preference as compared to loss problems and (2) emphasizing the probability of positive outcomes of a risky option leads to increased preference for this option compared to emphasizing the probability of negative outcomes. Results confirm the impact of both prospect framing and outcome salience and indicate that these effects should be understood in terms of distinct, independent processes.     

Strategies of clinical hypothesis testing

This research examines whether Psychology students, when they test clinical hypotheses, follow either confirmatory or disconfirmatory reasoning strategies. Two hundred and six psychology students, divided in four groups, participated. One group received information about the probability that the hypothesis was correct by means of verbal labels, and another group, by means of numerical expressions. An additional group received the information that getting a precise diagnosis was clinically important. In a last group, diagnostic tests allowed them to increase certainty about the hypothesis. Results show a partial use of confirmatory strategies because, although participants did not seek confirming information, they indeed avoided collecting disconfirming information. When the information increased certainty about the hypothesis, confirmatory strategies became more likely. Neither the increase in the task importance nor the numerical expression of the likelihood that the hypothesis was correct seemed to affect the testing strategy used.     

Revisions and Regret: The Cost of Changing your Mind

Decision reversals often imply improved decisions. Yet, people show a strong resistance against changing their minds. These are well‐established findings, which suggest that changed decisions carry a subjective cost, perhaps by being more strongly regretted. Three studies were conducted to explore participants' regret when making reversible decisions and to test the hypothesis that changing one's mind will increase post‐outcome regret. The first two studies employed the Ultimatum game and the Trust game. The third study used a variant of the Monty Hall problem. All games were conducted by individual participants playing interactively against a computer. The outcomes were designed to capture a common characteristic of real‐life decisions: they varied from rather negative to fairly positive, and for every outcome, it was possible to imagine both more and less profitable outcomes. In all experiments, those who changed their minds reported much stronger post‐outcome regret than those who did not change, even if the final outcomes were equally good (Experiments 2 and 3) or better (Experiment 1).This finding was not because of individual differences with respect to gender, tendency to regret, or tendency to maximize. Previous studies have found that those who change from a correct to wrong option regret more than those who select a wrong option directly. This study indicates that this finding is a special case of a more general phenomenon: changing one's mind seems to come with a cost, even when one ends up with favorable outcomes. Copyright © 2011 John Wiley & Sons, Ltd.     

American base‐rate neglect: It is not the math,but the context

We investigate cross‐cultural disparities in focalism bias through two studies of probability estimation. Using 60 American and 60 Chinese participants, Experiment 1 yields the standard finding that Americans manifest greater focalism bias by tending to neglect background probability base‐rates and to rely more heavily on obtained samples in estimating true probabilities, whereas Chinese participants show little tendency to ignore base‐rates. In Experiment 2, the phrasing of the probability‐estimation task is changed to bring base‐rates into the focus of the problem statement, again using a sample of 60 Americans and 60 Chinese. This allows us to test whether cross‐cultural differences result from a tendency to focus on the sample, and ignore ‘context’ (i.e., the background base‐rates), rather than simply a discrepancy in mathematical facility between the two groups. The results show far less base‐rate neglect for Americans, but essentially no change for Chinese (who always use base‐rate information, regardless of how presented). This outcome argues against the explanation that Americans are poorer Bayesians simply because they are weaker mathematicians. Implications are discussed.     

Randomness, game theory and free will

Libertarians claim that human behaviour is undetermined and cannot be predicted from knowledge of past history even in principle since it is based on the random movements of quantum mechanics. Determinists on the other hand deny thatmacroscopic phenomena can be activated bysub-microscopic events, and assert that if human action is unpredictable in the way claimed by libertarians, it must be aimless and irrational. This is not true of some types of random behaviour described in this paper. Random behaviour may make one unpredictable to opponents and may therefore be rational. Similarly, playing a game with a mixed strategy may have an unpredictable outcome in every single play, but the strategy is rational, in that it is meant to maximize the expected value of an objective, be it private or social. As to whether the outcome of such behaviour is genuinely unpredictable as in quantum mechanics, or predictable by a hypothetical outside observer knowing all natural laws, it is argued that it makes no difference in practice, as long as it is not humanly predictable. Thus we have a new version of libertarianism which is compatible with determinism.     

Hypothesis testing for coefficient alpha: An SEM approach

We show how to test hypotheses for coefficient alpha in three different situations: (1) hypothesis tests of whether coefficient alpha equals a prespecified value, (2) hypothesis tests involving two statistically independent sample alphas as may arise when testing the equality of coefficient alpha across groups, and (3) hypothesis tests involving two statistically dependent sample alphas as may arise when testing the equality of alpha across time or when testing the equality of alpha for two test scores within the same sample. We illustrate how these hypotheses may be tested in a structural equation-modeling framework under the assumption of normally distributed responses and also under asymptotically distribution free assumptions. The formulas for the hypothesis tests and computer code are given for four different applied examples. Supplemental materials for this article may be downloaded from http://brm.psychonomic-journals.org/content/supplemental.     

Testing for negligible interaction: A coherent and robust approach

Researchers often want to demonstrate a lack of interaction between two categorical predictors on an outcome. To justify a lack of interaction, researchers typically accept the null hypothesis of no interaction from a conventional analysis of variance (ANOVA). This method is inappropriate as failure to reject the null hypothesis does not provide statistical evidence to support a lack of interaction. This study proposes a bootstrap‐based intersection–union test for negligible interaction that provides coherent decisions between the omnibus test and post hoc interaction contrast tests and is robust to violations of the normality and variance homogeneity assumptions. Further, a multiple comparison strategy for testing interaction contrasts following a non‐significant omnibus test is proposed. Our simulation study compared the Type I error control, omnibus power and per‐contrast power of the proposed approach to the non‐centrality‐based negligible interaction test of Cheng and Shao (2007, Statistica Sinica, 17, 1441). For 2 × 2 designs, the empirical Type I error rates of the Cheng and Shao test were very close to the nominal α level when the normality and variance homogeneity assumptions were satisfied; however, only our proposed bootstrapping approach was satisfactory under non‐normality and/or variance heterogeneity. In general a × b designs, although the omnibus Cheng and Shao test, as expected, is the most powerful, it is not robust to assumption violation and results in incoherent omnibus and interaction contrast decisions that are not possible with the intersection–union approach.     

Sensitivity to alternatives in social hypothesis-testing

Past research has documented a hypothesis-testing strategy wherein evidence is sought to the extent that it is probable under the hypothesis. This strategy may yield nondiagnostic information and even biased confirmation of the hypothesis if the simultaneous probability of the evidence under the alternatives is disregarded. The results of three experiments demonstrated that hypothesis-testers were in fact sensitive to the probability of the evidence under the alternatives. In the first experiment, subjects tested a hypothesis under which two kinds of personal features, A-features and B-features, were highly probable. Subjects could test their hypothesis by selecting questions from a list of questions about A-features and B-features. The rerults showed that subjects' questions depended on the probability of the features under the alternative. Specifically, when the hypothesis shared A-features with the alternative, subjects preferred questions about B-features, but when the hypothesis shared B-features with the alternative, subjects preferred questions about A-features. Experiment 2 extended these findings to self-generated questions about a broader range of hypotheses and alternatives. Experiment 3 found that subjects who were provided with a specific alternative asked more diagnostic questions than subjects who were not provided with a specific alternative. Together, these results suggest that the process of generating and evaluating alternatives plays a crucial role in social hypothesis-testing and categorization.     

A mixture hierarchical model for response times and response accuracy

In real testing, examinees may manifest different types of test‐taking behaviours. In this paper we focus on two types that appear to be among the more frequently occurring behaviours – solution behaviour and rapid guessing behaviour. Rapid guessing usually happens in high‐stakes tests when there is insufficient time, and in low‐stakes tests when there is lack of effort. These two qualitatively different test‐taking behaviours, if ignored, will lead to violation of the local independence assumption and, as a result, yield biased item/person parameter estimation. We propose a mixture hierarchical model to account for differences among item responses and response time patterns arising from these two behaviours. The model is also able to identify the specific behaviour an examinee engages in when answering an item. A Monte Carlo expectation maximization algorithm is proposed for model calibration. A simulation study shows that the new model yields more accurate item and person parameter estimates than a non‐mixture model when the data indeed come from two types of behaviour. The model also fits real, high‐stakes test data better than a non‐mixture model, and therefore the new model can better identify the underlying test‐taking behaviour an examinee engages in on a certain item.     

The double meaning of a single act: influences of the perceiver and the perceived on cooperative behaviour

The present research examines how a single behaviour that is informative of both the morality and intelligence of a person influences impressions, degree of cooperative behaviour expected from that person, and degree of cooperative behaviour displayed toward that person in a mixed‐motive interdependence situation (i.e., a social dilemma). Furthermore, it is investigated how individual differences in social value orientation influence these processes. Participants were provided with behavioural information that could be construed in terms of both morality (high/low) and intelligence (high/low). Consistent with the morality‐importance hypothesis, participants assigned greater weight to morality than to intelligence aspects of the information. Congruent with the social value orientation hypothesis (i) only proselfs and not prosocials expected more cooperation from unintelligent than from intelligent others, and (ii) prosocials attended more strongly to morality aspects than proselfs in deciding on own cooperation. Finally, consistent with the relative benefit hypothesis, people overall expected more cooperation than they were willing to display, and this tendency was especially pronounced with others described by moral/unintelligent behaviour, and for people with a proself value orientation. The authors discuss a model describing influences of the perceiver and the perceived on cooperative behaviour. Copyright © 1999 John Wiley & Sons, Ltd.