Further evaluating the conditional decision rule for comparing two independent means

Many books on statistical methods advocate a ‘conditional decision rule’ when comparing two independent group means. This rule states that the decision as to whether to use a ‘pooled variance’ test that assumes equality of variance or a ‘separate variance’ Welch t test that does not should be based on the outcome of a variance equality test. In this paper, we empirically examine the Type I error rate of the conditional decision rule using four variance equality tests and compare this error rate to the unconditional use of either of the t tests (i.e. irrespective of the outcome of a variance homogeneity test) as well as several resampling‐based alternatives when sampling from 49 distributions varying in skewness and kurtosis. Several unconditional tests including the separate variance test performed as well as or better than the conditional decision rule across situations. These results extend and generalize the findings of previous researchers who have argued that the conditional decision rule should be abandoned.     

Replicated alternatives and the role of confusion,chasing, and regret in decisions from experience

The current paper explores choice among alternatives that can be classified into distinct classes. All the members of a particular class were ‘replicated alternatives’: they promised the same payoff distribution. Information to decision makers was limited to feedback concerning the realized (obtained and foregone) payoffs. Experiment 1 demonstrates that increasing the number of replicated alternatives creates confusion (which facilitates random choice) and changes the implications of the tendency to chase recent returns (i.e., select the alternative with the best recent outcomes). This effect, termed ‘confused chasing,’ facilitates risk seeking even when this behavior impairs expected earnings. Experiment 2 reveals that increasing the number of replicated alternatives can reduce (but does not eliminate) the tendency to underweight rare events. Experiment 3 shows that the relative importance of chasing and confusion is sensitive to the likelihood of realizing lower payoffs than the forgone payoffs. The main results are summarized with a simple model assuming that payoff sensitivity decreases with experienced regret. Copyright © 2006 John Wiley & Sons, Ltd.     

No expectation bias in elite-level cricket umpires’ leg-before-wicket (LBW) decisions according to batting order position

ObjectivesPrevious studies have shown that sport officials’ decisions can be impacted by biases associated with expectations. The aim of this study was to determine whether elite cricket umpires’ decisions are also influenced by expectations associated with batter skill.MethodsLBW decisions (n = 5578) from actual elite level cricket matches in Australia between 2009 and 2016 were analysed in a multi-level binomial logistic regression paradigm. In our first model, we predicted the likelihood that an umpire will answer ‘out’ for batters in the top order (1–4), middle order (5–7), and low order (8-11). In our second model, we controlled for the correctness of a decision.ResultsUmpires were more likely to respond ‘out’ as the batting order progressed, however this did not appear to be due to biased decision-making. Instead, as batting order progressed, batters were more likely to actually be ‘out’.ConclusionsCricket umpires do not seem to be impacted by expectation bias associated with batting order. This study highlights the importance of controlling for the correctness of a decision when exploring bias in sport officials’ decisions.     

Even better than the real thing: Alternative outcome bias affects decision judgements and decision regret

Three experiments demonstrated that decisions resulting in considerable amounts of profit, but missed alternative outcomes of greater profits, were rated lower in quality and produced more regret than did decisions that returned lesser (or equal) amounts of profit but either did not miss or missed only slightly better alternatives. These effects were mediated by upward counterfactuals and moderated by participants’ orientation to the decision context. That decision evaluations were affected by the availability and magnitude of alternative outcomes rather than the positivity of actual outcomes is counter to the outcome bias effect—a bias in which decisions are rated more positively when they led to more positive outcomes (despite a priori probabilities associated with the decision outcomes). Experiment 3 demonstrated that these effects represent a bias that occurs even when it is clear that the process by which decisions were made followed rational decision processes. This research suggests that when alternative worlds are even better than the desirable outcomes experienced, affect and cognition may be more strongly linked to the magnitude of alternative realities than to obtained outcomes.     

Structuring decision problems and the ‘bias heuristic’

This paper examines our understanding of the decomposition of immediate acts when structuring decision problems. Seven different types of uncertainties are identified, and four of these are shown to be taken explicitly into account in models within the province of decision theory, described in terms of four interlocking systems interfaced with semantic memory (a core act-event system, and systems buffering utilities, probabilities and events, respectively). Requisite decision modeling is shown to require that the remaining three types of uncertainty (procedural uncertainty; how the decision maker will feel about subsequent acts; agency for changing subsequent states of the world) are also resolved. Methods for ‘fixing’ structure are discussed in terms of aiming at a common understanding about the ‘small world’ in which a decision problem is located. Difficulties in resolving uncertainties in doing this are described. An alternative approach, common in studies invoking ‘behavioural decision theory’ is contrasted: imposing structure, assuming common understanding. The latter approach is shown to involve (i) the ‘naturalisation’ of the small world in which the decision problem is located, and (ii) the utilisation of normative models as ‘ideal types’, leading to the use of the ‘bias’ argument in discussing subjects' performance in decision tasks. Using this argument reflexively, the operation of the ‘bias heuristic’ is identified in a survey of published papers referencing this approach to the study of decision making. Effects identified are: availability of tasks, subjects and explanations; representativeness of findings; and anchoring and adjustment of explanations. Implications for practice are discussed throughout the paper.     

Learning From Peers’ Eye Movements in the Absence of Expert Guidance: A Proof of Concept Using Laboratory Stock Trading,Eye Tracking,and Machine Learning

Existing research shows that people can improve their decision skills by learning what experts paid attention to when faced with the same problem. However, in domains like financial education, effective instruction requires frequent, personalized feedback given at the point of decision, which makes it time‐consuming for experts to provide and thus, prohibitively costly. We address this by demonstrating an automated feedback mechanism that allows amateur decision‐makers to learn what information to attend to from one another, rather than from an expert. In the first experiment, eye movements of N = 100 subjects were recorded while they repeatedly performed a standard behavioral finance investment task. Consistent with previous studies, we found that a significant proportion of subjects were affected by decision bias. In the second experiment, a different group of N = 100 subjects faced the same task but, after each choice, they received individual, machine learning‐generated feedback on whether their pre‐decision eye movements resembled those made by Experiment 1 subjects prior to good decisions. As a result, Experiment 2 subjects learned to analyze information similarly to their successful peers, which in turn reduced their decision bias. Furthermore, subjects with low Cognitive Reflection Test scores gained more from the proposed form of process feedback than from standard behavioral feedback based on decision outcomes.     

Partial knowledge in the development of number word understanding

A common measure of number word understanding is the give‐N task. Traditionally, to receive credit for understanding a number, N, children must understand that N does not apply to other set sizes (e.g. a child who gives three when asked for ‘three’ but also when asked for ‘four’ would not be credited with knowing ‘three’). However, it is possible that children who correctly provide the set size directly above their knower level but also provide that number for other number words (‘N + 1 givers’) may be in a partial, transitional knowledge state. In an integrative analysis including 191 preschoolers, subset knowers who correctly gave N + 1 at pretest performed better at posttest than did those who did not correctly give N + 1. This performance was not reflective of ‘full’ knowledge of N + 1, as N + 1 givers performed worse than traditionally coded knowers of that set size on separate measures of number word understanding within a given timepoint. Results support the idea of graded representations (Munakata, Trends in Cognitive Sciences, 5, 309–315, 2001.) in number word development and suggest traditional approaches to coding the give‐N task may not completely capture children's knowledge.     

Hubert's multi-rater kappa revisited

There is a frequent need to measure the degree of agreement among R observers who independently classify n subjects within K nominal or ordinal categories. The most popular methods are usually kappa-type measurements. When R = 2, Cohen's kappa coefficient (weighted or not) is well known. When defined in the ordinal case while assuming quadratic weights, Cohen's kappa has the advantage of coinciding with the intraclass and concordance correlation coefficients. When R > 2, there are more discrepancies because the definition of the kappa coefficient depends on how the phrase ‘an agreement has occurred’ is interpreted. In this paper, Hubert's interpretation, that ‘an agreement occurs if and only if all raters agree on the categorization of an object’, is used, which leads to Hubert's (nominal) and Schuster and Smith's (ordinal) kappa coefficients. Formulae for the large-sample variances for the estimators of all these coefficients are given, allowing the latter to illustrate the different ways of carrying out inference and, with the use of simulation, to select the optimal procedure. In addition, it is shown that Schuster and Smith's kappa coefficient coincides with the intraclass and concordance correlation coefficients if the first coefficient is also defined assuming quadratic weights.     

Generating consensus priority interval vectors for group decision‐making in the AHP

Many practical and important decision‐making problems are complicated by at least two factors: (1) the qualitative/subjective nature of some criteria often results in uncertainty in the individual ratings; and (2) group decision‐making is involved and some means of aggregating individual ratings is required. Traditionally, both individual and group priorities have been represented as point estimates, but this approach presents severe limitations for accommodating imprecision in the decision‐making process. This paper examines the group decision‐making problem in the context where priorities are represented as numeric intervals. A set of techniques that could be used at some of the phases of an analytic hierarchy process (AHP)‐based group decision‐making process, which has the objective of generating a ‘consensus’ priority that represents the group's opinion with regards to the relative importance of a set of N objects (e.g. criteria, alternatives), is presented. Copyright © 2000 John Wiley & Sons, Ltd.     

Explanatory Consolidation: From ‘Best’ to ‘Good Enough’

In science and everyday life, we often infer that something is true because it would explain some set of facts better than any other hypothesis we can think of. But what if we have reason to believe that there is a better way to explain these facts that we just haven’t thought of? Wouldn’t that undermine our warrant for believing the best available explanation? Many philosophers have assumed that we can solve such underconsideration problems by stipulating that a hypothesis should not only be ‘the best’ explanation available; rather, it should also be ‘good enough’. Unfortunately, however, the only current suggestion for what it might mean to say that an explanation is ‘good enough’ is, well, not good enough. This paper aims to provide a better account of what is required for an explanatory hypothesis to be considered ‘good enough’. In brief, the account holds that a ‘good enough’ hypothesis is one that has gone through a process that I call explanatory consolidation, in which accumulating evidence and failed attempts to formulate better alternatives gradually make it more plausible that the explanation we currently have is better than any other that could be formulated.     

The Effect of Backloading Instructions on Eyewitness Identification from Simultaneous and Sequential Lineups

The sequential lineup is multifaceted, including serial presentation of faces, multiple decisions, and often backloading (indicating to an eyewitness that a lineup contains more photos than there actually are). We evaluated the effect of backloading instructions on response bias and sensitivity with an eyewitness identification paradigm. Importantly, we included an ‘undisclosed’ condition that provided no information to participants about the number of lineup members to expect. Experiment 1 (N = 780) tested sequential lineups; Experiment 2 (N = 532) tested simultaneous lineups. As predicted, signal detection analysis showed that backloading induced participants to be more conservative in choosing from both lineup types, but did not affect d′. We conclude that the criminal justice system should be mindful of this shift in response bias, as it has implications for both guilty and innocent suspect identifications. Copyright © 2016 John Wiley & Sons, Ltd.     

Transitive inference of social dominance by human infants

It is surprising that there are inconsistent findings of transitive inference (TI) in young infants given that non‐linguistic species succeed on TI tests. To conclusively test for TI in infants, we developed a task within the social domain, with which infants are known to show sophistication. We familiarized 10‐ to 13‐month‐olds (M = 11.53 months) to a video of two dominance interactions between three puppets (bear > elephant; hippo > bear) consistent with a dominance hierarchy (hippo > bear > elephant; where ‘>’ denotes greater dominance). Infants then viewed interactions between the two puppets that had not interacted during familiarization. These interactions were either congruent (hippo > elephant) or incongruent (elephant > hippo) with the inferred hierarchy. Consistent with TI, infants looked longer to incongruent than congruent displays. Control conditions ruled out the possibility that infants’ expectations were based on stable behaviors specific to individual puppets rather than their inferred transitive dominance relations. We suggest that TI may be supported by phylogenetically ancient mechanisms of ordinal representation and visuospatial processing that come online early in human development.     

A meta-analysis of the criterion-related validity of prehire work experience

Organizations frequently screen or select job applicants based on their work experience. Despite this, surprisingly little is known about the criterion-related validity of prehire experience, which reflects the amount, duration, or type of experience workers have acquired before they enter a new organization. To address this critical gap in the literature, we used meta-analysis to synthesize data from 81 independent samples that reported relations between prehire experience and performance or turnover. Results revealed overall corrected correlations of .06 for job performance (k = 44, n = 11,785), .11 for training performance (k = 21, n = 8,176), and .00 for turnover (k = 32, n = 11,676). Measures that capture prehire experience with tasks, jobs, or occupations relevant to workers’ current position also are only weakly related to the outcomes (e.g.,  = .07 for job performance). Two exceptions to our main findings are that (a) prehire experience is somewhat more predictive of job performance when workers first start a new job and (b) measures of task-level experience predict training performance, although these results are based on small subsets of primary studies. Overall, the present findings suggest that the types of prehire experience measures organizations currently use to screen job applicants generally are poor predictors of future performance and turnover. We therefore caution organizations from selecting employees based on such measures unless more positive evidence emerges.     

Is the label ‘conspiracy theory’ a cause or a consequence of disbelief in alternative narratives?

Using the label ‘conspiracy theory’ is widely perceived to be a way of discrediting wild ideas and unsubstantiated claims. However, prior research suggests that labelling statements as conspiracy theories does not reduce people's belief in them. In four studies, we probed this effect further, and tested the alternative hypothesis that the label ‘conspiracy theory’ is a consequence rather than a cause of (dis)belief in conspiracy-related statements. Replicating prior research, Study 1 (N = 170) yielded no evidence that the label ‘conspiracy theory’ affects belief in statements. In Study 2 (N = 199), we discovered that the less people believed in statements, the more they favoured labelling them as ‘conspiracy theories’. In Studies 3 and 4 (Ns = 150 and 151), we manipulated the relative believability of statements and found that participants preferred the label ‘conspiracy theory’ for relatively less believable versus more believable statements. The current research therefore supports the hypothesis that prior (dis)agreement with a statement affects the use of the label ‘conspiracy theory’ more than the other way around.     

Judgmental asymmetry in the exercise of police discretion

Police officers and probationer constables rated 40 sets of circumstances as to how far each would influence them to prosecute or not prosecute a motorist stopped for speeding. The distribution of responses about the ipsative means exhibited an asymmetry, consistent with the maximum salience bias towards a negative proportion of 1/e (Tuohy and Stradling, 1987). This asymmetry occurred in both directions, differentiating between a minority group for whom ‘prosecute’ was the negative pole and a majority for whom ‘not prosecute’ was negative. These groups did not differ significantly on overall mean rating, but did so on the primary underlying factor (‘lack of deference’), where the former group was significantly more likely to prosecute. Significant differences were also found between probationer and experienced officers in their susceptibility to the bias, and in their distribution between the two groups.     

Home to men’s business and bosoms: philosophy and rhetoric in Francis Bacon’s Essayes

This article claims that today’s reading of Francis Bacon’s Essayes as a solely literary text turns upon philosophers’ having largely lost access to the renaissance culture which Bacon inherited, and the renaissance debates about the role of rhetoric in philosophy in which Bacon participated. The article has two parts. Building upon Ronald Cranes’ seminal contribution on the place of the Essayes in Bacon’s ‘great instauration’, Part 1 examines how the subjects of Bacon’s Essayes need to be understood as Baconian contributions to ‘morrall philosophye’ and ‘civile knowledge’, rather than rhetorical or poetic exercises. In Part 2, contesting the interpretations of Crane, Fish, Ferrari and others, I will argue that the Essayes’ striking rhetorical form needs to be conceptualized in light of Bacon’s renaissance account of the ‘duty and office’ of rhetoric in any moral and civil philosophy that would look to actively cure mental afflictions and cultivate the virtuous or canny conduct it extols. Bacon’s Essayes, in this light, are best understood as a legatee and transformation of the popular early modern genre of books of apothegms and maxims designed to guide conduct.     

Random error reduction in analytic hierarchies: a comparison of holistic and decompositional decision strategies

The principle of ‘divide and conquer’ (DAC) suggests that complex decision problems should be decomposed into smaller, more manageable parts, and that these parts should be logically aggregated to derive an overall value for each alternative. Decompositional procedures have been contrasted with holistic evaluations that require decision makers to simultaneously consider all the relevant attributes of the alternatives under consideration (Fischer, 1977 ). One area where decompositional procedures have a clear advantage over holistic procedures is in the reduction of random error (Ravinder, 1992 ; Ravinder and Kleinmuntz, 1991 ; Kleinmuntz, 1990 ). Adopting the framework originally developed by Ravinder and colleagues, this paper details the results of a study of the random error variances associated with another popular multi‐criteria decision‐making technique, the Analytic Hierarchy Process (AHP); (Saaty, 1977 , 1980 ), as well as the random error variances of a holistic version of the Analytic Hierarchy Process (Jensen, 1983 ). In addition, data concerning various psychometric properties (e.g. the convergent validity and temporal stability) and values of AHP inconsistency are reported for both the decompositional and holistic evaluations. The results of the study show that the Ravinder and Kleinmuntz ( 1991 ) error‐propagation framework extends to the AHP and decompositional AHP judgments are more consistent than their holistic counterparts. Copyright © 2001 John Wiley & Sons, Ltd.