Analytic Confidence and Political Decision‐Making: Theoretical Principles and Experimental Evidence From National Security Professionals

When making decisions under uncertainty, it is important to distinguish between the probability that a judgment is true and the confidence analysts possess in drawing their conclusions. Yet analysts and decision‐makers often struggle to define “confidence” in this context, and many ways that scholars use this term do not necessarily facilitate decision‐making under uncertainty. To help resolve this confusion, we argue for disaggregating analytic confidence along three dimensions: reliability of available evidence, range of reasonable opinion, and responsiveness to new information. After explaining how these attributes hold different implications for decision‐making in principle, we present survey experiments examining how analysts and decision‐makers employ these ideas in practice. Our first experiment found that each conception of confidence distinctively influenced national security professionals' evaluations of high‐stakes decisions. Our second experiment showed that inexperienced assessors of uncertainty could consistently discriminate among our conceptions of confidence when making political forecasts. We focus on national security, where debates about defining “confidence levels” have clear practical implications. But our theoretical framework generalizes to nearly any area of political decision‐making, and our empirical results provide encouraging evidence that analysts and decision‐makers can grasp these abstract elements of uncertainty.     

I don’t know where to look: the impact of intolerance of uncertainty on saccades towards non-predictive emotional face distractors

Attentional bias to uncertain threat is associated with anxiety disorders. Here we examine the extent to which emotional face distractors (happy, angry and neutral) and individual differences in intolerance of uncertainty (IU), impact saccades in two versions of the “follow a cross” task. In both versions of the follow the cross task, the probability of receiving an emotional face distractor was 66.7%. To increase perceived uncertainty regarding the location of the face distractors, in one of the tasks additional non-predictive cues were presented before the onset of the face distractors and target. We did not find IU to impact saccades towards non-cued face distractors. However, we found IU, over Trait Anxiety, to impact saccades towards non-predictive cueing of face distractors. Under these conditions, IU individuals’ eyes were pulled towards angry face distractors and away from happy face distractors overall, and the speed of this deviation of the eyes was determined by the combination of the cue and emotion of the face. Overall, these results suggest a specific role of IU on attentional bias to threat during uncertainty. These findings highlight the potential of intolerance of uncertainty-based mechanisms to help understand anxiety disorder pathology and inform potential treatment targets.     

Attention lapses and behavioural microsleeps during tracking,psychomotor vigilance,and dual tasks

This study examined the incidence of attention lapses and microsleeps under contrasting levels of task complexity during three tasks: PVT, 2-D tracking and a dual task combining the two. More attention lapses per participant (median 15 vs. 3; range 1–74 vs. 0–76, p = 0.001), with the greatest increase with time spent-on-task (p = 0.002), were evident on the more cognitively-demanding dual task than on the PVT. Conversely, fewer microsleeps (median 0 vs. 0; range 0–1 vs. 0–18, p = 0.022) occurred during the more complex task compared to the tracking task. An increase in microsleep rate with time spent-on-task (p = 0.035) was evident during the tracking task but not the dual task. These results indicate that the higher cognitive load, associated with an increase in task complexity, increased the likelihood of attention lapses, while a reduction in task complexity increased the likelihood of microsleeps.     

There is a 60% probability,but I am 70% certain: communicative consequences of external and internal expressions of uncertainty

Current theories of probability recognise a distinction between external (un)certainty (frequentistic probabilities) and internal (un)certainty (degrees of belief). The present studies investigated this distinction in lay people's judgements of probability statements formulated to suggest either an internal (“I am X% certain”) or an external (“It is X% certain” or “There is an X% probability”) interpretation. These subtle differences in wording influenced participants' perceptions and endorsements of such statements, and their impressions of the speaker. External expressions were seen to signal more reliable task duration estimates, and a lower degree of external than internal certainty was deemed necessary to advise a course of action. In conversations about football, internal expressions were perceived as signalling more personal interest, and were expected to be on the average 10% higher than corresponding external probabilities. Finally, people who reported their outcome expectations for two major sports events let their degree of interest in these events influence their internal but not their external certainty. These results have implications for the communication of uncertainty and probability.     

Work engagement in the mining industry in South Africa: The role of tasks and relationships

The platinum mining sector in South Africa presents a significant context for the study of work engagement, given the major changes and turmoil experienced by employees in this sector. The aim of this study was to investigate the effects of task and relational factors on work engagement in the platinum mining industry in South Africa. A stratified random sample (N = 564) comprising employees in a platinum mining organisation in South Africa was surveyed (females = 35.1%, blacks = 51.6%, middle managers = 40.8%). The employees completed a biographical questionnaire and several other measures: the Engagement Scale, Supervisory Support Scale, Social Support Scale and Job Diagnostic Survey. The results supported a reliable three-factor structure (consisting of autonomy support, competence support, and relatedness support) for the Supervisory Support Scale. The structural model confirmed that task characteristics (comprising task significance and task identity) were strongly related to work engagement. Supervisor support and co-worker support showed a positive relationship with work engagement, albeit to a lesser extent.     

Working on a cognitive task does not influence performance in a physical fitness test

PurposeA limited number of studies have looked at the negative effect that cognitive fatigue has on physical performance.Methods and resultsTwo studies were conducted to assess the impact of a cognitive task on performance in an externally paced running task. In study 1, 12 trained athletes completed a standardized shuttle run, once after a cognitively fatiguing task (unmatched stroop for 10 min) and once after an easy cognitive task (matched stroop for 10 min). Performance in the shuttle run test did not differ between the two conditions, and, surprisingly, perceived effort was significantly higher in the control condition. In study 2, the control condition was modified and the easy cognitive task replaced by watching a video. 11 trained athletes completed both sessions, however, there were again no differences in either performance or in perceived effort.ConclusionBoth studies failed to reveal an impact of cognitive fatigue on subsequent physical performance. These findings contribute to the growing body of literature in this area, showing that the relationship between cognitive and physical task completion is not straightforward, and that other important factors still remain for investigation.     

Cultural variation in implicit independence: An extension of Kitayama et al. ()

Previous research shows that European Americans are consistently more independent (or less interdependent) than Japanese when implicit indices are used to assess independence (vs. interdependence). The present work extended this evidence by including a novel implicit association test (IAT), as an index of implicit attitude towards independence and interdependence. Consistent with the previous findings, as compared to Japanese, Americans were significantly higher in multiple indices of implicit independence (vs. interdependence) including personal (vs. social) self‐definition, experience of disengaging (vs. engaging) emotions and personal (vs. social) form of happiness. Furthermore, as compared to Japanese, Americans had a significantly more positive implicit attitude towards independence assessed with the IAT. As also observed in the previous research, explicit measures showed inconsistent cross‐cultural patterns. Lastly, we observed little statistical within‐culture coherence among the implicit measures of independence (vs. interdependence), consistent with a view that the implicit indices capture alternative ways for individuals to achieve the cultural mandate of independence or interdependence.     

A Bayesian Power Analysis Procedure Considering Uncertainty in Effect Size Estimates from a Meta-analysis

In conventional frequentist power analysis, one often uses an effect size estimate, treats it as if it were the true value, and ignores uncertainty in the effect size estimate for the analysis. The resulting sample sizes can vary dramatically depending on the chosen effect size value. To resolve the problem, we propose a hybrid Bayesian power analysis procedure that models uncertainty in the effect size estimates from a meta-analysis. We use observed effect sizes and prior distributions to obtain the posterior distribution of the effect size and model parameters. Then, we simulate effect sizes from the obtained posterior distribution. For each simulated effect size, we obtain a power value. With an estimated power distribution for a given sample size, we can estimate the probability of reaching a power level or higher and the expected power. With a range of planned sample sizes, we can generate a power assurance curve. Both the conventional frequentist and our Bayesian procedures were applied to conduct prospective power analyses for two meta-analysis examples (testing standardized mean differences in example 1 and Pearson's correlations in example 2). The advantages of our proposed procedure are demonstrated and discussed.