Explaining purportedly irrational behavior by modeling skepticism in task parameters: an example examining confidence in forced-choice tasks

Many purported demonstrations of irrational behavior rely on the assumption that participants believe key task parameters that are merely asserted by experimenters. For example, previous researchers have found that participants who first reported confidence in items presented in a yes-no format did not change confidence to the degree prescribed by the normative model when those same items were later presented in a forced-choice format. A crucial assumption, however, was that participants fully believed the assertion that the forced-choice items were mutually exclusive and exhaustive. In this article, the authors derive and test a new normative model in which it is not assumed that participants fully believe the assertion. Two visual identification experiments show that the new normative model provides a compelling account of participants' confidence reports.     

Separating sensitivity from response bias: implications of comparisons of yes-no and forced-choice tests for models and measures of recognition memory

A fundamental challenge to psychological research is the measurement of cognitive processes uncontaminated by response strategies resulting from different testing procedures. Test-free estimates of ability are vital when comparing the performance of different groups or different conditions. The current study applied several sets of measurement models to both forced-choice and yes-no recognition memory tests and concluded that the traditional signal-detection model resulted in distorted estimates of accuracy. Two-factor models were necessary to separate memory sensitivity from response bias. These models indicated that (a) memory accuracy did not differ across the tests and (b) the tests relied on the same underlying memory processes. The results illustrate the pitfalls of using a single-component model to measure accuracy in tasks that reflect 2 or more underlying processes.     

Relation between judged and physical distance in multicue conditions as a function of instructions and tasks.

Using a rod adjustable in distance 48 Os produced equal depth intervals along a floor of a visual alley (interval reproduction) or bisected a given depth interval into two equal parts (bisection). Also, verbal reports were obtained of a depth interval located at a near and farther distance from O (absolute judgments). Two kinds of instructions, "apparent" and "objective," were used with different Os with each of the three tasks. There was an over-all tendency (p = .05 on two of three tasks) for apparent instructions to produce less over-constancy or more under-constancy of distance than that produced by objective instructions. This tendency is consistent with a cognitive interpretation of over-constancy of distance. Clear over-constancy was obtained only by the combination of objective instructions and the bisection task. Clear under-constancy was obtained only by the combination of apparent instructions and the method of absolute judgments. The reason for the effect of task on magnitude of distance constancy is not understood.     

The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks

In this article, the authors consider optimal decision making in two-alternative forced-choice (TAFC) tasks. They begin by analyzing 6 models of TAFC decision making and show that all but one can be reduced to the drift diffusion model, implementing the statistically optimal algorithm (most accurate for a given speed or fastest for a given accuracy). They prove further that there is always an optimal trade-off between speed and accuracy that maximizes various reward functions, including reward rate (percentage of correct responses per unit time), as well as several other objective functions, including ones weighted for accuracy. They use these findings to address empirical data and make novel predictions about performance under optimality.     

Taste-quality recognition and forced-choice response

The Collings (1974) procedure for measuring taste-quality recognition thresholds controls adequately for bias caused by variation in the willingness of subjects to report perceiving a taste quality. However, it fails to control for bias due to the preference of subjects for one response over another. In the present study, preferential selection among the response alternatives was reflected in the disproportionate occurrence of the four taste-quality names among incorrect responses. Response preferences were idiosyncratic and grew stronger with repeated testing. They were not eliminated by informing subjects of the correctness of their responses. Use of this procedure for measuring recognition thresholds is discouraged.     

Self-monitoring and the association between confidence and accuracy

A personality prediction task in which a confidence rating accompanies each judgment is used to test the hypotheses that self-monitoring is positively related to accuracy, positively related to confidence, negatively related to calibration, and capable of acting as a moderator of the relationship between accuracy and confidence. Data from 101 university students support two of these predictions: High self-monitors exhibit more confidence and poorer calibration than low self-monitors. Concern for appropriateness is found to be unrelated to all other variables studied. Results are discussed in the context of socioanalytic theory and recent theories of self-presentation, affectivity, and the relationship between illusions and well-being.     

ROCs obtained with two signal intensities presented in random order,and a comparison between yes-no and rating ROCs

Two experiments are presented. The first experiment investigated the effect of mixing signals of two different intensities within the same blocks of trials upon receiver operating characteristics (ROCs) obtained with the rating procedure. The ROCs obtained when the signals of differing intensity were mixed in blocks of trials were found to be quite similar to those obtained when only one signal intensity was presented in any given block of trials. In the second experiment ROCs were determined by having subjects adopt different decision criteria on different blocks of trials in a yes-no experiment. The data points obtained in this experiment fell quite close to the ROCs obtained from the same subjects with the rating procedure in the first experiment.     

Children in Asian cultures say yes to yes-no questions: Common and cultural differences between Vietnamese and Japanese children

We investigated whether children's response tendency toward yes-no questions concerning objects is a common phenomenon regardless of languages and cultures. Vietnamese and Japanese 2- to 5-year-old (N = 108) were investigated. We also examined whether familiarity with the questioning issue has any effect on Asian children's yes bias. As the result, Asian children showed a yes bias to yes-no questions. The children's response tendency changes dramatically with their age: Vietnamese and Japanese 2- and 3-year-olds showed a yes bias, but 5-year-olds did not. However, Asian 4-year-olds also showed a yes bias only in the familiar condition. Also, Asian children showed a stronger yes bias in the familiar condition than the unfamiliar condition. These two findings in Asian children were different from the previous finding investigated North American children (Fritzley & Lee, 2003). Moreover, there was a within-Asian cross-cultural difference. Japanese children showed different response tendencies, which were rarely observed in Vietnamese children. Japanese 2-year-olds and some 3-year-olds showed a "no answer" response: they tended not to respond to an interviewer's questions. Japanese 4- and 5-year-olds also showed an "I don't know" response when they were asked about unfamiliar objects. Japanese children tended to avoid a binary decision. We discussed the cross-cultural differences.     

Visual search in a forced-choice paradigm

The processing of visual information was investigated in the context of two visual search tasks. The first was a forced-choice task in which one of two alternative letters appeared in a visual display of from one to five letters. The second task included trials on which neither of the two alternatives was present in the display. Search rates were estimated from the slopes of best linear fits to response latencies plotted as a function of the number of items in the visual display. These rates were found to be much slower than those estimated in yes-no search tasks. This result was interpreted as indicating that the processes underlying visual search in yes-no and forced-choice tasks are not the same.     

The relationship between riskiness and confidence in choice dilemma decisions

The relationship between riskiness of decisions and confidence in decisions was examined using Choice Dilemma items. Graphs show a clear curvilinear relationship between riskiness and confidence; on both risky items and cautious items greater confidence is associate with extremely risky and extremely cautious decisions than with moderate ones. it was also demonstrated that individual subjects have more confidence in their extreme decisions than in their moderate ones.     

Highly correlated stimuli do not necessarily facilitate the measurement of unconscious perception: Exclusion failure is hard to find in forced-choice tasks

Persaud and McLeod (2008) report that unconscious perception is easier to measure with forced-choice exclusion tasks when the stimuli are highly similar, such as choosing between the letters ‘h’ and ‘b’. The high degree of stimulus similarity may decrease conscious awareness of the target stimuli while leaving unconscious cognition intact. The present experiments used forced-choice exclusion tasks (i.e., choosing the opposite of a masked target stimulus) with the aim of replicating these findings. No evidence of relevant perception – either conscious or unconscious – was obtained with short duration targets. The forced-choice exclusion task was correctly performed at longer target durations (25 ms and higher), which suggests conscious perception of the target stimuli. We conclude that increasing stimulus similarity does not reliably produce exclusion failure effects and does not appear to facilitate the measurement of unconscious cognition.     

The relationship between confidence and self-concept — Towards a model of response confidence

According to Stankov [Stankov, L. (2000). Complexity, metacognition and fluid intelligence. Intelligence, 28, 121–143.] response confidence in cognitive tests reflects a trait on the boundary of personality and abilities. However, several studies failed in relating confidence scores to other known traits, including self-concept. A model of response confidence is proposed which predicts that confidence judgements do reflect self-concept, but only to the extent that they do not reflect a calibration process based on task-inherent cues. In the current study, 101 students completed various tests of cognitive abilities and skills as well as scales of the SDQ III measuring general academic and problem-solving self-concept. As expected, self-concept predicts a significant proportion of variance in the confidence factor that is over and above the influence of test scores.     

Comparison of yes-no and latency measures of auditory intensity discrimination.

Rats discriminated auditory intensity differences of sinusoids at 3.0 kilohertz in a go/no-go signal detection procedure. Responses to the signal (hits) were reinforced with electrical brain stimulation, and misses produced a brief timeout. On intermixed noise trials, withholding of responses (correct rejections) was reinforced, and false alarms produced the time-out. In two test conditions, the signal was either the louder (100 decibels) or softer (90, 93, 96, or 99 decibels) of the pair of intensities presented within a set of trials. Each animal was first trained with signal value louder or softer, and reversed for the second condition so that the former noise value served as signal. Hits showed shorter latencies than false alarms, regardless of the relative intensity of signal and noise, and the magnitude of differentiation was proportional to signal-noise separation. Both hits and false alarms showed longer latencies as the discrimination became more difficult. Isosensitivity contours derived from the latencies showed close similarity across conditions; in comparison, the yes-no measure of detectability, d', showed greater variability. The similarity of latency differentiation across louder and softer signal conditions supports a detection model in which the observer's judgment is controlled by the distance of sensory effect from criterion on each trial, as opposed to the loudness of the tones per se.     

The variance of d' estimates obtained in yes-no and two-interval forced choice procedures

In studies of detection and discrimination, data are often obtained in the form of a 2 x 2 matrix and then converted to an estimate of d' based on the assumptions that the underlying decision distributions are Gaussian and equal in variance. The statistical properties of the estimate of d', d' are well understood for data obtained using the yes-no procedure, but less effort has been devoted to the more commonly used two-interval forced choice (2IFC) procedure. The variance associated with d' is a function of true d' in both procedures, but for small values of true d' the variance of d' obtained using the 2IFC procedure is predicted to be less than the variance of d' obtained using yes-no; for large values of true d', the variance of d' obtained using the 2IFC procedure is predicted to be greater than the variance of d' from yes-no. These results follow from standard assumptions about the relationship between the two procedures. The present paper reviews the statistical properties of d' obtained using the two standard procedures and compares estimates of the variance of d' as a function of true d' with the variance observed in values of d' obtained with a 2IFC procedure.     

Central and peripheral precuing of forced-choice discrimination

There are suggestions in the literature that spatial precuing of attention with peripheral and central cues may be mediated by different mechanisms. To investigate this issue, data from two previous papers were reanalysed to investigate the complete time course of precuing target location with either: (1) a peripheral cue that may draw attention reflexively, or (2) a central, symbolic cue that may require attention to be directed voluntarily. This analysis led to predictions that were tested in another experiment. The main result of this experiment was that a peripheral cue produced its largest effects on discrimination performance within 100 msec, whereas a central cue required approximately 300 msec to achieve maximum effects. In conjunction with previous findings, the present evidence for time differences between the two cuing conditions suggests that more than one process is involved in the spatial precuing of attention.     

List length and overlap effects in forced-choice associative recognition

Two experiments examined forced-choice associative recognition for OLAP and NOLAP test conditions. OLAP test trials consist of pairs with overlapping items (e.g., AB vs. AD), whereas NOLAP test trials contain no overlapping items (e.g., AB vs. CF). Previous results show better performance for NOLAP than for OLAP tests, contrary to the predictions of global memory models. The present experiments varied list length to examine the hypothesis that the NOLAP advantage is produced by recall-like retrieval processes. The use of longer lists either eliminated (Experiment 1) or greatly reduced (Experiment 2) the NOLAP advantage. However, a reliable OLAP advantage was not obtained. Implications for models are discussed.     

Event-related potentials in a two-interval forced-choice detection task

In an attempt to elucidate the nature of the subject’s strategy in a two-interval forcedchoice auditory detection task, event-related potentials were studied at two intensities which yielded mean accuracies of 82% and 98%. Subjects reported the observation interval in which they judged the signal to be present and the confidence of the judgment. Principal components varimax analyses yielded four components: a CZ maximal P300, a Slow Wave, a slow negative shift, and a late negative component. The P300 amplitude findings suggest that different strategies are utilized for high-confidence and low-confidence detections. At high confidence, P300 amplitude is large for the observation interval in which the signal is presented, indicating a strategy involving serial independent detection. However, the P300 latency findings at high confidence suggest that absence of the signal in the first observation interval is nonetheless noted: P300 latency in response to signal presence is shorter for the second observation interval than for the first observation interval. At low confidence, P300 is small or absent for both observation intervals, indicating a deferred decision, presumably arrived at through comparison of the two percepts.     

Intuitive confidence: choosing between intuitive and nonintuitive alternatives

People often choose intuitive rather than equally valid nonintuitive alternatives. The authors suggest that these intuitive biases arise because intuitions often spring to mind with subjective ease, and the subjective ease leads people to hold their intuitions with high confidence. An investigation of predictions against point spreads found that people predicted intuitive options (favorites) more often than equally valid (or even more valid) nonintuitive alternatives (underdogs). Critically, though, this effect was largely determined by people's confidence in their intuitions (intuitive confidence). Across naturalistic, expert, and laboratory samples (Studies 1-3), against personally determined point spreads (Studies 4-11), and even when intuitive confidence was manipulated by altering irrelevant aspects of the decision context (e.g., font; Studies 12 and 13), the authors found that decreasing intuitive confidence reduced or eliminated intuitive biases. These findings indicate that intuitive biases are not inevitable but rather predictably determined by contextual variables that affect intuitive confidence.