Learning to Make Selection and Detection Decisions: The Roles of Base Rate and Feedback

The effects of type of feedback and base rate on threshold learning in a multiple‐cue decision task were examined. In most such decision experiments, participants receive feedback after every trial (full feedback), and a single base rate (usually 0.5) is used. Our experiment explored conditional feedback (feedback only after positive decisions) representing common selection and detection tasks (such as hiring), where the decision maker receives no feedback unless the decision is positive (e.g., hire the applicant). We used three base rates (0.2, 0.5, and 0.8). As expected, performance was best in full feedback, but after 300 learning trials, the difference was small. Conditional feedback generally resulted in fewer positive decisions than full feedback, but this difference was not found in the low (0.2) base rate condition. There were interactions between base rates and types of feedback. Results provide partial support for the constructivist encoding hypothesis of Elwin and colleagues. Simulation results suggest that our results may reflect overconfidence when feedback is not given. With respect to rate of learning, when the base rate was 0.2, conditional feedback participants reached approximately the same selection rate but did so more slowly than the full feedback participants. Partial feedback participants learned slower and appeared to be still learning after 500 trials. When the base rate was 0.5 or 0.8, partial feedback was nearly as good as full feedback, but conditional feedback resulted in a systematically lower rate of positive decisions. Copyright © 2011 John Wiley & Sons, Ltd.     

Detection and Selection Decisions with Conditional Feedback: Interaction of Task Uncertainty and Base Rate

Effects of task uncertainty on decision thresholds in a multiple cue decision task were examined under two types of feedback and three base rate conditions. In most such decision experiments, participants receive feedback after every trial (full feedback) with a single (usually .5) base rate. Our experiment explored conditional (decision‐contingent) feedback, in a task representing a detection problem (passenger screening) in which the decision maker receives no feedback unless the decision is positive (search the passenger). Increased uncertainty made all dependent measures worse. Task uncertainty had detrimental effects on both judgment and decision making, and interacted with effects of feedback and base rate. Decision performance was better with full feedback than with conditional feedback, but not by much. There may be no single unifying explanation for results of our base rate manipulation. Conditional feedback generally resulted in fewer positive decisions than full feedback, but not in the low (.1) base rate condition. Results provide partial support for constructivist encoding and for accuracy maximization with moderate and high base rates, but not with a low base rate. Our results indicated lower selection rates with conditional feedback compared with full feedback in moderate and high base rate conditions, and a more exploratory strategy with higher selection rates with conditional feedback compared with full feedback when base rate was low. Copyright © 2018 John Wiley & Sons, Ltd.     

Exemplars in the mist: the cognitive substrate of the representativeness heuristic

The idea that people often make probability judgments by a heuristic short-cut, the representativeness heuristic, has been widely influential, but also criticized for being vague. The empirical trademark of the heuristic is characteristic deviations between normative probabilities and judgments (e.g., the conjunction fallacy, base-rate neglect). In this article the authors contrast two hypotheses concerning the cognitive substrate of the representativeness heuristic, the prototype hypothesis (Kahneman & Frederick, 2002) and the exemplar hypothesis (Juslin & Persson, 2002), in a task especially designed to elicit representativeness effects. Computational modelling and an experiment reveal that representativeness effects are evident early in training and persist longer in a more complex task environment and that the data are best accounted for by a model implementing the exemplar hypothesis.     

Optimal classifier feedback improves cost-benefit but not base-rate decision criterion learning in perceptual categorization

Unequal payoffs engender separate reward- and accuracy-maximizing decision criteria; unequal base rates do not. When payoffs are unequal, observers place greater emphasis on accuracy than is optimal. This study compares objective classifier (the objectively correct response) with optimal classifier feedback (the optimal classifier's response) when payoffs or base rates are unequal. It provides a critical test of Maddox and Bohil's (1998) competition between reward and accuracy maximization (COBRA) hypothesis, comparing it with a competition between reward and probability matching (COBRM) and a competition between reward and equal response frequencies (COBRE) hypothesis. The COBRA prediction that optimal classifier feedback leads to better decision criterion leaning relative to objective classifier feedback when payoffs are unequal, but not when base rates are unequal, was supported. Model-based analyses suggested that the weight placed on accuracy was reduced for optimal classifier feedback relative to objective classifier feedback. In addition, delayed feedback affected learning of the reward-maximizing decision criterion.     

Can attentional theory explain the inverse base rate effect? Comment on Kruschke (2001)

In J. K. Kruschke's (2001; see record 2001-18940-005) study, it is argued that attentional theory is the sole satisfactory explanation of the inverse base rate effect and that eliminative inference (P. Juslin, P. Wennerholm, & A. Winman, 2001; see record 2001-07828-016) plays no role in the phenomenon. In this comment, the authors demonstrate that, in contrast to the central tenets of attentional theory, (a) rapid attention shifts as implemented in ADIT decelerate learning in the inverse base-rate task and (b) the claim that the inverse base-rate effect is directly caused by an attentional asymmetry is refuted by data. It is proposed that a complete account of the inverse base-rate effect needs to integrate attention effects with inference rules that are flexibly used for both induction and elimination.     

Reconstructing Christian Ethics: Exploring Constructivist Practices for Teaching Christian Ethics in the Masters of Divinity Curriculum

This article reflects on an effort to incorporate constructivist pedagogies (learner‐centered, inquiry‐guided, problem‐based models of teaching) into an introductory class on Christian Ethics in an M.Div. curriculum. Although some students preferred more traditional pedagogies, the majority found that constructivist pedagogies better accommodated different life experiences, diverse learning styles, and other features of the M.Div. curriculum. Further, a qualitative assessment of one student exercise indicates that constructivist pedagogies have benefits over traditional pedagogies. Specifically, students' work on a learning‐group research project displayed creativity, depth, and breadth not found in individual research papers. Nonetheless, lukewarm student feedback also demonstrated the need to consider wider factors when attempting such innovations.     

Mode Transition and Change in Variable Use in Perceptual Learning

Runeson, Juslin, and Olsson (2000) proposed (a) that perceptual learning entails a transition from an inferential to a direct-perceptual mode of apprehension, and (b) that relative confidence—the difference between estimated and actual performance—indicates whether apprehension is inferential or direct. In 3 experiments participants received feedback on judgments of force; the results replicated Runeson et al.'s observed decrease in overconfidence but showed more overconfidence. Relative confidence depended on how performance was defined. An attempt to manipulate confidence failed, but trait confidence affected relative confidence. It was concluded that overconfidence does not necessarily signal inferential functioning and that a decrease in overconfidence might occur in a direct-perceptual mode. A theory of learning within the direct-perceptual mode, in addition to learning through a mode transition, appears necessary.     

Living and Learning: Reproducing Beliefs in Selective Experience

People's decisions shape their experience. For example, a recruitment officer decides between job applicants and cannot evaluate the suitability of rejected applicants. The selection decisions thus affect the content of the officer's experience of suitable and unsuitable applicants, and experiential learning is achieved from a selective sample of experiences. It is suggested that people's beliefs are sensitive to the content of the experienced sample, but the mind cannot adjust for the selectivity of the sample even when it results from the individual's own decisions. Two experiments with a recruitment task showed that incorrect prior beliefs survive experiential learning when the beliefs are reproduced and thus appear to be confirmed, in actual experience. When the task was to achieve high performance, incorrect prior beliefs persisted because they were reproduced in a smaller sample of selected job applicants. In contrast, when the task was focused on learning, a greater number of applicants were selected, and a more representative experience therefore revised incorrect beliefs. The actual content of the experienced sample is thus crucial for the persistence, as well as for the revision, of incorrect beliefs. Further, as predicted by the hypothesis of constructivist coding, when feedback was absent for rejected applicants, participants constructed “internal feedback” in line with the expectation that the rejected applicant was unsuitable. Thus, when fewer applicants were hired, participants came to believe that the actual proportion of suitable applicants was low. Finally, the implications for efforts to reduce bias and improve experiential learning are discussed. Copyright © 2012 John Wiley & Sons, Ltd.     

A comparison of correlation, calibration, and diagnosticity as measures of the confidence-accuracy relationship in witness identification

The relationship between witness confidence and accuracy (CA) has traditionally been measured by the point-biserial correlation (rpb). Recently, 2 alternative indices for measuring the CA relation have been proposed, namely calibration and diagnosticity analyses (e.g., P. Juslin, N. Olsson, & A. Winman, 1996). In this study, the 3 measures were compared quantitatively using 52 independent data sets. The measures rpb and calibration were weakly correlated, whether computed across earwitness data sets, eyewitness data sets, or all data. Thus, when applied to the same data, these 2 measures sometimes yield different conclusions. A modest relation was observed between the rpb and the diagnosticity of confidence. Finally, calibration and degree of over- and underconfidence covaried with task difficulty, consistent with K. A. Deffenbachers' (1980) optimality hypothesis.     

Calibration of sensory and cognitive judgments: Two different accounts

The purpose of this study was to test predictions of two recent theories of realism of confidence. Ecological approaches to realism of confidence in one's general knowledge (Gigerenzer et al. , 1991; Juslin, in press a ; Björkman, in press) predict good calibration or, in the case of poor cognitive adjustment, overconfidence, within the cognitive domain. The subjective distance theory of confidence in sensory discriminations (Björkman et al. , 1992) predicts a pervasive underconfidence bias for sensory discriminations. Empirical data are reported showing that: (a) Calibration for sensory judgments is considerably poorer than calibration for well adapted cognitive judgements, a difference that can be entirely traced to underconfidence in the sensory domain. (b) While an initial overconfidence bias in the cognitive domain is removed by outcome feedback, the bias observed in sensory discriminations is unaffected even by a prolonged feedback session. It is suggested that the nature of confidence in sensory discriminations is different from the nature of confidence in cognitive judgments.     

Strength of German accent under altered auditory feedback.

Borden's (1979, 1980) hypothesis that speakers with vulnerable speech systems rely more heavily on feedback monitoring than do speakers with less vulnerable systems was investigated. The second language (L2) of a speaker is vulnerable, in comparison with the native language, so alteration to feedback should have a detrimental effect on it, according to this hypothesis. Here, we specifically examined whether altered auditory feedback has an effect on accent strength when speakers speak L2. There were three stages in the experiment. First, 6 German speakers who were fluent in English (their L2) were recorded under six conditions--normal listening, amplified voice level, voice shifted in frequency, delayed auditory feedback, and slowed and accelerated speech rate conditions. Second, judges were trained to rate accent strength. Training was assessed by whether it was successful in separating German speakers speaking English from native English speakers, also speaking English. In the final stage, the judges ranked recordings of each speaker from the first stage as to increasing strength of German accent. The results show that accents were more pronounced under frequency-shifted and delayed auditory feedback conditions than under normal or amplified feedback conditions. Control tests were done to ensure that listeners were judging accent, rather than fluency changes caused by altered auditory feedback. The findings are discussed in terms of Borden's hypothesis and other accounts about why altered auditory feedback disrupts speech control.     

The Calibration Issue: Theoretical Comments on Suantak,Bolger, and Ferrell (1996)

In this comment on a recent article in this journal on calibration of subjective probabilities by Suantak, Bolger, and Ferrell (1996) we point to methodological problems with the data presented in the article, with computer simulations we demonstrate that the results claimed by the authors to refute the ecological models (Gigerenzer, Hoffrage, & Kleinbölting, 1991; Juslin, 1994) are actually predicted by them, and we respond to the criticisms of the notion of “ecological cue validity.” It is further argued that contrary to the claims by the authors: Thedecision variable partition modeladvocated by the authors fails to account for the results reported in the article, but there is one error model—thecombined error model(Juslin, Olsson, & Björkman, 1997)—that predicts the observed symmetric hard-easy effect with crossover at proportion correct .75. Finally, an analysis of 44 cognitive and 21 sensory tasks demonstrates that there is a difference in calibration for sensory and cognitive tasks.     

Strength of German accent under altered auditory feedback

Borden’s (1979, 1980) hypothesis that speakers with vulnerable speech systems rely more heavily on feedback monitoring than do speakers with less vulnerable systems was investigated. The second language (L2) of a speaker is vulnerable, in comparison with the native language, so alteration to feedback should have a detrimental effect on it, according to this hypothesis. Here, we specifically examined whether altered auditory feedback has an effect on accent strength when speakers speak L2. There were three stages in the experiment. First, 6 German speakers who were fluent in English (their L2) were recorded under six conditions—normal listening, amplified voice level, voice shifted in frequency, delayed auditory feedback, and slowed and accelerated speech rate conditions. Second, judges were trained to rate accent strength. Training was assessed by whether it was successful in separating German speakers speaking English from native English speakers, also speaking English. In the final stage, the judges ranked recordings of each speaker from the first stage as to increasing strength of German accent. The results show that accents were more pronounced under frequency-shifted and delayed auditory feedback conditions than under normal or amplified feedback conditions. Control tests were done to ensure that listeners were judging accent, rather than fluency changes caused by altered auditory feedback. The findings are discussed in terms of Borden’s hypothesis and other accounts about why altered auditory feedback disrupts speech control.     

Direct perception and the predictive mind

Predictive approaches to the mind claim that perception, cognition, and action can be understood in terms of a single framework: a hierarchy of Bayesian models employing the computational strategy of predictive coding. Proponents of this view disagree, however, over the extent to which perception is direct on the predictive approach. I argue that we can resolve these disagreements by identifying three distinct notions of perceptual directness: psychological, metaphysical, and epistemological. I propose that perception is plausibly construed as psychologically indirect on the predictive approach, in the sense of being constructivist or inferential. It would be wrong to conclude from this, however, that perception is therefore indirect in a metaphysical or epistemological sense on the predictive approach. In the metaphysical case, claims about the inferential properties of constructivist perceptual mechanisms are consistent with both direct and indirect solutions to the metaphysical problem of perception (e.g. naïve realism, representationalism, sense datum theory). In the epistemological case, claims about the inferential properties of constructivist perceptual mechanisms are consistent with both direct and indirect approaches to the justification of perceptual belief. In this paper, I demonstrate how proponents of the predictive approach have conflated these distinct notions of perceptual directness and indirectness, and I propose alternative strategies for developing the philosophical consequences of the approach.     

A model for realism of confidence judgments: Implications for underconfidence in sensory discrimination

In a recent issue of this journal, Björkman, Juslin, and Winman (1993) presented a model of the calibration of subjective confidence judgments for sensory discrimination which they called “subjective distance theory.” They proposed that there was a robust underconfidence bias in such judgments, that the model predicted such a bias, and that two different models were needed for the calibration of subjective confidence for cognitive judgments and for sensory ones. This paper addresses issues they raised. It points out that they have not presented a new model, but rather a portion of a more general one, the “decision-variable partition model” originally proposed in Ferrell and McGoey (1980). This paper explores properties of the model and shows, contrary to Björkman, Juslin, and Winman’s hypotheses, that the model does not predict under-confidence, that the “hard-easy effect” can be observed with sensory discriminations, and that the model fits not only sensory, but also cognitive judgments.     

Groups perform better than the best individuals on letters-to-numbers problems: effects of group size

Individuals and groups of 2, 3, 4, or 5 people solved 2 letters-to-numbers problems that required participants, on each trial, to identify the coding of 10 letters to 10 numbers by proposing an equation in letters, receiving the answer in letters, proposing a hypothesis, and receiving feedback on the correctness of the hypothesis. Groups of 3, 4, and 5 people proposed more complex equations and had fewer trials to solution than the best of an equivalent number of individuals. Groups of 3, 4, and 5 people had fewer trials to solution than 2-person groups but did not differ from each other. These results suggest that 3-person groups are necessary and sufficient to perform better than the best individuals on highly intellective problems.     

A Sensorimotor Basis for Motor Learning: Evidence Indicating Specificity of Practice

Our previous work (Proteau, Marteniuk, Girouard, & Dugas, 1987) was concerned with determining whether with relatively extensive practice on a movement aiming task, as the skill theoretically starts becoming open-loop, there would be evidence for a decreasing emphasis on visual feedback for motor control. We eliminated vision of the moving limb after moderate and extensive practice and found that the movement became more dependent on this feedback with greater amounts of practice. In the present study, we wished to test the hypothesis, developed from our previous work, that at the base of movement learning is a sensorimotor representation that consists of integrated information from central processes and sensory feedback derived from previous experiences on the movement task. A strong test of this hypothesis would be the prediction that for an aiming task, the addition of vision, after moderate and relatively extensive practice without vision, would lead to an increasingly large movement decrement, relative to appropriate controls. We found good support for this prediction. From these and previous results, and the idea of the sensorimotor representation underlying learning, we develop the idea that learning is specific to the conditions that prevail during skill acquisition. This has implications for the ideas of generalized motor program and schema theory.     

Reduced-frequency concurrent and terminal feedback: a test of the guidance hypothesis

In 2 experiments, the authors manipulated the frequency of concurrent feedback to discern the effects on learning. In each experiment, participants (N = 48, Experiment 1; N = 36, Experiment 2) attempted to reproduce a criterion force-production waveform (5 s in duration) presented on the computer monitor. Consistent with the guidance hypothesis, the results of Experiment 1 indicated very strong guiding effects of concurrent feedback and strong dependence on the feedback, as indicated by participants' extremely poor performance upon feedback withdrawal in retention. As predicted by the guidance hypothesis, dependence on the feedback was reduced as a result of reducing the frequency of the concurrent feedback. The results of Experiment 2 indicated that one can enhance learning by providing concurrent and terminal feedback on 1 trial, with no feedback on the subsequent trial. In that way, the strong guiding effects of concurrent feedback could be realized and the beneficial effects of terminal feedback could also be achieved.     

Groups perform better than the best individuals on letters-to-numbers problems: informative equations and effective strategies

One-hundred 3-person groups and 300 individuals solved 2 letters-to-numbers problems, requiring identification of the coding of 10 letters to 10 numbers by proposing an equation in letters, receiving the answer in letters, proposing a hypothesis, and receiving feedback on the hypothesis on each trial. There were 5 instruction conditions: (a). standard, (b). use at least 3 letters on all equations, (c). use at least 4 letters on all equations, (d). number 1 known before beginning problem, and (e). number 9 known before beginning problem. The groups had fewer trials to solution, proposed more complex equations, and identified more letters per equation than the best individuals. Performance was best under instructions to use at least 4 letters and with the number 9 known.     

False physiological feedback and emotion: Experimenter demand and salience effects

Four experiments examined the hypothesis that the Valins (1966) false physiological feedback effect with attractiveness ratings of slides is due to experimenter demand. Experiments 1 and 2 showed significant feedback effects with 5-sec feedback periods, previously reported by Barefoot and Straub (1971) to be too brief a time to search the slides for a cause of the apparent physiological arousal. Experiments 3 and 4 had 17 variations of instructions (emotional, nonemotional), stimuli (slides of people, scenic tourist slides), and type of feedback information (heart rate, eyeblink, or none). The typical false feedback effect was found under many conditions that did not seem to meet the presumptive attributional requirements for the effect. In Experiment 4, only subjects who said they were supposed to rate feedback slides higher showed the effect, regardless of instructions, stimuli, or type of feedback. The overall results are interpreted in terms of experimenter demand and stimulus salience effects.