Learning to Make Selection and Detection Decisions: The Roles of Base Rate and Feedback

The effects of type of feedback and base rate on threshold learning in a multiple‐cue decision task were examined. In most such decision experiments, participants receive feedback after every trial (full feedback), and a single base rate (usually 0.5) is used. Our experiment explored conditional feedback (feedback only after positive decisions) representing common selection and detection tasks (such as hiring), where the decision maker receives no feedback unless the decision is positive (e.g., hire the applicant). We used three base rates (0.2, 0.5, and 0.8). As expected, performance was best in full feedback, but after 300 learning trials, the difference was small. Conditional feedback generally resulted in fewer positive decisions than full feedback, but this difference was not found in the low (0.2) base rate condition. There were interactions between base rates and types of feedback. Results provide partial support for the constructivist encoding hypothesis of Elwin and colleagues. Simulation results suggest that our results may reflect overconfidence when feedback is not given. With respect to rate of learning, when the base rate was 0.2, conditional feedback participants reached approximately the same selection rate but did so more slowly than the full feedback participants. Partial feedback participants learned slower and appeared to be still learning after 500 trials. When the base rate was 0.5 or 0.8, partial feedback was nearly as good as full feedback, but conditional feedback resulted in a systematically lower rate of positive decisions. Copyright © 2011 John Wiley & Sons, Ltd.     

Expert Judgement and Expert Disagreement

As Hammond has argued, traditional explanations for disagreement among experts (incompetence, venality, and ideology) are inadequate. The character and fallibilities of the human judgement process itself lead to persistent disagreements even among competent, honest, and disinterested experts. Social Judgement Theory provides powerful methods for analysing such judgementally based disagreements when the experts' judgement processes can be represented by additive models involving the same cues. However, the validity and usefulness of such representations depend on several conditions: (a) experts must agree on a problem definition, (b) experts must have access to the same information, and (c) experts must use the same organising principles. When these conditions are not met, methods for diagnosing and treating disagreement are poorly understood. As a start towards developing such an understanding, sources of expert disagreement are discussed and categorised.     

Detection and Selection Decisions with Conditional Feedback: Interaction of Task Uncertainty and Base Rate

Effects of task uncertainty on decision thresholds in a multiple cue decision task were examined under two types of feedback and three base rate conditions. In most such decision experiments, participants receive feedback after every trial (full feedback) with a single (usually .5) base rate. Our experiment explored conditional (decision‐contingent) feedback, in a task representing a detection problem (passenger screening) in which the decision maker receives no feedback unless the decision is positive (search the passenger). Increased uncertainty made all dependent measures worse. Task uncertainty had detrimental effects on both judgment and decision making, and interacted with effects of feedback and base rate. Decision performance was better with full feedback than with conditional feedback, but not by much. There may be no single unifying explanation for results of our base rate manipulation. Conditional feedback generally resulted in fewer positive decisions than full feedback, but not in the low (.1) base rate condition. Results provide partial support for constructivist encoding and for accuracy maximization with moderate and high base rates, but not with a low base rate. Our results indicated lower selection rates with conditional feedback compared with full feedback in moderate and high base rate conditions, and a more exploratory strategy with higher selection rates with conditional feedback compared with full feedback when base rate was low. Copyright © 2018 John Wiley & Sons, Ltd.