Toward a unified theory of decision criterion learning in perceptual categorization

Optimal decision criterion placement maximizes expected reward and requires sensitivity to the category base rates (prior probabilities) and payoffs (costs and benefits of incorrect and correct responding). When base rates are unequal, human decision criterion is nearly optimal, but when payoffs are unequal, suboptimal decision criterion placement is observed, even when the optimal decision criterion is identical in both cases. A series of studies are reviewed that examine the generality of this finding, and a unified theory of decision criterion learning is described (Maddox & Dodd, 2001). The theory assumes that two critical mechanisms operate in decision criterion learning. One mechanism involves competition between reward and accuracy maximization: The observer attempts to maximize reward, as instructed, but also places some importance on accuracy maximization. The second mechanism involves a flat-maxima hypothesis that assumes that the observer's estimate of the reward-maximizing decision criterion is determined from the steepness of the objective reward function that relates expected reward to decision criterion placement. Experiments used to develop and test the theory require each observer to complete a large number of trials and to participate in all conditions of the experiment. This provides maximal control over the reinforcement history of the observer and allows a focus on individual behavioral profiles. The theory is applied to decision criterion learning problems that examine category discriminability, payoff matrix multiplication and addition effects, the optimal classifier's independence assumption, and different types of trial-by-trial feedback. In every case the theory provides a good account of the data, and, most important, provides useful insights into the psychological processes involved in decision criterion learning.     

On the relation between base-rate and cost-benefit learning in simulated medical diagnosis.

Observers completed a series of simulated medical diagnosis tasks that differed in category discriminability and base-rate/cost-benefit ratio. Point, accuracy, and decision criterion estimates were closer to optimal (a) for category d' = 2.2 than for category d' = 1.0 or 3.2, (b) when base-rates as opposed to cost-benefits were manipulated, and (c) when the cost of an incorrect response resulted in no point loss (nonnegative cost) as opposed to a point loss (negative cost). These results support the "flat-maxima" and competition between reward and accuracy (COBRA) hypotheses. A hybrid model that instantiated simultaneously both hypotheses was applied to the data. The model parameters indicated that (a) the reward-maximizing decision criterion quickly approached the optimal criterion, (b) the importance placed on accuracy maximization early in learning was larger when the cost of an incorrect response was negative as opposed to nonnegative, and (c) by the end of training the importance placed on accuracy was equal for negative and nonnegative costs.     

Probability matching, accuracy maximization, and a test of the optimal classifier’s independence assumption in perceptual categorization

Observers completed perceptual categorization tasks that included 25 base-rate/payoff conditions constructed from the factorial combination of five base-rate ratios (1:3, 1:2, 1:1, 2:1, and 3:1) with five payoff ratios (1:3, 1:2, 1:1, 2:1, and 3:1). This large database allowed an initial comparison of the competition between reward and accuracy maximization (COBRA) hypothesis with a competition between reward maximization and probability matching (COBRM) hypothesis, and an extensive and critical comparison of the flat-maxima hypothesis with the independence assumption of the optimal classifier. Model-based instantiations of the COBRA and COBRM hypotheses provided good accounts of the data, but there was a consistent advantage for the COBRM instantiation early in learning and for the COBRA instantiation later in learning. This pattern held in the present study and in a reanalysis of Bohil and Maddox (2003). Strong support was obtained for the flat-maxima hypothesis over the independence assumption, especially as the observers gained experience with the task. Model parameters indicated that observers’ reward-maximizing decision criterion rapidly approaches the optimal value and that more weight is placed on accuracy maximization in separate base-rate/payoff conditions than in simultaneous base-rate/payoff conditions. The superiority of the flat-maxima hypothesis suggests that violations of the independence assumption are to be expected, and are well captured by the flat-maxima hypothesis, with no need for any additional assumptions.     

Category discriminability, base-rate, and payoff effects in perceptual categorization

The optimality of perceptual categorization performance under manipulations of category discriminability (i.e., d' level), base rates, and payoffs was examined. Base-rate and payoff manipulations across two category discriminabilities allowed a test of the hypothesis that the steepness of the objective reward function affects performance (i.e., the flat-maxima hypothesis), as well as the hypothesis that observers combine base-rate and payoff information independently. Performance was (1) closer to optimal for the steeper objective reward function, in line with the flat-maxima hypothesis, (2) closer to optimal in base-rate conditions than in payoff conditions, and (3) in partial support of the hypothesis that base-rate and payoff knowledge is combined independently. Implications for current theories of base-rate and payoff learning are discussed.     

Probability matching, accuracy maximization, and a test of the optimal classifier's independence assumption in perceptual categorization

Observers completed perceptual categorization tasks that included 25 base-rate/payoff conditions constructed from the factorial combination of five base-rate ratios (1:3, 1:2, 1:1, 2:1, and 3:1) with five payoff ratios (1:3, 1:2, 1:1, 2:1, and 3:1). This large database allowed an initial comparison of the competition between reward and accuracy maximization (COBRA) hypothesis with a competition between reward maximization and probability matching (COBRM) hypothesis, and an extensive and critical comparison of the flat-maxima hypothesis with the independence assumption of the optimal classifier. Model-based instantiations of the COBRA and COBRM hypotheses provided good accounts of the data, but there was a consistent advantage for the COBRM instantiation early in learning and for the COBRA instantiation later in learning. This pattern held in the present study and in a reanalysis of Bohil and Maddox (2003). Strong support was obtained for the flat-maxima hypothesis over the independence assumption, especially as the observers gained experience with the task. Model parameters indicated that observers' reward-maximizing decision criterion rapidly approaches the optimal value and that more weight is placed on accuracy maximization in separate base-rate/payoff conditions than in simultaneous base-rate/payoff conditions. The superiority of the flat-maxima hypothesis suggests that violations of the independence assumption are to be expected, and are well captured by the flat-maxima hypothesis, with no need for any additional assumptions.     

On the generality of optimal versus objective classifier feedback effects on decision criterion learning in perceptual categorization

Biased category payoff matrices engender separate reward- and accuracy-maximizing decision criteria Although instructed to maximize reward, observers use suboptimal decision criteria that place greater emphasis on accuracy than is optimal. In this study, objective classifier feedback (the objectively correct response) was compared with optimal classifier feedback (the optimal classifier's response) at two levels of category discriminability when zero or negative costs accompanied incorrect responses for two payoff matrix multiplication factors. Performance was superior for optimal classifier feedback relative to objective classifier feedback for both zero- and negative-cost conditions, especially when category discriminability was low, but the magnitude of the optimal classifier advantage was approximately equal for zero- and negative-cost conditions. The optimal classifier feedback performance advantage did not interact with the payoff matrix multiplication factor. Model-based analyses suggested that the weight placed on accuracy was reduced for optimal classifier feedback relative to objective classifier feedback and for high category discriminability relative to low category discriminability. In addition, the weight placed on accuracy declined with training when feedback was based on the optimal classifier and remained relatively stable when feedback was based on the objective classifier. These results suggest that feedback based on the optimal classifier leads to superior decision criterion learning across a wide range of experimental conditions.     

A test of the optimal classifier's independence assumption in perceptual categorization

Observers completed perceptual categorization tasks that included separate base-rate/payoff manipulations, corresponding simultaneous base-rate/payoff manipulations, and conflicting simultaneous base-rate/payoff manipulations. Performance (1) was closer to optimal for 2:1 than for 3:1 base-rate/payoff ratios and when base rates as opposed to payoffs were manipulated, and (2) was more in line with the predictions from the flat-maxima hypothesis than from the independence assumption of the optimal classifier in corresponding and conflicting simultaneous base-rate/payoff conditions. A hybrid model that instantiated simultaneously the flat-maxima and the competition between reward and accuracy maximization (COBRA) hypotheses was applied to the data. The hybrid model was superior to a model that incorporated the independence assumption, suggesting that violations of the independence assumption are to be expected and are well captured by the flat-maxima hypothesis without requiring any additional assumptions. The parameters indicated that observers' reward-maximizing decision criterion rapidly approaches the optimal value and that more weight is placed on accuracy maximization in separate and corresponding simultaneous base-rate/payoff conditions than in conflicting simultaneous base-rate/payoff conditions.     

A test of the optimal classifier’s independence assumption in perceptual categorization

Observers completed perceptual categorization tasks that included separate base-rate/payoff manipulations, corresponding simultaneous base-rate/payoff manipulations, and conflicting simultaneous base-rate/payoff manipulations. Performance (1) was closer to optimal for 2:1 than for 3:1 baserate/ payoff ratios and when base rates as opposed to payoffs were manipulated, and (2) was more in line with the predictions from the flat-maxima hypothesis than from the independence assumption of the optimal classifier in corresponding and conflicting simultaneous base-rate/payoff conditions. A hybrid model that instantiated simultaneously the flat-maxima and the competition between reward and accuracy maximization (COBRA) hypotheses was applied to the data. The hybrid model was superior to a model that incorporated the independence assumption, suggesting that violations of the independence assumption are to be expected and are well captured by the flat-maxima hypothesis without requiring any additional assumptions. The parameters indicated that observers’ reward-maximizing decision criterion rapidly approaches the optimal value and that more weight is placed on accuracy maximization in separate and corresponding simultaneous base-rate/payoff conditions than in conflicting simultaneous base-rate/payoff conditions.     

A theoretical framework for understanding the effects of simultaneous base-rate and payoff manipulations on decision criterion learning in perceptual categorization

Observers completed perceptual categorization tasks in which base rates and payoffs were manipulated separately or simultaneously across a range of category discriminabilities. Decision criterion estimates from the simultaneous base-rate/payoff conditions were closer to optimal than those predicted from the independence assumption, in line with predictions from the flat-maxima hypothesis. A hybrid model that instantiated the flat-maxima and competition between reward and accuracy maximization hypotheses was applied to the data as well as used in a reanalysis of C. J. Bohil and W.J. Maddox's (2001) study. The hybrid model was superior to a model that incorporated the independence assumption, suggesting that violations of the independence assumption are to be expected and are well captured by the flat-maxima hypothesis, without requiring any additional assumptions.     

Effects of musicality and motivational orientation on auditory category learning: A test of a regulatory-fit hypothesis

Two experiments investigated the effects of musicality and motivational orientation on auditory category learning. In both experiments, participants learned to classify tone stimuli that varied in frequency and duration according to an initially unknown disjunctive rule; feedback involved gaining points for correct responses (a gains reward structure) or losing points for incorrect responses (a losses reward structure). For Experiment 1, participants were told at the start that musicians typically outperform nonmusicians on the task, and then they were asked to identify themselves as either a “musician” or a “nonmusician.” For Experiment 2, participants were given either a promotion focus prime (a performance-based opportunity to gain entry into a raffle) or a prevention focus prime (a performance-based criterion that needed to be maintained to avoid losing an entry into a raffle) at the start of the experiment. Consistent with a regulatory-fit hypothesis, self-identified musicians and promotion-primed participants given a gains reward structure made more correct tone classifications and were more likely to discover the optimal disjunctive rule than were musicians and promotion-primed participants experiencing losses. Reward structure (gains vs. losses) had inconsistent effects on the performance of nonmusicians, and a weaker regulatory-fit effect was found for the prevention focus prime. Overall, the findings from this study demonstrate a regulatory-fit effect in the domain of auditory category learning and show that motivational orientation may contribute to musician performance advantages in auditory perception.     

Optimal classifier feedback improves cost-benefit but not base-rate decision criterion learning in perceptual categorization

Unequal payoffs engender separate reward- and accuracy-maximizing decision criteria; unequal base rates do not. When payoffs are unequal, observers place greater emphasis on accuracy than is optimal. This study compares objective classifier (the objectively correct response) with optimal classifier feedback (the optimal classifier's response) when payoffs or base rates are unequal. It provides a critical test of Maddox and Bohil's (1998) competition between reward and accuracy maximization (COBRA) hypothesis, comparing it with a competition between reward and probability matching (COBRM) and a competition between reward and equal response frequencies (COBRE) hypothesis. The COBRA prediction that optimal classifier feedback leads to better decision criterion leaning relative to objective classifier feedback when payoffs are unequal, but not when base rates are unequal, was supported. Model-based analyses suggested that the weight placed on accuracy was reduced for optimal classifier feedback relative to objective classifier feedback. In addition, delayed feedback affected learning of the reward-maximizing decision criterion.     

Response processes in information-integration category learning

Much recent evidence suggests that human category learning is mediated by multiple systems. Evidence suggests that at least one of these depends on procedural learning within the basal ganglia. Information-integration categorization tasks are thought to load heavily on this procedural-learning system. The results of several previous studies were interpreted to suggest that response positions are learned in information-integration tasks. This hypothesis was tested in two experiments. Experiment 1 showed that information-integration category learning was slowed but not disrupted when the spatial location of the responses varied randomly across trials. Experiment 2 showed that information-integration learning was impaired if category membership was signaled by responding to a Yes/No question and the category label had no consistent spatial location. These results suggest that information-integration category learning does not require consistent response locations. In these experiments, a consistent association between a category and a response feature was sufficient. The implication of these results for the neurobiology of information-integration category learning is discussed.     

Feedback interference and dissociations of classification: Evidence against the multiple-learning-systems hypothesis

Researchers have argued that different categorization problems are learned by separate and distinct cognitive systems. They propose that an explicit system is responsible for learning rule-based categories and that a separate implicit system learns information-integration categories. One source of supporting evidence involves experiments in which observers perform a concurrent memory-scanning task that interferes with the processing of feedback. Researchers have reported a dissociation in which this manipulation impairs learning of a rule-based category but not an information-integration category. In the present research, we test the hypothesis that the dissociation was the result of lowered perceptual discriminability in the rule-based structure in comparison with the information-integration one. We demonstrate an example of an alternate rule-based category with easy-to-discriminate stimuli in which performance is unaffected by the interfering memory-scanning task. Furthermore, we demonstrate that learning of an information-integration category with low perceptual discriminability is impaired by the memory-scanning task. These demonstrations of the reverse dissociation challenge the interpretation that rule-based and information-integration category structures are learned by separate cognitive systems.     

Null category-length and target–lure relatedness effects in episodic recognition: A constraint on item-noise interference models

There are two main classes of model of interference effects in recognition memory: item-noise and context-noise. Item-noise models predict that a loss of memory discriminability will occur with an increase in the number of studied items from the same taxonomic category (category length, CL) and that forced-choice recognition performance will be higher when the target and lure are related rather than unrelated. Context-noise models, however, predict null effects for both of these manipulations. Although results from some recent experiments suggest that CL and target–lure relatedness have a trivial or no effect on memory discriminability when the related items from the same taxonomic category are “not back to back in the study list but are separated (spaced) by interleaving items from other semantic categories,” these experiments have methodological limitations that were eliminated in the present experiment in which exemplars representing category lengths of 2, 8, or 14 were presented spaced apart within the same study list. Recognition was tested using a yes/no recognition test or a two-alternative forced-choice recognition test in which the target and lure were either related or unrelated. In yes/no recognition, d′ decreased as CL increased, replicating prior research. However, when the slope of the z-ROC function is less than 1.0, as is typically so and was so in the present results, d′ differences can arise due to criterion shifts and are not necessarily due to memory discriminability differences. When the more appropriate measure of memory discriminability, d _a , was computed, CL had no effect in yes/no recognition, nor did it have an effect in forced-choice recognition, which also was not affected by target–lure relatedness. Thus, the present results are congruent with context-noise models and pose a challenge for item-noise models.     

Deepened extinction following compound stimulus presentation: noradrenergic modulation

Behavioral extinction is an active form of new learning involving the prediction of nonreward where reward has previously been present. The expression of extinction learning can be disrupted by the presentation of reward itself or reward-predictive stimuli (reinstatement) as well as the passage of time (spontaneous recovery) or contextual changes (renewal). The following experiments replicated the demonstration that presenting multiple previously rewarded stimuli in compound during extinction enhances extinction learning. To explore the pharmacological basis for this we next examined the effects of pharmacological treatments that either facilitated or blocked noradrenergic activity to test the hypothesis that increased noradrenergic activity at the time of extinction training would improve, whereas blockade of noradrenergic activity would impair the extinction of appetitive stimulus-reward memories. Different groups of rats were trained in a discriminative stimulus paradigm to lever-press for food reward. Once stable responding was achieved, responding was extinguished for 2 d. Prior to a third extinction session, rats received systemic administration of either saline, yohimbine (α2 antagonist), atomoxetine (norepinephrine reuptake inhibitor), or propranolol (β-receptor antagonist). Spontaneous recovery of responding to the stimuli was tested 4 wk later. Our results indicate that increasing noradrenergic activity during extinction augments extinction learning resulting in less recovery of responding at test. These results have important implications for models of relapse to drug seeking and the development of extinction-based therapies.     

Bias of phencyclidine discrimination by the schedule of reinforcement.

Pigeons, trained to discriminate phencyclidine from saline under a procedure requiring the bird to track the location of a color, received cumulative doses of phencyclidine, pentobarbital, or d-amphetamine with a variety of schedules of reinforcement in effect (across phases). When the same second-order schedules were used to reinforce responding after either saline or phencyclidine administration, stimulus control by phencyclidine did not depend on the schedule parameter. When different second-order schedules were used that biased responding toward the phencyclidine-correlated key color, pigeons responded on the phencyclidine-correlated key at lower doses of phencyclidine and pentobarbital than when the second-order schedule biased responding toward the saline key color. A similar but less marked effect was obtained with d-amphetamine. Attempts to produce bias by changing reinforcement magnitude (duration of food availability) were less successful. A signal-detection analysis of dose-effect curves for phencyclidine under two of the second-order schedules employed suggested that at low doses of phencyclidine, response bias is a major determinant of responding. As doses were increased, position preferences occurred and response bias decreased; at higher doses both response bias and position preference decreased and discriminability increased. With low doses of pentobarbital, responding again was biased but increasing doses produced position preference with only small increases in discriminability. At low doses of d-amphetamine responding also was biased, but bias did not decrease consistently with dose nor did discriminability increase. These experiments suggest that the schedule of reinforcement can be used to bias responding toward or away from making the drug-correlated response in drug discrimination experiments, and that signal-detection analysis and analysis of responding at a position can be used to separate the discriminability of the drug state from other effects of the drug on responding.     

Blocking in category learning

Many theories of category learning assume that learning is driven by a need to minimize classification error. When there is no classification error, therefore, learning of individual features should be negligible. The authors tested this hypothesis by conducting three category-learning experiments adapted from an associative learning blocking paradigm. Contrary to an error-driven account of learning, participants learned a wide range of information when they learned about categories, and blocking effects were difficult to obtain. Conversely, when participants learned to predict an outcome in a task with the same formal structure and materials, blocking effects were robust and followed the predictions of error-driven learning. The authors discuss their findings in relation to models of category learning and the usefulness of category knowledge in the environment.     

Learning conditional frequencies in a probability learning task

The study was prompted by a theoretical discussion of probability learning by Estes (1976). In three separate experiments, subjects were presented with frequency information in the form of wins and losses among 3 teams, and later predicted future wins and losses. Frequencies were devised so that conditional win frequencies for a pair of teams were either inconsistent or consistent with marginal win frequencies for each team. In experiment 1, when subjects predicted future events on the basis of known past frequencies, predictions were generally based on conditional frequencies. In experiment 2 six blocks of observations were presented, with predictions following each block. What little learning did occur was in the direction of the conditional frequencies. Subjects in experiment 3 were able to learn conditional frequencies when given explicit instructions to do so. Results were discussed in terms of a two-stage hypothesis generation model that might operate within the framework of an associative theory of probability learning.     

What is pressure? Evidence for social pressure as a type of regulatory focus

Previous research (Markman, Maddox, & Worthy, 2006) suggests that pressure leads to choking when one is learning to classify items on the basis of an explicit rule, but it leads to excelling when one is learning to classify items on the basis of an implicit strategy. In this article, we relate social pressure to regulatory focus theory. We propose that the effects of pressure on performance arise because pressure induces a prevention focus that interacts with the more local reward structure of the task. To test this hypothesis, we repeated previous research, but using a losses reward structure, so that participants under pressure were in a regulatory fit. We also successfully replicated previous results by using a gains reward structure. In contrast with participants who attempted to maximize gains on each trial, participants who attempted to minimize losses choked on the implicit-learning task but excelled on the explicit-learning task. The results suggest a three-way interaction between pressure level, task type, and reward structure.