Probability matching, accuracy maximization, and a test of the optimal classifier’s independence assumption in perceptual categorization

Observers completed perceptual categorization tasks that included 25 base-rate/payoff conditions constructed from the factorial combination of five base-rate ratios (1:3, 1:2, 1:1, 2:1, and 3:1) with five payoff ratios (1:3, 1:2, 1:1, 2:1, and 3:1). This large database allowed an initial comparison of the competition between reward and accuracy maximization (COBRA) hypothesis with a competition between reward maximization and probability matching (COBRM) hypothesis, and an extensive and critical comparison of the flat-maxima hypothesis with the independence assumption of the optimal classifier. Model-based instantiations of the COBRA and COBRM hypotheses provided good accounts of the data, but there was a consistent advantage for the COBRM instantiation early in learning and for the COBRA instantiation later in learning. This pattern held in the present study and in a reanalysis of Bohil and Maddox (2003). Strong support was obtained for the flat-maxima hypothesis over the independence assumption, especially as the observers gained experience with the task. Model parameters indicated that observers’ reward-maximizing decision criterion rapidly approaches the optimal value and that more weight is placed on accuracy maximization in separate base-rate/payoff conditions than in simultaneous base-rate/payoff conditions. The superiority of the flat-maxima hypothesis suggests that violations of the independence assumption are to be expected, and are well captured by the flat-maxima hypothesis, with no need for any additional assumptions.     

A test of the optimal classifier's independence assumption in perceptual categorization

Observers completed perceptual categorization tasks that included separate base-rate/payoff manipulations, corresponding simultaneous base-rate/payoff manipulations, and conflicting simultaneous base-rate/payoff manipulations. Performance (1) was closer to optimal for 2:1 than for 3:1 base-rate/payoff ratios and when base rates as opposed to payoffs were manipulated, and (2) was more in line with the predictions from the flat-maxima hypothesis than from the independence assumption of the optimal classifier in corresponding and conflicting simultaneous base-rate/payoff conditions. A hybrid model that instantiated simultaneously the flat-maxima and the competition between reward and accuracy maximization (COBRA) hypotheses was applied to the data. The hybrid model was superior to a model that incorporated the independence assumption, suggesting that violations of the independence assumption are to be expected and are well captured by the flat-maxima hypothesis without requiring any additional assumptions. The parameters indicated that observers' reward-maximizing decision criterion rapidly approaches the optimal value and that more weight is placed on accuracy maximization in separate and corresponding simultaneous base-rate/payoff conditions than in conflicting simultaneous base-rate/payoff conditions.     

A test of the optimal classifier’s independence assumption in perceptual categorization

Observers completed perceptual categorization tasks that included separate base-rate/payoff manipulations, corresponding simultaneous base-rate/payoff manipulations, and conflicting simultaneous base-rate/payoff manipulations. Performance (1) was closer to optimal for 2:1 than for 3:1 baserate/ payoff ratios and when base rates as opposed to payoffs were manipulated, and (2) was more in line with the predictions from the flat-maxima hypothesis than from the independence assumption of the optimal classifier in corresponding and conflicting simultaneous base-rate/payoff conditions. A hybrid model that instantiated simultaneously the flat-maxima and the competition between reward and accuracy maximization (COBRA) hypotheses was applied to the data. The hybrid model was superior to a model that incorporated the independence assumption, suggesting that violations of the independence assumption are to be expected and are well captured by the flat-maxima hypothesis without requiring any additional assumptions. The parameters indicated that observers’ reward-maximizing decision criterion rapidly approaches the optimal value and that more weight is placed on accuracy maximization in separate and corresponding simultaneous base-rate/payoff conditions than in conflicting simultaneous base-rate/payoff conditions.     

A theoretical framework for understanding the effects of simultaneous base-rate and payoff manipulations on decision criterion learning in perceptual categorization

Observers completed perceptual categorization tasks in which base rates and payoffs were manipulated separately or simultaneously across a range of category discriminabilities. Decision criterion estimates from the simultaneous base-rate/payoff conditions were closer to optimal than those predicted from the independence assumption, in line with predictions from the flat-maxima hypothesis. A hybrid model that instantiated the flat-maxima and competition between reward and accuracy maximization hypotheses was applied to the data as well as used in a reanalysis of C. J. Bohil and W.J. Maddox's (2001) study. The hybrid model was superior to a model that incorporated the independence assumption, suggesting that violations of the independence assumption are to be expected and are well captured by the flat-maxima hypothesis, without requiring any additional assumptions.     

Category discriminability, base-rate, and payoff effects in perceptual categorization

The optimality of perceptual categorization performance under manipulations of category discriminability (i.e., d' level), base rates, and payoffs was examined. Base-rate and payoff manipulations across two category discriminabilities allowed a test of the hypothesis that the steepness of the objective reward function affects performance (i.e., the flat-maxima hypothesis), as well as the hypothesis that observers combine base-rate and payoff information independently. Performance was (1) closer to optimal for the steeper objective reward function, in line with the flat-maxima hypothesis, (2) closer to optimal in base-rate conditions than in payoff conditions, and (3) in partial support of the hypothesis that base-rate and payoff knowledge is combined independently. Implications for current theories of base-rate and payoff learning are discussed.     

Linear transformations of the payoff matrix and decision criterion learning in perceptual categorization

The effects of payoff-matrix multiplication, payoff-matrix addition, the presence of long-run gains versus long-run losses, category discriminability, and base rate on decision criterion learning were examined in 2 perceptual categorization experiments. Observers were found to be sensitive to the effects of payoff-matrix multiplication (and category discriminability) on the steepness of the objective reward function in line with predictions from the flat-maxima hypothesis and contrary to the predictions from the payoff-variance hypothesis. Decision criterion learning was best in base-rate conditions, was worst when losses were associated with incorrect responding, and was intermediate when no losses were associated with incorrect responding. This performance profile was well captured by the competition between reward and accuracy (COBRA) hypothesis. A hybrid model framework that instantiates both the flat-maxima and COBRA hypotheses was necessary to account for the data from both experiments.     

Toward a unified theory of decision criterion learning in perceptual categorization

Optimal decision criterion placement maximizes expected reward and requires sensitivity to the category base rates (prior probabilities) and payoffs (costs and benefits of incorrect and correct responding). When base rates are unequal, human decision criterion is nearly optimal, but when payoffs are unequal, suboptimal decision criterion placement is observed, even when the optimal decision criterion is identical in both cases. A series of studies are reviewed that examine the generality of this finding, and a unified theory of decision criterion learning is described (Maddox & Dodd, 2001). The theory assumes that two critical mechanisms operate in decision criterion learning. One mechanism involves competition between reward and accuracy maximization: The observer attempts to maximize reward, as instructed, but also places some importance on accuracy maximization. The second mechanism involves a flat-maxima hypothesis that assumes that the observer's estimate of the reward-maximizing decision criterion is determined from the steepness of the objective reward function that relates expected reward to decision criterion placement. Experiments used to develop and test the theory require each observer to complete a large number of trials and to participate in all conditions of the experiment. This provides maximal control over the reinforcement history of the observer and allows a focus on individual behavioral profiles. The theory is applied to decision criterion learning problems that examine category discriminability, payoff matrix multiplication and addition effects, the optimal classifier's independence assumption, and different types of trial-by-trial feedback. In every case the theory provides a good account of the data, and, most important, provides useful insights into the psychological processes involved in decision criterion learning.     

Optimal classifier feedback improves cost-benefit but not base-rate decision criterion learning in perceptual categorization

Unequal payoffs engender separate reward- and accuracy-maximizing decision criteria; unequal base rates do not. When payoffs are unequal, observers place greater emphasis on accuracy than is optimal. This study compares objective classifier (the objectively correct response) with optimal classifier feedback (the optimal classifier's response) when payoffs or base rates are unequal. It provides a critical test of Maddox and Bohil's (1998) competition between reward and accuracy maximization (COBRA) hypothesis, comparing it with a competition between reward and probability matching (COBRM) and a competition between reward and equal response frequencies (COBRE) hypothesis. The COBRA prediction that optimal classifier feedback leads to better decision criterion leaning relative to objective classifier feedback when payoffs are unequal, but not when base rates are unequal, was supported. Model-based analyses suggested that the weight placed on accuracy was reduced for optimal classifier feedback relative to objective classifier feedback. In addition, delayed feedback affected learning of the reward-maximizing decision criterion.     

On the relation between base-rate and cost-benefit learning in simulated medical diagnosis.

Observers completed a series of simulated medical diagnosis tasks that differed in category discriminability and base-rate/cost-benefit ratio. Point, accuracy, and decision criterion estimates were closer to optimal (a) for category d' = 2.2 than for category d' = 1.0 or 3.2, (b) when base-rates as opposed to cost-benefits were manipulated, and (c) when the cost of an incorrect response resulted in no point loss (nonnegative cost) as opposed to a point loss (negative cost). These results support the "flat-maxima" and competition between reward and accuracy (COBRA) hypotheses. A hybrid model that instantiated simultaneously both hypotheses was applied to the data. The model parameters indicated that (a) the reward-maximizing decision criterion quickly approached the optimal criterion, (b) the importance placed on accuracy maximization early in learning was larger when the cost of an incorrect response was negative as opposed to nonnegative, and (c) by the end of training the importance placed on accuracy was equal for negative and nonnegative costs.     

Gambling primates: reactions to a modified Iowa Gambling Task in humans,chimpanzees and capuchin monkeys

Humans will, at times, act against their own economic self-interest, for example, in gambling situations. To explore the evolutionary roots of this behavior, we modified a traditional human gambling task, the Iowa Gambling Task (IGT), for use with chimpanzees, capuchin monkeys and humans. We expanded the traditional task to include two additional payoff structures to fully elucidate the ways in which these primate species respond to differing reward distributions versus overall quantities of rewards, a component often missing in the existing literature. We found that while all three species respond as typical humans do in the standard IGT payoff structure, species and individual differences emerge in our new payoff structures. Specifically, when variance avoidance and reward maximization conflicted, roughly equivalent numbers of apes maximized their rewards and avoided variance, indicating that the traditional payoff structure of the IGT is insufficient to disentangle these competing strategies. Capuchin monkeys showed little consistency in their choices. To determine whether this was a true species difference or an effect of task presentation, we replicated the experiment but increased the intertrial interval. In this case, several capuchin monkeys followed a reward maximization strategy, while chimpanzees retained the same strategy they had used previously. This suggests that individual differences in strategies for interacting with variance and reward maximization are present in apes, but not in capuchin monkeys. The primate gambling task presented here is a useful methodology for disentangling strategies of variance avoidance and reward maximization.     

Pressure to cooperate: Is positive reward interdependence really needed in cooperative learning?

Background. Despite extensive research on cooperative learning, the debate regarding whether or not its effectiveness depends on positive reward interdependence has not yet found clear evidence. Aims. We tested the hypothesis that positive reward interdependence, as compared to reward independence, enhances cooperative learning only if learners work on a ‘routine task’; if the learners work on a ‘true group task’, positive reward interdependence induces the same level of learning as reward independence. Sample. The study involved 62 psychology students during regular workshops. Method. Students worked on two psychology texts in cooperative dyads for three sessions. The type of task was manipulated through resource interdependence: students worked on either identical (routine task) or complementary (true group task) information. Students expected to be assessed with a Multiple Choice Test (MCT) on the two texts. The MCT assessment type was introduced according to two reward interdependence conditions, either individual (reward independence) or common (positive reward interdependence). A follow‐up individual test took place 4 weeks after the third session of dyadic work to examine individual learning. Results. The predicted interaction between the two types of interdependence was significant, indicating that students learned more with positive reward interdependence than with reward independence when they worked on identical information (routine task), whereas students who worked on complementary information (group task) learned the same with or without reward interdependence. Conclusions. This experiment sheds light on the conditions under which positive reward interdependence enhances cooperative learning, and suggests that creating a real group task allows to avoid the need for positive reward interdependence.     

基础比率和认知风格对贝叶斯推理影响的眼动研究

选取35名本科生为被试,采用EyeLink II眼动仪,考察了基础比率和认知风格对贝叶斯推理的影响,探讨了基础比率作用机制的争论。实验采用2（基础比率：高、低）×2（认知风格：场依存、场独立）双因素被试间设计,要求每位被试完成一道贝叶斯推理题,问题内容为疾病情境。将推理材料划为AOI1（描述基础比率）、AOI2（描述击中率）、AOI3（描述虚报率）、AOI4（提问）4个兴趣区,分别记录被试的总注视时间、注视次数等。结果发现：（1）在总注视时间和注视次数指标上,基础比率和认知风格的主效应均不显著,两者的交互作用显著;（2）在总注视时间和回视次数指标上,各兴趣区差异显著,关注程度从高到低依次为：AOI2＞AOI3＞AOI1＞AOI4。这说明,在贝叶斯推理中,并没有完全忽视基础比率;对于不同认知风格的个体而言,基础比率所起的作用是不同的。这也给我们一点启示：或许我们不能单一地去考虑基础比率的作用机制,而应该将它与个体的某些因素结合起来综合加以讨论。     

On adaptation, maximization, and reinforcement learning among cognitive strategies

Analysis of binary choice behavior in iterated tasks with immediate feedback reveals robust deviations from maximization that can be described as indications of 3 effects: (a) a payoff variability effect, in which high payoff variability seems to move choice behavior toward random choice; (b) underweighting of rare events, in which alternatives that yield the best payoffs most of the time are attractive even when they are associated with a lower expected return; and (c) loss aversion, in which alternatives that minimize the probability of losses can be more attractive than those that maximize expected payoffs. The results are closer to probability matching than to maximization. Best approximation is provided with a model of reinforcement learning among cognitive strategies (RELACS). This model captures the 3 deviations, the learning curves, and the effect of information on uncertainty avoidance. It outperforms other models in fitting the data and in predicting behavior in other experiments.     

Asymmetric attributions for compliance: Reward vs punishment

It was proposed that people attribute an individual's behavior more to internal factors when that individual's actions are influenced by reward than when those actions are influenced by punishment. Previous research has failed to control for the power of reward versus punishment which, in effect, creates a confounding of behavioral base rates (consensus) with the reward-punishment manipulation. The current research created reward and punishment contingencies that were equal in their base rates for producing a compliant response. In Experiment 1, subjects (n = 63) who produced the base-rate data also made attributions regarding a compliant target person. The results supported the reward-punishment attributional asymmetry hypothesis in that the target person was held more responsible for his actions in the reward than in the punishment conditions. A second experiment (n = 72) provided some attributors with information regarding base rates for compliance and measured perceived base rates for compliance. Knowledge of the base rates for compliance eliminated the reward-punishment attributional asymmetry phenomenon. Subjects not provided with such knowledge erroneously assumed different base rates for reward and punishment and maintained the perception of reward-punishment attributional asymmetry. Using subjects' estimates of base rate for compliance as a covariate eliminated the attributional asymmetry effect. It is suggested that erroneous base-rate assumptions mediate the attributional asymmetry phenomenon.     

Pseudodiagnosticity in judgment under uncertainty

The study investigates the extent to which the false alarm (i.e., ) is utilized in judgment under uncertainty. The main findings are (1) this cue is utilized by subjects when provided with a numerically low base-rate (i.e., P(H)) and a high hit-rate (i.e., P(D/H)). Under these conditions the false alarm helps resolve the inconsistency between the implications of these probabilistic cues. (2) The false alarm is ignored by subjects when provided with a numerically high base-rate and a high hit-rate. Under these conditions both latter probabilities are consistent and imply strong support for the focal hypothesis. The false alarm is either not consistent with these cues or redundant and ignored. In addition, three experiments provided evidence regarding base-rate utilization. When comparing conditions (1) and (2) it is demonstrated that the base-rate has a significant effect on judgments. Finally, the experiments provided evidence suggesting that the base-rate is not ignored when the datum is not diagnostic. The results are discussed in relation to the base-rate fallacy, pseudodiagnosticity, and cue consistency.     

Encoding variability theory and the spacing effect in associate learning

Encoding variability theory accounts for the spacing effect by assuming that, as the lag between repetitions increases, the memorial representations approach independence. A method for testing this assumption in determining repetition effects in associative learning is suggested. In the independence hypothesis, we simply assume that each presentation of a word pair is represented independently, so that the expected proportion recalled is simple, P=P1+P2 - PIP2. Two studies are reported using a continuous paired-associate task representing a factorial combination of lag and retention interval as well as single-presentation control conditions. The results indicate that for moderately long retention intervals, as the lag between repetitions increases, the observed proportion recalled is first less, then eventually exceeds, and then returns to the level expected by an independence hypothesis.     

The distracting effect of material reward: An alternative explanation for the superior performance of reward groups in probability learning

To determine whether the distraction effect associated with material rewards in discrimination learning can account for the superior performance of reward groups in probability learning, the performance of 144 school children (preschool, second, and fifth grades) on a two-choice successive discrimination task was compared under three reinforcement conditions (material reward, marker, and knowledge of results). The two events in the task had different frequencies of occurrence, as in probability learning, yet they appeared in a constant order to make 100% payoff possible. The subjects in the reward and marker groups learned the task more slowly, and the nonlearners among them used stereotyped alternation patterns to a greater degree than subjects in the knowledge-of-results condition. These findings suggest that a distracting effect of material reward is present in probability learning and may explain the superior performance of reward groups typically found in probability learning studies.     

On the logic of testing the independence assumption in the process-dissociation procedure

Theprocess-dissociation procedure is designed to provide quantitative estimates of the influence of explicit and implicit memory in a variety of tasks. The procedure relies on the assumption that these two forms of memory produce independent influences on performance. Prior investigators have attempted to test this assumption by determining whether the parameter representing the influence of implicit memory (denotedA) is constant across experimental conditions. I argue that the constancy of A cannot provide an appropriate test of the independence assumption, because (1) the prediction of constancy can be generated without the assumption of independence, obviating the need to posit independence; and (2) the constancy ofA does not necessarily imply independence, even if one assumes that a dependency hypothesis, supplemented by ancillary assumptions (Curran & Hintzman, 1995), predicts differences inA. I close by emphasizing that we can test the independence assumption by using standard procedures that compare the fit of a model that assumes independence with the fit of a model that assumes dependence.     

Decreased responsiveness to reward in depression

We have interpreted the literature showing left anterior hypoactivation in depression as reflecting a decrease in approach-related motivation and behaviour among depressed subjects. In support of this model, we have previously demonstrated a decreased responsiveness to reward in subclinically depressed dysphoric subjects. The current study was designed to replicate and extend those findings. Clinically depressed subjects who met DSM-IV criteria for major depression were compared to a group of nondepressed control subjects on a verbal memory task under three monetary payoff conditions: neutral, reward, and punishment. Although control subjects changed their pattern of responding in both the reward and punishment conditions, relative to the neutral condition, so as to maximise their earnings, depressed subjects did not do so during reward. The two groups did not differ during the punishment condition. These findings provide additional evidence of a decreased responsiveness to reward in depressed individuals, and are consistent with the hypothesis that the left prefrontal hypoactivation observed in depression reflects a deficit in approach-related behaviour.