A test of the optimal classifier’s independence assumption in perceptual categorization

Observers completed perceptual categorization tasks that included separate base-rate/payoff manipulations, corresponding simultaneous base-rate/payoff manipulations, and conflicting simultaneous base-rate/payoff manipulations. Performance (1) was closer to optimal for 2:1 than for 3:1 baserate/ payoff ratios and when base rates as opposed to payoffs were manipulated, and (2) was more in line with the predictions from the flat-maxima hypothesis than from the independence assumption of the optimal classifier in corresponding and conflicting simultaneous base-rate/payoff conditions. A hybrid model that instantiated simultaneously the flat-maxima and the competition between reward and accuracy maximization (COBRA) hypotheses was applied to the data. The hybrid model was superior to a model that incorporated the independence assumption, suggesting that violations of the independence assumption are to be expected and are well captured by the flat-maxima hypothesis without requiring any additional assumptions. The parameters indicated that observers’ reward-maximizing decision criterion rapidly approaches the optimal value and that more weight is placed on accuracy maximization in separate and corresponding simultaneous base-rate/payoff conditions than in conflicting simultaneous base-rate/payoff conditions.     

Probability matching, accuracy maximization, and a test of the optimal classifier’s independence assumption in perceptual categorization

Observers completed perceptual categorization tasks that included 25 base-rate/payoff conditions constructed from the factorial combination of five base-rate ratios (1:3, 1:2, 1:1, 2:1, and 3:1) with five payoff ratios (1:3, 1:2, 1:1, 2:1, and 3:1). This large database allowed an initial comparison of the competition between reward and accuracy maximization (COBRA) hypothesis with a competition between reward maximization and probability matching (COBRM) hypothesis, and an extensive and critical comparison of the flat-maxima hypothesis with the independence assumption of the optimal classifier. Model-based instantiations of the COBRA and COBRM hypotheses provided good accounts of the data, but there was a consistent advantage for the COBRM instantiation early in learning and for the COBRA instantiation later in learning. This pattern held in the present study and in a reanalysis of Bohil and Maddox (2003). Strong support was obtained for the flat-maxima hypothesis over the independence assumption, especially as the observers gained experience with the task. Model parameters indicated that observers’ reward-maximizing decision criterion rapidly approaches the optimal value and that more weight is placed on accuracy maximization in separate base-rate/payoff conditions than in simultaneous base-rate/payoff conditions. The superiority of the flat-maxima hypothesis suggests that violations of the independence assumption are to be expected, and are well captured by the flat-maxima hypothesis, with no need for any additional assumptions.     

Probability matching, accuracy maximization, and a test of the optimal classifier's independence assumption in perceptual categorization

Observers completed perceptual categorization tasks that included 25 base-rate/payoff conditions constructed from the factorial combination of five base-rate ratios (1:3, 1:2, 1:1, 2:1, and 3:1) with five payoff ratios (1:3, 1:2, 1:1, 2:1, and 3:1). This large database allowed an initial comparison of the competition between reward and accuracy maximization (COBRA) hypothesis with a competition between reward maximization and probability matching (COBRM) hypothesis, and an extensive and critical comparison of the flat-maxima hypothesis with the independence assumption of the optimal classifier. Model-based instantiations of the COBRA and COBRM hypotheses provided good accounts of the data, but there was a consistent advantage for the COBRM instantiation early in learning and for the COBRA instantiation later in learning. This pattern held in the present study and in a reanalysis of Bohil and Maddox (2003). Strong support was obtained for the flat-maxima hypothesis over the independence assumption, especially as the observers gained experience with the task. Model parameters indicated that observers' reward-maximizing decision criterion rapidly approaches the optimal value and that more weight is placed on accuracy maximization in separate base-rate/payoff conditions than in simultaneous base-rate/payoff conditions. The superiority of the flat-maxima hypothesis suggests that violations of the independence assumption are to be expected, and are well captured by the flat-maxima hypothesis, with no need for any additional assumptions.     

A theoretical framework for understanding the effects of simultaneous base-rate and payoff manipulations on decision criterion learning in perceptual categorization

Observers completed perceptual categorization tasks in which base rates and payoffs were manipulated separately or simultaneously across a range of category discriminabilities. Decision criterion estimates from the simultaneous base-rate/payoff conditions were closer to optimal than those predicted from the independence assumption, in line with predictions from the flat-maxima hypothesis. A hybrid model that instantiated the flat-maxima and competition between reward and accuracy maximization hypotheses was applied to the data as well as used in a reanalysis of C. J. Bohil and W.J. Maddox's (2001) study. The hybrid model was superior to a model that incorporated the independence assumption, suggesting that violations of the independence assumption are to be expected and are well captured by the flat-maxima hypothesis, without requiring any additional assumptions.     

Category discriminability, base-rate, and payoff effects in perceptual categorization

The optimality of perceptual categorization performance under manipulations of category discriminability (i.e., d' level), base rates, and payoffs was examined. Base-rate and payoff manipulations across two category discriminabilities allowed a test of the hypothesis that the steepness of the objective reward function affects performance (i.e., the flat-maxima hypothesis), as well as the hypothesis that observers combine base-rate and payoff information independently. Performance was (1) closer to optimal for the steeper objective reward function, in line with the flat-maxima hypothesis, (2) closer to optimal in base-rate conditions than in payoff conditions, and (3) in partial support of the hypothesis that base-rate and payoff knowledge is combined independently. Implications for current theories of base-rate and payoff learning are discussed.     

Linear transformations of the payoff matrix and decision criterion learning in perceptual categorization

The effects of payoff-matrix multiplication, payoff-matrix addition, the presence of long-run gains versus long-run losses, category discriminability, and base rate on decision criterion learning were examined in 2 perceptual categorization experiments. Observers were found to be sensitive to the effects of payoff-matrix multiplication (and category discriminability) on the steepness of the objective reward function in line with predictions from the flat-maxima hypothesis and contrary to the predictions from the payoff-variance hypothesis. Decision criterion learning was best in base-rate conditions, was worst when losses were associated with incorrect responding, and was intermediate when no losses were associated with incorrect responding. This performance profile was well captured by the competition between reward and accuracy (COBRA) hypothesis. A hybrid model framework that instantiates both the flat-maxima and COBRA hypotheses was necessary to account for the data from both experiments.     

Toward a unified theory of decision criterion learning in perceptual categorization

Optimal decision criterion placement maximizes expected reward and requires sensitivity to the category base rates (prior probabilities) and payoffs (costs and benefits of incorrect and correct responding). When base rates are unequal, human decision criterion is nearly optimal, but when payoffs are unequal, suboptimal decision criterion placement is observed, even when the optimal decision criterion is identical in both cases. A series of studies are reviewed that examine the generality of this finding, and a unified theory of decision criterion learning is described (Maddox & Dodd, 2001). The theory assumes that two critical mechanisms operate in decision criterion learning. One mechanism involves competition between reward and accuracy maximization: The observer attempts to maximize reward, as instructed, but also places some importance on accuracy maximization. The second mechanism involves a flat-maxima hypothesis that assumes that the observer's estimate of the reward-maximizing decision criterion is determined from the steepness of the objective reward function that relates expected reward to decision criterion placement. Experiments used to develop and test the theory require each observer to complete a large number of trials and to participate in all conditions of the experiment. This provides maximal control over the reinforcement history of the observer and allows a focus on individual behavioral profiles. The theory is applied to decision criterion learning problems that examine category discriminability, payoff matrix multiplication and addition effects, the optimal classifier's independence assumption, and different types of trial-by-trial feedback. In every case the theory provides a good account of the data, and, most important, provides useful insights into the psychological processes involved in decision criterion learning.     

On the relation between base-rate and cost-benefit learning in simulated medical diagnosis.

Observers completed a series of simulated medical diagnosis tasks that differed in category discriminability and base-rate/cost-benefit ratio. Point, accuracy, and decision criterion estimates were closer to optimal (a) for category d' = 2.2 than for category d' = 1.0 or 3.2, (b) when base-rates as opposed to cost-benefits were manipulated, and (c) when the cost of an incorrect response resulted in no point loss (nonnegative cost) as opposed to a point loss (negative cost). These results support the "flat-maxima" and competition between reward and accuracy (COBRA) hypotheses. A hybrid model that instantiated simultaneously both hypotheses was applied to the data. The model parameters indicated that (a) the reward-maximizing decision criterion quickly approached the optimal criterion, (b) the importance placed on accuracy maximization early in learning was larger when the cost of an incorrect response was negative as opposed to nonnegative, and (c) by the end of training the importance placed on accuracy was equal for negative and nonnegative costs.     

On the generality of optimal versus objective classifier feedback effects on decision criterion learning in perceptual categorization

Biased category payoff matrices engender separate reward- and accuracy-maximizing decision criteria Although instructed to maximize reward, observers use suboptimal decision criteria that place greater emphasis on accuracy than is optimal. In this study, objective classifier feedback (the objectively correct response) was compared with optimal classifier feedback (the optimal classifier's response) at two levels of category discriminability when zero or negative costs accompanied incorrect responses for two payoff matrix multiplication factors. Performance was superior for optimal classifier feedback relative to objective classifier feedback for both zero- and negative-cost conditions, especially when category discriminability was low, but the magnitude of the optimal classifier advantage was approximately equal for zero- and negative-cost conditions. The optimal classifier feedback performance advantage did not interact with the payoff matrix multiplication factor. Model-based analyses suggested that the weight placed on accuracy was reduced for optimal classifier feedback relative to objective classifier feedback and for high category discriminability relative to low category discriminability. In addition, the weight placed on accuracy declined with training when feedback was based on the optimal classifier and remained relatively stable when feedback was based on the objective classifier. These results suggest that feedback based on the optimal classifier leads to superior decision criterion learning across a wide range of experimental conditions.     

Optimal classifier feedback improves cost-benefit but not base-rate decision criterion learning in perceptual categorization

Unequal payoffs engender separate reward- and accuracy-maximizing decision criteria; unequal base rates do not. When payoffs are unequal, observers place greater emphasis on accuracy than is optimal. This study compares objective classifier (the objectively correct response) with optimal classifier feedback (the optimal classifier's response) when payoffs or base rates are unequal. It provides a critical test of Maddox and Bohil's (1998) competition between reward and accuracy maximization (COBRA) hypothesis, comparing it with a competition between reward and probability matching (COBRM) and a competition between reward and equal response frequencies (COBRE) hypothesis. The COBRA prediction that optimal classifier feedback leads to better decision criterion leaning relative to objective classifier feedback when payoffs are unequal, but not when base rates are unequal, was supported. Model-based analyses suggested that the weight placed on accuracy was reduced for optimal classifier feedback relative to objective classifier feedback. In addition, delayed feedback affected learning of the reward-maximizing decision criterion.     

Overestimation of base-rate differences in complex perceptual categories

The optimality of multidimensional perceptual categorization performance was examined for several base-rate ratios, for both integral and separable dimension stimuli, and for complex category structures. In all cases, the optimal decision bound was highly nonlinear. Observers completed several experimental sessions, and all analyses were performed at the single-observer level using a series of nested models derived from decision-bound theory (Maddox, 1995; Maddox & Ashby, 1993). In every condition, all observers were found to be sensitive to the base-rate manipulations, but the majority of observers appeared to overestimate the base-rate difference. These findings converge with those for cases in which the optimal decision bound was linear (Maddox, 1995) and suggest that base-rates are learned in a similar fashion regardless of the complexity of the optimal decision bound. Possible explanations for the consistent overestimate of the base-rate difference are discussed. Several continuous-valued analogues of Kruschke’s (1996) theory of base-rate learning with discrete-valued stimuli were tested. These models found some support, but in all cases were outperformed by a version of decision-bound theory that assumed accurate knowledge of the category structure and an overestimate of the base-rate difference.     

基础比率和认知风格对贝叶斯推理影响的眼动研究

选取35名本科生为被试,采用EyeLink II眼动仪,考察了基础比率和认知风格对贝叶斯推理的影响,探讨了基础比率作用机制的争论。实验采用2（基础比率：高、低）×2（认知风格：场依存、场独立）双因素被试间设计,要求每位被试完成一道贝叶斯推理题,问题内容为疾病情境。将推理材料划为AOI1（描述基础比率）、AOI2（描述击中率）、AOI3（描述虚报率）、AOI4（提问）4个兴趣区,分别记录被试的总注视时间、注视次数等。结果发现：（1）在总注视时间和注视次数指标上,基础比率和认知风格的主效应均不显著,两者的交互作用显著;（2）在总注视时间和回视次数指标上,各兴趣区差异显著,关注程度从高到低依次为：AOI2＞AOI3＞AOI1＞AOI4。这说明,在贝叶斯推理中,并没有完全忽视基础比率;对于不同认知风格的个体而言,基础比率所起的作用是不同的。这也给我们一点启示：或许我们不能单一地去考虑基础比率的作用机制,而应该将它与个体的某些因素结合起来综合加以讨论。     

Base-rate training without case cues reduces base-rate neglect

Base-rate neglect is a persistent phenomenon in which subjects do not place sufficient weight on the probabilities of occurrence of relevant events. Two experiments with college students support the hypothesis that base-rate neglect may be minimized by providing base-rate training in the absence of case, or witness, cues, prior to introducing (or reintroducing) these cues. In Experiment 1, the hypothesis was supported by both within-subjects and between-groups assessments; in Experiment 2, the hypothesis was supported while the effects of instructions and a correction procedure were found to be minimal. In Experiment 1, but not in Experiment 2, training with case cues present also reduced base-rate neglect, but this effect was not sufficient to account for the effect of cue-absent base-rate training. Correction trials led some subjects to detect that the task contingencies were random; however, neither this nor actually telling subjects after the experiment that the task was indeed random led invariably to subjects’ describing the optimal strategy (which was to choose the richer alternative exclusively).     

On the logic of testing the independence assumption in the process-dissociation procedure

Theprocess-dissociation procedure is designed to provide quantitative estimates of the influence of explicit and implicit memory in a variety of tasks. The procedure relies on the assumption that these two forms of memory produce independent influences on performance. Prior investigators have attempted to test this assumption by determining whether the parameter representing the influence of implicit memory (denotedA) is constant across experimental conditions. I argue that the constancy of A cannot provide an appropriate test of the independence assumption, because (1) the prediction of constancy can be generated without the assumption of independence, obviating the need to posit independence; and (2) the constancy ofA does not necessarily imply independence, even if one assumes that a dependency hypothesis, supplemented by ancillary assumptions (Curran & Hintzman, 1995), predicts differences inA. I close by emphasizing that we can test the independence assumption by using standard procedures that compare the fit of a model that assumes independence with the fit of a model that assumes dependence.     

基础概率在贝叶斯推理中的作用

针对贝叶斯推理研究中基础概率忽略现象的争论,直接操纵基础概率水平,考察被试在贝叶斯推理时是否加工了基础概率信息。结果表明,低基础概率组后验概率估计显著低,无基础概率组后验概率估计反应时显著短于有基础概率组。被试在各种条件下估计的正确率都较低,说明对基础概率在贝叶斯推理中的作用考虑不充分。     

On a problematic procedure to manipulate response biases in recognition experiments: the case of “implied” base rates

The experimental manipulation of response biases in recognition-memory tests is an important means for testing recognition models and for estimating their parameters. The textbook manipulations for binary-response formats either vary the payoff scheme or the base rate of targets in the recognition test, with the latter being the more frequently applied procedure. However, some published studies reverted to implying different base rates by instruction rather than actually changing them. Aside from unnecessarily deceiving participants, this procedure may lead to cognitive conflicts that prompt response strategies unknown to the experimenter. To test our objection, implied base rates were compared to actual base rates in a recognition experiment followed by a post-experimental interview to assess participants’ response strategies. The behavioural data show that recognition-memory performance was estimated to be lower in the implied base-rate condition. The interview data demonstrate that participants used various second-order response strategies that jeopardise the interpretability of the recognition data. We thus advice researchers against substituting actual base rates with implied base rates.     

Pseudodiagnosticity in judgment under uncertainty

The study investigates the extent to which the false alarm (i.e., ) is utilized in judgment under uncertainty. The main findings are (1) this cue is utilized by subjects when provided with a numerically low base-rate (i.e., P(H)) and a high hit-rate (i.e., P(D/H)). Under these conditions the false alarm helps resolve the inconsistency between the implications of these probabilistic cues. (2) The false alarm is ignored by subjects when provided with a numerically high base-rate and a high hit-rate. Under these conditions both latter probabilities are consistent and imply strong support for the focal hypothesis. The false alarm is either not consistent with these cues or redundant and ignored. In addition, three experiments provided evidence regarding base-rate utilization. When comparing conditions (1) and (2) it is demonstrated that the base-rate has a significant effect on judgments. Finally, the experiments provided evidence suggesting that the base-rate is not ignored when the datum is not diagnostic. The results are discussed in relation to the base-rate fallacy, pseudodiagnosticity, and cue consistency.     

The combining of multiple hemispheric resources in learning-disabled and skilled readers' recall of words: a test of three information-processing models

Three theoretical models (additive, independence, maximum rule) that characterize and predict the influence of independent hemispheric resources on learning-disabled and skilled readers' simultaneous processing were tested. Predictions related to word recall performance during simultaneous encoding conditions (dichotic listening task) were made from unilateral (dichotic listening task) presentations. The maximum rule model best characterized both ability groups in that simultaneous encoding produced no better recall than unilateral presentations. While the results support the hypothesis that both ability groups use similar processes in the combining of hemispheric resources (i.e., weak/dominant processing), ability group differences do occur in the coordination of such resources.