Base rates versus sample accuracy: competition for control in human matching to sample

People often place undue weight on specific sources of information (case cues) and insufficient weight on more global sources (base rates) even when the latter are highly predictive, a phenomenon termed base-rate neglect. This phenomenon was first demonstrated with paper-and-pencil tasks, and also occurs in a matching-to-sample procedure in which subjects directly experience case sample (cue) accuracy and base rates, and in which discrete, nonverbal choices are made. In two nonverbal experiments, subjects were exposed to hundreds of trials in which they chose between two response options that were both probabilistically reinforced. In Experiment 1, following one of two possible samples (the unpredictive sample), either response was reinforced with a .5 probability. The other sample (predictive) provided reinforcement for matching on 80% of the trials in one condition but in only 20% of the trials in another condition. Subjects' choices following the unpredictive sample were determined primarily by the contingencies in effect for the predictive sample: If matching was reinforced following the predictive sample, subjects tended to match the unpredictive sample as well; if countermatching the predictive sample was generally reinforced, subjects tended to countermatch the unpredictive sample. These results demonstrate only weak control by base rates. In Experiment 2, base rates and sample accuracy were simultaneously varied in opposite directions to keep one set of conditional probabilities constant. Subjects' choices were determined primarily by the overall accuracy of the sample, again demonstrating only weak control by base rates. The same pattern of choice occurred whether this pattern increased or decreased rate of reinforcement. Together, the results of the two experiments provide a clear empirical demonstration of base-rate neglect.     

Principles for Designing Randomized Preventive Trials in Mental Health: An Emerging Developmental Epidemiology Paradigm

An emerging population-based paradigm is now being used to guide the design of preventive trials used to test developmental models. We discuss elements of the designs of several ongoing randomized preventive trials involving reduction of risk for children of divorce, for children who exhibit behavioral or learning problems, and for children whose parents are being treated for depression. To test developmental models using this paradigm, we introduce three classes of design issues: design for prerandomization, design for intervention, and design for postintervention. For each of these areas, we present quantitative results from power calculations. Both scientific and cost implications of these power calculations are discussed in terms of variation among subjects on preintervention measures, unit of intervention, assignment, balancing, number of pretest and posttest measures, and the examination of moderation effects.     

Assessment Centre Observation Procedures: An Experimental Comparison of Traditional,Checklist and Coding Methods

Concern about the construct validity of assessment centre judgements has led to calls for research into observers’ cognitive processes. In an experiment comparing the Traditional observation procedure against use of a Behavioural Checklist and of Behavioural Coding, six outcome variables were examined. The methods were found to yield similar outcomes in terms of accuracy of judgement, accuracy of written evidence, correlation between dimension ratings, and attitude toward the method employed. However, significant between-method differences were observed in variability between observers and in their willingness to recommend a procedure. It is argued that Behavioural Coding has special merit in assessment centres; the method reduces the cognitive demands placed on assessors, and valuably structures their information-processing. Traditional objections to simultaneous observation and classification are shown to be inappropriate.     

Matching-to-sample accuracy on fixed-ratio schedules.

Pigeons performing on a matching-to-sample procedure were exposed to six fixed-ratio (FR) schedules (FR 1, 5, 10, 20, 40, and 60) of food reinforcement for correct matching responses. During both a correction and a noncorrection procedure without an intertrial interval (ITI), matching accuracy was lower on FR 1 and FR 60 than at intermediate ratios. With the FR 1 schedule, both a 5-sec and a 25-sec ITI resulted in higher matching accuracy than without an ITI; accuracy, with an ITI, was fairly constant for ratios of 1 to 20 but declined at higher ratios. The results suggest that the presence or absence of an ITI in matching to sample may account for inconsistencies obtained in earlier studies of the relationship of matching accuracy to ratio size.     

Methodological issues in judgment and decision-making research: Concurrent verbal protocol validity and simultaneous traces of process

This paper examines two dimensions of concurrent verbal protocol validity. First, whether verbalization affects process and outcome (reactivity) was examined by comparing concurrent verbal protocol traces with those from a computer search process tracing method, the latter being a complete trace of information acquisition from experimental materials. Earlier findings that verbalization affects time were confirmed. However, verbalization did not affect amount and pattern of acquisition or accuracy of judgments. Second, whether concurrent verbal protocols are complete was examined by comparing concurrent verbal protocol and computer traces that were simultaneously obtained in a treatment in which subjects verbalized as they acquired information from the computer. The verbal traces less completely captured information acquisition behavior than computer search. This suggests that, although concurrent verbal protocols provide greater insight into decision behavior than computer search, the latter is a more reliable information-acquisition trace. Thus, if information acquisition is of primary interest and if computer search activities can be naturally integrated into performing the primary task, computer search is preferred to concurrent verbal protocols. However, if information use or retrieval from long-term memory is of primary interest, concurrent verbal protocols are preferred to computer search. Finally, this paper examined whether the simultaneous use of concurrent verbal protocols and computer search provides traces of information acquisition and use that are as complete as when each method is independently applied. Although computer search tended to limit subjects verbalizations of evaluative operators, this effect may be eliminated by practice on the computer prior to collecting data.     

Effects of training to increase self-monitoring accuracy

We examined the effects of teaching children to monitor and record their attending behavior accurately. On the basis of their low levels of attention to task and agreement with an observer about their attending, we selected four students from a class of 11, all of whom were practicing self-recording. The classroom teacher trained the students to make their judgments about their attention to task correspond with hers. The accuracy training, which was applied according to a multiple baseline (across subjects) design, produced salutary increases in levels of attention to task. But changes in accuracy of the students' self-recording were small and unstable. Although the procedures used in this study produced high levels of reactivity, they do not support the position that accuracy in self-monitoring is required for reactivity.     

个体和群体方式绩效评估的准确性研究

本研究通过对 2 1 0名大学生以个体和群体方式对两个学科评估结果的研究 ,分析了个体与群体评估方式以及不同的评估标准对评估过程和评估结果的影响。研究结果表明 :1群体评估方式有利于减少评估中的晕轮效应 ;2群体方式有利于提高对评估信息的利用水平 ;3群体方式有利于提高评估者评分倾向 ,即代表性认知的一致程度 ;4群体方式有利于提高评估者理解绩效信息意义的一致程度 ;5群体方式评估时评估者对绩效信息结构的理解更为合理 ;6提供较为详细的评估标准有利于提高对信息的利用水平 ,有利于提高评估者评分倾向的一致程度以及评估者对学科差异的敏感性。     

Young children combine sensory cues with learned information in a statistically efficient manner: But task complexity matters

Human adults are adept at mitigating the influence of sensory uncertainty on task performance by integrating sensory cues with learned prior information, in a Bayes‐optimal fashion. Previous research has shown that young children and infants are sensitive to environmental regularities, and that the ability to learn and use such regularities is involved in the development of several cognitive abilities. However, it has also been reported that children younger than 8 do not combine simultaneously available sensory cues in a Bayes‐optimal fashion. Thus, it remains unclear whether, and by what age, children can combine sensory cues with learned regularities in an adult manner. Here, we examine the performance of 6‐ to 7‐year‐old children when tasked with localizing a ‘hidden’ target by combining uncertain sensory information with prior information learned over repeated exposure to the task. We demonstrate that 6‐ to 7‐year‐olds learn task‐relevant statistics at a rate on par with adults, and like adults, are capable of integrating learned regularities with sensory information in a statistically efficient manner. We also show that variables such as task complexity can influence young children's behavior to a greater extent than that of adults, leading their behavior to look sub‐optimal. Our findings have important implications for how we should interpret failures in young children's ability to carry out sophisticated computations. These ‘failures’ need not be attributed to deficits in the fundamental computational capacity available to children early in development, but rather to ancillary immaturities in general cognitive abilities that mask the operation of these computations in specific situations.     

Hick's Law in a Stochastic Race Model with Speed-Accuracy Tradeoff

We present an analytic solution for a race model of n stochastic accumulators for multiple choice reaction time. We show that to maintain a constant level of accuracy, the response criterion needs to be increased approximately logarithmically with n, to compensate for the increase with n in the likelihood that an incorrect alternative will be most active after any fixed amount of time accumulating information. Assuming that participants monitor and maintain a constant level of performance can then explain the logarithmic dependency of the response latency on n as specified by Hick's law. Moreover, we show that for short time intervals, the Shannon information that observers extract from a stimulus, is predicted to increase linearly with processing time.