A signal detection framework for the evaluation of probabilistic forecasts

In this paper I formulate an approach for evaluating probabilistic forecasts in terms of signal detection theory. Signal detection theory provides a powerful perspective for this type of problem, and a rich empirical background including methodological tools as well as an extensive body of research in many domains. I propose procedures which emphasize the maximization of expected utility for the decision maker who uses the forecasts. Further, I suggest approaches to obtaining indices of calibration and resolution within this framework. I also present arguments that the proposed indices will exhibit the same basic properties as do decompositions of Brier's (1950, Monthly Weather Review, 78, 1–3) mean probability score. However, the properties may be reflected in different ways, and hence, the present methods may lead to different conclusions about forecasting ability. Finally, I argue that the use of an expected utility loss function makes this approach more appropriate for practical applications as well as for theoretical research than other procedures with more arbitrary loss functions.     

The retention of nouns,adjectives, and verbs of sentences

This research compared the retention of nouns, adjectives and verbs of sentences. A model of the structure underlying the retention of sentences and in which deep structure constituent propositions are hierarchically related, was used to make predictions concerning the retention of these word classes. The model was not supported by the data and it was determined that latencies to recognize the words of a previously presented sentence were the same for all classes of words. This result indicated that for recognition (but not necessarily for other memory tasks), each word of a sentence is seperately or equally accessible (as long as inter-word imagery is the same). However, a further experiment showed that, although word class is not a variable, the image-arousing capacity of a word is. Latencies and error rates were much lower for high-imagery words than for low-imagery ones. The findings on error rates in recognizing words were interesting and pointed to a different model where the meaning representations and syntatic functions of words have an active role in sentence retention.     

Predictability,controllability, and inoculation against learned helplessness

Nondepressed human subjects were divided into seven groups. On a series of discrimination problems, a helplessness group received insoluble problems, a solvable group received contingent feedback, and a no treatment control group received no feedback. For two other groups insoluble problems were preceded by success feedback on a different task presented according to a fixed ratio (FR) or variable ratio (VR) schedule of reinforcement. Two control groups received either FR or VR schedules of success but were not examined on the discrimination problems. All groups were tested for escape/avoidance performance on a human shuttle box. Both FR and VR schedules produced an inoculation against learned helplessness; escape performance by the helplessness group was significantly worse than that of FR and VR inoculation groups. These latter groups performed similar to the solvable and three control groups. Significantly worse than that of FR and VR inoculation groups. These latter groups performed similar to the solvable and three control groups. Significantly fewer subjects in the VR inoculation group exhibited avoidance responses than their counterparts in the FR inoculation group. despite similar escape performance. The findings indicate that learned helplessness can be prevented in humans and suggest different sources of interference produced by unpredictable and uncontrollable events.     

Comparative judgments with numerical reference points

A model of subjective magnitude comparisons is explored, which assumes that subjects compare symbolic stimulus magnitudes with respect to a reference point. The reference point may be established implicitly by the question (e.g., “Which is larger?” vs “Which is smaller?”) or be presented explicitly (e.g., “Choose the stimulus closer to X.”). The model was tested in five experiments in which subjects judged which of two comparison digits was closer to (or further from) a reference digit. Regression analyses in three experiments revealed that reaction time depended on the ratio of the distances from the comparison items to the reference point. The other two experiments provided evidence that subjects can strategically vary the processes by which they compare stimuli to a reference point. The results indicated that subjects can perform various types of “analog arithmetic” using either the linear number scale or a nonlinear scale of subjective digit magnitude.     

Development of natural language concepts: IV. The relationship between semantic organization and learning

In Experiment 1, 30 subjects at each of three age levels were shown pictures of unfamiliar “animals”. They were told two semantic attributes for each animal (e.g., kind and strong) and were asked to learn these sets so that, when shown a picture, the appropriate attributes could be recalled. The results indicated that the dimensional structure of the attributes had a strong effect on new learning for young children. Incongruent pairs of attributes (e.g., kind and ugly) were difficult to learn compared to congruent (e.g., kind and beautiful) or unrelated (e.g., kind and tall) sets. These results indicate: (a) Attribute structure is a factor in new learning; (b) 6-year-olds tend to organize the verbal labels for attributes into bipolar dimensions, rather than into independent clusters of attributes. The latter findings require reexamination of the meaning of previous word-association data which had been interpreted as indicating that 6-year-olds tend not to organize meanings bipolarly. Experiment 2 showed that the disruptive effects of incongruence on new learning disappears by college age.     

Training children's self-control: A field experiment in self-monitoring and goal-setting in the classroom

A field experiment investigated the effects of training in self-monitoring and goal-setting skills on classroom study behavior and academic achievement among elementary school students in an individualized mathematics program. In the Self-Monitoring Conditions, students were shown a simple system for observing and maintaining daily records of their own study behavior during their math classes; orthogonally, in the Goal-Setting Conditions, students were shown a simple method of setting and recording daily performance goals during their math classes. Exposure to self-monitoring procedures produced significant increases in both appropriate study behavior and in actual achievement in the mathematics program, while exposure to goal-setting procedures had no effect on either study behavior or academic achievement. Nor, contrary to expectation, did exposure to the goal-setting instructions enhance the effectiveness of the self-monitoring system. The implications of these results in terms of implicit and explicit models of self-reinforcement processes were discussed.     

Developmental constraints associated with children's use of the keyword method of foreign language vocabulary learning

Second- and sixth-grade (ages 7 and 11 years, respectively) subjects were instructed to learn simple Spanish vocabulary nouns using the keyword method. To remember a foreign word translation the keyword method user must: (1) associate the foreign word to an English word (the keyword) that sounds like part of the foreign word; and (2) remember an image of the keyword and translation referents interacting. Second-grade keyword users who were provided with interactive pictures remembered more vocabulary items than those who generated their own imagery links when given separate pictures of the keyword and translation referents. Second graders who generated their own linking images when given only the keywords and translation words recalled fewer items than both picture groups, and were not significantly different from control subjects. Sixth graders in the three imagery-link variations performed at comparable levels and better than control subjects. The results are in complete accord with previous speculations about the development of imagery strategies in children.     

Transfer of compounds and components in the discriminative learning of retardates

The discriminative learning and transfer of compound and component problems were assessed in retarded subjects at two levels of intelligence. It was found that brighter subjects (MA = 10 years) found the component problems relatively easier than the compound problems to a greater extent than did the less developed subjects (MA = 5 years). The learning of compound solutions was shown to transfer positively to other compound problems and negatively to component problems. Component solutions were found to transfer positively to other component solutions and negatively to compound solutions. These dimensional transfer effects were interpreted to mean that both compound and component solutions were mediated and conceptual in nature. The developmental differences in relative usages of compound and component aspects of stimuli were viewed as the result of differences in direction of attention rather than of differences in complexity of processing.     

Social influence on attitudes and on anchoring of congeneric attitude scales

Criteria for distinguishing between attitude and response scale effects in “after-only” studies of source-recipient discrepancy were developed and applied in two experiments. These criteria refer to congeneric scales (i.e., multiple measures of the the same property, which may differ in origin, unit, and reliability). Persuasive information may function as a scale anchor, in which case it is expected to affect only that response scale to which it directly pertains. Alternatively, it may convice recipients to change attitudinal positions, and produce effects on all congeneric scales. In one experiment, the self-proclaimed leniency-sternness of a presiding judge in a manslaughter case was varied along with the sentence he imposed and his trustworthiness. In the second study, only the sentence was manipulated, while both leniency ratings and sentencing were measured as dependent variables. Both analyses revealed support for the response scale interpretation, in that effects were observed only on those scales that were directly manipulated.