A signal detection framework for the evaluation of probabilistic forecasts

In this paper I formulate an approach for evaluating probabilistic forecasts in terms of signal detection theory. Signal detection theory provides a powerful perspective for this type of problem, and a rich empirical background including methodological tools as well as an extensive body of research in many domains. I propose procedures which emphasize the maximization of expected utility for the decision maker who uses the forecasts. Further, I suggest approaches to obtaining indices of calibration and resolution within this framework. I also present arguments that the proposed indices will exhibit the same basic properties as do decompositions of Brier's (1950, Monthly Weather Review, 78, 1–3) mean probability score. However, the properties may be reflected in different ways, and hence, the present methods may lead to different conclusions about forecasting ability. Finally, I argue that the use of an expected utility loss function makes this approach more appropriate for practical applications as well as for theoretical research than other procedures with more arbitrary loss functions.     

Comparative judgments with numerical reference points

A model of subjective magnitude comparisons is explored, which assumes that subjects compare symbolic stimulus magnitudes with respect to a reference point. The reference point may be established implicitly by the question (e.g., “Which is larger?” vs “Which is smaller?”) or be presented explicitly (e.g., “Choose the stimulus closer to X.”). The model was tested in five experiments in which subjects judged which of two comparison digits was closer to (or further from) a reference digit. Regression analyses in three experiments revealed that reaction time depended on the ratio of the distances from the comparison items to the reference point. The other two experiments provided evidence that subjects can strategically vary the processes by which they compare stimuli to a reference point. The results indicated that subjects can perform various types of “analog arithmetic” using either the linear number scale or a nonlinear scale of subjective digit magnitude.     

Neuropsychological significance of phonetically accurate and phonetically inaccurate spelling errors in younger and older retarded spellers

The performances of two groups of retarded spellers (who were equated with respect to the level of deficient performance in spelling, but who differed in the level of phonetic accuracy of their misspellings) at two different age levels were compared to those of normal spellers who had been matched for age and WISC Performance IQ with the retarded spelling groups. The results indicated very marked differences in a number of linguistic skills between the normal spellers and the retarded spellers whose misspellings were characterized by phonetic inaccuracy. The performances of the retarded spellers who produced a preponderance of phonetically accurate misspellings differed markedly from those of the normal spellers only when the task demands involved fairly complex linguistic-cognitive operations. For the most part, the differences among the three groups were in evidence only at the older age level studied.     

Selective attention by children and adults to pictures specified by color

Four conditions were used to investigate developmental trends in the ability to establish and use a color set to direct the selective processing of pictures. In three conditions, 6-year-old, 9-year-old, and adult subjects viewed a series of pairs of pictures, with one red and one black line drawing in each pair. Subjects were asked to look either at the red pictures only, the black pictures only, or both pictures. In a fourth condition, subjects viewed a series of singly presented red and black pictures. Pictures of both colors were included in a subsequent recognition memory test. At all ages recognition memory was comparable for pictures of each color in the both and single conditions but was higher for pictures of the specified color in the selective red and selective black conditions. There was no evidence at any age that memory for pictures of the specified color was decreased by the presence of the second picture. These results, showing roughly comparable selectivity at all ages, were discussed in relation to findings of developmental trends in selective attention on more “traditional” central-incidental learning tasks.     

Central deafness and subcortical motor aphasia

Clinical and autopsy studies were made on a right-handed man who had central deafness and subcortical motor aphasia, and the literature on central deafness and on subcortical motor aphasia was analyzed. Central deafness is due to bilateral destruction of the primary auditory cortex. It is sometimes difficult to distinguish from word deafness and from auditory agnosia, which are due to pathology in other parts of the temporal lobes. There is almost always some preserved hearing in central deafness, possibly from some auditory pathway other than the classical pathway. In this patient the subcortical motor aphasia was due to bilateral destruction of the motor cortex for the mouth and throat. In some other cases subcortical motor aphasia was due to the same pathology that usually causes Broca's aphasia; in these cases the unexpected preservation of writing was perhaps related to some difference in how language functions were organized in the brain.     

Effects of knowledge that attributions will be observed by others

The current study was conducted to determine if attribution statements would be affected by subjects' knowledge that their attributions of success or failure would be observed by an opposite-sex peer. At the time subjects recorded their attributions, half of them anticipated that their attributions would soon be observed in their presence by an opposite-sex peer, while the other half recorded their attributions anonymously. Results indicated that attributions of success and failure were affected by the social context. Observed subjects evidenced less tendency to attribute their failure to low ability than did nonobserved subjects. Subjects who succeeded on an identity-relevant task reported higher ability attributions under observation conditions than under nonobservation conditions. Observed subjects evidenced significantly greater willingness to attribute failure to lack of effort than did nonobserved subjects. For a task intended to be of minimal relevance to subjects' identities, nonobserved subjects attributed failure to task difficulty to a significantly greater degree than did observed subjects. Results were discussed in relation to Bradley's contention that self-serving biases in attribution can usefully be conceptualized as strategic self-presentations.     

Sensitivity of 5- and 7-month-old infants to pictorial depth information

An Ames (1951, Psychological Monographs, 65(1, Whole No. 324)) static trapezoidal window, under monocular view, was used to test young infants' responsiveness to pictorial depth. When adults view this display monocularly with the smaller side of the window rotated toward them, they report that the orientation of the display becomes ambiguous: When the head is moved, the window may appear to be in the fronto-parallel plane or either side may appear closer. The 7-month-olds we tested appeared to experience a similar ambiguity; they reached to the near side of the rotated trapezoidal window with significantly less consistency or directedness than infants in a control group tested with a rotated object that lacked pictorial depth information. When 5-month-olds were tested, however, they reached with equal consistency to the closer side of the trapezoidal window and of the control display, apparently uninfluenced by the pictorial depth information available in the trapezoidal window. Thus, sensitivity to the pictorial information for depth that is present in the trapezoidal window appears to develop after the age of 22 weeks.     

Performance of dyslexic children on speech perception tests

Several researchers who have compared the performance of dyslexic and normal-reading children on a variety of different tasks have suggested that dyslexic children may have subtle deficits in the phonemic analysis of spoken as well as written language. Thus it is of interest to know how children who have extraordinary difficulty learning to read can perform explicity auditory-phonetic tasks. Seventeen dyslexic children (10 years of age) and a group of 17 controls were administered tests of identification and discrimination of synthesized voiced stop consonants differing in place of articulation. These were tests of the type used to study categorical perception in adults, adapted for use with young children. Significant differences between dyslexics and controls were found in both kinds of tasks; the pattern of identification and discrimination differences suggests an inconsistency in the dyslexics' phonetic classification of auditory cues. A significant relationship was found between reading level and speech discrimination.     

Children's detection and awareness of textual anomaly

Children of 8 and 11 years were assessed in two experiments for their sensitivity to textual anomaly. In Experiment 1, subjects read stories containing two target lines, one appropriate and the other anomalous in relation to previously given information. Both age groups read the anomalous line more slowly than the appropriate line, but in a subsequent test of comprehension monitoring, the older group was more likely than the younger group to pick out the anomalous line as not fitting in with the rest of the story. Experiment 2 produced similar results: both 8- and 11-year-old children read an anomalous line more slowly, but 11-year-olds were more likely than 8-year-olds to cite the anomalous line or part of it when questioned about the possible presence of a line that did not fit in with the rest of the story. The results indicate that an age change in comprehension monitoring as indexed by citation or selection of a textual anomaly need not be contingent upon a parallel age change in constructive processing as indexed by modulation of reading rate.