A signal detection framework for the evaluation of probabilistic forecasts

In this paper I formulate an approach for evaluating probabilistic forecasts in terms of signal detection theory. Signal detection theory provides a powerful perspective for this type of problem, and a rich empirical background including methodological tools as well as an extensive body of research in many domains. I propose procedures which emphasize the maximization of expected utility for the decision maker who uses the forecasts. Further, I suggest approaches to obtaining indices of calibration and resolution within this framework. I also present arguments that the proposed indices will exhibit the same basic properties as do decompositions of Brier's (1950, Monthly Weather Review, 78, 1–3) mean probability score. However, the properties may be reflected in different ways, and hence, the present methods may lead to different conclusions about forecasting ability. Finally, I argue that the use of an expected utility loss function makes this approach more appropriate for practical applications as well as for theoretical research than other procedures with more arbitrary loss functions.     

The effects of feedback consistency and feedback favorability on self-perceived task competence and perceived feedback accuracy

The present study, employing a 2 × 2 true-experimental design and regression analyses assessed the main and interactive effects of feedback consistency (consistent vs inconsistent), and feedback favorability (acceptable vs superior), for feedback given at two time periods on measures of perceived feedback accuracy and self-perceived task competence. Among the study's findings were that (a) the perceived accuracy of feedback at the second period was a function of the favorability of feedback received at the first time period, (b) the self-perceived task competence levels of subjects were jointly determined by the favorability of feedback received at both time periods, and (c) the perceived accuracy of feedback was a function of its consistency. Organizational implications of the study's results are offered.     

Childhood stress: Perceptions of children and professionals

An assessment instrument was developed to measure children's perceptions of stressors in their lives, as well as their experiences and emotional reactions to these stressors. Two approaches from the adult literature, major life events and daily hassles, were compared for children. In addition, a comparison between children's and professionals' ratings of the severity of various stressors was made, as well as an examination of the relationship of experience and self-reported reaction to children's ratings. Results indicate that children's ratings of the severity of stressors do differ from adults', particularly when children have experienced the stressor. Further investigation in the study of childhood stress incorporating children's perceptions is proposed.     

Interviewer evaluations of interviewee nonverbal behavior

Personnel representatives (N = 52) were shown one of four videotaped job interviews in which the verbal content of the 16-min interview was identical, but the interviewee's nonverbal behavior was manipulated. A “low nonverbal” interviewee was defined by minimal eye contact, low energy level, lack of affect and voice modulation, and a lack of speech fluency. The “high nonverbal” interviewee demonstrated the opposite behavior on each of these components. The subjects were asked to rate the videotaped candidates on dimensions previously identified as critical in influencing a job interviewer's decisions. Nonverbal behavior was found to have a significant effect on almost every rating made by subjects in this study. After reviewing the entire 16-min interview, 23 of the 26 subjects who saw the “high nonverbal” candidate would have invited him/her for a second interview. All 26 of the subjects who saw the “low nonverbal” candidate would not have recommended a second interview.     

Relationships between perceptions of organizational size and the organizational choice process

Expectancy theory concepts were used to predict (a) the attractiveness of, (b) the amount of effort directed toward entering, and (c) the eventual choice of a job in relatively big and small work organizations. It was found, among a group of graduating seniors from a technical college, that antecedent perceptions regarding organizational size (expectancy, valence, instrumentality) were significantly related to the relative attractiveness of different-sized organizations and the number of job interviews taken with relatively big and small companies. In addition, economically tied extrinsic outcomes were seen as more readily attainable in big organizations, whereas certain intrinsic outcomes were perceived to be more easily obtained in small organizations.     

When is an easy task difficult and vice versa? A reply to lane

Lane has argued that kantowitz and Knight incorrectly predicted an interaction between primary and secondary task difficulty for the variable-allocation model of attention. It is shown that such an interaction is indeed obtained if task-performance difficulty is defined in terms of capacity and only reasonable transformations of data are permitted. This interaction holds for any set of monotonic resource operating characteristics.     

The retention of nouns,adjectives, and verbs of sentences

This research compared the retention of nouns, adjectives and verbs of sentences. A model of the structure underlying the retention of sentences and in which deep structure constituent propositions are hierarchically related, was used to make predictions concerning the retention of these word classes. The model was not supported by the data and it was determined that latencies to recognize the words of a previously presented sentence were the same for all classes of words. This result indicated that for recognition (but not necessarily for other memory tasks), each word of a sentence is seperately or equally accessible (as long as inter-word imagery is the same). However, a further experiment showed that, although word class is not a variable, the image-arousing capacity of a word is. Latencies and error rates were much lower for high-imagery words than for low-imagery ones. The findings on error rates in recognizing words were interesting and pointed to a different model where the meaning representations and syntatic functions of words have an active role in sentence retention.     

Clustering of skills and occupations: A generic skills approach to occupational training

A report on the process and procedures of determining generic occupational skills and clusters of occupations for optimal occupational training needs is presented. Four hundred and ninety workers and their supervisors representing 37 different occupations were interviewed using a structured questionnaire consisting of items on mathematics, communication, reasoning, and interpersonal and manipulative skills. Respondents were asked to indicate whether a specific skill was a prerequisite to enter a job and also whether that specific skill was required on the job. Six, three, and two hierarchical skill factors and nine occupational clusters were identified. Training and curricular implications were explored.