Self-serving bias effects on job analysis ratings

The purpose of this study was to investigate whether worker-oriented job analysis importance ratings were influenced by subject matter experts' (SME) standing (as measured by self-rated performance) on a competency. This type of relationship (whereby SMEs indicate that the traits they have are important for successful job performance) is an example of the self-serving bias (which is widely described in the social cognition literature and rarely described in the industrial/organizational psychology literature). An archival dataset covering 57 clerical and technical occupations with 26,682 participants was used. Support was found for the relationship between self-rated performance and importance ratings. Significant relationships (typically in the .30s) were observed for all 31 competencies that were studied. Controls were taken to account for common method bias and differences in the competencies required for each of the 57 occupations. Past research has demonstrated the effects of the self-serving bias on personality-based job analysis ratings. This study was the first to extend these findings to traditional job analysis, which covers other competencies in addition to personality. In addition, this study is the first to use operational field data instead of laboratory data.     

The value of relative-time-spent ratings in task-oriented job analysis

Recent studies have called for the abandonment of the relative-time-spent scale in task inventories. This recommendation is based on findings that the job profile created with the scale data is highly correlated with the profile created from a much simpler “Do you perform this task?” checklist. We examined this issue using 3 inventories and 42 jobs (N=2252). Profile correlations were computed on only the tasks actually performed by incumbents to avoid possibly inflated rs due to including irrelevant tasks. The specificity of task inventory items was proposed as an explanation for the high correlation between the two job profiles. Specificity of items was examined by looking at both the type (job duties versus tasks) and the amount (number of items in job profile and average number of items relevant to each job) of items used in the inventory. Correlations between time spent and checklist profiles were in the .80's and .90's regardless of the number of irrelevant tasks or the specificity of tasks. We agree with previous military research and conclude that the relative-time-spent scale has limited incremental utility beyond a dichotomous checklist.     

Relationships between job evaluation ratings and self-ratings of job characteristics

Self-ratings by 308 shop workers of their job variety, autonomy, task identity, and skill challenge correlate significantly with ratings by observers of the jobs' skill requirements. Self-ratings, however, are unrelated to observers' estimates of the physical demands of the jobs or to environmental conditions of the work. The self-ratings and observer measures provide complementary, not substitute, information. Self-ratings can signal person-job dysfunctions, and more objective job measures can identify possible sources of the dysfunctions. Modifications to job characteristics measures are discussed and suggestions are made to begin a new phase of programmatic job characteristics research.     

Effects of amount of information on judgment accuracy and confidence

When a person evaluates his or her confidence in a judgment, what is the effect of receiving more judgment-relevant information? We report three studies that show when judges receive more information, their confidence increases more than their accuracy, producing substantial confidence–accuracy discrepancies. Our results suggest that judges do not adjust for the cognitive limitations that reduce their ability to use additional information effectively. We place these findings in a more general framework of understanding the cues to confidence that judges use and how those cues relate to accuracy and calibration.     

Effects of variable-interval value and amount of training on stimulus generalization.

In Experiment 1 pigeons pecked a key that was illuminated with a 501-nm light and obtained food by doing so according to a variable-interval (VI) schedule of reinforcement, the mean value of which differed across groups: either 30 s, 120 s, or 240 s. The pigeons in all three groups were trained for 10 50-min sessions. Generalization testing was conducted in extinction with different wavelengths of light. Absolute and relative generalization gradients were similar in shape for the three groups. Experiment 2 was a systematic replication of Experiment 1 using line orientation as the stimulus dimension and a mean VI value of either 30 s or 240 s. Again, gradients of generalization were similar for the two groups. In Experiment 3 pigeons pecked a key that was illuminated with a 501-nm light and obtained food reinforcers according to either a VI 30-s or a 240-s schedule. Training continued until response rates stabilized (> 30 sessions). For subjects trained with the 30-s schedule, generalization gradients were virtually identical regardless of whether training was for 10 sessions (Experiment 1) or until response rates stabilized. For subjects trained with the VI 240-s schedule, absolute generalization gradients for subjects trained to stability were displaced upward relative to gradients for subjects trained for only 10 sessions (Experiment 1), and relative generalization gradients were slightly flatter. These results indicate that the shape of a generalization gradient does not necessarily depend on the rate of reinforcement during 10-session single-stimulus training but that the effects of prolonged training on stimulus generalization may be schedule dependent.     

Prediction of job satisfactoriness for workers with severe handicaps from aptitudes,personality, and training ratings

The predictive validity of General Aptitude Test Battery and Sixteen Personality Factor Questionnaire scores were compared to standard training ratings made by vocational instructors against the criterion of work performance measured by the Minnesota Satisfactoriness Scales for a sample of 106 employees with severe handicaps. The psychometric test variables were not correlated with the criterion; however, the training ratings were consistently predictive of the job satisfactoriness scores. These results suggest that the employment potential of job applicants with disabilities can be assessed more accurately using situational training ratings, as opposed to standardized psychometric test scores.     

Overhabituation and dishabituation: Effects of extraversion and amount of training

The Eysenck (1967) hypothesis that introverts have higher levels of cortical activation than extraverts has received support in a number of psychophysiological studies, though there is not complete consistency. The present study extended the habituation paradigm used in some previous work to include an examination of differences between extraverts and introverts during and following long-term overhabituation. Electrodermal activity was recorded while 72 extraverts and 72 introverts were subjected to criterion auditory habituation to a 1000-Hz, 90-db tone, followed by 60, 100, or 140 trials of overhabituation. The overhabituation run was followed by a test stimulus of 7000 Hz and a final repetition of the standard stimulus. Results showed that introverts increased in response frequency with increasing overhabituation exposure, while extraverts showed an increase only from 100 to 140 trials. In addition, the hypothesis that differences in test response amplitude favoring introverts would be eliminated by extended overhabituation training was supported. Specifically, extraverts showed increased responses at 100 and 140 trials, while introverts remained approximately constant across the three conditions. Finally, results supported the Sokolov hypothesis that overhabituation training produces test responses of larger amplitude.     

Understanding the latent structure of job performance ratings

This study quantified the effects of 5 factors postulated to influence performance ratings: the ratee's general level of performance, the ratee's performance on a specific dimension, the rater's idiosyncratic rating tendencies, the rater's organizational perspective, and random measurement error. Two large data sets, consisting of managers (n = 2,350 and n = 2,142) who received developmental ratings on 3 performance dimensions from 7 raters (2 bosses, 2 peers, 2 subordinates, and self) were used. Results indicated that idiosyncratic rater effects (62% and 53%) accounted for over half of the rating variance in both data sets. The combined effects of general and dimensional ratee performance (21% and 25%) were less than half the size of the idiosyncratic rater effects. Small perspective-related effects were found in boss and subordinate ratings but not in peer ratings. Average random error effects in the 2 data sets were 11% and 18%.     

The effect of dimension content on observation and ratings of job performance

Recent theory and research in the performance appraisal area (e.g., Denisi, Cafferty, & Meglino, 1984; Feldman, 1986; Ilgen & Feldman, 1983; Williams, Denisi, & Blencoe, 1985) have suggested that providing information regarding the performance dimension to be rated will cause raters to select appropriate observational schemata and, as a result, produce higher quality ratings. A study was conducted to determine if giving raters dimension-relevant information prior to performance observation would affect their attention processes and rating quality. Prior to watching a videotape of an instructor giving a lecture, 156 subjects were given either: 1) correct information, 2) incorrect information, or 3) no information regarding dimensions of performance they would subsequently be asked to rate. The results indicated that giving prior information regarding dimension content affected subjects' attention processes. Further, raters receiving no information and those receiving misinformation prior to performance observation produced less accurate ratings compared to expert raters. Ratings produced by subjects receiving correct information did not differ significantly from experts' ratings. These results are discussed in terms of both their practical and theoretical implications.     

Effects of amount of helplessness training and internal-external locus of control on mood and performance.

Based on Wortman and Brehm's integration of reactance theory with Seligman's model of learned helplessness, an investigation was conducted to examine the effects of amount of helplessness training and internal--external locus of control on subsequent task performance and on self-ratings of mood. Subjects were divided into "internal" and "external" groups and were then given either high, low, or no helplessness training on a series of concept-formation problems. After completing a mood checklist, all subjects worked on an anagram task presented as a second experiment by a second experimenter. The results revealed that internals exhibited greater performance decrements and reported greater depression under high helplessness than did externals. In the low helplessness conditions, internals tended to perform better than control subjects, while externals tended to perform worse than control subjects; low helplessness subjects also reported the highest levels of hostility. The results are discussed within the context of Wortman and Brehm's integration of reactance and learned helplessness theories.     

Effects of training intensity on observers' ratings of anxiety, social skills, and alcohol-specific coping skills

Few studies have addressed the amount of training needed to obtain reliable ratings in behavioral observation data. The current study examined the effects of differing intensities of frame-of-reference (FOR) rater training on observers' ratings of anxiety, social skills, and alcohol-specific coping skills in community volunteers with and without social anxiety and alcohol use disorders. Interrater reliability was assessed by comparing three training conditions (no-training, moderate FOR, and intensive FOR) on discrepancies between raters'scores and the strength of association between raters'scoring patterns. The discrepancies between raters' scores were significantly larger in the control condition than in the intensive and moderate training conditions. Generally, small and nonsignificant differences were found between intensive and moderate training's discrepancy scores. Strength of association results showed significantly lower correlations in the control group compared to the intensive group. However, these correlational results showed less consistent differences between the moderate and other training conditions; differences when found were in the expected directions. Study findings suggest that differing training intensities can affect rating scores and that interrater reliability may be meaningfully assessed through multiple methods.     

Chronic job stressors and job control: Effects on event‐related coping success and well‐being

The present study investigates how chronic work characteristics relate to situational work characteristics, and their effect on outcomes such as well‐being, problem solving, and calming down. It also examines how the relationship between situational work characteristics and coping strategies affect these outcomes. Based on an event‐sampling approach, 23 employees of a counselling agency reported 120 stressful events over 7 days, yielding 92 work‐related and 28 private events. Multi‐level analyses revealed that with regard to chronic conditions, job control was positively associated with successful calming down and problem solving in stressful situations, whereas job stressors were negatively related to immediate well‐being. Chronic stressors and job control can be seen as ‘background’ variables that influence the response to stressful events. For situational factors, stressfulness was negatively associated and controllability was positively associated with immediate well‐being and problem solving. In addition, problem‐focused coping predicted positively situation‐related success in calming down and problem solving. An effect on well‐being was found when situational controllability was high, confirming the active coping rule of Perrez and Reicherts (1992) . Palliative coping was positively related to successful calming down.     

Averaging processes in ratings and choices based on numerical information

Ss were presented sets of numbers said to represent sample price information from grocery stores. Ss rated the relative desirability of shopping at each of a series of stores and were also asked to make paired-comparison preference choices. In Experiment I, some sets contained favorable information, some contained unfavorable information, and some contained neutral information added to the favorable or unfavorable information. The addition of neutral information led to a decrease in the polarity of responses. This is consistent with an averaging, but not an adding, formulation of how the information is integrated. In Experiment II, sets of favorable or unfavorable information varied in size and a set-size effect was obtained. The greater the amount of favorable or unfavorable information, the more extreme the response. This was accounted for by assuming that an initial neutral expectancy is averaged with the information presented.