AN EVALUATION OF ALTERNATE SCORING METHODS FOR THE MIXED STANDARD SCALE

This study investigated the psychometric properties of three methods of scoring a Mixed Standard Scale (MSS) performance evaluation: the patterned procedure as corrected by Saal (1979); a simple nonpatterned scoring procedure suggested by Prien, Jones, and Miller (1977), which gives equal weights to the performance statements; and a procedure that assigned differential weights to each statement on the basis of scale values provided by a panel of subject matter experts. Interrater reliabilities, scale variances for averaged ratings, and a convergent/discriminant validity analysis, which included an alternate method of job skill ratings, indicated no difference in the score distribution variance, interrater reliability, or validity of different method scores.     

Validation of a test battery for manufacturing machine operators

The present study determined if it was possible to develop a selection procedure which would effectively select machine operator applicants. A concurrent validity study was conducted on 140 employees with a job simulation as the criterion. The results indicated a correlation with the criterion of .71, and the Lawshe (1983) test of fairness revealed no significant bias for minority or female applicants. The utility analysis indicated a 16% increase in output would occur with the use of the validated selection system.     

Self-presentation processes in job analysis: a field experiment investigating inflation in abilities, tasks, and competencies

Although job analysis is a widely used organizational data collection technique, little research has investigated the extent to which job analysis information is affected by self-presentation processes. This study represents the first direct test of the propositions offered by F. P. Morgeson and M. A. Campion (1997) concerning self-presentation in job analysis measurement. Using an experimental design, the authors examined job incumbent response differences across ability, task, and competency statements. Results indicated that ability statements were more subject to inflation than were task statements across all rating scales. Greater endorsement of nonessential ability statements was responsible for the differences. This produced higher endorsement of ability items but lower mean ratings. Finally, frequency and importance ratings of global competency statements were generally higher than decomposed ability and task scales, but required-at-entry judgments demonstrated the opposite relationship.     

EVALUATION OF TASK AND JOB SKILL LINKAGE JUDGMENTS USED TO DEVELOP TEST SPECIFICATIONS

The research on job analysis judgments, such as "time spent," has been relatively limited, particularly with reference to external criteria remote from the job analysis operation. The more complex job analysis judgment linking a job skill to specific tasks or duties has not been systematically examined. While it would appear that a simple scaling of importance of a skill for a task or duty or a retranslation judgment would suffice, the fact is that a single job skill may be a prerequisite for performance in a variety of tasks, and any one task may require multiple skills of varying levels for effective performance. With a multiple assignment of tasks to job skills, the evaluation becomes considerably more difficult. In the present study, a sequence of statistical evaluations was conducted to examine, first, the reliability of the subject matter expert (SME) panel's association of tasks and job skills and, second, the factor structure of the task by job skill relationship. The results are discussed with reference to developing selection test specifications and test budgets.     

A COMPARISON OF GLOBAL JUDGMENT VS. TASK ORIENTED APPROACHES TO JOB CLASSIFICATION

The purpose of this study was to determine the extent to which direct judgments of similarity by supervisors and incumbents could provide the same job classification results as a more elaborate job analysis procedure involving measures of task overlap among jobs. To accomplish this, 8 foreman jobs in a chemical processing plant were analyzed and compared on 237 task statements. In addition, 15 foremen incumbents and 17 supervisors evaluated the similarities among the same 8 foremen jobs in a paired comparisons rating task. The task-oriented job analysis required hundreds of man-hours to complete; the rating task took 15 minutes. Results using hierarchical cluster analysis and multidimensional scaling analysis revealed that the global judgments and the task-oriented data led to identical conclusions. Also, it was found that incumbent ratings produced the same results as ratings from supervisors. Uses, advantages, and disadvantages of the procedure are outlined.     

The Relationship Between SME Job Experience and Job Analysis Ratings: Findings with and without Statistical Control

This investigation examined the relationship between SME job experience and job analysis ratings. Managerial job incumbents (N = 209) provided job analysis ratings which measured 9 task frequency and 6 skill importance components. Job experience was operationalized as both job tenure and residualized job tenure (i.e., statistically-controlled for SME characteristics correlated with job tenure). Respondents were categorized into low, medium, and high groups for each tenure measure. MANOVA results indicated that task ratings differed across residualized tenure groups, while skill ratings did not. Statistical control of the job tenure measure made a substantial difference in results. For the non-residualized tenure groups, findings were reversed: skill ratings differed across non-residualized tenure groups, while task ratings did not. The results of this research are integrated with the existing literature and implications for future research and practice are offered.     

Political Skill and Work Outcomes: A Theoretical Extension,Meta‐Analytic Investigation,and Agenda for the Future

This quantitative review explored the political skill construct and its predictive ability across a number of organizational outcomes. First, we extended the Ferris et al. meta‐theoretical framework of political skill. Next, incorporating meta‐analysis, we found political skill is positively related to self‐efficacy, job satisfaction, organizational commitment, work productivity, organizational citizenship behavior (OCB), career success, and personal reputation, and negatively related to physiological strain. Political skill was not significantly related to psychological strain or perceptions of organizational politics. Using meta‐regression and dominance analyses, political skill predicted task performance after controlling for the Big Five personality characteristics and general mental ability. In a test of indirect relationships, our results suggest that personal reputation and self‐efficacy partially mediate the political skill–task performance relationship. Finally, in a post hoc test of political skill dimensions, we found that networking ability, interpersonal influence, and apparent sincerity (but not social astuteness) predicted task performance. Our findings provide a comprehensive assessment of theory and research to date on political skill and extend theoretical foundations to stimulate new inquiry into the operation of this important construct.     

An Alternate Method for Content-Oriented Test Construction: An Empirical Evaluation

This study empirically evaluates the relationships between rationally and empirically estimated item indices used in the development of a test of land navigation knowledge for the US Marine Corps. Three land navigation instructors provided ratings on three item indices (item content validity ratios [CVRs], difficulty, and discrimination). These ratings were correlated with empirically derived item indices (difficulty, discrimination, and item-criterion correlations) obtained from the scores of 359 Marines on measures of land navigation knowledge, skill, and performance. Contrary to previous research, item CVRs were significantly correlated with all three empirically determined item indices. The increased effectiveness of rational item indices obtained in the present study over those from previous research appears to be related to the following areas: qualifications of expert raters, match of raters to the rating task, and the nature of the rating task. The implications for content-oriented test design and for judgmental test validation strategies are discussed.     

Deterioration of planning ability with age in Japanese monkeys (Macaca fuscata)

To investigate the ability of aged monkeys to plan and the effect of aging on this ability, performance in a food retrieval task was assessed in aged and younger Japanese monkeys (Macaca fuscata). In this task, the monkeys had to retrieve food items by selecting from a set of 9 holes, each of which contained 1 food item. Results showed that task performance declined significantly with age. All monkeys showed, to a greater or lesser extent, some consistent patterns in their sequence of selecting holes for retrieving the food item. An analysis of these selection patterns indicated that the younger monkeys showed more consistent sequences in selection than the aged monkeys. Furthermore, success in the task performance correlated strongly with higher consistency in the sequence of selecting holes. The authors simulated performance for this task by monkeys without any strategies or plans. The results suggest that the empirical data were far more systematic than the simulated data. Thus, the authors conclude that Japanese monkeys have the ability to plan and that this ability to plan deteriorates with age.     

The mutual effects between job resources and mental health: a prospective study among Dutch youth

The current study construes the relation between job resources (such as job autonomy, skill utilization, and variety) and mental health as a reciprocal process. Following conservation of resources theory (S. E. Hobfoll, 1989), the author expected job resources to have a positive effect on mental health. Poor mental health was assumed to impede workers' attempts to maintain or enhance their job resources either by changing the content of their jobs or attempting to find another (less distressing) job. Longitudinal data from a representative sample of 699 Dutch youth were analyzed with analysis of variance and covariance structure modeling. The results provided good support for the proposition that job resources and mental health mutually influence each other. Limitations and implications of the study are discussed.     

Linking Job Characteristics and Mental Health among Middle Level Indian Managers: Testing the Mediating Role of Psychological Empowerment

The present study examined the relationship between job characteristics and mental health by testing the mediating role of psychological empowerment. Data were obtained on a sample of middle level Indian managers (n=290) from two manufacturing organizations of North India. Results of the hierarchical regression analyses revealed that all the core job characteristics significantly positively predicted psychological empowerment and mental health after controlling for the effects of demographic variables. The results also showed that psychological empowerment significantly positively predicted mental health over and above the demographic variables. The mediated regression analyses procedure developed by Baron and Kenny (Journal of Personality and Social Psychology 51:1173–1182, 1986) was used to test the mediating effect of psychological empowerment. Results of mediated regression analyses indicated that psychological empowerment significantly mediated the relationship between overall job characteristics and mental health. Further it partially mediated the relationship between core job characteristics (skill variety, task identity, task significance, feedback from work) and mental health. In addition the Sobel test for mediation confirmed the statistical significance of the mediated relationships. The implications of the findings are discussed in order to confirm the importance of core job characteristics to empower employees.     

任职者任务绩效水平对其工作分析结果的影响

在控制了个体人口统计学变量和排除组织水平变量的影响下,探讨了电厂设计和编辑两个职位任职者的任务绩效水平对于他们Occupational Information Network (以下缩写为O*NET)工作分析问卷评价结果的影响。层次回归结果发现,控制了相关因素后,发电厂设计人员的任务绩效水平能显著影响其对技术性技能的水平评价,编辑的任务绩效水平能显著影响信息处理的重要性评价和水平评价,在一定程度上证实了“工作分析结果差异来源于真实差异”的理论     

How generation affects source memory

Generation effects (better memory for self-produced items than for provided items) typically occur in item memory. Jurica and Shimamura (1999) reported a negative generation effect in source memory, but their procedure did not test participants on the items they had generated. In Experiment 1, participants answered questions and read statements made by a face on a computer screen. The target word was unscrambled, or letters were filled in. Generation effects were found for target recall and source recognition (which person did which task). Experiment 2 extended these findings to a condition in which the external sources were two different faces. Generation had a positive effect on source memory, supporting an overlap in the underlying mechanisms of item and source memory.     

Managerial Work, Job Analysis, and Holland's RIASEC Vocational Environment Dimensions

We examined relations between the environmental dimensions underlying Holland's theory of vocational choice and skill requirements, context characteristics, and task frequency ratings for managerial jobs. The Holland environmental constructs were measured by the recently developedPosition Classification Inventory(PCI). The task, skill requirement, and context variables were measured using traditional job analysis surveys. Ten judges provided estimates of the expected correlations between the job analysis variables and the Holland constructs. The profile of observed correlations was generally consistent with the judges’ expectations based on Holland's theory, providing support for both that framework and the construct validity of the PCI. The one Holland dimension for which the data were least consistent with predictions was “Realistic.” Results provide a detailed picture of the work content, skills, and context variables within managerial work that are likely to be associated with the RIASEC dimensions. Implications for management development are discussed.     

JOB CLASSIFICATION APPROACHES AND THE IMPLEMENTATION OF VALIDITY GENERALIZATION RESULTS

This paper compares two job classification methods for showing the appropriateness of cognitive tests in settings that were not involved in supplying data for a validity generalization analysis. One approach was an elaborate quantitative procedure that involved a lengthy job inventory and a multivariate item analysis. This approach was shown to be highly successful when applied to the responses from 1179 job inventories collected in 54 petroleum-petrochemical plants from 30 different companies. The other procedure involved simple job classification judgments by supervisors and incumbents. This latter approach was shown to be as effective, but was much less time consuming and costly. Professional and legal implications of these findings are discussed.     

ITEM BIAS INDICES BASED ON TOTAL TEST SCORE AND JOB PERFORMANCE ESTIMATES OF ABILITY

Researchers (e.g., Ironson, 1982; Tenopyr, 1990) have suggested that item bias investigators equate subgroups on external criteria such as job performance rather than total test scores before considering subgroup passing rates on test items. In a study comparing these two approaches to studies of item bias, we found little evidence of bias using total test score as the estimate of overall examinee ability, but nearly all items were biased in comparisons of white and African-American subgroups on Numerical, Verbal, and Mechanical Reasoning tests and in male-female comparisons on a Mechanical Reasoning test when job performance was used to select "equally able" examinees. However, the use of job performance as the ability index is analogous to performance-based approaches to test bias (Hartigan & Wigdor, 1989; Thorndike, 1971) and directly equivalent to the Darlington (1971) and Cole (1973) test bias definition, the logical inconsistencies of which have been previously described (Hunter & Schmidt, 1976; Peterson & Novick, 1976). We conclude that performance matching as a basis of forming "equal ability" groups is inappropriate.     

Technology and the structuring of jobs: Employee satisfaction,performance, and influence

This research investigated the relationships between technology, interdependence, job characteristics, and employee satisfaction, performance, and influence. Technology was operationalized at the individual level of analysis to include the dimensions of input uncertainty, conversion uncertainty, and output uncertainty. Pooled, sequential, and reciprocal interdependence was assessed. The results indicated that input and conversion uncertainty and interdependence were strongly related to such job characteristics as autonomy, skill variety, task identity, task significance, and task feedback. While the job characteristics related positively to employee satisfaction, input and conversion uncertainty related negatively to satisfaction, thus creating a mutual suppression effect. Although the uncertainty dimensions did not relate significantly to performance, there is some evidence that this negative relationship was being suppressed by the positive relationship between job characteristics and performance. Both the job characteristics and the technology dimensions related positively to influence. Results were discussed in terms of appropriate matches between technology and job characteristics.     

The revelation effect: A meta-analytic test of hypotheses

Judgments can depend on the activity directly preceding them. An example is the revelation effect whereby participants are more likely to claim that a stimulus is familiar after a preceding task, such as solving an anagram, than without a preceding task. We test conflicting predictions of four revelation-effect hypotheses in a meta-analysis of 26 years of revelation-effect research. The hypotheses’ predictions refer to three subject areas: (1) the basis of judgments that are subject to the revelation effect (recollection vs. familiarity vs. fluency), (2) the degree of similarity between the task and test item, and (3) the difficulty of the preceding task. We use a hierarchical multivariate meta-analysis to account for dependent effect sizes and variance in experimental procedures. We test the revelation-effect hypotheses with a model selection procedure, where each model corresponds to a prediction of a revelation-effect hypothesis. We further quantify the amount of evidence for one model compared to another with Bayes factors. The results of this analysis suggest that none of the extant revelation-effect hypotheses can fully account for the data. The general vagueness of revelation-effect hypotheses and the scarcity of data were the major limiting factors in our analyses, emphasizing the need for formalized theories and further research into the puzzling revelation effect.