Supervisors as SMEs: The Relationship Between Supervisor Job Experience and Ratings of Subordinate Skill Importance

This study investigated the relationship between supervisor job experience and ratings of the importance of eight skill components for performing subordinates' jobs. When examining these correlations, statistical control was utilized to ensure that supervisor and subordinate characteristics confounded with supervisor job experience were not affecting hypothesized relationships. As predicted, there were statistically significant positive correlations between supervisor job experience and ratings on 6 of the 8 components. Also as predicted, controlling for the characteristics confounded with supervisor job experience made a substantial difference in results; 13 of the 16 correlations using statistical control significantly differed from correlations that did not. This illustrates that future research on SME characteristics should consider controlling variables confounded with the focal characteristic(s) under study in order to more thoroughly understand characteristic–rating relationships. Implications of these findings for research and practice are discussed and suggestions for future research are offered.     

任职者任务绩效水平对其工作分析结果的影响

在控制了个体人口统计学变量和排除组织水平变量的影响下,探讨了电厂设计和编辑两个职位任职者的任务绩效水平对于他们Occupational Information Network (以下缩写为O*NET)工作分析问卷评价结果的影响。层次回归结果发现,控制了相关因素后,发电厂设计人员的任务绩效水平能显著影响其对技术性技能的水平评价,编辑的任务绩效水平能显著影响信息处理的重要性评价和水平评价,在一定程度上证实了“工作分析结果差异来源于真实差异”的理论     

工作满意度、情感承诺和工作投入对工作技能评价结果的影响

通过对从事人力资源管理工作的公务员、软件工程师、网络编辑和报纸广告销售人员四个职业的272名任职者调查数据的层次回归分析,文章发现控制了职业和人口统计学变量的影响之后,工作满意度、情感承诺和工作投入三个工作态度变量对工作技能的重要性和水平评价有显著影响。进一步对比发现,工作满意度对于技能的重要性和水平评价的影响效应较大。该研究对多个职业的分析结果拓展了人们对工作分析结果影响因素的认识,并对未来工作分析研究和实践有重要的启示作用     

JOB ANALYSIS: THE COMPOSITION OF SME SAMPLES

It is common for job analysts to solicit information from incumbents and supervisors (Subject Matter Experts or SMEs) when conducting a job analysis. These SMEs are asked to provide ratings on salient dimensions (e.g., frequency and importance of tasks that comprise the job). In constructing samples of SMEs for this purpose, it is reasonable to consider any possible influences that might bias or systematically influence the task ratings. The present paper considers the possible influence of SME demographic characteristics on task ratings of frequency. The tasks comprising the job of patrol officer in a large city were rated for frequency by approximately 700 incumbents. These ratings were gathered in two different years, 1982 and 1984. The total data set was used to conduct a components analysis of the 444-item task inventory. The first eight principal components were considered the dependent variables and four demographic characteristics the independent variables in an analysis of the 1982 data set. Analysis of variance and follow-up tests indicated that incumbent experience had a substantial influence on task ratings. Educational level and race had minimal effects on ratings. The sex factor was confounded by the experience factor, making interpretation of the sex effect equivocal. Mechanisms that might account for the experience effect are discussed. It was concluded that incumbent experience is a salient issue in job analysis using SME groups.     

Does longer job tenure help or hinder job performance?

There are two competing theoretical perspectives on how job tenure might affect job performance. Human capital theory suggests that as knowledge and skill increase with greater tenure, job performance will improve as well. In contrast, the literature on job design suggests that as job tenure increases, employees are likely to become more bored and less motivated at work. Consequently, the gains from human capital acquisition might be offset by losses of motivation. To examine these competing perspectives, we conducted meta-analyses on the relationships of job tenure with four types of job performance: core task performance, citizenship behavior, creativity and innovative behavior, and counterproductive work behavior. The results support the second perspective, as both the linear and curvilinear relationships of job tenure with the four types of job performance are weak. Further, the results do not change in strength across type of research design, job industry, age, or gender.     

Does longer job tenure help or hinder job performance?

There are two competing theoretical perspectives on how job tenure might affect job performance. Human capital theory suggests that as knowledge and skill increase with greater tenure, job performance will improve as well. In contrast, the literature on job design suggests that as job tenure increases, employees are likely to become more bored and less motivated at work. Consequently, the gains from human capital acquisition might be offset by losses of motivation. To examine these competing perspectives, we conducted meta-analyses on the relationships of job tenure with four types of job performance: core task performance, citizenship behavior, creativity and innovative behavior, and counterproductive work behavior. The results support the second perspective, as both the linear and curvilinear relationships of job tenure with the four types of job performance are weak. Further, the results do not change in strength across type of research design, job industry, age, or gender.     

Managerial Work, Job Analysis, and Holland's RIASEC Vocational Environment Dimensions

We examined relations between the environmental dimensions underlying Holland's theory of vocational choice and skill requirements, context characteristics, and task frequency ratings for managerial jobs. The Holland environmental constructs were measured by the recently developedPosition Classification Inventory(PCI). The task, skill requirement, and context variables were measured using traditional job analysis surveys. Ten judges provided estimates of the expected correlations between the job analysis variables and the Holland constructs. The profile of observed correlations was generally consistent with the judges’ expectations based on Holland's theory, providing support for both that framework and the construct validity of the PCI. The one Holland dimension for which the data were least consistent with predictions was “Realistic.” Results provide a detailed picture of the work content, skills, and context variables within managerial work that are likely to be associated with the RIASEC dimensions. Implications for management development are discussed.     

The Impact of Gender on the Review of the Curricula Vitae of Job Applicants and Tenure Candidates: A National Empirical Study

The purpose of this study was to determine someof the factors that influence outside reviewers andsearch committee members when they are reviewingcurricula vitae, particularly with respect to the gender of the name on the vitae. The participants inthis study were 238 male and female academicpsychologists who listed a university address in the1997 Directory of the American PsychologicalAssociation. They were each sent one of four versions of acurriculum vitae (i.e., female job applicant, male jobapplicant, female tenure candidate, and male tenurecandidate), along with a questionnaire and aself-addressed stamped envelope. All the curricula vitaeactually came from a real-life scientist at twodifferent stages in her career, but the names werechanged to traditional male and female names. Althoughan exclusively between-groups design was used to avoidsparking genderconscious responding, the resultsindicate that the participants were clearly able todistinguish between the qualifications of the jobapplicants versus the tenure candidates, as evidenced bysuggesting higher starting salaries, increasedlikelihood of offering the tenure candidates a job,granting them tenure, and greater respect for theirteaching, research, and service records. Both men andwomen were more likely to vote to hire a male jobapplicant than a female job applicant with an identicalrecord. Similarly, both sexes reported that the male job applicant had done adequate teaching,research, and service experience compared to the femalejob applicant with an identical record. In contrast,when men and women examined the highly competitive curriculum vitae of the real-life scientist whohad gotten early tenure, they were equally likely totenure the male and female tenure candidates and therewas no difference in their ratings of their teaching, research, and service experience. There was nosignificant main effect for the quality of theinstitution or professional rank on selectivity inhiring and tenuring decisions. The results of this study indicate a gender bias for both men and womenin preference for male job applicants.     

Employees' Political Skill and Job Performance: An Inverted U‐Shaped Relation?

During the past decade, the construct of political skill has attracted a lot of attention. In particular, its relation to job performance has been examined. With regard to this link, it is typically proposed that political skill affects job performance in a positive linear manner. However, in this article it is suggested that intermediate levels of employees' political skill yield the highest job performance, implying that this association is in fact represented by an inverted U‐shape. Findings from two field studies (N₁ = 178, N₂ = 115 employee‐supervisor‐colleague triads) that incorporated different sample characteristics (early career employees, established employees), job performance dimensions (overall, task, contextual, and adaptive performance), and rating sources (supervisors and colleagues) supported this idea. Across different analyses, employees with intermediate levels of political skill received higher job performance ratings compared to those with lower and higher levels, respectively. In addition, the nature of the relationships between employees and their raters was found to moderate this curvilinear effect. Specifically, besides the fact that employees who had close working relationships with their raters generally received higher job performance ratings, the decreases in the job performance ratings of employees high in political skill were less pronounced when they had close relationships.     

Methodology for Identifying Abilities for Job Classification: An Application of Job Analysis

There has been a resurgence of interest in using cognitive and personality factors to improve prediction of job performance. Small gains in prediction efficiency can have enormous benefits for organizations that make many employee placements. It is prohibitively expensive to conduct empirical validity studies for every newly proposed factor for every job. A cost-effective alternative is the method of expert judgments in which subject matter experts (SMEs) rate the importance of various abilities for task components. This rational specification technique was applied to four Air Force enlisted and four officer occupations using the survey mail-out method to obtain SME ratings of the importance of various abilities for successful job performance. Tasks (e.g., calculate serial dilutions, evacuate refrigerant subsystems) were grouped by verbs to define activity modules. Reliable and reasonable ratings of the importance of abilities were obtained by the survey mail-out method.     

SME Committee vs. Field Job Analysis Ratings: Convergence, Cautions, and a Call

This investigation examined the agreement between job analysis ratings completed by a small committee of subject matter experts (SMEs) and ratings made by a larger sample of field respondents. The results indicated there were very high levels of agreement across six commonly used job analysis rating indices. These findings converge with those in prior studies suggesting that for purposes of obtaining task and KSA ratings, a small committee of SMEs may provide empirical results that agree with those obtained from a larger sample of field respondents. However, cautions are presented for applying a committee only strategy in job analysis. Given convergence in conclusions across studies such as the current one thus far, a call for future research which addresses more specific issues is presented.     

EVALUATION OF TASK AND JOB SKILL LINKAGE JUDGMENTS USED TO DEVELOP TEST SPECIFICATIONS

The research on job analysis judgments, such as "time spent," has been relatively limited, particularly with reference to external criteria remote from the job analysis operation. The more complex job analysis judgment linking a job skill to specific tasks or duties has not been systematically examined. While it would appear that a simple scaling of importance of a skill for a task or duty or a retranslation judgment would suffice, the fact is that a single job skill may be a prerequisite for performance in a variety of tasks, and any one task may require multiple skills of varying levels for effective performance. With a multiple assignment of tasks to job skills, the evaluation becomes considerably more difficult. In the present study, a sequence of statistical evaluations was conducted to examine, first, the reliability of the subject matter expert (SME) panel's association of tasks and job skills and, second, the factor structure of the task by job skill relationship. The results are discussed with reference to developing selection test specifications and test budgets.     

A Multi‐Source,Multi‐Study Investigation of Job Performance Prediction by Political Skill

Political skill is a social effectiveness construct with a demonstrated capacity to predict job performance. However, because performance prediction research in this area to date has made exclusive use of self‐reports of political skill, and due to frequent distrust of self‐ratings of constructs in important personnel decisions, there is a need to investigate how multiple alternative sources of political skill and job performance measures relate, thus raising both theoretical and methodological issues. In three studies, employing a triadic data collection methodology, and utilising both cross‐sectional and longitudinal designs, this research tested the hypotheses that employee political skill, measured from the perspective of employees' assessor A, will positively predict job performance rated by assessor B (i.e. Hypothesis 1a), and vice versa, that employee political skill measured by assessor B will predict job performance ratings measured by assessor A (i.e. Hypothesis 1b).     

Fit of Political Skill to the Work Context: A Two‐Study Investigation

A two‐study investigation was designed to examine the role of job type (i.e. fit of political skill to work context) as a contextual moderator of the political skill–job performance relationship. Specifically, it was hypothesised that political skill operates most effectively in enterprising job contexts, and thus is most predictive of job performance in such contexts, but political skill would demonstrate no relationship with job performance in job contexts that did not emphasise interpersonal interaction and effectiveness. In Study 1, enterprising job demands interacted with political skill to affect job performance. That is, political skill positively and significantly predicted job performance in enterprising job contexts, as hypothesised. Study 2 selected one specific job context (i.e. insurance sales) high in enterprising job demands, and hypothesised that political skill would significantly predict objective measures of insurance sales (i.e. sales volume, performance‐based income, performance‐based commission rate, and performance‐based status). The results demonstrated significant predictive effects of political skill (i.e. beyond age, sex, education level, tenure on the job, and experience in sales) on all four measures of sales performance. Contributions and implications of this research, strengths and limitations, and directions for future study are discussed.     

Progress Toward Understanding the Structure and Determinants of Job Performance: A focus on task and citizenship performance

Job performance criterion models have become more popular and more often investigated. The current research adds to this literature by testing performance models focusing on ability and personality predictors of task and citizenship job performance. Air traffic controllers (N = 928) completed cognitive ability and personality predictor measures, were administered a performance test assessing two possible mediators, and were evaluated using supervisor and peer ratings of two task performance, one citizenship performance, and an overall performance dimension. Path analyses largely confirmed that ability predicts primarily task performance and personality predicts primarily citizenship performance. The mediation related to task knowledge and skill on the task performance side was confirmed, and the overall fit of the performance models was quite good. Discussion focuses on how the present research furthers progress toward mapping relations among individual differences and individual performance constructs, beyond using overall performance as the endogenous performance variable.     

Self‐Ratings of Political Skill in Job Application: A within‐ and between‐subjects field experiment

Political skill is the ability to understand others and use that knowledge to influence others to act in ways that enhance one's personal and/or organizational objectives. The Political Skill Inventory ( Ferris et al., 2005 ) assesses this talent, and has potential for use in personnel decision making. However, central organizational stakeholders are concerned about the distortion of self‐rating scores in job application; consequently, we examined the effects of a job application situation on self‐ratings of political skill in a field experiment with 205 job incumbents. The findings showed consistently that the relationship of self‐ratings of political skill and job performance ratings by supervisors (ρ=.30, p<.01) were not distorted in job application. Implications and limitations are discussed.     

Personality-Based Job Analysis and The Self-Serving Bias

This study tested the hypothesis that personality-based job analysis (PBJA) ratings are correlated with subject matter expert (SME) personality scale scores (through a self-serving bias). Some support was found for the hypothesis. However, PBJA ratings were somewhat predictive of the criterion-related validity of scale scores for the big five personality dimensions.     

Managerial Experience and the Measurement Equivalence of Performance Ratings

ABSTRACT: Establishing the measurement equivalence of instruments is a prerequisite to making meaningful comparisons between individuals or within individuals over time. Whereas previous research has investigated the effects of rater characteristics on the measurement equivalence of performance ratings, the current study investigated a ratee characteristic—ratee job experience. Using confirmatory factor analysis and item response theory methods with replication, the measurement equivalence of supervisor ratings of 7,200 managers with differing levels of managerial experience was assessed. Overall, results indicated a high degree of measurement equivalence suggesting that meaningful comparison may be made across ratees with different levels of job experience.     

Assessing Training Needs: Do Work Experience and Capability Matter?

Despite the crucial nature of needs assessment in training design and development, very little empirical work examining factors that influence such ratings has been undertaken. We investigated the impact of individuals' levels of work experience, self-efficacy, and skill proficiency on their subsequent ratings of training needs. Our results indicate that self-efficacy and skill proficiency are positively related to importance and frequency ratings for both skills and job tasks. However, when considered collectively, skill proficiency was a more potent influence, suggesting that an individual's actual capability may have a greater impact on ratings of training needs than his or her perceived capability (i.e., self-efficacy). Implications for future research and training needs assessment practice are discussed.