Personality-Based Job Analysis and The Self-Serving Bias

This study tested the hypothesis that personality-based job analysis (PBJA) ratings are correlated with subject matter expert (SME) personality scale scores (through a self-serving bias). Some support was found for the hypothesis. However, PBJA ratings were somewhat predictive of the criterion-related validity of scale scores for the big five personality dimensions.     

Supervisory opportunity to observe moderates criterion‐related validity estimates

Supervisors' opportunity to observe incumbents' performance (i.e., how often a supervisor typically sees an employee's performance) has been suggested to be important for accurate performance rating and to be a moderator of criterion‐related validity. Here we test these suggestions and present empirical evidence for the effects of opportunity to observe. In Study 1, supervisors in a multi‐occupation/organization criterion‐related validation study for a biodata measure indicated the opportunity they had to observe incumbents. The data were split according to different levels of opportunity to observe. Higher validities were found when opportunity to observe was maximal. In Study 2, this finding was replicated using a cognitive ability test. These results suggest that psychologists should consider measuring opportunity to observe in criterion‐related validity studies.     

Generalization of Cognitive and Noncognitive Validities across Personality‐based Job Families

The positive relationship between complexity of work and the validity of general mental ability (GMA) measures across a variety of occupations is well supported by research and provides important practical and theoretical support for cognitive ability measures. However, there is currently no research demonstrating a systematic relationship between the size of the validities of any personality measure and the personality requirements of jobs, thus leaving open to question the predictive and construct validity of personality measures for applicant selection. We compared the validities of two biodata measures – one scored to measure social competence and one to measure GMA – across six job families that varied in social requirements. The validity of the GMA measure generalized at approximately the same level across the job families while the validity of the social competence measure decreased as social requirements diminished.     

Scoring Biodata: Is it rational to be quasi‐rational?

Five quasi‐rational biodata scales were developed by empirically keying biodata items to predict scores on a measure of the Big Five dimensions of personality. The criterion‐related validities of the quasi‐rational scales were compared to empirical and rational biodata keying methods using supervisory ratings of job performance as the criterion. Empirical keying outperformed the quasi‐rational and rational methods (which had similar validities).     

Self-serving bias effects on job analysis ratings

The purpose of this study was to investigate whether worker-oriented job analysis importance ratings were influenced by subject matter experts' (SME) standing (as measured by self-rated performance) on a competency. This type of relationship (whereby SMEs indicate that the traits they have are important for successful job performance) is an example of the self-serving bias (which is widely described in the social cognition literature and rarely described in the industrial/organizational psychology literature). An archival dataset covering 57 clerical and technical occupations with 26,682 participants was used. Support was found for the relationship between self-rated performance and importance ratings. Significant relationships (typically in the .30s) were observed for all 31 competencies that were studied. Controls were taken to account for common method bias and differences in the competencies required for each of the 57 occupations. Past research has demonstrated the effects of the self-serving bias on personality-based job analysis ratings. This study was the first to extend these findings to traditional job analysis, which covers other competencies in addition to personality. In addition, this study is the first to use operational field data instead of laboratory data.     

Nonlinear Personality–Performance Relationships and the Spurious Moderating Effects of Traitedness

Abstract This article evaluated the validity of two traditional traitedness measures (i.e., standard deviation traitedness measures and global, single‐item traitedness measures) and three new traitedness measures (i.e., multi‐item, ranking, and facet traitedness measures). Overall, there was poor construct validity for both the traditional and new measures of traitedness. Furthermore, traitedness was a spurious moderator of the relationship between openness to experience and academic performance. Finally, evidence of nonlinear personality‐academic performance relationships was discovered.     

Video‐based Testing: A high‐fidelity job simulation that demonstrates reliability,validity, and utility

A conceptual rationale is presented for the use of a high‐fidelity simulation: a video‐based test (VBT), in which applicants view job‐related scenarios, respond orally, and are later scored on five dimensions by trained raters. Using applicant data, we empirically investigate the criterion‐related validity, reliability, and utility of the VBT. Overall, we found that the VBT predicted performance in training and on the job. In addition, different raters strongly agreed with one another on each applicant's scores. Despite somewhat higher development costs, the VBT demonstrated high utility. We suggest that organizations consider the use of VBTs as a selection instrument for jobs that require the use of applied social and interpersonal skills.