The stability of validity coefficients over time: Ackerman's (1988) model and the general aptitude test battery

This study examined P. L. Ackerman's (1988) model of skill acquisition within an applied setting. Differences were examined between jobs on the basis of task consistency: changes in performance variability across experience, learning curves, and stability of ability-performance correlations across experience. Results showed the degree of task consistency influenced the shape of learning curves, with jobs composed of primarily consistent tasks improving more rapidly and reaching asymptote sooner. In addition, trends in ability-performance correlations were moderated by the degree of task consistency within ajob. Specifically, forjobs with primarily consistent tasks, general cognitive ability best predicted early performance whereas psychomotor ability best predicted later performance. In contrast, general cognitive ability was the strongest predictor across experience for jobs with primarily inconsistent tasks.     

Predictors of Managerial Performance: More Than Cognitive Ability

This study investigated the discriminant validity of cognitive ability, managerial potential, fear of negative evaluation, interaction anxiousness, and audience anxiousness as predictors of managerial performance. As hypothesized cognitive ability significantly predicted only thinking and knowledge criteria. However, managerial potential was significantly correlated with all the criteria. As predicted, fear of negative evaluation predicted thinking and knowledge criteria, interaction anxiousness, and audience anxiousness. In turn, as predicted, interaction anxiousness significantly predicted interpersonal behavior and audience anxiousness significantly predicted communication behavior. Managerial potential and fear of negative evaluation were the only two predictors that were significantly correlated with overall performance.     

Predicting individual differences in complex skill acquisition: dynamics of ability determinants.

Substantial controversy exists about ability determinants of individual differences in performance during and subsequent to skill acquisition. This investigation addresses the controversy. An information-processing examination of ability-performance relations during complex task acquisition is described. Included are ability testing (including general, reasoning, spatial, perceptual speed, and perceptual/psychomotor abilities) and skill acquisition over practice on the terminal radar approach controller simulation. Results validate and extend Ackerman's (1988) theory of cognitive ability determinants of individual differences in skill acquisition. Benefits of ability component and task component analyses over global analyses of ability-skill relations are demonstrated. Implications are discussed for selection instruments to predict air traffic controller success and for other tasks with inconsistent information-processing demands.     

Aptitude is not enough: How personality and behavior predict academic performance

The study investigated the incremental validity of Big Five personality traits for predicting academic criteria (college GPA, course performance) while controlling for academic ability (SAT). Results showed that conscientiousness incrementally predicted each criterion over SAT. Results also showed that behavior (attendance) incrementally predicted GPA and course performance and it mediated the relationship between conscientiousness and both academic criteria. Personality measures are promising predictors of academic outcomes and they may have usefulness in admissions and student development.     

Effectively Coping With Task Stress: A Study of the Validity of the Trait Emotional Intelligence Questionnaire–Short Form (TEIQue–SF)

In this study, we investigated the validity of the Trait Emotional Intelligence Questionnaire–Short Form (TEIQue–SF; Petrides, 2009) in the context of task-induced stress. We used a total sample of 225 volunteers to investigate (a) the incremental validity of the TEIQue–SF over other predictors of coping with task-induced stress, and (b) the construct validity of the TEIQue–SF by examining the mechanisms via which scores from the TEIQue–SF predict coping outcomes. Results demonstrated that the TEIQue–SF possessed incremental validity over the Big Five personality traits in the prediction of emotion-focused coping. Results also provided support for the construct validity of the TEIQue–SF by demonstrating that this measure predicted adaptive coping via emotion-focused channels. Specifically, results showed that, following a task stressor, the TEIQue–SF predicted low negative affect and high task performance via high levels of emotion-focused coping. Consistent with the purported theoretical nature of the trait emotional intelligence (EI) construct, trait EI as assessed by the TEIQue–SF primarily enhances affect and performance in stressful situations by regulating negative emotions.     

Predictive Criterion-Related Validity of Observer Ratings of Personality and Job-Related Competencies Using Multiple Raters and Multiple Performance Criteria

The purpose of this study was to assess the predictive validity of observer ratings of personality and job-related competencies in a selection setting. Based on ratings from multiple raters of both the predictors and the criteria in a sample of MBA students, results indicated that observer ratings of Conscientiousness, Emotional Stability, leadership, and interpersonal skills predicted work performance, team performance, and academic performance. For work performance and team performance, a composite of the four predictors had incremental predictive validity over general mental ability, even after controlling for how well the rater knew the ratee.     

Teacher-student relationship quality type in elementary grades: Effects on trajectories for achievement and engagement

Teacher, peer, and student reports of the quality of the teacher-student relationship were obtained for an ethnically diverse and academically at-risk sample of 706 second- and third-grade students. Cluster analysis identified four types of relationships based on the consistency of child reports of support and conflict in the relationship with reports of others: Congruent Positive, Congruent Negative, Incongruent Child Negative, and Incongruent Child Positive. The cluster solution evidenced good internal consistency and construct validity. Group membership predicted growth trajectories for teacher-rated engagement and standardized achievement scores over the following three years, above prior performance. The predictive associations between child reports of teacher support and conflict and the measured outcomes depended on whether child reports were consistent or inconsistent with reports of others. Study findings have implications for theory development, assessment of teacher-student relationships, and teacher professional development.     

The five-factor model of personality traits and organizational citizenship behaviors: a meta-analysis

Using meta-analytic tests based on 87 statistically independent samples, we investigated the relationships between the five-factor model (FFM) of personality traits and organizational citizenship behaviors in both the aggregate and specific forms, including individual-directed, organization-directed, and change-oriented citizenship. We found that Emotional Stability, Extraversion, and Openness/Intellect have incremental validity for citizenship over and above Conscientiousness and Agreeableness, 2 well-established FFM predictors of citizenship. In addition, FFM personality traits predict citizenship over and above job satisfaction. Finally, we compared the effect sizes obtained in the current meta-analysis with the comparable effect sizes predicting task performance from previous meta-analyses. As a result, we found that Conscientiousness, Emotional Stability, and Extraversion have similar magnitudes of relationships with citizenship and task performance, whereas Openness and Agreeableness have stronger relationships with citizenship than with task performance. This lends some support to the idea that personality traits are (slightly) more important determinants of citizenship than of task performance. We conclude with proposed directions for future research on the relationships between FFM personality traits and specific forms of citizenship, based on the current findings.     

A Review and Extension of Current Models of Dynamic Criteria

An important issue in personnel selection and test validation has been the nature of performance criteria and more specifically the existence of dynamic criteria. There is a continuing debate regarding the extent to which performance and validity coefficients remain stable over time. We examine research within work, laboratory, and academic settings and evaluate existing models of dynamic criteria. Building on previous models, we propose an integrative model of dynamic criteria that identifies important issues for ability and performance constructs and discusses how variables related to the task, job, and organization can affect the temporal stability of criterion performance and the ability-performance relationship.     

DYNAMIC CRITERIA REVISITED: A LONGITUDINAL STUDY OF PERFORMANCE STABILITY AND PREDICTIVE VALIDITY

The concept of dynamic criteria has been the subject of a recent debate regarding both the definition and prevalence of the phenomenon (Austin, Humphreys, & Hulin, 1989; Barrett & Alexander, 1989; Barrett, Caldwell, & Alexander, 1985). The present paper questions the adequacy of the conceptual framework underlying the debate and provides data supporting a refined concept of dynamic criteria. The incidence and possible causes of change in relative performance were investigated using weekly performance data from 509 sewing machine operators. Analyses were conducted to determine the degree of performance consistency, potential moderators of consistency, and the stability of predictor-criteria relationships using multiple predictors and criteria. Results revealed a steady decline in performance stability coefficients as the interval between measures increased. This decay was evident regardless of employees' prior job experience, cognitive ability, or psychomotor ability. Analyses of predictive validity coefficients revealed temporal changes in validity for both objective and subjective criteria, but not in the expected direction. The validity of cognitive ability increased, the validity of psychomotor ability was stable, and that of prior job experience decreased over time. Implications for theory and research are discussed.     

Validating Personnel Selection: A Process Model for Research and Practice

This article highlights the need for a clearer conceptual appreciation of the relationships among and between predictors and criteria, and for a better understanding of measurement issues in selection. A process model of selection is outlined that includes recent research on the changing relationships of ability to performance at different stages of skill acquisition, the interaction between type of task and validity, the role of motivational constructs in the measurement and validation process, and the reciprocally interacting influences of work experience and individual characteristics. Comment is offered on issues such as general versus specific abilities, the ‘big five’in personality, aptitude by treatment or training interactions and maximal versus typical performance. Measurement issues are discussed throughout, but particularly in relation to the assessment process.     

Validity of Six Openness Facets in Predicting Work Behaviors: A Meta-Analysis

To illustrate the importance of facet-level investigations in predicting organizational outcomes and the need for more primary studies on this topic, we conducted a meta-analysis that took an exploratory look at differential relationships among 7 organizational criteria and Openness traits varying in breadth. Nine Openness predictors—the global dimension, 2 aspects (intellect and culture), and 6 facets (intellectual efficiency, ingenuity, curiosity, aesthetics, tolerance, and depth)—were examined in relations with various organizational criteria such as traditional performance outcomes (task performance, contextual performance, counterproductive work behavior), turnover, leadership effectiveness, training performance, and adaptive performance. Our results support the idea that Openness facets could exhibit differential validity for many organizational outcomes.     

Candidates' Ability to Identify Criteria in Nontransparent Selection Procedures: Evidence from an assessment center and a structured interview

In selection procedures like assessment centers (ACs) and structured interviews, candidates are often not informed about the targeted criteria. Previous studies have shown that candidates' ability to identify these criteria (ATIC) is related to their performance in the respective selection procedure. However, past research has studied ATIC in only one selection procedure at a time, even though it has been assumed that ATIC is consistent across situations, which is a prerequisite for ATIC to contribute to selection procedures' criterion‐related validity. In this study, 95 candidates participated in an AC and a structured interview. ATIC scores showed cross‐situational consistency across the two procedures and accounted for part of the relationship between performance in the selection procedures. Furthermore, ATIC scores in one procedure predicted performance in the other procedure even after controlling for cognitive ability. Implications and directions for future research are discussed.     

IMPLICATIONS OF THE MULTIDIMENSIONAL NATURE OF JOB PERFORMANCE FOR THE VALIDITY OF SELECTION TESTS: MULTIVARIATE FRAMEWORKS FOR STUDYING TEST VALIDITY

Although most studies of criterion-related validity focus on univariate relationships, the complex and multidimensional nature of the performance construct and the widespread use of multiple selection devices argue in favor of multivariate frameworks for evaluating validity. Using a Monte Carlo simulation we estimated the validity of general cognitive ability tests and personality tests in predicting "job performance," where performance is conceptualized as a composite of multiple performance measures (i.e., individual job task performance and organizational citizenship behaviors). The validity of a selection battery varies substantially as a function of the relative weight given to both predictors and criteria; the 95% confidence interval for validities ranged from .20 to .78. The effective weights given to performance dimensions accounted for 34% of the variance in selection battery validities; depending on precisely how "performance" is defined, the same test battery can have relatively high or relatively low levels of validity. Our model suggests that the way an organization defines job performance is a source of true and important variability in validities, and that the validity of selection tests for predicting complex performance criteria may show considerably less generalizability that current metaanalysis of univariate validities would suggest.     

WE (SOMETIMES) KNOW NOT HOW WE FEEL: PREDICTING JOB PERFORMANCE WITH AN IMPLICIT MEASURE OF TRAIT AFFECTIVITY

In this study we examined relationships between trait affectivity and work performance. However, because trait affectivity is believed to operate primarily outside awareness, we assessed it using techniques designed to measure content at explicit  and  implicit levels. Although results were consistent across the explicit and implicit measures (i.e., positive affectivity was positively related to task performance and citizenship behavior, whereas negative affectivity was negatively related to task performance and positively related to counterproductive behavior), the implicit measure predicted greater proportions of variance in supervisor-rated criteria and did so incremental to the explicit measure. We discuss the implications of these results for theory and practice, and highlight the potential usefulness of implicit measures for applied research.     

The dark and the bright sides of proactive work behaviour and situational constraints: Longitudinal relationships with task conflicts

We investigated relationships of a job stressor (situational constraints) and specific proactive behaviours with change in task conflicts over time. We introduced two distinct types of proactive work behaviour (promotion-oriented initiative and prevention-oriented initiative) and examined their positive but also negative relationships with change in task conflicts. Study 1 supported the construct validity of promotion-oriented initiative and prevention-oriented initiative for 363 student assistants: Confirmatory factor analyses indicated that both types of proactive behaviour were distinct constructs and also distinct from active coping; patterns of correlations were similar to those of a prior concept of proactive behaviour. Study 2 was a longitudinal online survey with 197 employees over 3 weeks. Consistent with our hypotheses, situational constraints and promotion-oriented initiative predicted an increase in task conflicts, whereas prevention-oriented initiative predicted a decrease in task conflicts.     

PROJECT A VALIDITY RESULTS: THE RELATIONSHIP BETWEEN PREDICTOR AND CRITERION DOMAINS

A predictor battery of cognitive ability, perceptual-psychomotor ability, temperament/personality, interest, and job outcome preference measures was administered to enlisted soldiers in nine Army jobs. These measures were summarized in terms of 24 composite scores. The relationships between the predictor composite scores and five components of job performance were analyzed. Scores from the cognitive and perceptual-psychomotor ability tests provided the best prediction of job-specific and general task proficiency, while the temperament/personality composites were the best predictors of giving extra effort, supporting peers, and exhibiting personal discipline. Composite scores derived from the interest inventory were correlated more highly with task proficiency than with demonstrating effort and peer support. In particular, vocational interests were among the best predictors of task proficiency in combat jobs. The results suggest that the Army can improve the prediction of job performance by adding non-cognitive predictors to its present battery of predictor tests.     

When cognitive exertion does not yield cognitive gain: toward an informational explanation of learned helplessness

This study tested a new information-processing explanation of learned helplessness that proposes that an uncontrollable situation produces helplessness symptoms because it is a source of inconsistent, self-contradictory task information during problem-solving attempts. The flow of such information makes hypothesis-testing activity futile. Prolonged and inefficient activity of this kind leads in turn to the emergence of a state of cognitive exhaustion, with accompanying performance deficits. In 3 experiments, Ss underwent informational helplessness training (IHT): They were sequentially exposed to inconsistent task information during discrimination problems. As predicted, IHT was associated with subjective symptoms of irreducible uncertainty and resulted in (a) performance deterioration on subsequent avoidance learning, (b) heightened negative mood, and (c) subjective symptoms of cognitive exhaustion.     

Adolescent risk-taking is predicted by individual differences in cognitive control over emotional,but not non-emotional,response conflict

While much research on adolescent risk behaviour has focused on the development of prefrontal self-regulatory mechanisms, prior studies have elicited mixed evidence of a relationship between individual differences in the capacity for self-regulation and individual differences in risk taking. To explain these inconsistent findings, it has been suggested that the capacity for self-regulation may be, for most adolescents, adequately mature to produce adaptive behaviour in non-affective, “cold” circumstances, but that adolescents have a more difficult time exerting control in affective, “hot” contexts. To further explore this claim, the present study examined individual differences in self-control in the face of affective and non-affective response conflict, and examined whether differences in the functioning of cognitive control processes under these different conditions was related to risk taking. Participants completed a cognitive Stroop task, an emotional Stroop task, and a risky driving task known as the Stoplight game. Regression analyses showed that performance on the emotional Stroop task predicted laboratory risk-taking in the driving task, whereas performance on the cognitive Stroop task did not exhibit the same trend. This pattern of results is consistent with theories of adolescent risk-taking that emphasise the impacts of affective contextual influences on the ability to enact effective cognitive control.     

The Incremental Validity of Honesty–Humility Over Cognitive Ability and the Big Five Personality Traits

The present study examines the incremental validity of Honesty–Humility (H-H), a measure of the tendency to be fair and genuine in dealing with others, for supervisory ratings of job performance (including both task and contextual performance) over cognitive ability and the Big Five personality traits. Specifically, we explore the incremental validity of H-H in predicting contextual performance. Results based on 217 South Korean military officer candidates are consistent with previous conclusions that Conscientiousness is the strongest predictor of contextual performance and that cognitive ability is the strongest predictor of task performance. More important, our results reveal that H-H offers moderate incremental validity for contextual performance but no incremental validity for task performance over the effects of cognitive ability and the Big Five personality traits.