Nonlinear Personality–Performance Relationships and the Spurious Moderating Effects of Traitedness

Abstract This article evaluated the validity of two traditional traitedness measures (i.e., standard deviation traitedness measures and global, single‐item traitedness measures) and three new traitedness measures (i.e., multi‐item, ranking, and facet traitedness measures). Overall, there was poor construct validity for both the traditional and new measures of traitedness. Furthermore, traitedness was a spurious moderator of the relationship between openness to experience and academic performance. Finally, evidence of nonlinear personality‐academic performance relationships was discovered.     

Metatraits: Enhancing Criterion‐Related Validity Through the Assessment of Traitedness1

With the growing use of personality tests as predictors of job performance, the reliable measurement of these personality variables has become important to selection researchers and practitioners. Metatrait researchers have examined the possibility that personality traits are not equally relevant for all people. This study sought to examine whether considering the relevance of a trait to an individual (i.e., traitedness) would enhance the predictive validity of a measure. Specifically, traitedness was expected to moderate the relationship between predictor and criterion scores such that the predictor‐criterion relationship would be stronger for traited than for untraited individuals. Consistent with this hypothesis, a strong moderating effect was found for traitedness between a measure of personality and an objective criterion. Contrary to what was predicted, traitedness only provided incremental validity, above personality, for supervisor ratings of job performance.     

The Analysis of Consistency in Personality Assessment

ABSTRACT In recent years psychologists have treated not only the mean (or sum) but also the variability of a person's item scores on multi-item scales as significant individual attributes Variability measures and comparable statistics, interpreted as indicators of (in)consistency, have been used as a basis for rejecting test results or trait constructs In cognitive assessment the concern has been with test results, in personality assessment the emphasis has been primarily on traits and "traitedness" Some of the assessment procedures are described and discussed For personality assessment the main conclusions are ( a ) the use of intra-individual variability measures for identifying invalid test data could be explored and exploited more, ( b ) intra-individual variability is for many reasons to be considered a poor indicator of low traitedness rather than its hallmark, and ( c ) traitedness itself is a promising concept and certain alternative approaches to traitedness assessment seem promising and integrative     

Improving the measurement of spatial aptitude by dynamic testing

Can spatial aptitude scores be increased by test-related training? Is construct validity influenced? The traditional psychometric view of test-related training as coaching, which threatens test validity, contrasts sharply with the view that performance modifiability measures a theoretically important construct that increases test validity (e.g., Vygotsky, 1978). This article reports on three studies of the dynamic testing of spatial ability in which training on the physical analogue of the mental folding task intervenes between a pretest and a posttest. Experiment 1 found that spatial ability can be significantly and substantially increased. Experiment 2 examined the impact of the dynamic testing procedure on the relationship of ability scores to other measures. Although sex differences in spatial ability were unchanged, the dynamic testing procedure increased predictive validity for training in text editing. Experiment 3 examined the impact on construct representation. The mathematical models of task difficulty indicated that the posttest more clearly represented rotational processing while the psychometric models supported the measurement of two abilities (i.e., initial status and gain). The implications for measuring learning ability by dynamic testing are discussed.     

Introduction

This article reports a comprehensive meta-analysis of the criterion-oriented validity of the Big Five personality dimensions assessed with forced-choice (FC) inventories. Six criteria (i.e., performance ratings, training proficiency, productivity, grade-point average, global occupational performance, and global academic performance) and three types of FC scores (i.e., normative, quasi-ipsative, and ipsative) served for grouping the validity coefficients. Globally, the results showed that the Big Five assessed with FC measures have similar or slightly higher validity than the Big Five assessed with single-stimulus (SS) personality inventories. Quasi-ipsative measures of conscientiousness (K = 44, N = 8794, ρ = .40) are found to be better predictors of job performance than normative and ipsative measures. FC inventories also showed similar reliability coefficients to SS inventories. Implications of the findings for theory and practice in academic and personnel decisions are discussed, and future research is suggested.     

Specificity Matters: Criterion-Related Validity of Contextualized and Facet Measures of Conscientiousness in Predicting College Student Performance

To enhance the predictive validity of self-report personality measures, 2 distinct ways of increasing specificity of personality measures have been proposed in the literature—contextual specificity (i.e., providing a contextual referent) and content specificity (i.e., focusing on more specific constructs such as the Big Five facets). This study extends this line of research by examining whether there is an optimal way to configure, align, or integrate contextual and content specificity using measures of conscientiousness to predict college student success. A sample of 478 undergraduate students completed 4 measures of conscientiousness that varied in the level of content and contextual specificity. These forms of specificity were crossed to yield 4 distinct measures of conscientiousness. We then evaluated and compared the relative importance and the incremental importance of these different measures in the prediction of academic success. Superior predictive validity was found for both contextualized and facet measures of conscientiousness compared to a measure of global conscientiousness in predicting grade-point average and a broader behavioral criterion of student performance. When contextual and content specificity approaches were compared and combined, we observed the strongest predictive validity when the level of specificity is appropriately matched between predictor and criterion.     

Coloured Letters and Numbers (CLaN): A reliable factor-analysis based synaesthesia questionnaire

Synaesthesia is a heterogeneous phenomenon, even when considering one particular sub-type. The purpose of this study was to design a reliable and valid questionnaire for grapheme-colour synaesthesia that captures this heterogeneity. By the means of a large sample of 628 synaesthetes and a factor analysis, we created the Coloured Letters and Numbers (CLaN) questionnaire with 16 items loading on 4 different factors (i.e., localisation, automaticity/attention, deliberate use, and longitudinal changes). These factors were externally validated with tests which are widely used in the field of synaesthesia research. The questionnaire showed good test–retest reliability and construct validity (i.e., internally and externally). Our findings are discussed in the light of current theories and new ideas in synaesthesia research. More generally, the questionnaire is a useful tool which can be widely used in synaesthesia research to reveal the influence of individual differences on various performance measures and will be useful in generating new hypotheses.     

Better (or worse) for some than others: Individual differences in the positivity offset and negativity bias

People tend to respond with more positive than negative affect to mildly emotional stimuli (i.e., positivity offset) and respond more strongly to very negative than to matched positive stimuli (i.e., negativity bias). In the current study, the authors examine individual differences in the positivity offset and negativity bias and demonstrate that both are stable over time and generalize across different kinds of stimuli (e.g., pictures, sounds, words, games of chance). Furthermore, the positivity offset and negativity bias are not redundant with traditional personality measures and exhibit differential predictive validity, such that both types of measures predict behavior in meaningful ways. Implications for a comprehensive understanding of affect and emotion and their relationship to physical and mental health are discussed.     

Further explorations of perceptual speed abilities in the context of assessment methods, cognitive abilities, and individual differences during skill acquisition

Measures of perceptual speed ability have been shown to be an important part of assessment batteries for predicting performance on tasks and jobs that require a high level of speed and accuracy. However, traditional measures of perceptual speed ability sometimes have limited cost-effectiveness because of the requirements for administration and scoring of paper-and-pencil tests. There have also been concerns about the validity of previous computer approaches to administering perceptual speed tests (e.g., see Mead & Drasgow, 1993). The authors developed two sets of computerized perceptual speed tests, with touch-sensitive monitors, that were designed to parallel several paper-and-pencil tests. The reliability and validity of the tests were explored across three empirical studies (N = 167, 160, and 117, respectively). The final study included two criterion tasks with 4.67 and 10 hours of time-on-task practice, respectively. Results indicated that these new measures provide both high levels of reliability and substantial validity for performance on the two skill-learning tasks. Implications for research and application for computerized perceptual speed tests are discussed.     

The Impact of Contextual Self‐Ratings and Observer Ratings of Personality on the Personality–Performance Relationship1

This study examined 2 possible ways of increasing the predictive validity of personality measures: using observer (i.e., supervisor and coworker) ratings and work‐specific self‐ratings of Big Five personality factors. Results indicated that among general self‐ratings of Big Five personality dimensions, Conscientiousness was the best predictor of in‐role performance, and Agreeableness and Emotional Stability were the best predictors of organizational citizenship behavior (OCB). Observer ratings of personality accounted for incremental variance in job performance (in‐role performance and OCB) beyond that accounted for by general self‐ratings. However, contrary to our expectations, work‐specific (i.e., contextual) self‐ratings of personality generally did not account for incremental variance in job performance beyond that accounted for by general self‐ratings.     

IN SUPPORT OF PERSONALITY ASSESSMENT IN ORGANIZATIONAL SETTINGS

Personality constructs have been demonstrated to be useful for explaining and predicting attitudes, behaviors, performance, and outcomes in organizational settings. Many professionally developed measures of personality constructs display useful levels of criterion-related validity for job performance and its facets. In this response to Morgeson et al. (2007) , we comprehensively summarize previously published meta-analyses on (a) the optimal and unit-weighted multiple correlations between the Big Five personality dimensions and behaviors in organizations, including job performance; (b) generalizable bivariate relationships of Conscientiousness and its facets (e.g., achievement orientation, dependability, cautiousness) with job performance constructs; (c) the validity of compound personality measures; and (d) the incremental validity of personality measures over cognitive ability. Hundreds of primary studies and dozens of meta-analyses conducted and published since the mid 1980s indicate strong support for using personality measures in staffing decisions. Moreover, there is little evidence that response distortion among job applicants ruins the psychometric properties, including criterion-related validity, of personality measures. We also provide a brief evaluation of the merits of alternatives that have been offered in place of traditional self-report personality measures for organizational decision making. Given the cumulative data, writing off the whole domain of individual differences in personality or all self-report measures of personality from personnel selection and organizational decision making is counterproductive for the science and practice of I-O psychology.     

Variations in the Understanding of Interpersonal Behavior: Adherence to the Interpersonal Circle as a Moderator of the Rigidity–Psychological Well‐Being Relation

ABSTRACT The idiothetic structure of interpersonal trait perceptions was examined as it moderated the interpersonal rigidity–psychological well‐being relation. The focus was on the extent to which individuals' perceptions of the similarity of interpersonal behavior fits (i.e., adhered to) the normative interpersonal circle. In two samples of college students, individual differences in adherence to the interpersonal circle moderated the relation of interpersonal rigidity with various indices of psychological well‐being. We found that those individuals whose perceptions of interpersonal traits were better represented by the interpersonal circle had negative relations between interpersonal rigidity and satisfaction with life, self‐confidence, self‐liking, and complementarity and positive relations with interpersonal problems. The results suggest that adherence to the interpersonal circle may be a new means of viewing traitedness and that cognitive interpretation of traits may have an important moderating function.     

Level validity of self-report whole-family measures

This article introduces an approach to testing the level validity of family assessment instruments (i.e., whether a family instrument measures family functioning at the level of the system it purports to assess). Two parents and 2 adolescents in 69 families rated the warmth in each of their family relationships and in the family as a whole. Family members' ratings of whole-family warmth assessed family functioning not only at the family level (i.e., characteristics of the family as a whole) but also at the individual level of analysis (i.e., characteristics of family members as raters), indicating a lack of level validity. Evidence was provided for the level validity of a latent variable based on family members' ratings of whole-family warmth. The findings underscore the importance of assessing the level validity of individual ratings of whole-family functioning.     

Examining the Merit of Grit in Women's Soccer: Questions of Theory,Measurement, and Application

The purpose of the current study was to examine the factorial validity of existing grit measures and to examine differences in the grit level of athletes across performance tiers (i.e., collegiate, professional, Olympic). Female soccer players (N = 305) of varying competitive levels completed measures of grit. Results revealed preliminary support for the 8-item Grit Scale factorial validity, as well as a significant age-adjusted difference in grit level between Division I and Division II collegiate athletes. Taken together, study results prompt questions regarding the theoretical underpinnings, and consequent measurement, of the grit construct in sport psychology practice.     

Rate as a Measure of Academic Success for Mildly Handicapped Students

In large numbers, students variously categorized as learning disabled, behavior disordered, or mentally retarded, are being educated in the "least restrictive environment" (LRE). The article examines the relationship between carrying out instruction in the LRE and traditional versus contemporary assessment practices. It is shown that the selection of performance measures, i.e., measures of frequency, percent, or rate of academic responding, should be linked ot the learning stage of the student(s), i.e., acquisition to proficiency. An argument is made for wider use of rate measures and establish peer-referenced standards for guiding decisions regarding functional performance criteria. Impliactions for the increased use of rate standards within and across special and mainstream classroom settings are discussed.     

Team effectiveness: beyond skills and cognitive ability.

On the basis of job analysis results, the validity of using measures of general cognitive ability, job-specific skills, and personality traits jointly at both the individual level and the group level to predict the performance of 79 four-person, human resource work teams was evaluated. Team member trait and job skill scores were aggregated with a conjunctive model of task performance. At the individual level of analysis, measures of personality (i.e., Agreeableness and Conscientiousness) predicted peer ratings of team member performance beyond measures of job-specific skills and general cognitive ability. Similarly, at the group level of analysis, both Agreeableness and Conscientiousness predicted supervisor ratings of work team performance, objective measures of work team accuracy, and work completed. At both the individual and group levels, the trait of Agreeableness predicted Interpersonal Skills.     

A META-ANALYSIS OF WORK SAMPLE TEST VALIDITY: UPDATING AND INTEGRATING SOME CLASSIC LITERATURE

Work sample tests have been used in applied psychology for decades as important predictors of job performance, and they have been suggested to be among the most valid predictors of job performance. As we examined classic work sample literature, we found the narrative review by Asher and Sciarrino (1974) to be plagued by many methodological problems. Further, it is possible that data used in this study may have influenced the results (e.g., r = .54) reported by Hunter and Hunter in their seminal work in 1984. After integrating all of the relevant data, we found an observed mean correlation between work sample tests and measures of job performance of .26. This value increased to .33 when measures of job performance (e.g., supervisory ratings) were corrected for attenuation. Our results suggest that the level of the validity for work sample tests may not be as large as previously thought (i.e., approximately one third less than previously thought). Further, our work also summarizes the relationship of work sample exams to measures of general cognitive ability. We found that work sample tests were associated with an observed correlation of .32 with tests of general cognitive ability.     

Predicting Military Performance from Specific Personality Measures: A validity study

This study examined the criterion‐related validity of the dimensions of a customized personality inventory in predicting different aspects of military performance. A cross‐validity study was also carried out to establish the stability of the obtained regression weights. The sample consisted of 733 non‐commissioned officers who filled out the personality inventory. Three different measures of performance (i.e., commendations, reprimands, and cumulative ranking) were used as the criteria in validating the inventory. Among the 11 dimensions of the inventory, emotional stability, directing and monitoring, self‐discipline, and military bearing predicted commendations; determination was the only predictor for reprimands, and cumulative ranking was predicted by determination and adaptability. The cross‐validity procedure showed that the obtained regression weights were stable, providing further support for the validity of the personality measures. Results yielded support for the argument that specific personality measures could be respectable predictors of different aspects of military performance.