A closer look at the frame-of-reference effect in personality scale scores and validity

This article contributes to the understanding of why the use of a frame-of-reference leads to increased criterion-related validity of personality inventories. Two competing explanations are described and tested. A between-subjects (N = 337) and a within-subject (N = 105) study are conducted to test the hypothesized effects of use of a frame of reference on reliability and validity. Regarding the effects on reliability, use of a frame of reference reduces within-person inconsistency (instead of between-person variability) in responding to generic items. Use of a frame of reference further leads to higher validity as a result of the reduction of between-person variability and within-person inconsistency. Yet, reducing these inconsistencies is not enough. It is also important to use a frame of reference that is conceptually relevant to the criterion. Besides implications for contextualized personality inventories, these results provide an explanation for the moderate validities of generic personality inventories.     

A field study of frame-of-reference effects on personality test validity

As part of a test validation study at a major U.S.-based airline, the authors tested the effects of providing an "at work" frame-of-reference on the validity of the NEO Five-Factor Inventory among a sample of customer service supervisors (N = 206). Frame-of-reference moderated the validity of the Extraversion and Openness to Experience subscales after controlling for cognitive ability. In addition, the frame-of-reference personality test showed incremental validity over cognitive ability (deltaR2 = .16), but the standard personality test did not (deltaR2 = .05). The authors' discussion focuses on implications for personality theory and research and on implications for increasing the validity of personality tests in organizational settings.     

Psychopathy and the Five-factor model of personality: a replication and extension

It has recently been argued that psychopathy can be understood and represented using common dimensions of personality taken from the Five-factor model (FFM). In this research, we examined this possibility by using the Revised NEO Personality Inventory (NEO-PI-R; Costa & McCrae, 1992) to assess psychopathy in an undergraduate sample. Specifically, we matched individuals' NEO-PI-R profiles with an expert-generated psychopathy prototype to yield a psychopathy score. These scores were correlated with self-reports of drug use, delinquency, risky sex, aggression, and several laboratory tasks. FFM psychopathy was significantly related to all forms of deviance, although the effects tended to be small in size. Moreover, individuals who more closely resembled the prototypic FFM psychopath were more aggressive in a laboratory aggression task, less willing to delay gratification in a time discounting task, and demonstrated a preference for aggressive responses in a social information-processing paradigm.     

Incremental validity of the Psychopathy Checklist facet scores: predicting release outcome in six samples

The incremental validity of the 4 facet scores (Interpersonal, Affective, Lifestyle, Antisocial) of the Psychopathy Checklist-Revised (PCL-R; R. D. Hare, 1991, 2003) and the Psychopathy Checklist: Screening Version (PCL:SV; S. D. Hart, D. N. Cox, & R. D. Hare, 1995) was evaluated in 6 forensic/correctional samples with average follow-ups ranging from 20 weeks to 10 years. Results indicated that whereas Facet 4 (Antisocial) achieved incremental validity relative to the first 3 facets (Interpersonal, Affective, and Lifestyle) in predicting recidivism in all 6 samples, a block of the first 3 facets achieved incremental validity relative to the 4th facet in only 1 sample. Thus, although there was consistent support for the incremental validity of Facet 4 above and beyond the first 3 facets, there was minimal support for the incremental validity of Facets 1, 2, and 3 above and beyond Facet 4. The implications of these findings for the psychopathy construct in general and the PCL-R/SV in particular are discussed.     

Incremental validity of person-organization fit over the Big Five personality measures

Few studies have provided the validity evidence of a measure of objective person-organization fit (P-O fit) as a selection tool. The present study used a concurrent validation design to examine the criterion-related validity and the incremental validity of a P-O fit measure beyond the validity of the Big Five personality test for predicting job performance (task performance and organizational citizenship behavior) and employee commitment (organizational commitment and supervisory commitment) for a group of high-tech professional employees in Taiwan. Results showed that P-O fit predicted the contextual component of overall job performance and was significantly related to two types of employee commitment. Moreover, P-O fit had an incremental validity beyond that of the personality measures for predicting some of our outcome variables.     

Fully contextualized,frequency-based personality measurement: A replication and extension

We compared the predictive validity of two types of Frame-of-Reference personality measures to each other and to a baseline generic measure. Each version of the measures used a unique response-format referred to as frequency-based estimation that allowed the behavioral consistency of responses to be gauged. Generic personality scales, tagged scales with “at school”, and completely modified scales were compared in their prediction of academic performance, counterproductive academic behavior, and participant reactions. Results showed that completely contextualized measures were the most predictively valid and, contrary to our expectations, behavioral consistency did not moderate the relationships. Face validity and to a lesser extent perceived predictive validity improved with increasing contextualization. We discuss the implications of our results for personality assessment in applied settings.     

On the validity of culture‐level personality and stereotype scores

In response to comments by Perugini and Richetin and by Ashton, we discuss the reference‐group effect as a potential source of distortion in cross‐cultural comparisons and suggest some research designs to test its nature and importance. We argue that laboratory studies of personality are of limited utility in understanding personality questionnaire responses in real life. We summarise evidence in favour of the validity of aggregate personality traits and suggest that more scepticism is needed with regard to the accuracy of national character stereotypes. Published in 2007 by John Wiley & Sons, Ltd.     

Patterns of adaptation to conflict and schizoid personality scale scores

A previous investigation gave no evidence of a significant relationship of patterns of adaptation to conflict, as measured with the Serial Color-Word Test, with the Schizoid Personality Scale of the Coolidge Axis II Inventory. As a new scoring algorithm has subsequently been proposed for the latter scale, a replication was done with the modified schizoid scale. A group of 75 consecutive nonpsychotic women outpatients was given the Serial Color-Word Test and Coolidge's inventory. Both multiple and logistic regressions selected two significant predictors of schizoid personality, corresponding to high values of linear change in reading times during Trials 3 and 5 of the Serial Color-Word Test, i.e., to an increasingly rigid and inflexible style of the adaptive process. A multivariate analysis of variance yielded an effect size of .22 (partial eta2).     

Further assessment of the construct validity of four measures of narcissism: replication and extension

CT. The authors build on earlier research by L. S. Mullins and R. E. Kopelman (1988) and R. E. Kopelman and L. S. Mullins (1992) to reexamine the construct validity of four narcissism scales: the Margolis-Thomas Measure of Narcissism (MT; H. D. Margolis & V. A. Thomas, 1980), the Narcissistic Personality Disorder Scale (NPDS; H. U. Ashby, R. R. Lee, & E. H. Duke, 1979), the Narcissism-Hypersensitivity subscale of the MMPI, Scale 5, Masculinity-Femininity (NHMF; K. Serkownek, 1975), and the Narcissistic Personality Inventory (NPI; R. Raskin & C. S. Hall, 1979). The present analysis included the revised NPI and its factors along with four measures of satisfaction and a number of other previously assessed variables. The MT exhibited the strongest validity, correlating positively with conceptually related constructs such as Machiavellianism, nonsignificantly with unrelated measures like the need to achieve, and inversely with all four satisfaction scales. Contrasts between the NPI and NPDS and NHMF seemed to parallel recent differentiations between overt and covert narcissism, but data for the NPI factors suggested instead that the four narcissism scales helped describe a complex psychological continuum related to adjustment.     

Incremental validity of the MMPI-2 PSY-5 scales in assessing self-reported personality disorder criteria

The Minnesota Multiphasic Personality Inventory-2 (MMPI-2) Personality Psychopathology-Five (PSY-5) scales were developed to measure abnormal personality symptomatology. The present study examines the incremental validity of the PSY-5 scales beyond the clinical and content scales in assessing criteria associated with personality disorders. The current sample includes 240 male and 407 female clients from private practice settings who completed the MMPI-2 and the Multiaxial Diagnostic Inventory (MDI), a self-report checklist of Diagnostic and Statistical Manual of Mental Disorders (3rd ed., revised) symptoms. Six of the MDI personality disorder scales, conceptually related to the PSY-5 scales, are used as criteria. Hierarchical regression analyses determine the incremental validity of each PSY-5 scale. In most analyses, PSY-5 scales add a significant increment of variance to the clinical and content scales. Implications of the results are discussed.     

A new frame for frame-of-reference training: enhancing the construct validity of assessment centers

The authors undertook a comprehensive examination of the construct validity of an assessment center in this study by (a) gathering many different types of evidence to evaluate the strength of the inference between predictor measures and constructs (e.g., reliability, accuracy, convergent and discriminant relationships), (b) introducing a theoretically relevant intervention (frame-of-reference [FOR] training) aimed at improving construct validity, and (c) examining the effect of this intervention on criterion-related validity (something heretofore unexamined in the assessment center literature). Results from 58 assessees and 122 assessors suggested that FOR training was effective at improving the reliability, accuracy, convergent and discriminant validity, and criterion-related validity of assessment center ratings. Findings are discussed in terms of implications and future directions for both FOR training and assessment center practice.     

Obsessional slowness: a replication and extension

The treatment of an obsessionally slow patient is described. A combination of pacing and self instruction was used in order to speed up behaviour. This method is adopted and extended from that reported by Rachman (1974). Suggestions as to the maintenance of the treatment effects are discussed.     

An extension of construct validity for personality scales using twin-based criteria

A twin-based analysis of personality inventory scales yields information relevant to construct validity. If scales validly measure dimensions influenced developmentally either by environmental factors common to cotwins or by genetic similarities, these influences should be apparent in patterns of intraclass correlations and within-pair variance ratios for monozygotic and dizygotic pairs. Criteria for construct validity derived from adolescent twin statistics are illustrated with Wiggins' MMPI Content Scales. A type of discriminant validity indicates that some scales are not distinct in terms of the influences of familial similarity. Sex differences are examined.     

Comparing applicants and incumbents: Effects of response distortion on mean scores and validity of personality measures

This study examined the negative effect of likely applicant distortion on mean scores and validity of personality measures. The personality test scores and performance ratings of applicants were directly compared to those of incumbents with the same occupation in four different samples. The results showed that applicant mean scores were higher and validity coefficients were lower than for incumbents.