Research validity scales for the NEO-PI-R: development and initial validation

The purpose of this investigation was to develop a set of research validity scales for use with the NEO Personality Inventory-Revised (NEO-PI-R; Costa & McCrae, 1992b). In study 1 we used the existing NEO-PI-R item pool to select items for three validity scales: positive presentation management, negative presentation management, and inconsistency. Several iterative item and scale analyses were conducted, using multiple criteria for item selection. These analyses resulted in 10-item scales. In Study 2, the internal consistency, interscale relationships, and normative characteristics were examined in a separate sample of working adults. In Study 3, the validity of the scales was examined by contrasting five sets of NEO-PI-R protocols: from a separate sample of working adults, from a sample of 100 NEO-PI-R protocols with randomly produced responses, and from three samples of undergraduates completing the NEO-PI-R under different instructional sets. Analyses revealed that both the research validity scales and the NEO-PI-R domain scales were sensitive to group differences. issues relating to the appropriate use of the scales are discussed.     

The detection of malingering in criminal forensic groups: MMPI validity scales

Despite the value of the MMPI to the forensic assessment of malingering (exaggeration) of psychopathology, few studies have assessed the accuracy of the MMPI validity scales in criminal forensic populations. We administered the MMPI to 35 insanity defendants undergoing evaluation for fitness to stand trial and/or sanity at the time of the crime, who stood to benefit from being assessed as psychologically disturbed, and 39 subjects previously found not guilty by reason of insanity (NGRI), who did not stand to gain from such an assessment, Insanity defendants showed significantly more malingering than NGRI subjects, p less than .05. Racial differences did not affect the data. These findings support the efficacy of MMPI validity scales in assessing malingering within criminal forensic groups, and support the generalizability of the scales across race.     

The utility of the NEO-PI-R validity scales to detect response distortion: a comparison with the MMPI-2

In this psychometric study, we compared the recently developed Validity Scales from the Revised NEO Personality Inventory (NEO PI-R; Costa & McCrae, 1992b) with the MMPI-2 (Butcher, Dahstrom, Graham, Tellegen, & Kaemmer, 1989) Validity Scales. We collected data from clients (n = 74) who completed comprehensive psychological evaluations at a university-based outpatient mental health clinic. Correlations between the Validity Scales of the NEO-PI-R and MMPI-2 were significant and in the expected directions. The relationships provide support for convergent and discriminant validity of the NEO-PI-R Validity Scales. The percent agreement of invalid responding on the two measures was high, although the diagnostic agreement was modest (kappa = .22-.33). Finally, clients who responded in an invalid manner on the NEO-PI-R Validity Scales produced significantly different clinical profiles on the NEO-PI-R and MMPI-2 than clients with valid protocols. These results provide additional support for the clinical utility of the NEO-PI-R Validity Scales as indicators of response bias.     

Research Validity Scales for the NEO-PI-R: additional evidence for reliability and validity

We examined the reliability and validity of the research validity scales (Schinka, Kinder, & Kremer, 1997) for the NEO-Personality Inventory-Revised (NEO-PI-R) in a clinical sample. The Negative Presentation Management (NPM) and Positive Presentation Management (PPM) scales were found to have satisfactory internal consistency reliability. Support for the validity of these scales was provided by the pattern of convergent and discriminant correlations with respective Personality Assessment Inventory (PAI) validity scales. Finally, PAI profiles of individuals with invalid NPM scores were found to differ significantly from those with valid NPM scores. Comparisons of the invalid profiles with profiles from other clinical samples provided additional support for the use of the NPM scale as a measure of negative impression management.     

Detection of overreported psychopathology with the MMPI-2-RF [corrected] validity scales

We examined the utility of the validity scales on the recently released Minnesota Multiphasic Personality Inventory-2 Restructured Form (MMPI-2 RF; Ben-Porath & Tellegen, 2008) to detect overreported psychopathology. This set of validity scales includes a newly developed scale and revised versions of the original MMPI-2 validity scales. We used an analogue, experimental simulation in which MMPI-2 RF responses (derived from archived MMPI-2 protocols) of undergraduate students instructed to overreport psychopathology (in either a coached or noncoached condition) were compared with those of psychiatric inpatients who completed the MMPI-2 under standardized instructions. The MMPI-2 RF validity scale Infrequent Psychopathology Responses best differentiated the simulation groups from the sample of patients, regardless of experimental condition. No other validity scale added consistent incremental predictive utility to Infrequent Psychopathology Responses in distinguishing the simulation groups from the sample of patients. Classification accuracy statistics confirmed the recommended cut scores in the MMPI-2 RF manual (Ben-Porath & Tellegen, 2008).     

Classification accuracy of MMPI-2 validity scales in the detection of pain-related malingering: a known-groups study

The purpose of this study was to determine the accuracy of Minnesota Multiphasic Personality Inventory 2nd edition (MMPI-2; Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) validity indicators in the detection of malingering in clinical patients with chronic pain using a hybrid clinical-known groups/simulator design. The sample consisted of patients without financial incentive (n = 23), nonmalingering patients with financial incentive (n = 34), patients definitively determined to be malingering based on published criteria ( n = 32), and college students asked to simulate pain-related disability (n = 26). The MMPI-2 validity scales differentiated malingerers from nonmalingerers with a high degree of accuracy. Hypochondriasis and Hysteria were also effective. For all variables except Scale L, more extreme scores were associated with higher specificity. This study demonstrates that the MMPI-2 is capable of differentiating intentional exaggeration from the effects on symptom report of chronic pain, genuine psychological disturbance, and concurrent stress associated with pursuing a claim in a medico-legal context.     

Effectiveness of NEO-PI-R Research Validity Scales for Discriminating Analog Malingering and Genuine Psychopathology

We investigated the research validity scales for the NEO Personality Inventory-Revised (NEO-PI-R) proposed by Schinka, Kinder, and Kremer (1997): Positive Presentation Management (PPM) and Negative Presentation Management (NPM). Additionally, an experimental analog to the Minnesota Multiphasic Personality Inventory-2's (MMPI-2; Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) F - K index was calculated by subtracting the raw score on PPM from the raw score on NPM (NPM - PPM). In 2 studies, all indexes showed significant between-group differences when samples of analog malingerers (n = 97) were contrasted with psychiatric outpatients (n = 272). The sensitivity and specificity of these validity indexes indicated that although none performed well in extremely low base rate environments, the NPM and NPM - PPM indexes showed promise when the base rate of faking bad rose to higher levels.     

Maximizing internal and external validity in MMPI malingering research: a study of a military population

The authors investigated the effectiveness of various commonly used Minnesota Multiphasic Personality Inventory (MMPI; Hathaway & McKinley, 1943) indices of exaggeration and malingering in detecting suspected malingering in a military sample of 121 enlisted men. To maximize external validity, only men undergoing psychological evaluation were used as participants. Forty-one participants were identified as suspected malingerers through multiple criteria and were contrasted with schizophrenic-spectrum and clinic outpatient groups. To improve internal validity, the 41 suspected malingering participants were asked to retake the test without exaggerating. Results revealed that there were many false positives and fewer, but nonetheless many, false negatives with standard malingering indices. It appeared that the Gough Dissimulation scale (Gough, 1947) might hold the most promise as a measure of malingering, but other scales are also useful. Individual comparisons between different samples and implications for MMPI-2 (Butcher et al., 1989) are presented.     

Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences

Using a self-administered questionnaire, 149 respondents rated service elements associated with a recently visited store or restaurant on scales that differed only in the number of response categories (ranging from 2 to 11) and on a 101-point scale presented in a different format. On several indices of reliability, validity, and discriminating power, the two-point, three-point, and four-point scales performed relatively poorly, and indices were significantly higher for scales with more response categories, up to about 7. Internal consistency did not differ significantly between scales, but test-retest reliability tended to decrease for scales with more than 10 response categories. Respondent preferences were highest for the 10-point scale, closely followed by the seven-point and nine-point scales. Implications for research and practice are discussed.     

On the validity and utility of discriminating among impulsivity-like traits

The ability to make precise distinctions among related personality constructs helps clarify theory and increases the utility of clinical assessment. In three studies, the authors evaluated the validity of distinctions among four impulsivity-like traits: sensation seeking, lack of planning, lack of persistence, and urgency (acting rashly when distressed). Factor analyses indicated that lack of planning and lack of persistence are two distinct facets of one broader trait, whereas urgency and sensation seeking are both very modestly related to each other and to the planning/persistence measures. The authors developed interview assessments of each, and multitrait, multimethod matrix results indicated clear convergent and discriminant validity among the constructs. The distinctions among them were useful: The traits accounted for different aspects of risky behaviors. Sensation seeking appeared to relate to the frequency of engaging in risky behaviors, and urgency appeared to relate to problem levels of involvement in those behaviors.     

Profile validity standards for MMPI and MMPI-2 F scales

Although numerous indices of validity have been developed for the MMPI and MMPI-2, interest in the F scale and its variants continues, especially among practicing clinicians. The use of the binomial for assessing standards for random answering and possibly for judgments of malingering offers another approach for the interpretation of F-scale scores. The theoretical binomial distribution and Monte Carlo data are in accord. Cut-off scores of 24 for the MMPI and 23 for the MMPI-2 suggest random responses, and scores of 40 and 37, respectively, suggest clinical interpretation rather than randomness of responding.     

Incremental validity and the assessment of psychopathology in adults

Results from studies on incremental validity are described for (a) interviews, (b) personality inventories, (c) projective techniques, and (d) brief self-rated and clinician-rated measures. In some of the studies (clinical judgment studies), psychologists were given increasing amounts of information. In other studies (statistical prediction studies), increasing amounts of assessment information were entered into a statistical prediction rule. Although relatively little research has been conducted on incremental validity, results that have been obtained tend to favor the use of interviews, personality inventories (e.g., the Minnesota Multiphasic Personality Inventory--2; J. N. Butcher, W. G. Dahlstrom, J. R., Graham, A. Tellegen, & B. Kaemmer, 1989), and brief self-rated measures (e.g., the Anxiety Sensitivity Index, S. Reiss, R. A. Peterson, D. M. Gursky, & R. M. McNally, 1986). Results are generally less encouraging for projective techniques.     

Finding the "genuine" schizotype: a model and method for resolving heterogeneity in performance on laboratory measures in experimental psychopathology research

Heterogeneity in the performance of persons affected with schizophrenia or schizotypic psychopathology on various laboratory tasks has long been recognized, both for its consistency across tasks and studies and for the massive methodological and substantive challenges it poses for experimental psychopathology, genetic, and other investigations. Traditional multivariate techniques, such as factor analysis, discriminant function analysis, and cluster analysis, have all been deemed inadequate for resolving heterogeneity, because of one or another statistical limitation. Here, an objective statistical approach based on a formal statistical model that uses the ubiquitous and well-developed expectation-maximization (EM) algorithm (A. P. Dempster, N. M. Laird, & D. B. Rubin, 1977) is presented, which enables one effectively to partition a group of experimental subjects, in this case identified initially using the well-known Perceptual Aberration Scale (L. J. Chapman, J. P. Chapman, & M. L. Raulin, 1978), in a manner that reduces heterogeneity and allows for the separation of what are termed genuine and false-positive schizotypes. The validity of the parsing strategy was supported by reference to other laboratory indexes of relevance to schizophrenia and schizotypy that were not included in the initial EM-based analyses. The potential utility of this approach is discussed with reference to future schizophrenia and schizotypy research.     

Hierarchical linear modeling analyses of the NEO-PI-R scales in the Baltimore Longitudinal Study of Aging

The authors examined age trends in the 5 factors and 30 facets assessed by the Revised NEO Personality Inventory in Baltimore Longitudinal Study of Aging data (N=1,944; 5,027 assessments) collected between 1989 and 2004. Consistent with cross-sectional results, hierarchical linear modeling analyses showed gradual personality changes in adulthood: a decline in Neuroticism up to age 80, stability and then decline in Extraversion, decline in Openness, increase in Agreeableness, and increase in Conscientiousness up to age 70. Some facets showed different curves from the factor they define. Birth cohort effects were modest, and there were no consistent Gender x Age interactions. Significant nonnormative changes were found for all 5 factors; they were not explained by attrition but might be due to genetic factors, disease, or life experience.     

Semantic response consistency and protocol validity in structured personality assessment: the case of the NEO-PI-R

In this study we tested the hypothesis that groups of NEO Personality Inventory-Revised (NEO-PI-R; Costa & McCrae, 1992a) protocols identified as potentially invalid by an inconsistency scale (INC; Schinka, Kinder, & Kremer, 1997) would show reduced reliability and validity according to a series of psychometric tests. Data were obtained from 2 undergraduate student samples, a self-report group (n = 132) who provided NEO-PI-R self-ratings on 2 occasions separated by a 7- to 14-day interval and an informant group (n = 109) who provided ratings of well-known friends or relatives on 2 occasions separated by a 6 month interval. INC scores from the Time 1 protocols were used to divide these samples into low, moderate, and elevated inconsistency groups. In both samples, these 3 groups showed equivalent levels of reliability and validity as measured by: contingency coefficients for the 20 INC item responses across occasions; test-retest intraclass correlations of NEO-PI-R domain scores; convergent correlations with Goldberg's (1992) Bipolar Adjective Scale scores; and discriminant correlations between the 5 NEO-PI-R domain scores. The similarity of results across self-report and informant assessment contexts provides additional evidence that semantic consistency approaches to assessing protocol validity may overestimate the prevalence of random or careless response behavior in standard administration conditions. Several theories are discussed that accommodate the existence of valid inconsistency in structured personality assessment.     

Convergent validity of the MCMI-III personality disorder scales and the MMPI-2 scales

The MCMI-III personality disorder scales (Millon, 1994) were empirically validated in a sample of prisoners, psychiatric inpatients, and outpatients (N = 477). The scale intercorrelations were congruent with those obtained by Millon, Davis, and Millon (1997). We conclude that our Flemish/Dutch version shows no significant differences with the original version of the MCMI-III as far as intercorrelations are concerned. Convergent validity of the MCMI-III personality disorder scales was evaluated by the correlational data between the MCMI-III personality disorder scales and the MMPI-2 clinical (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) and personality disorder (Somwaru & Ben-Porath, 1995) scales. Improved convergence was obtained compared with previous versions of the MCMI-I. Only the compulsive MCMI-III personality disorder scale remains problematic. The scale even showed negative correlations with some of the related clinical scales and with the corresponding personality disorder scales of the MMPI-2.