Examining the impact of unscorable item responses on the validity and interpretability of MMPI-2/MMPI-2-RF Restructured Clinical (RC) Scale scores

This article examined the impact of unscorable item responses on the psychometric validity and practical interpretability of scores on the Restructured Clinical (RC) Scales of the Minnesota Multiphasic Personality Inventory-2/Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2/MMPI-2-RF). In analyses conducted with five archival samples, we found that relatively large proportions of unscorable responses (defined as 10% or more of the items scored on a scale) were relatively uncommon, occurring most often in forensic samples. Simulated unscorable responses were inserted in varying proportions (10% to 90%) in place of the responses of participants in two of the archival samples for which criterion data were available. Analyses were conducted to gauge the impact of unscorable responses on the criterion validity of scores on these scales and their interpretability. Impact on validity was evaluated by examining correlations with extra-test variables as a function of increasing levels of unscorable responding. Interpretability was evaluated by examining the proportion of participants who produced clinically elevated RC Scale scores as a function of unscorable responding. Results indicate that whereas scale score validity was relatively robust up to a level of 50% unscorable responses, interpretability was substantially compromised at only 10% unscorable responding. This suggests that prorated scores may be used to correct for the impact of unscorable responses on the interpretability of RC Scale scores at levels as high as 50% unscorable responses. Classification analyses supported this possibility. Further steps needed to explore the feasibility of using prorated scores are discussed.     

MMPI-2-RF characteristics of custody evaluation litigants

The Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF) is a 338-item objective self-report measure drawn from the 567 items of the MMPI-2. Although there is a substantial MMPI-2 literature regarding child custody litigants, there has been only one previously published study using MMPI-2-RF data in this population that focused on Validity scales L-r and K-r. The current study evaluated the MMPI-2-RF results of 344 child custody litigants and showed substantial consistency between T-score elevations typically found on MMPI-2 Validity scales L and K, and comparable elevations for MMPI-2-RF validity scales L-r and K-r. Mean T-scores well within normal limits characterized results for clinical scales on both instruments. The RC scale intercorrelation patterns, and alpha coefficient values found for MMPI-2-RF scales in a custody population, were also found to be very similar to those reported for other populations. Directions for future research are presented.     

The factor structure of the MMPI-2 Restructured Clinical (RC) scales

We investigated the MMPI-2 Restructured Clinical (RC) scales (Tellegen et al., 2003) to determine if they had a more differentiated factor structure than the MMPI-2 Clinical scales. When factored alone, the RC scales had a 5-dimensional structure; the Clinical scales had 3 dimensions. When factored in combination with the Content scales, both sets of scales produced 5 dimensions. However, the RC and Content factors generally provided more efficient and logical markers of psychopathology than the Clinical and Content factors. We discuss interpretive considerations.     

MMPI-2 at the crossroads: aging technology or radical retrofitting?

Paradigmatic shifts are uncommon in psychological assessment, which is often steeped more in tradition than innovative practices. The introduction of the Restructured Clinical (RC) Scales by Tellegen et al. (2003) represents a radical shift in the development and validation of the MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989). In broadening the debate over the merit of the RC Scales, in this article, we examine current challenges to MMPI-2 interpretation including issues with discriminant validity and imprecision (e.g., standard errors of measurement).We evaluate the RC alternative in the context of these interpretive challenges. Although we note several improvements, the incremental validity for MMPI-2 interpretative statements remains to be established. MMPI-2 researchers may wish to test several innovative models for their comparative strengths in establishing scale properties and providing empirically based interpretations.     

Psychometric evaluation of the MMPI-2 Restructured Clinical scales in two Dutch samples

In this article, we evaluate internal validity, internal consistency, and test-retest reliability of the MMPI-2 Restructured Clinical (RC) scales in the Dutch MMPI-2 normative sample (N = 1,244) and a Dutch outpatient psychiatric sample (N = 1,066). We pay special attention to a critique regarding construct drift of RC3 and the redundancy of the RC scales with existing MMPI-2 scales. The results indicate that the RC scales in both samples show comparable or better internal consistencies than the Clinical scales. Also, in both samples, the RC scales demonstrate lower scale-level intercorrelations than the Clinical scales. As to the structural characteristics, principal component analysis of the RC scales provided a clearer pattern than an analysis of the Clinical scales. Furthermore, mean raw scores on the RC scales for men in the Dutch normative sample corresponded highly with those in the U.S. normative sample except for RC2 and RC4. Less correspondence was found for women. Overall, we conclude that the RC scales show satisfactory reliability and promising internal validity in our Dutch samples. We suggest that U.S. validation studies on the RC scales may be generalized to the Dutch-language version of the MMPI-2 RC scales.     

Associations between Symptom Validity Test failure and scores on the MMPI-2-RF validity and substantive scales

This study examined the association between Symptom Validity Test (SVT) failure and the Minnesota Multiphasic Personality Inventory-2 Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen, 2008), in the Forensic Disability Claimant samples described in the MMPI-2-RF Technical Manual (Tellegen & Ben-Porath, 2008 a, 2008b). SVTs used included the Word Memory Test (Green, 2003), the Computerized Assessment of Response Bias (Allen, Conder, Green, & Cox, 1997), the Medical Symptom Validity Test (Green, 2004), and the Test of Memory Malingering (Tombaugh, 1996). SVT failure was associated with significant elevations throughout the MMPI-2-RF overreporting validity scales and substantive scales. Pairwise contrasts between groups failing 0 and 3 SVTs revealed predominantly large effect sizes for the overreporting validity scales (d = 0.78-1.11), and many of the substantive scales, including the Cognitive Complaints (COG) scale. Results of this study demonstrate an association between SVT performance and elevated scores on the MMPI-2-RF. These results suggest that exaggeration of cognitive symptoms as demonstrated by SVT failure is also associated with overreported emotional, somatic, and neurocognitive complaints on the MMPI-2-RF.     

Further validation of the MMPI-2 and MMPI-2-RF Response Bias Scale: findings from disability and criminal forensic settings

The present study extends the validation of the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) and the Minnesota Multiphasic Personality Inventory-2 Restructured Form (MMPI-2-RF) Response Bias Scale (RBS; R. O. Gervais, Y. S. Ben-Porath, D. B. Wygant, & P. Green, 2007) in separate forensic samples composed of disability claimants and criminal defendants. Using cognitive symptom validity tests as response bias indicators, the RBS exhibited large effect sizes (Cohen's ds = 1.24 and 1.48) in detecting cognitive response bias in the disability and criminal forensic samples, respectively. The scale also added incremental prediction to the traditional MMPI-2 and the MMPI-2-RF overreporting validity scales in the disability sample and exhibited excellent specificity with acceptable sensitivity at cutoffs ranging from 90T to 120T. The results of this study indicate that the RBS can add uniquely to the existing MMPI-2 and MMPI-2-RF validity scales in detecting symptom exaggeration associated with cognitive response bias.     

Bridging the divide: comments on the Restructured Clinical Scales of the MMPI-2

In this comment, I discuss the target articles written by Nichols (2006/this issue) and Rogers, Sewell, Harrison, and Jordan (2006/this issue) regarding the Restructured Clinical (RC) Scales (Tellegen et al., 2003) of the MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989). Both articles provide thoughtful discussion of the extent to which the RC Scales achieved the primary goals set forth by Tellegen et al., but they set different tones and come to somewhat different conclusions. I argue that the initial studies conducted on the RC Scales suggest a promising future for them. However, further studies are needed to explore the construct validity of the scales, assess their incremental validity with respect to other more established MMPI-2 scale sets, and determine their interpretive meaning in clinical contexts.     

An investigation of the psychometric properties of the MMPI-2 Restructured Clinical (RC) scales with mental health inpatients

In this study, we compared the internal psychometric properties and external correlates of the MMPI-2 (Butcher et al., 2001; Tellegen et al., 2003) Restructured Clinical (RC) scales and corresponding MMPI-2 non-K-corrected Clinical scales in a sample of 280 men and 264 women admitted to inpatient psychiatric units. We evaluated the internal psychometric properties of the RC and Clinical scales by intercorrelation analyses and alpha coefficients. We investigated external correlates of scores from the RC and Clinical scales using selected chart variables, selected Symptom Checklist 90-Revised (Derogatis, 1983, 1994) items, and clinician ratings on the Brief Psychiatric Rating Scale (Overall & Gorham, 1988). Results showed psychometric support for the RC scales in terms of improved intercorrelation patterns and comparable or greater alpha coefficients. Further, external correlational analyses revealed varying degrees of evidence of improvement in convergent and discriminant validity.     

Susceptibility of the MMPI-2 clinical, restructured clinical (RC), and content scales to overreporting and underreporting

The authors examined and compared the susceptibility of three Minnesota Multiphasic Personality Inventory-2 (MMPI-2) scale sets (Clinical, Restructured Clinical [RC], and Content) to over- and underreporting using five analog samples. Two samples of 85 and 191 undergraduate students, respectively, took the MMPI-2 under underreporting versus standard instructions. Three samples consisting of 42 undergraduates, 73 psychiatric inpatients, and 84 medical patients took the MMPI-2 under overreporting versus standard instructions. A comparison of the effect sizes across the three sets of scales indicated that Clinical Scale scores are not less susceptible to distortion than the Content or RC Scales. An apparent lesser susceptibility to underreporting for the Clinical Scales was an artifact of the subtle items' effect on these scales.     

Effectiveness of the MMPI-2-RF Validity Scales for Feigned Mental Disorders and Cognitive Impairment: A Known-Groups Study

The MMPI and MMPI-2 validity scales have long been accepted as standard tools in the assessment of feigned mental disorders (FMD) based on their extensive empirical validation. Studies are now examining MMPI-2-RF with modified validity scales plus the new Infrequent Somatic Responses Scale (F_S) and the recently-adapted Response Bias Scale (RBS). The current investigation used a known-groups design to examine the effectiveness of the MMPI-2-RF for differentiating FMD and feigned cognitive impairment (FCI) from patients with genuine disorders for a large civil forensic sample. Criterion measures included the Structured Interview of Reported Symptoms-2 (SIRS-2) for the FMD group, and below-chance performances on the Victoria Symptom Validity Test (VSVT) and the Test of Memory Malingering (TOMM) for the FCI group. For FMD, both F-r and F_P-r produced very large effect sizes (ds > 2.00). Moreover, the absence of severe elevations (≥80 T) on F-r proved effective at ruling-out most FMD. For the current study, a F_P-r cut score ≥90 T for FMD produced virtually no false-positives (0.01) and only a moderate level of false-alarms. As predicted by its detection strategies, most MMPI-2-RF validity scales have limited effectiveness with the FCI group. However, FBS-r and RBS may be useful in conjunction with other clinical data for ruling out FCI for genuine neuropsychological consults. An entirely separate concern is whether certain diagnostic groups, such as major depression, will have marked elevations on MMPI-2-RF scales thereby increasing the likelihood of false-positives. On this point, F_P-r performed exceptionally well with unelevated scores (Ms < 55 T) consistently across diagnostic categories.     

A comparison of MMPI--2 measures of psychopathic deviance in a forensic setting

We examined the convergent and discriminant validity of the Minnesota Multiphasic Personality Inventory--2 (MMPI--2) measures of psychopathy, including the Clinical Scale 4, Restructured Clinical Scale 4 (RC4), Content Scale Antisocial Practices (ASP), and Personality Psychopathology Five Scale Disconstraint (DISC). Comparisons of the empirical correlates of these scales were conducted with 2 samples of participants evaluated at a criminal court clinic. The 2 samples included 59 men and 19 women and 913 men and 327 women, respectively. Two types of criteria (clinician ratings and archival record review) were utilized in the analyses. Relative to Clinical Scale 4, RC4 had significantly greater convergent validity in predicting psychopathy as measured by the Psychopathy Checklist--Screening Version (S. D. Hart, D. N. Cox, & R. D. Hare, 1995) and behavioral criteria associated with this construct. RC4 also showed substantially improved discriminant validity when compared with its Clinical Scale counterpart. Among all the MMPI-2 scales studied, RC4 was the best measure of the social deviance traits of psychopathy.     

What do the MMPI-2 restructured clinical scales reliably measure? Answers from multiple research settings

The Restructured Clinical (RC; Tellegen et al., 2003) scales were developed to improve measurement of the core constructs of the MMPI-2 (Butcher et al., 2001) Clinical scales by removing "demoralization," hypothesized to affect these scales adversely. Using 25 samples with MMPI-2 responses from 78,159 subjects across diverse clinical settings, we found that each RC scale was highly correlated with a Supplementary, Content, or Personality Psychopathology 5 (PSY-5; Harkness, McNulty, & Ben-Porath, 1995) scale: higher, in fact, than the correlation between the RC scale and its parent scale. Furthermore, for over half the RC scales (i.e., RC1, RC3, RC7, RC8, and RCd), the correlations were strong enough to conclude that the RC scales replicate MMPI-2 scales with rich empirical foundations; the remaining RC scales were not redundant. Next, we examined reliability estimates using alpha coefficients and interitem correlations and did not reveal superior reliability for most of the RC scales over existing MMPI-2 scales.     

The MMPI-2 Restructured Clinical (RC) Scales and restraints to innovation, or "what have they done to my song?"

Rogers, Sewell, Harrison, and Jordan (2006/this issue) largely replicate in an independent clinical sample the MMPI-2 Restructured Clinical (RC) Scales developed by Tellegen et al. (2003). Nichols (2006/this issue) raises numerous concerns about the development and utility of the RC Scales, which on close appraisal did not change our view that the scales are well conceived and potentially valuable to researchers and clinicians alike. We present two case studies in which the RC Scales helped clarify complex MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) profiles with multiple elevations on the Clinical, Content, Supplementary, and Personality Psychopathology Five (Harkness, McNulty, Ben-Porath, & Graham, 2001) scales. When interpretations refined by the RC Scales were discussed with the clients, each seemed to feel deeply understood. Reservations about instrument innovation can be appreciated as helping to counterbalance change and thereby ensure the MMPI-2's successful ongoing evolution. We discuss specific ways the MMPI-2 community could avoid polarization about the RC Scales.     

The deconstruction of the Hy Scale of MMPI-2: failure of RC3 in measuring somatic symptom expression

The MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) Clinical Scales have a long history in psychological assessment. Recently, Tellegen et al. (2003) conducted a series of analyses to restructure the scales to reduce what they considered to be problems that limit scale functioning. In a critique of the Restructured Clinical (RC) Scales published in this issue, Nichols (2006/this issue) questions a number of aspects of the approach Tellegen et al. took including their theoretical assumptions, methods of analysis, and failures to report important information needed for scale evaluation such as relationships with existing scales. We concur with many points raised by Nichols. In our analysis of the performance of the RC3 scale, we found that it has "drifted" so far from the original Hy scale as to be a completely different measure- a scale of cynical attitudes that is already well represented in existing MMPI-2 measures. In this article, we take these concerns a step further and examine the history and construct validity of the Hy scale in evaluating the somatic expression of problems that the original authors (McKinley & Hathaway, 1944) intended. We also include new information from a medical setting, an application not represented in Tellegen et al.'s RC Scale monograph. In agreement with Rogers et al. (2006/this issue), it is our conclusion that some RC Scales do not represent the measurement domain of the original scales and should not be relied on for or used to refine traditional interpretation, particularly in medical or forensic situations (such as personal injury cases) because of their confusing and conflicting results.     

Differentiating Bipolar Disorder from Major Depressive Disorder Using the MMPI-2-RF: A Receiver Operating Characteristics (ROC) Analysis

We examined the utility of the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF) to differentiate patients with lifetime bipolar disorder (BD) from patients with lifetime major depressive disorder (MDD), and to differentiate patients with BD who are currently depressed from patients with current MDD. Sixty-one patients with BD (BD-I n = 51, BD-II n = 10) and 381 patients with MDD were administered the MMPI-2; MMPI-2-RF scale scores were derived from these MMPI-2 protocols. Receiver operating characteristics analysis revealed that the MMPI-2-RF Activation (ACT) scale had the largest Area Under the Curve (AUC), which was 0.74. Using a cut-off score of 4 on the ACT scale resulted in 71% of patients being correctly classified as having BD or MDD (sensitivity = 0.67, specificity = 0.71). An examination of currently depressed patients with BD (n = 29) and a randomly selected sample of MDD patients (n = 29) revealed that the ACT scale correctly classified 72% of patients (AUC = 0.75, sensitivity = 0.69, specificity = 0.76). Implications and limitations of these findings were discussed.     

The MMPI-2 Restructured Clinical Scales: a paradigmatic shift in scale development

Tellegen et al. (2003) proposed fundamental changes in MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) scale development by discarding empirical scale development in favor of construct validation via Jackson's (1970) sequential system of scale development. As a result of their efforts, a general distress factor (Demoralization) was identified and 8 Restructured Clinical (RC) Scales were developed. Using 7,330 clinical cases from Caldwell's (1997) data set, in this study, we sought to cross-validate the MMPI-2 RC Scales. Scale homogeneity was confirmed with high alpha coefficients and interitem correlations in the expected range. We also achieved a major objective of reducing interscale correlations. In replicating Tellegen et al.'s principal components analysis, we achieved a high concordance for 6 of the 8 RC Scales. We critically examine these results in light of Jackson's construct validation. We discuss the clinical usefulness of the MMPI-2 RC Scales within the context of current and future research.     

The Personality Psychopathology-Five (PSY-5): recent constructive replication and assessment literature review

The Personality Psychopathology-Five (PSY-5; Harkness & McNulty, 1994) is a model of individual differences relevant to adaptive functioning in both clinical and non-clinical populations. In this article, we review the development of the PSY-5 model (Harkness, 1992; Harkness & McNulty, 1994) and discuss the ways in which the PSY-5 model is related to and distinct from other 5-factor models. Using different methods and measures, the dimensions of the PSY-5 model have been constructively replicated (Lykken, 1968) by Tackett, Silberschmidt, Krueger, and Sponheim (2008) and by Watson, Clark, and Chmielewski (2008), and dimensions congruent with the PSY-5 have even been suggested for the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; Krueger et al., 2011). PSY-5 Scales can be scored from the Minnesota Multiphasic Personality Inventory-2 (MMPI-2; Butcher et al., 2001), the MMPI-Adolescent version (MMPI-A; Butcher et al., 1992), and the Restructured Form of the MMPI-2 (MMPI-2-RF; Ben-Porath & Tellegen, 2008). Because the largest body of research exists for the MMPI-2-based scales, we focus our review of the literature on the MMPI-2-based PSY-5 scales (Harkness, McNulty, & Ben-Porath, 1995), but we briefly cover the small, but growing, body of MMPI-A and MMPI-2-RF PSY-5 scales research. We show that the PSY-5 research literature includes a wide variety of psychometric methodologies as well as diverse samples and clinical problems. An integrative summary reprises the theory behind each PSY-5 construct and links it to the reviewed literature. Advantages and limitations of MMPI-2-based PSY-5 scales are discussed.     

Syndromal complexity, paradigm shifts, and the future of validation research: comments on Nichols and Rogers, Sewell, Harrison, and Jordan

In this comment, I address a number of the points raised in the reviews of the MMPI-2 Restructured Clinical (RC; Tellegen et al., 2003) Scales by Nichols (2006/this issue) and Rogers, Sewell, Harrison, and Jordan (2006/this issue), and I advocate for changes in assessment validation research. There is little evidence that the "syndromal complexity" Nichols ascribes to the original MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) Clinical Scales is worth preserving. Although their construction does not constitute the paradigm shift claimed by Rogers et al., the RC Scales are promising, psychometrically defensible measures of core features of the original MMPI-2 Clinical Scales. However, validation of inferences from multiscale inventories such as the MMPI-2 is limited at present by a disconnection between the integrative manner in which MMPI-2 profiles are interpreted and the scale-by-scale nature of most MMPI-2 validation studies. Q-sort procedures show promise for operationalizing integrated MMPI-2 interpretations, with both research and teaching applications.