Coming to grips with negative evidence for the Comprehensive System for the Rorschach: a comment on Gacono, Loving, and Bodholdt; Ganellen; and Bornstein

The Comprehensive System (CS; Exner, 1991, 1993) for the Rorschach is currently engulfed in controversy. This comment article responds to 3 articles by Rorschach proponents in this issue of the Journal of Personality Assessment. Contrary to the claims of Gacono, Loving, and Bodholdt (this issue), CS scores do not bear a well-demonstrated relationship to psychopathy, antisocial personality disorder, or conduct disorder. Contrary to the claims of Ganellen (this issue), both the original and the revised CS Depression Index (Exner, 1993) bear little or no relationship to depression diagnoses. Furthermore, the scoring reliability of some CS scores is problematic. Although we agree with Bornstein (this issue) that Rorschach scores generally bear little or no relation to psychiatric diagnoses or self-report questionnaires, we believe this lack of relationship tends to disconfirm hypotheses concerning the validity of the Rorschach. In the spirit of the philosopher Sir Karl Popper, the Rorschach community should not minimize negative evidence or engage in post hoc arguments to immunize the CS against falsification.     

Psychopathic and nonpsychopathic violent offenders on the rorschach: discriminative features and comparisons with schizophrenic inpatient and university student samples

We examined discriminant and convergent validity of theoretically relevant Rorschach Comprehensive System (CS) variables (Exner, 2003) and Meloy and Gacono's (1992) aggression variables in distinguishing between imprisoned violent offenders (VO) who were psychopathic (P-VO) and nonpsychopathic (NP-VO) under psychiatric treatment, schizophrenic inpatients (ISs), and university students (USs). A total of 7 of 12 variables discriminated significantly between P-VO and NP-VO, which suggests more aggressive, cognitive, and interpersonal disturbances among P-VO. We also found significant differences between VOs, ISs, and USs. Logistic regression analyses revealed that AgPast accumulated incrementally in the classification of P-VO versus NP-VO, and AgC accumulated incrementally in the classification of VO versus IS when entered after CS variables. The findings support the view that psychopathy is a distinctive form of antisocial personality disorder (Diagnostic and Statistical Manual of Mental Disorders; American Psychiatric Association, 1994) organized at a more severe pathological level.     

Predicting DMS-IV cluster B personality disorder criteria from MMPI-2 and Rorschach data: a test of incremental validity

Despite their frequent conjoint clinical use, the incremental validity of Rorschach (Rorschach, 1921/1942) and MMPI (Hathaway & McKinley, 1943) data has not been adequately established, nor has any study to date explored the incremental validity of these tests for predicting Diagnostic and Statistical Manual of Mental Disorders (4th ed. [DSM-IV]; American Psychiatric Association, 1994) personality disorders (PDs). In a reanalysis of existing data, we used select Rorschach variables and the MMPI PD scales to predict DSM-IV antisocial, borderline, histrionic, and narcissistic PD criteria in a sample of treatment-seeking outpatients. The correlational findings revealed alimited relation between Rorschach and MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) variables, with only 5 of 30 correlations reaching significance (p <.05). Hierarchical regression analyses showed that both the MMPI and Rorschach data add incrementally in the prediction of DSM-IV borderline and narcissistic PD total criteria scores. The findings were less clear for the incremental value of Rorschach and MMPI-2 data in predicting the total number of DSM-IV histrionic PD criteria, which were best predicted by Rorschach data, and antisocial PD criteria, which were best predicted by MMPI-2 data. In addition to providing evidence of the incremental validity of Rorschach data, these findings also shed light on the psychological characteristics of the DSM-IV Cluster B PDs.     

Predicting DSM-IV Cluster B Personality Disorder Criteria From MMPI-2 and Rorschach Data: A Test of Incremental Validity

Despite their frequent conjoint clinical use, the incremental validity of Rorschach (Rorschach, 1921/1942) and MMPI (Hathaway & McKinley, 1943) data has not been adequately established, nor has any study to date explored the incremental validity of these tests for predicting Diagnostic and Statistical Manual of Mental Disorders (4th ed. [DSM-IV]; American Psychiatric Association, 1994) personality disorders (PDs). In a reanalysis of existing data, we used select Rorschach variables and the MMPI PD scales to predict DSM-IV antisocial, borderline, histrionic, and narcissistic PD criteria in a sample of treatment-seeking outpatients. The correlational findings revealed a limited relation between Rorschach and MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) variables, with only 5 of 30 correlations reaching significance (p < .05). Hierarchical regression analyses showed that both the MMPI and Rorschach data add incrementally in the prediction of DSM-IV borderline and narcissistic PD total criteria scores. The findings were less clear for the incremental value of Rorschach and MMPI-2 data in predicting the total number of DSM-IV histrionic PD criteria, which were best predicted by Rorschach data, and antisocial PD criteria, which were best predicted by MMPI-2 data. In addition to providing evidence of the incremental validity of Rorschach data, these findings also shed light on the psychological characteristics of the DSM-IV Cluster B PDs.     

Rorschach variables related to blood glucose control in insulin-dependent diabetes patients

In this article, we explored relations between selected Rorschach variables and blood glucose control in Insulin-Dependent Diabetes Mellitus (IDDM) patients. Three domains of psychological functioning are taken into consideration: emotional distress, coping and problem solving, and cognitive efficiency. Seventy-one IDDM patients (38 men, 33 women; mean age 42.2 +/- 14.9 years) from an outpatient unit took the Rorschach Comprehensive System (Exner, 1986). Nine variables were selected as independent variables. Blood glucose, a dependent variable, was measured by the proportion of glycated hemoglobin. Results of 2 regression analyses show that Y and C' correlate to higher blood glucose. WSum C and the absence of texture related to lower levels of blood glucose. A confounding effect of complications was observed. The findings suggest that emotional factors should be accorded greater attention in behavioral self-regulations in diabetes.     

On Conducting Construct Validity Meta-Analyses for the Rorschach: A Reply to Tibon Czopp and Zeligman (2016)

We respond to Tibon Czopp and Zeligman's (2016) critique of our systematic reviews and meta-analyses of 65 Rorschach Comprehensive System (CS) variables published in Psychological Bulletin (2013). The authors endorsed our supportive findings but critiqued the same methodology when used for the 13 unsupported variables. Unfortunately, their commentary was based on significant misunderstandings of our meta-analytic method and results, such as thinking we used introspectively assessed criteria in classifying levels of support and reporting only a subset of our externally assessed criteria. We systematically address their arguments that our construct label and criterion variable choices were inaccurate and, therefore, meta-analytic validity for these 13 CS variables was artificially low. For example, the authors created new construct labels for these variables that they called “the customary CS interpretation,” but did not describe their methodology nor provide evidence that their labels would result in better validity than ours. They cite studies they believe we should have included; we explain how these studies did not fit our inclusion criteria and that including them would have actually reduced the relevant CS variables’ meta-analytic validity. Ultimately, criticisms alone cannot change meta-analytic support from negative to positive; Tibon Czopp and Zeligman would need to conduct their own construct validity meta-analyses.     

Reliability and validity of rorschach aggression variables with a sample of adjudicated adolescents

In this investigation, we assessed the reliability and validity of 5 Rorschach aggression variables (AG, A1, A2, AgC, and AgPast) in a sample of adjudicated, mostly conduct-disordered adolescents (N = 150). More specifically, we assessed the interrelationships of Rorschach aggression variables using correlational analyses and factor analysis and assessed the relationships between Rorschach variables and a measure of aggressive potential (the Manifest Aggression scale from the Jesness Inventory; Jesness, 1996) as well as a measure of real-world aggression/violence (the Violence Rating Scale-Revised [VRS-R]; Young, Justice, & Erdberg, 1997). Two of Gacono and Meloy's (1994) Rorschach aggression variables were dropped from the study (AgPot and SM) due to a low frequency of occurrence. All 5 of the remaining Rorschach aggression variables and the VRS-R were rated reliably, and factor analysis of the Rorschach variables revealed 2 distinct factors accounting for 71% of the total variance. Only the AgC variable concurrently predicted aggressive potential and aggressive/violent behavior. Results lend further support for the inclusion of AgC in the Comprehensive System's (Exner, 1993) list of Special Scores.     

Exploring MMPI-Rorschach Relationships

Limitations exist in the current literature examining Minnesota Multiphasic Personality Inventory (MMPI)—Rorschach interrelationships. In this article I attempt to identify several issues to be addressed by researchers in this area. These include restricting conclusions about MMPI—Rorschach relationships to research using the Comprehensive System; considering the relative reliability and validity of these tests; comparing the predictive power of the Rorschach and MMPI; examining the effects extreme outliers have on statistical analyses; and determining the appropriate level of analysis for research, the level of individual test scores, or psychological constructs. The implications of these issues for research design, statistical analysis, and applied clinical work are discussed.     

Geographical Directory of Membership of the Society for Projective Techniques United States and Territories

We examined empirical findings related to the integration of the Rorschach and the MMPI in assessing adolescents. Intercorrelations between 50 Rorschach variables and the 13 MMPI basic scales are reported for a clinical sample of I97 adolescents. Significant correlations only slightly exceeded the number expected by chance. A review of six additional studies of adolescent samples also generally revealed either very modest or nonsignificant relationships between the Rorschach and the MMPI. These findings leave open the possibility that combining data from the two instruments may increase incremental validity, an issue that should be assessed by using multivariate analyses. Guidelines for integrating Rorschach and MMPI data in clinical practices with adolescents are provided.     

罗夏综合系统的元分析*

为考察罗夏墨迹测验Exner综合系统常模的客观性和准确性,运用元分析的方法,将来自16个国家的18篇罗夏综合系统正常成人的研究结果,与Exner新版常模进行比较。结果显示,在综合系统的全部113个变量中,有23个变量在联合样本与Exner常模间存在显著差异,包括被试的资源、能力、认知水平和人际关系等方面,且均为Exner常模样本结果更加优秀。结论:新版罗夏综合系统常模具有一定的偏差,存在偏好倾向,使用时应注意避免对被试的低估甚至做出假阳性的诊断。     

A critique of Lilienfeld et al.'s (2000) "The scientific status of projective techniques"

Lilienfeld, Wood, and Garb (2000) published a largely negative critique of the validity and reliability of projective methods, concentrating on the Comprehensive System for the Rorschach (Exner, 1993), 3 systems for coding the Thematic Apperception Test (TAT; Murray, 1943) cards, and human figure drawings. This article is an effort to document and correct what I perceive as errors of omission and commission in the Lilienfeld et al. article. When projective measures are viewed in the light of these corrections, the evidence for the validity and clinical usefulness of the Rorschach and TAT methods is more robust than Lilienfeld et al. represented.     

Schizophrenic, depressive, and nonpatient personality organizations described by Rorschach factor structures

Rorschachs of 155 schizophrenics, 102 depressives, and 186 nonpatients were collected and scored using the Comprehensive System. Twenty-seven Rorschach variables considered central to interpretation were selected for study. Factor analyses and partial correlations controlling for number of responses, R, were generated in order to investigate the personality organization of each of the three groups. A three factor solution was found to be optimal for each group. Patterns of personality organization were discussed within the three groups and factor structures were compared and contrasted. While similarities occur between the three groups, each group also manifests a unique personality organization. These findings support the hypothesis that schizophrenics and depressives differ from nonpatients in kind, rather than degree of disorder.     

Rorschach Depression indices with children and adolescents: concurrent validity findings.

Exner (1983, 1986) developed and recently revised (1990b) a Rorschach Depression Index based on scores from variables in the Comprehensive System. This study evaluated both the original and the revised DEPIs for child and adolescent outpatient (n = 67) and inpatient (n = 99) samples in order to assess the diagnostic utility of these indices. There were no significant relationships between the original form and the revised form of the DEPI and clinical elevations on the Depression scale of the Personality Inventory for Children in the outpatient sample or treatment team diagnostic judgments in the inpatient sample. These findings sound a strong cautionary note for using only Rorschach Depression indices to diagnose depression in children and adolescents.     

A method to optimize the response range while maintaining Rorschach Comprehensive System validity

Brief and lengthy Rorschach records have been identified as common problems in protocol administration. Clinicians have debated how to prevent overly short and long records, but they have been reluctant to alter standardized administration for fear of introducing bias. The present study examines a nonintrusive method for constraining responses by prompting for an extra response when only one is offered per card and by removing the card after four responses are given. Among patients who typically produce brief records, consisting of a residential sample of civil and forensic patients with a range of disordered thinking, the alternative administration method demonstrated improved Comprehensive System validity in assessing thought disorder and eliminated the need to readminister the test due to fewer than 14 responses. The findings have clinical implications for protocol administration with thought-disordered populations that typically produce brief records.     

A Method to Optimize the Response Range While Maintaining Rorschach Comprehensive System Validity

Brief and lengthy Rorschach records have been identified as common problems in protocol administration. Clinicians have debated how to prevent overly short and long records, but they have been reluctant to alter standardized administration for fear of introducing bias. The present study examines a nonintrusive method for constraining responses by prompting for an extra response when only one is offered per card and by removing the card after four responses are given. Among patients who typically produce brief records, consisting of a residential sample of civil and forensic patients with a range of disordered thinking, the alternative administration method demonstrated improved Comprehensive System validity in assessing thought disorder and eliminated the need to readminister the test due to fewer than 14 responses. The findings have clinical implications for protocol administration with thought-disordered populations that typically produce brief records.     

Cross-Cultural Validation of the Rorschach Developmental Index

The Developmental Index (DI) has recently been introduced as a composite Rorschach measure of psychological development and maturation, which can be used both with the Comprehensive System (Exner, 2003), and with the recently developed Rorschach Performance Assessment System (Meyer, Viglione, Mihura, Erard, & Erdberg, 2011). As the DI is new, and its validity has not yet been investigated with independent non-U.S. samples, we tested the correlation between DI and age using 3 relatively large samples, 2 of which were from outside the United States (total N = 902). Other Rorschach variables presumably associated with maturation, such as complexity and productivity, were also investigated. As expected, the DI significantly correlated with age, with small variations across the 3 samples. Importantly, the correlation between DI and age remained statistically significant also after controlling for productivity (i.e., the number of responses) and complexity.     

Rorschach administration: a comparison of the effect of two instructions given to an inpatient sample of drug addicts

The effect of administering the Rorschach Inkblot Method under two instructional sets was compared on three classes of outcome variables: the frequency with which subjects asked questions about the test; the frequency of brief protocols (fewer than 14 responses); and 17 traditional Rorschach structural summary scores. Sixty subjects, obtained from three inpatient psychiatric clinics treating drug addicts, randomly received either the short pre‐testing instruction “What might this be?” originally developed by Herman Rorschach and recommended in the Comprehensive System, or a longer and more elaborated instruction, which for many years has been the standard instruction in Norway. Compared with the Norwegian instruction, the short instruction produced significantly more questions to the examiner about the test. For the other outcome measures no differences were observed.