Measuring Mental Toughness in Sport: A Psychometric Examination of the Psychological Performance Inventory–A and Its Predecessor

The Rorschach Inkblot Method (RIM) is a performance-based personality assessment instrument used in both clinical and research settings worldwide. This investigation examines response-level, interrater reliability of U.S. graduate students enrolled in the same doctoral program and divided into two unique participant groups: 20 of those with previous Comprehensive System (CS) training background who are now trained in Rorschach Performance Assessment System (R-PAS; initial coders N = 11, blind coders N = 9), and 19 of those trained solely with R-PAS (initial coders N = 10, blind coders N = 9). Trainees’ coding was analyzed based on a sample of 50 protocols (half per group) representing a total of 1,189 responses. Blind coding was used among trainees and coding between the two investigated training groups was analyzed for each variable at the response level and expressed as percent agreement, Gwet's AC, and kappa values. Rates of agreement between the trainees were similar across both groups, and most of the concordance rates can be characterized as excellent. Overall, these results suggest that trainees may learn how to code R-PAS variables with similar reliability levels whether or not they had prior CS knowledge.     

Rorschach Comprehensive System data for a sample of 111 adult nonpatients from Romania

In this study Rorschach data from a Romanian sample of 111 respondents was collected and analyzed in a first attempt to establish national norms. The protocols were collected in a 5-year period (2002-2006) by the author. Interrater reliability statistics are presented for a sample of 20 cases, along with scores for the Rorschach Comprehensive System (CS; Exner, 1993). These results can be used for cross-cultural comparisons of the CS.     

The impact of administration and inquiry on Rorschach Comprehensive System protocols in a national reference sample

We investigated the impact of administration and inquiry skills on Rorschach Comprehensive System (CS; Exner, 1974, 1991, 1993) protocols collected for the Italian adult nonpatient reference sample. The same research team collected CS protocols on two occasions. The initial reference sample (N = 212; Lis, Rossi, & Priha, 1998) was collected under the supervision of experienced psychologists who carefully studied CS administration and scoring procedures (Exner, 1986, 1990, 1993). The second sample (N = 101; Lis, Zennaro, Calvo, & Salcuni, 2001) was collected after the team obtained additional and sustained CS training from Rorschach workshops certified instructors. Both samples were scored, reliably but they showed large differences on many codes, with protocols from the second sample being richer and more complex than the first. The results indicate that administration skills can have a dramatic impact on CS protocols and may contribute to variations in samples collected by different investigators. Training standards should be devised to insure uniform administration procedures are followed when collecting CS protocols.     

An Investigation of Interrater Reliability for the Rorschach Performance Assessment System (R–PAS) in a Nonpatient U.S. Sample

The Rorschach Performance Assessment System (R–PAS) aims to provide an evidence-based approach to administration, coding, and interpretation of the Rorschach Inkblot Method (RIM). R–PAS analyzes individualized communications given by respondents to each card to code a wide pool of possible variables. Due to the large number of possible codes that can be assigned to these responses, it is important to consider the concordance rates among different assessors. This study investigated interrater reliability for R–PAS protocols. Data were analyzed from a nonpatient convenience sample of 50 participants who were recruited through networking, local marketing, and advertising efforts from January 2013 through October 2014. Blind recoding was used and discrepancies between the initial and blind coders' ratings were analyzed for each variable with SPSS yielding percent agreement and intraclass correlation values. Data for Location, Space, Contents, Synthesis, Vague, Pairs, Form Quality, Populars, Determinants, and Cognitive and Thematic codes are presented. Rates of agreement for 1,168 responses were higher for more simplistic coding (e.g., Location), whereas agreement was lower for more complex codes (e.g., Cognitive and Thematic codes). Overall, concordance rates achieved good to excellent agreement. Results suggest R–PAS is an effective method with high interrater reliability supporting its empirical basis.     

A study of nonpatient preadolescent Rorschach protocols

There is a paucity of peer-reviewed, nonpatient Rorschach studies in psychological journals, particularly for children. This study examined 100 preteenage children, using the Comprehensive System to code their Rorschach protocols. The participants were also evaluated with the Conners Parent Rating Scale-93 (Conners, 1989) and 5 exclusionary behavior criteria. The 100 participants demonstrated better than average behaviors. The results presented include all the ratios and indexes found in the nonpatient statistics for the Comprehensive System. We discuss interrater reliability issues and offer recommendations for further cross-validating research.     

Beyond “Objective” and “Projective”: A Logical System for Classifying Psychological Tests: Comment on Meyer and Kurtz (2006)

Using a diverse sample of 4,786 protocols obtained with the Rorschach Performance Assessment System (Meyer, Viglione, Mihura, Erard, &; Erdberg, 2011), we provide a contemporary overview of how people organize Rorschach's inkblots into identifiable regions while formulating responses. After examining how frequently each location was used across all cards in this sample, we examined the consistency of their use by computing parallel information in 17 samples (N = 4,701) obtained using the Comprehensive System (CS; Exner, 2003), including clinical, nonclinical, and adult, child, and adolescent data. Even though the CS data could only record a single location for each response, the average correlation of location use across samples was .96. The results also document continuous dimensionality in use rather than any discontinuities that would demarcate a boundary between common and uncommon locations. Implications of this notable reproducibility and dimensionality are discussed for future conceptualization of location typicality, including location coding considerations and possibilities for improved measures of perceptual fit.     

Rorschach Comprehensive System data for a sample of 517 adults from Spain (Barcelona)

This study is composed of 517 adult voluntary participants, from Barcelona and its surroundings, all of whom are from Spain, Barcelona, and Catalonia. The study builds on a previously published report for a smaller sample (N = 250; see Alvarez, Baeza, Campo, Garcia, Guardia, et al., 1993) and is the result of collaboration between students and professors at the Barcelona Rorschach School (Catalan Rorschach Society). Our aim is to offer local data that should help to understand possible differences with existing reference data for the Rorschach Comprehensive System (CS; Exner, 2005), which is the system that has been taught at this school since 1978. Inclusion and exclusion criteria are described and interrater reliability statistics at the response level are presented, along with scores for the CS, followed by a discussion and very brief comparison with other international reference samples.     

Rorschach Intercoder Reliability for Protocol-Level Comprehensive System Variables in an International Sample

This study examines the intercoder reliability of Rorschach Comprehensive System (CS; Exner, 2001) protocol-level variables. A large international sample was combined to obtain intercoder agreement for 489 Rorschach protocols coded using the CS. Intercoder agreement was calculated using an Iota coefficient, a statistical coefficient similar to kappa that is corrected for chance. Iota values for the variables analyzed ranged from .31 to 1.00, with 2 in the poor range of agreement, 4 in the fair range, 25 in the good range, and 116 in the excellent range of agreement. Discrepancies between variables are discussed.     

Rorschach intercoder reliability for protocol-level comprehensive system variables in an international sample

This study examines the intercoder reliability of Rorschach Comprehensive System (CS; Exner, 2001) protocol-level variables. A large international sample was combined to obtain intercoder agreement for 489 Rorschach protocols coded using the CS. Intercoder agreement was calculated using an Iota coefficient, a statistical coefficient similar to kappa that is corrected for chance. Iota values for the variables analyzed ranged from .31 to 1.00, with 2 in the poor range of agreement, 4 in the fair range, 25 in the good range, and 116 in the excellent range of agreement. Discrepancies between variables are discussed.     

Rorschach Comprehensive System data for a sample of 141 adult nonpatients from Denmark

A sample (n = 141) of Danish nonpatients 25-50 years of age, never hospitalized with a psychiatric diagnosis and currently employed, was demographically representative of two geographical areas of Copenhagen with different social strain. The sample, as well as 45 persons not currently employed, was tested with the Rorschach (Exner, 1995), MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989), Word Association Test (Ivanouw, 1999b), WAIS Comprehension subtest (Hess, 1974), and SCL-90-R (Olsen, Mortenson, & Bech, 2006). Half of the persons contacted volunteered for the study. There was no difference in rate of volunteering between a standard no-feedback condition and a feedback condition; the latter, however, tended to attract more psychologically resourceful persons. The employed persons tended to appear healthier than the not employed. Response style of the subjects, quality of the Rorschach protocols, reliability of scoring, and the effect of the data being grouped on geographical area and examiner were examined. Form level, color, texture, cooperative movement, and EA were lower than in the Comprehensive System (CS; n = 450) sample, but higher than in nine international nonpatient Rorschach studies. Unique for the Danish sample was a low number of animal movement answers. The Rorschach data showed women to be healthier than men. Differences in Rorschach variables based on educational level were small.     

Scoring accuracy using the comprehensive system for the Rorschach.

The Comprehensive System (CS; Exner, 1974, 1978) for scoring Rorschach responses is the most widely taught and most widely accepted system in use today. The complexity and labor- intensive nature of the CS makes the issue of scoring accuracy a central concern. Twenty-one graduate psychology students and 12 professionals scored 20 Rorschach responses drawn from normal and clinical protocols. In general. accuracy scores for both students and professionals were below acceptable levels. Accuracy scores were clearly better for the code categories of Location, DQ, Pairs, Popular, and Z than for Determinants, FQ, Content, and Special Scores. Responses from clinical protocols were subject to more error. The results suggest that high levels of scoring errors may exist in the field use of the CS. Training standards may need to be devised to insure scoring competence.     

The interclinician reliability of Rorschach interpretation in four data sets

To examine agreement on Rorschach Comprehensive System (CS; Exner, 2004) interpretations, 55 patient protocols were interpreted by 3 to 8 clinicians across 4 data sets on a representative set of 29 characteristics. Substantial reliability was observed across data sets, although a problematic design produced lower results in one. Unexpectedly, a Q-sort task had slightly lower reliability than a simple rating task. As expected, scales that summarized judgments had higher agreement than judgments to individual interpretive statements, and some clinicians produced more generalizable inferences than others. Interpretations for all clinicians were more strongly associated with patients' psychometric true scores (aggregated judgment M range = .82 to .92) than with the judgments of other clinicians (range = .76 to .89). Compared to meta-analyses of interrater reliability in psychology and medicine, the findings indicate these clinicians could reliably interpret Rorschach CS data.     

Rorschach Comprehensive System data for a sample of 283 adult nonpatients from the United States

This article presents an update on adult nonpatients living in central California previously described by Shaffer, Erdberg, and Haroian (1999). In this study an additional 160 nonpatients were administered the Rorschach for a total sample size of 283. Graduate students enrolled in a 2-year Rorschach research seminar administered the Rorschach (Exner, 1995), WAIS-R (Weschler, 1981), and MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989), and were provided with extensive supervision and ongoing quality control. All protocols were recoded by the first two authors and a psychologist with extensive Rorschach experience. Exclusion criteria are described. Interrater reliability statistics at the response level are presented along with scores for Rorschach Comprehensive System (CS), WAIS-R, and MMPI-2 variables.     

Rorschach Comprehensive System data for a sample of 190 Japanese nonpatient children at five ages

This study is a part of our series of studies of nonpatient, Japanese children. In total, 346 children were administered the Rorschach. Thirty-two children had fewer than 10 responses, 112 children had more than 10 but fewer than 14 responses, and 12 children were maladjusted or mentally retarded, for a total of 156. In this study, we analyzed the remaining 190 children not excluded by the above criteria in the following age groups: 5 years (n = 24), 8 years (n = 43), 9 years (n = 42), 12 years (n = 42), and 14 years (n = 39). Japanese children showed few responses generally, with an average of 18 ~ 20 responses. In addition to having fewer responses, other features of Japanese children are high Lambda, low EA, very high X-%, and low X+%. The fact that the Rorschach test depends on verbal expression may be one factor in all these features, and we should consider this study in view of these Japanese cultural factors. Interrater reliability findings and Rorschach Comprehensive System (CS; Exner, 1995; Matsumoto, 2003; Matsumoto et al., 2002) data are presented.     

Rorschach Comprehensive System data for 100 nonpatient children from the United States in two age groups

Building on our previously published study (Hamel, Shaffer, & Erdberg, 2000), which provided data on 100 nonpatient children aged 6 to 12 from the United States, we here provide reference data for two more homogeneous age subgroups: 6 to 9 (N = 50) and 10 to 12 (N = 50). Inclusion criteria are described, and expanded interrater reliability statistics at the response level are presented along with scores for the Rorschach Comprehensive System (CS; Exner, 2001) at each age grouping. In addition to the children being administered the Rorschach, their parents were given the Conners' Parent Rating Scale-93 (CPRS-93; Conners, 1989), and these results are presented as well.     

The Rorschach Comprehensive System Egocentricity Index in psychiatric inpatients

This paper reports an exploratory study of potential correlates of Exner's Rorschach Egocentricity Index, a measure of self-focusing, in a sample of 70 psychiatric inpatients. We examined the relationship of the Index of the MMPI Ego Strength scale and to other MMPI and Exner Comprehensive System Rorschach variables, using Pearson Product-Moment correlations with partialling of the number of Rorschach responses. There were seven meaningful significant correlations between the Index and Rorschach variables: M, FM, X +%, F +%, Lambda, D, and A%. The Index-MMPI correlations were not significant. Sex differences and differences between these intercorrelations and some reported by Exner (1983) are discussed.