Errors in scoring objective personality tests

Given the paucity of previous research, we examined the occurrence of scoring error on widely used objective personality tests and examined its possible relation to two factors: scoring procedure complexity (SPC) and commitment to accuracy (CTA). We double-checked the scoring of three tests (MMPI, Beck Depression Inventory, Spielberger State/Trait Anxiety Inventory) across three settings. Each of the tests were misscored at a surprisingly high rate in at least one setting, and some such errors altered major interpretive implications. Tests of higher SPC showed greater error rates, but high CTA greatly reduced the occurrence of error across levels of SPC. Unexpected sources of error were also uncovered, such as commercial computer scoring errors and disagreement in scoring standards among test publishers. Practical suggestions for improving scoring accuracy are offered.     

Personal problem-solving scoring of the TAT: sensitivity to training

The Thematic Apperception Test (TAT; Murray, 1943) is one of the most frequently taught and administered projective instruments in the United States. Nevertheless, formal scoring systems are not often used when evaluating story responses; thus, the reliability and validity of response interpretations remain controversial. The sensitivity-to-training of a procedure that uses the TAT to assess personal problem-solving skills, namely the Personal Problem-Solving System (PPSS; Ronan, 1990), was evaluated. In Experiment 1, undergraduate students received either detailed (n = 30) or minimal (n = 29) training in the problem-solving skill of generating alternative solutions to personal problems and subsequently wrote responses to three TAT cards. In Experiment 2, subjects received either detailed (n = 22) or minimal (n = 22) training in a utility model of decision making and thereafter responded to the same three TAT cards. Subjects who received detailed training, when compared to their minimally trained counterparts, reported an increase in their knowledge of the relevant problem-solving skill, and this increased knowledge influenced their responses to the TAT cards as measured by the PPSS scores. Implications for the construct validity of the PPSS are discussed.     

Personal problem-solving system for scoring TAT responses: preliminary validity and reliability data

Personal problem solving has emerged as an important construct in the cognitive-behavioral literature, yet there is a lack of clinically useful, performance-based measures practitioners can use to assess the personal problem-solving skills of their clients. Two studies evaluated the validity and reliability of a scoring system for measuring personal problem-solving processes via the Thematic Apperception Test (TAT; Morgan & Murray, 1935). In Experiment 1, undergraduate students (N = 87) completed two measures of personal problem solving, as well as three TAT cards, which were scored using the Personal Problem-Solving System (PPSS; Renan, 1990). In Experiment 2, an additional group of undergraduates (N = 56) responded to three TAT cards on two separate occasions and also completed a different measure of personal problem solving. Results from both studies supported the use of the PPSS for scoring TAT responses to assess personal problem-solving processes. Suggestions for future research are highlighted.     

The creativity quotient: An objective scoring of ideational fluency

A classical test for accessing the potential creativity of an individual is based on ideational fluency, where a person is asked to generate all possible uses for a familiar item like a piece of paper. In scoring the results, it is intuitive that the suggested uses should not be weighted equally. Those suggested in radically different categories are “worth more”; than those suggested within the same category only. We used information theory to derive a simple mathematical expression for a more objective measure of ideational fluency. We call this the creativity quotient (CQ). This innovative measure was examined using a small sample of participants, and is illustrated by the responses of two typical individuals from an ideational fluency task. The CQ accounts for the number of ideas (fluency), plus the number of categories (flexibility). Ongoing research will examine the independence of CQ from established measures of intelligence and personality.     

Clinical assessment of object relations using the TAT

Despite its widespread use, the Thematic Apperception Test (TAT) has rarely been used systematically as a clinical instrument. This article describes a multidimensional measure devised for the clinical assessment of object relations using the TAT. Four scales assessing different dimensions of object relations, which have been validated in several studies, are described: Complexity of Representations of People, Affect-Tone of Relationship Paradigms, Capacity for Emotional Investment in Relationships and Moral Standards, and Understanding of Social Causality. Use of the scales is illustrated by applying them to TAT responses of a borderline patient and a normal subject.     

An objective procedure for the hand scoring of scalp average evoked potentials

A method is described for identifying and scoring average evoked response (AER) components. The method uses background EEG activity as a guide for visual identification of AER components. This procedure provides an advantage in objectivity and precision over most other methods, thus producing an increase in reliability and greater likelihood for interlaboratory agreement in experiments employing measures based upon the AER components.     

Psychological systems questionnaire: An objective personality test designed for on-line computer presentation,scoring, and interpretation

Previous research efforts to use on-line computer systems for personality assessment are briefly reviewed. Shortcomings in the conversion of paper-and-pencil testing forms to computer media are noted. Construction of a new instrument, specifically designed for use with an on-line computer system, is described. Advantages of this approach are noted.     

A Basic TAT Set

The 10 TAT cards judged most valuable for a basic test set were selected by a total of 170 highly experienced psychologists. The judges' choices, separately determined for adults and children, were very consistent. For example, 86.7% of judges in the adult series and 92.5% of judges in the child series listed Picture 1 (Boy-Violin) within their first 10 choices. Rankings of the 10 most frequently chosen cards were closely parallel in the adult and child series. Utilizing these findings a recommended Basic TAT set of eight cards (Pictures 1, 2, 3BM, 4, 6BM, 7BM, 13MF, 8BM) is proposed to enhance development of the TAT for clinical research and teaching purposes.     

Exploring fantasies of TAT reliability

It has been argued that low TAT stability estimates are an artifact of retest instructions to be original and creative. This view was supported in a recent study by Winter and Stewart (1977) who found that test-retest reliability was higher when subjects were told on the second occasion to write the same stories than when they were told to write different stories. This study attempted to replicate Winter and Stewart's results and also to determine whether the similarity of stories did vary as hypothesized. The replication was only partially successful. More importantly, it was concluded that while instructions to write similar stories did improve reliability, the increase reflected only subjects' memory capabilities and not greater stability in motive expression. While true TAT stability may still be commonly underestimated, altering retest instructions to improve reliability does not appear to be the solution.