Personal problem-solving scoring of the TAT: sensitivity to training

The Thematic Apperception Test (TAT; Murray, 1943) is one of the most frequently taught and administered projective instruments in the United States. Nevertheless, formal scoring systems are not often used when evaluating story responses; thus, the reliability and validity of response interpretations remain controversial. The sensitivity-to-training of a procedure that uses the TAT to assess personal problem-solving skills, namely the Personal Problem-Solving System (PPSS; Ronan, 1990), was evaluated. In Experiment 1, undergraduate students received either detailed (n = 30) or minimal (n = 29) training in the problem-solving skill of generating alternative solutions to personal problems and subsequently wrote responses to three TAT cards. In Experiment 2, subjects received either detailed (n = 22) or minimal (n = 22) training in a utility model of decision making and thereafter responded to the same three TAT cards. Subjects who received detailed training, when compared to their minimally trained counterparts, reported an increase in their knowledge of the relevant problem-solving skill, and this increased knowledge influenced their responses to the TAT cards as measured by the PPSS scores. Implications for the construct validity of the PPSS are discussed.     

Sociotropy and autonomy: preliminary evidence for construct validity using TAT narratives

In this investigation, we evaluated the construct validity of sociotropy and autonomy as assessed by the revised Personal Style Inventory (PSI; Robins et al., 1994). Stories given to 6 cards of the Thematic Apperception Test (Murray, 1943) were coded for need for Achievement (McClelland, Atkinson, Clark, & Lowell, 1953) and need for Affiliation (Heyns, Veroff, & Atkinson, 1958). These scores were correlated with PSI Sociotropy and Autonomy, along with their component subscales. The construct validity of Sociotropy, Autonomy, and 5 of 6 component subscales were supported as hypoth sized. Consistent with past research, there was no support for the construct validity of the Perfectionism/Self-Criticism subscale of Autonomy. In addition, separate analyses by gender suggested that the construct validity of sociotropy may be greater for women than for men. The results represent an important finding in that nonquestionnaire measures of interpersonal and achievement-related concerns were found to support the validity of the PSI, a need identified by the questionnaire's authors.     

Measuring socioeconomic status: reliability and preliminary validity for different approaches

This study investigated issues related to commonly used socioeconomic status (SES) measures in 140 participants from three cities (Atlanta, Boston, and Toronto) in two countries (United States and Canada). Measures of SES were two from the United States (four-factor Hollingshead scale, Nakao and Treas scale) and one from Canada (Blishen, Carroll, and Moore scale). Reliability was examined both within (interrater agreement) and across (intermeasure agreement) measures. Interrater reliability and classification agreement was high for the total sample (ranger = .86 to .91), as were intermeasure correlations and classification agreement (range r = .81 to .88). The weakest agreement across measures was found when families had one wage earner who was female. Validity data for these SES measures with academic and intellectual measures also were obtained. Some support for a simplified approach to measuring SES was found. Implications of these findings for the use of SES in social and behavioral science research are discussed.     

The MAVRIC-C and MAVRIC-P: a preliminary reliability and validity study

We administered the Measure of Aggression, Violence, and Rage in Children (MAVRIC; Bass, Geenens, & Popper, 1993a, 1993b), a questionnaire assessing the severity of reactive, impulsive aggression, to 28 prepubertal psychiatrically hospitalized children and 54 prepubertal lowrisk nonpatients and their mothers. Cross-informant reliability was supported between the MAVRIC-Child Version (MAVRIC-C; Bass et al., 1993a) and MAVRIC-Parent Version (MAVRIC-P; Bass et al., 1993b), r = .62, p < .001. Convergent validity was supported with the Aggressive Behavior factor of the Child Behavior Checklist (Achenbach, 1991a) for the MAVRIC-C, r = .62, p < .001, and MAVRIC-P, r = .74, p < .001. Both versions were also associated with disruptive behavior disorders, inpatient status, and assaultive behavior. We discuss the differences in the perception of aggression as a function of informant.     

Alternatives to switch-cost scoring in the task-switching paradigm: Their reliability and increased validity

In the task-switching paradigm, the latency switch-cost score—the difference in mean reaction time between switch and nonswitch trials—is the traditional measure of task-switching ability. However, this score does not reflect accuracy, where switch costs may also emerge. In two experiments that varied in response deadlines (unlimited vs. limited time), we evaluated the measurement properties of two traditional switch-cost scoring methods (the latency switch-cost score and the accuracy switch-cost score) and three alternatives (a rate residual score, a bin score, and an inverse efficiency score). Scores from the rate residual, bin score, and inverse efficiency methods had comparable reliability for latency switch-cost scores without response deadlines but were more reliable than latency switch-cost scores when higher error rates were induced with a response deadline. All three alternative scoring methods appropriately accounted for differences in accuracy switch costs when higher error rates were induced, whereas pure latency switch-cost scores did not. Critically, only the rate residual and bin score methods were more valid indicators of task-switching ability; they demonstrated stronger relationships with performance on an independent measure of executive functioning (the antisaccade analogue task), and they allowed the detection of larger effect sizes when examining within-task congruency effects. All of the three alternative scoring methods provide researchers with a better measure of task-switching ability than do traditional scoring methods, because they each simultaneously account for latency and accuracy costs. Overall, the three alternative scoring methods were all superior to the traditional latency switch-cost scoring method, but the strongest methods were the rate residual and bin score methods.     

The new MAACL scales with adolescents: preliminary reliability and validity determinations

This study reports on the reliability and concurrent validity of both the trait and state forms of the Multiple Affect Adjective Check List (MAACL) with a high school population, age range of 14-16 years and educational range of 9th and 10th grades (N = 403). Alphas of the four scales (anxiety, depression, hostility, and positive affect) ranged from .67 to .91. Correlations with a check-list of stress-related symptoms and an inventory of the number of extracurricular activities in which the students participated were significant and in the expected direction. Means and standard deviations on the MAACL scales were similar to those of college students. Sex differences were found on some of the scales. The findings indicate that the MAACL is sufficiently reliable and valid to warrant additional use with an adolescent population.     

The developmental profile: preliminary results on interrater reliability and construct validity

This study presents the preliminary results of research into the interrater reliability and construct validity of the Developmental Profile (DP). In the DP a number of developmental lines, such as Object-Relations, Self-Images, and Problem-Solving Capacities, are assessed and classified according to the level of functioning. A total of 108 profiles were assessed, drawn from three different categories of patients. The weighted kappa values for interrater reliability were sufficient. On the adaptive level, but also on the maladaptive levels Symbiosis and Resistance, significant differences were found between psychiatric patients, "normal controls" (dental patients) and somatic patients. No differences were recorded between the latter two groups. The conclusion is that the DP is a promising instrument, of which the reliability and validity has to be further investigated in order to contribute to scientific support for psychodynamic theory formation.     

Measures of Hindu pathways: development and preliminary evidence of reliability and validity

The authors examined religious practices of Hindus in the United States and developed measures of their religious pathways. Based on reviews of the psychology of religion, Hindu literature, and interviews with Hindus (N=15), 4 religious pathways were identified: devotion, ethical action, knowledge, and physical restraint/yoga. Items reflecting these pathways were generated and administered to a pilot sample (N=42) and then mailed to a sample across the United States (N=182). Consistent with Hindu theology, participants endorsed 4 religious pathways. Results indicated that the measures of the religious pathways possessed adequate psychometric properties and were predictive of mental and physical well-being. Additional findings emphasized the need to attend to age, marital status, and acculturation when studying religious practices among Hindus.     

PyParse: A semiautomated system for scoring spoken recall data

Studies of human memory often generate data on the sequence and timing of recalled items, but scoring such data using conventional methods is difficult or impossible. We describe a Python-based semiautomated system that greatly simplifies this task. This software, called PyParse, can easily be used in conjunction with many common experiment authoring systems. Scored data is output in a simple ASCII format and can be accessed with the programming language of choice, allowing for the identification of features such as correct responses, prior-list intrusions, extra-list intrusions, and repetitions.     

The reliability and validity of a system for family assessment

The Darlington Family Assessment System (DFAS) is based upon the principles of multisystem-multimethod (MSMM) assessment. In practice it consists of a structured family interview with an integrated rating scale, a number of self-report questionnaries, and a task with an integrated behaviour coding system. This article summarizes the results of a series of empirical evaluations of the DFAS, which concern evaluations of the system as an aid to clinical work with families and as a method for training (at a basic level) in family assessment. The results are presented in terms of their implications for the reliability and validity of the assessment system and discussed from the perspective of their generalizability.     

A system for assessing personal responsibility: validity, reliability and rater trainability

Summary: Sixty-two subjects completed the California Psjrchological Inventory, the Rotter External-Internal locus of control scale and an audio-taped discussion of their personal problems. The audio-taped problems were rated on a five-point level of personal responsibility scale and were compared wi.th the scores on the California Personality Inventory and the Internal-External locus of control scale in a correlation matrix which was subjected to a factor analysis. The results from these analyses suppovted the hypothesis that the Personal Responsibility Rating System has construct validity as a measure of psychological health. Study I1 assessed the trainability of the Personal Responsibility System. With a four-hour training program it was found that graduate students could be taught to rate personal responsibility in a reliable manner.     

Consensus scoring, correct responses and reliability of the MSCEIT V2

The scoring protocol adopted by the MSCEIT V2 has been criticised since its development. The present study raises questions regarding the value of consensus scoring by analysing responses within the categorical subscales of Changes and Blends using the Optimal Scaling technique within the Categorical Principal Components Analysis (CATPCA) via Statistical Package for the Social Sciences (SPSS), (n = 206). On a number of occasions, there was no clear agreement as to the “correct” response to items within these categorical subscales. Such an issue seems integral to the application of the MSCEIT V2 and one which deserves more attention. On a more positive note, improvements were made to the reliabilities of the subscales of Changes and Blends, using Optimal Scaling, but less so for Changes. Nevertheless, this raises the possibility of improving the reliabilities of other subscales in the MSCEIT V2 and in turn improving the power of subsequent statistical tests.     

Personal Problem-Solving Scoring of the TAT: Sensitivity to Training

The Thematic Apperception Test (TAT; Murray, 1943) is one of the most frequently taught and administered projective instruments in the United States. Nevertheless, formal scoring systems are not often used when evaluating story responses; thus, the reliability and validity of response interpretations remain controversial. The sensitivity-to-training of a procedure that uses the TAT to assess personal problem-solving skills, namely the Personal Problem-Solving System (PPSS; Ronan, 1990), was evaluated. In Experiment 1, undergraduate students received either detailed (n = 30) or minimal (n = 29) training in the problem-solving skill of generating alternative solutions to personal problems and subsequently wrote responses to three TAT cards. In Experiment 2, subjects received either detailed (n = 22) or minimal (n = 22) training in a utility model of decision making and thereafter responded to the same three TAT cards. Subjects who received detailed training, when compared to their minimally trained counterparts, reported an increase in their knowledge of the relevant problem-solving skill, and this increased knowledge influenced their responses to the TAT cards as measured by the PPSS scores. Implications for the construct validity of the PPSS are discussed.     

A consideration of the validity and reliability of suicide mortality data

The question of the validity and reliability of suicide statistics may be considered at three levels: (1) Are suicide deaths misidentified or differentially identified across jurisdictions or over time? (2) To what degree are suicide deaths misidentified? and (3) Is the degree to which suicides are misidentified sufficient to threaten the validity of research based on suicide statistics? There is general agreement that suicides are likely to be undercounted, both for structural reasons (the burden-of-proof issue, the requirement that the coroner or medical examiner suspect the possibility of suicide) and for sociocultural reasons. There is also substantial anecdotal and empirical evidence suggesting that the mode of death for some true suicides is in fact certified as other than suicide. Overall, it does not seem that very many true nonsuicides are incorrectly certified as suicides. There is not, however, much agreement as to the degree to which true suicides are undercounted. At least some of the inconsistencies in the findings of different investigators arise because the validity of suicide certification seems to vary from place to place. But the source of apparent conflicts in many of the findings is undoubtedly the lack of a "gold standard" against which the verdicts of any given death certification process can be measured. At best, we can estimate that the sensitivity with which coroners and medical examiners certify true suicides varies from approximately 55% to 99%. A central question in estimating the sensitivity of suicide certification is this: What proportion of true suicides are either equivocal or likely to go unsuspected by the coroner or medical examiner? Very little has been done to investigate this issue. Yet the sensitivity of suicide certification clearly varies for equivocal versus unequivocal suicides. As shown in Table 1.2, specificity is also at issue when it comes to certifying equivocal cases. The final question--whether the degree of undercounting of suicide deaths is so great that it threatens the validity of research based on official statistics--is at the crux of the general concern about suicide certification. There are examples of studies in which conclusions based on crude comparisons of reported suicide statistics appear to be invalid. For the most part, these are comparisons among nations with substantially differing death certification procedures. When official statistics are interpreted with a degree of caution and an understanding of the source and direction of biases likely to affect the published rates, however, it seems unlikely that major conclusions based on these statistics will be in error.(ABSTRACT TRUNCATED AT 400 WORDS)