Clinical judgment, clinical training, and professional experience

Reviews studies on training, experience, and clinical judgment. The results on the validity of judgments generally fail to support the value of on-the-job experience in mental health fields. The validity results do provide limited support for the value of training. Other results suggest that experienced clinicians are better than less experienced judges at knowing which of their judgments are likely to be correct and which are likely to be wrong. Reasons why clinicians have trouble learning from experience are given. Recommendations are made for improving training and clinical practice.     

Introduction to the special section on the use of computers for making judgments and decisions

Computers have the potential to greatly increase the accuracy of judgments and decisions that are made in the area of psychological assessment. The validity of present-day computer programs are described in the first 2 articles of this Special Section. Both computer-based test interpretation programs and mechanical prediction rules (e.g., statistical prediction rules) are described. Limitations of present-day computer programs are described in the 3rd article along with methodological recommendations for building more powerful rules. In the 4th article, a statistical analysis (neural network modeling) is described that may allow us to build better rules.     

Incremental validity and the assessment of psychopathology in adults

Results from studies on incremental validity are described for (a) interviews, (b) personality inventories, (c) projective techniques, and (d) brief self-rated and clinician-rated measures. In some of the studies (clinical judgment studies), psychologists were given increasing amounts of information. In other studies (statistical prediction studies), increasing amounts of assessment information were entered into a statistical prediction rule. Although relatively little research has been conducted on incremental validity, results that have been obtained tend to favor the use of interviews, personality inventories (e.g., the Minnesota Multiphasic Personality Inventory--2; J. N. Butcher, W. G. Dahlstrom, J. R., Graham, A. Tellegen, & B. Kaemmer, 1989), and brief self-rated measures (e.g., the Anxiety Sensitivity Index, S. Reiss, R. A. Peterson, D. M. Gursky, & R. M. McNally, 1986). Results are generally less encouraging for projective techniques.     

Cognitive complexity and the validity of clinicians' judgments

Efforts to explain why some neuropsychologists are more accurate than others have been unsuccessful. In the present study, the relation between the cognitive complexity of clinicians and the validity of their judgments was investigated. Forty-two neuropsychologists completed a measure of cognitive complexity. The hypothesis that neuropsychologists who are high in cognitive complexity will be more accurate than those who are low in cognitive complexity was not supported. It is not clear why cognitive complexity was not significantly related to the validity of diagnostic ratings: One explanation is that the measure of cognitive complexity used in this study was inadequate but another possibility is that neuropsychologists who are low in cognitive complexity may still be able to achieve high levels of accuracy if they attend to normative data when they interpret test results.     

The Validity of the Rorschach and the Minnesota Multiphasic Personality Inventory: Results from Meta-Analyses

Results from meta-analyses have been widely cited to defend the validity of the Rorschach. However, the meta-analyses have been flawed. For example, one meta-analysis included results that were obtained by calculating correlations but not results that were obtained by conducting t tests or analyses of variance. When we reanalyzed the data from the most widely cited meta-analysis (Parker, Hanson, & Hunsley, 1988), we found that for confirmatory studies (also called convergent-validity studies), the Minnesota Multiphasic Personality Inventory (MMPI) explained 23% to 30% of the variance, whereas the Rorschach explained only 8% to 13% of the variance. These results indicate that the Rorschach is not as valid as the MMPI.     

Judgment research and the dimensional model of personality

Comments on the original article "Plate tectonics in the classification of personality disorder: Shifting to a dimensional model," by T. A. Widiger and T. J. Trull. The purpose of this comment is to address (a) whether psychologists know how personality traits are currently assessed by clinicians and (b) the reliability and validity of those evaluations. Although Widiger and Trull argued that the shift to a dimensional model will improve the diagnosis of personality disorders, we can also expect that it will lead to improvements in evaluating personality traits.     

Computer-administered interviews and rating scales

To evaluate the value of computer-administered interviews and rating scales, the following topics are reviewed in the present article: (a) strengths and weaknesses of structured and unstructured assessment instruments, (b) advantages and disadvantages of computer administration, and (c) the validity and utility of computer-administered interviews and rating scales. Computer-administered evaluations are more comprehensive and reliable and less biased than evaluations routinely conducted in clinical practice. Also, the use of continuous monitoring systems, which increasingly entail the use of computer administration, has been related to improved treatment outcome. However, the use of computer-administered interviews and rating scales will sometimes lead to false positive diagnoses, and for this reason, it is recommended that computer assessment be combined with clinical judgment.     

Unresolved questions concerning the effectiveness of psychological assessment as a therapeutic intervention: comment on Poston and Hanson (2010)

In a recent article in this journal, Poston and Hanson (2010) reported a meta-analysis of 17 studies on the use of psychological assessment as a therapeutic intervention (PATI) and concluded that "psychological assessment procedures--when combined with personalized, collaborative, and highly involving test feedback--have positive, clinically meaningful effects on treatment" (Poston & Hanson, 2010, p. 203). Although extant data suggest that PATI can sometimes exert positive effects, Poston and Hanson's (2010) meta-analysis may overstate the magnitude of these effects because the authors (a) included several studies that combined assessment with treatment components that are irrelevant to PATI, sometimes rendering it impossible to attribute any observed effects to PATI per se and (b) excluded numerous nonsignificant results. Moreover, the studies Poston and Hanson (2010) reviewed neglected to rule out Barnum effects as alternative explanations for client improvement, raising the possibility that PATI works for reasons other than those proposed by its advocates. We conclude that Poston and Hanson's (2010) review leaves a number of lingering questions concerning the treatment utility of PATI unanswered.     

The Shedler-Westen Assessment Procedure-200 as a basis for modifying DSM personality disorder categories

The Shedler-Westen Assessment Procedure-200 (SWAP) is a Q-sort instrument designed to assess personality pathology on the basis of clinician ratings. On the basis of research with the SWAP, its creators have proposed a group of 12 personality disorder (PD) diagnoses that can be used to replace or modify current Axis II categories of the Diagnostic and Statistical Manual of Mental Disorders (DSM). The authors discuss conceptual and empirical issues that require clarification before this proposal can be properly evaluated. They identify problematic psychometric features of the SWAP, including its unrepresentative normative sample, its reliance on a fixed skewed distribution, and anomalies in its T-score approach to diagnoses. In addition, a review of research on SWAP-based PD categories indicates that important information regarding diagnostic coverage, validity, and temporal stability is presently lacking. The authors conclude that research evidence is currently insufficient to justify the use of SWAP-based PD categories to guide revision of the DSM.