不同类型的测评维度对评价中心结构效度的影响研究

本研究将测评维度分为行为能力和心理特质两组,考察对评价中心结构效度的不同影响。A公司216名部门副经理级员工接受了本次管理素质评价中心测评,采用公文筐、角色扮演和无领导小组讨论三种方法,测查行为能力和心理特质两组维度,各自有三个维度被一种以上的方法测量。多质多法和验证性因素分析的结果表明,评价中心以行为能力比以心理特质为测评维度结构效度好;以行为能力为测评维度时,会聚效度和区分效度都较好。     

A survey of neuropsychologists’ use of validity tests with children and adolescents

A web-based survey of validity test use by North American neuropsychologists was conducted, with 282 participants meeting inclusion criteria. Respondents indicated that they use a median of one stand-alone performance validity test (PVT), one embedded PVT, and one symptom validity test (SVT) per pediatric assessment. The vast majority of respondents indicated they give at least one PVT (92%) and at least one SVT (88%) during each pediatric assessment. A meaningful difference in validity use (i.e., at least a medium effect size) was only found for those who engage in forensic work, with those clinicians giving more stand-alone PVTs than those who do not conduct forensic work. The most frequently used validity measures in pediatric assessments are presented, as are reasons participants reported for both using and not using validity tests. Limitations and qualitative comparisons to other surveys on validity test use with adults are discussed.     

Introduction to the special section on incremental validity and utility in clinical assessment

This special section focuses on the incremental validity and utility of clinical assessment data. The lack of replicated incremental validity research limits the ability of psychologists to establish their assessment practices on a solid empirical footing. The articles in this section deal with conceptual, methodological, and content issues in the development and use of clinical measures. The authors addressed several aspects of incremental validity research, including (a) the evaluation of the magnitude of validity increments, (b) the use of incremental validity data for test development and validation, (c) the costs of assessment, (d) the use of multi-informant and multimethod assessments, and (e) the treatment utility of assessment. Obstacles limiting the use of incremental validity research to inform and guide clinical practice are also emphasized.     

Model-Based Approaches for Teaching and Practicing Personality Assessment

Psychological assessment is a complex professional skill. Competence in assessment requires an extensive knowledge of personality, neuropsychology, social behavior, and psychopathology, a background in psychometrics, familiarity with a range of multimethod tools, cognitive flexibility, skepticism, and interpersonal sensitivity. This complexity makes assessment a challenge to teach and learn, particularly as the investment of resources and time in assessment has waned in psychological training programs over the last few decades. In this article, we describe 3 conceptual models that can assist teaching and learning psychological assessments. The transtheoretical model of personality provides a personality systems-based framework for understanding how multimethod assessment data relate to major personality systems and can be combined to describe and explain complex human behavior. The quantitative psychopathology—personality trait model is an empirical model based on the hierarchical organization of individual differences. Application of this model can help students understand diagnostic comorbidity and symptom heterogeneity, focus on more meaningful high-order domains, and identify the most effective assessment tools for addressing a given question. The interpersonal situation model is rooted in interpersonal theory and can help students connect test data to here-and-now interactions with patients. We conclude by demonstrating the utility of these models using a case example.     

Validity of self-reports of alcohol and other drug use: a multitrait-multimethod assessment

Four methods (self-reported rating, self-reported intake, peer-reported rating, peer-reported intake) and multiple analytical criteria were employed to assess the construct validity of reports of three types of substance use (alcohol, marijuana, and cigarettes) in a study of 194 college students who were members of friendship pairs. The resulting multitrait-multimethod matrix was analyzed in the traditional fashion as well as with confirmatory factor analysis. Convergent and discriminant validity for the three substance use traits were quite adequate, although the validity of one measure (the peer-reported intake measure of alcohol use) may be considered inadequate according to some criteria. This study demonstrates the usefulness of multimethod assessment and confirmatory factor analysis for research on construct validity and for derivation of more accurate measures of drug use.     

Psychological Assessment Training in Clinical Psychology Doctoral Programs

We surveyed American Psychological Association–accredited clinical psychology doctoral programs' (n = 83) training in psychological assessment—specifically, their coverage of various assessment topics and tests in courses and practica, and whether the training was optional or required. We report results overall and separately per training model (clinical science, scientist-practitioner, and practitioner-focused). Overall, our results suggest that psychological assessment training is as active, or even more active, than in previous years. Areas of increased emphasis include clinical interviewing and psychometrics; multimethod, outcomes, health, and collaborative or therapeutic assessment; and different types of cognitive and self-report personality tests. All or almost all practice-focused programs offered training with the Thematic Apperception Test and Rorschach compared to about half of the scientist-practitioner programs and a third of the clinical science programs. Although almost all programs reported teaching multimethod assessment, what constitutes different methods of assessing psychopathology should be clarified in future studies because many programs appear to rely on one method—self-report (especially clinical science programs). Although doctoral programs covered many assessment topics and tests in didactic courses, there appears to be a shortage of program-run opportunities for students to obtain applied assessment training. Finally, we encourage doctoral programs to be familiar with (a) internships' assessment expectations and opportunities, (b) the professional guidelines for assessment training, and (c) the American Psychological Association's requirements for preinternship assessment competencies.     

Effects of Situational Judgment Test Format on Reliability and Validity

Single-response situational judgment tests (SRSJTs) differ from multiple-response SJTs (MRSJTS) in that they present test takers with edited critical incidents and simply ask test takers to read over the action described and evaluate it according to its effectiveness. Research comparing the reliability and validity of SRSJTs and MRSJTs is thus far extremely limited. The study reported here directly compares forms of a SRSJT and MRSJT and explores the reliability, convergent validity, and predictive validity of each format. Results from this investigation present preliminary evidence to suggest SRSJTs may produce internal consistency reliability, convergent validity, and predictive validity estimates that are comparable to those achieved with many traditional MRSJTs. We conclude by discussing practical implications for personnel selection and assessment, and future research in psychological science more broadly.     

Four fatal flaws of brain injury assessment in the cross-cultural context

This analytical review serves to highlight pitfalls clinicians in multicultural contexts may encounter when using normed ability tests for the assessment of cognitive dysfunction due to brain injury. Four sources of erroneous practice are: (i) Failure to recognize within-race group heterogeneity on cognitive test performance in association with factors such as quality of education; (ii) Lack of rigorous attention being paid to the exact mode of test administration used in a derived normative base to be used for assessment purposes; (iii) Estimating an expected level of performance for an individual on diverse cognitive tests based on an IQ score; (iv) Adopting a thoroughly rule-bound actuarial approach to test interpretation, over a contextually based clinical approach. Even with locally derived norms, the validity of actuarially based interpretive conclusions in the absence of a conceptually coherent case analysis is not supported.     

Analyzing the convergent and discriminant validity of states and traits: development and applications of multimethod latent state-trait models

The analysis of convergent and discriminant validity is an integral part of the construct validation process. Models for analyzing the convergent and discriminant validity have typically been developed for cross-sectional data. There exist, however, only a few approaches for longitudinal data that can be applied for analyzing the construct validity of fluctuating states. In this article, the authors show how models of latent state-trait theory can be combined with models of multitrait-multimethod analysis to develop a model that allows for analyzing convergent and discriminant validity in time: the multimethod latent state-trait model. The model allows for identifying different sources of variance (trait consistency, trait-method specificity, occasion-specific consistency, occasion-specific method specificity, and unreliability). It is applied to the repeated measurement of depression and anxiety in children, which was assessed by self and teacher reports (N = 375). The application shows that the proposed models fit the data well and allow a deeper understanding of method effects in clinical assessment.     

Convergent and discriminant validity of selected masculinity and femininity scales

Using Campbell and Fiske's (1959) multitrait-multimethod matrix technique and Jackson's (1975) multimethod factor analysis, convergent and discriminant validity were investigated for masculinity and femininity scales from the Adjective Check List, Bem Sex Role Inventory, and PRF ANDRO Scale as compared with measures of dominance and nurturance. Results indicated very good convergent and discriminant validity for the masculinity and femininity scales from the Adjective Check List and Bem Inventory, but not for the PRF ANDRO Scale. Implications of these results for research on sex roles and psychological androgyny are discussed.     

Incremental validity in the psychological assessment of children and adolescents

Incremental validity in the process of psychological assessment of children and adolescents is explored. The authors highlight the dependence of the incremental validity of assessment information on factors such as goal of assessment, other information available, base rate of the problem or outcome, age or gender of the child, and type of problem being assessed. The authors discuss the incremental validity of assessment information from alternate sources, methods, and constructs. In view of the limited number of studies directly relevant to incremental validity in child clinical assessments, the authors call for more clinically relevant research. To have the greatest impact on child and adolescent services, this research must be readily generalized and immediately relevant to actual clinical practice.     

Infant Distress During Immunization: A Multimethod Assessment

The purpose of this study was to comprehensively describe infant procedural distress and pain across assessment modalities, and to compare similarities and differences across measures. A multimethod assessment of distress was conducted to investigate infants (N = 37) undergoing routine immunizations. Measures of infant distress included Parent report, nurse report, infant heart rate, and an observational measure of infant distress. Parents rated their infant's distress and pain significantly higher than did nurses. Observational and physiological ratings of infant distress were found to vary significantly by phase, and there were no correlations between adult ratings of pain and distress and physiological ratings. Findings suggest that infant procedural distress can be assessed in a number of manners. The discordance between these measures emphasizes the need for multimethod assessment of pediatric procedural distress in both research and clinical settings. Given the differences between parent and nurse ratings, clinicians should be aware that different assessment methods might lead to different conclusions about infant procedural distress.     

Journeys through the valley of death: multimethod psychological assessment and personality transformation in long-term psychotherapy

The Psychodynamic Diagnostic Manual (PDM Task Force, 2006) is based on the assumption that an in-depth understanding of clients' underlying emotional, personality, and interpersonal patterns will facilitate their treatment. In this article I show how such an understanding can be achieved through multimethod psychological assessment, and how useful such information can be in long-term psychotherapy with high-achieving, successful clients who struggle with forming and maintaining intimate relationships. Such treatments are extremely difficult, because when these clients attach to their psychotherapists, many of them temporarily become more symptomatic. I illustrate these points with a detailed account of my long-term therapy with a resilient but highly traumatized young man. Repeated use of the Minnesota Multiphasic Personality Disorder-2 (MMPI-2; Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) and Rorschach with my client helped guide us in our work, and also helped create an important therapeutic "opening" into the underlying traumatic material. This and other experiences have convinced me that it is extremely useful for psychologists to have training in both assessment and psychotherapy.     

Toward a process-focused model of test score validity: improving psychological assessment in science and practice

Although definitions of validity have evolved considerably since L. J. Cronbach and P. E. Meehl's classic (1955) review, contemporary validity research continues to emphasize correlational analyses assessing predictor-criterion relationships, with most outcome criteria being self-reports. The present article describes an alternative way of operationalizing validity--the process-focused (PF) model. The PF model conceptualizes validity as the degree to which respondents can be shown to engage in a predictable set of psychological processes during testing, with those processes dictated a priori by the nature of the instrument(s) used and the context in which testing takes place. In contrast to the traditional approach wherein correlational methods are used to quantify the relationship between test score and criterion, the PF model uses experimental methods to manipulate variables that moderate test score-criterion relationships, enabling researchers to draw more definitive conclusions regarding the impact of underlying psychological processes on test scores. By complementing outcome-based validity assessment with a process-driven approach, researchers will not only improve psychology's assessment procedures but also enhance their understanding of test bias and test score misuse by illuminating the intra- and interpersonal factors that lead to differential performance (and differential prediction) in different groups.     

Training and Consultation in Psychological Assessment With Professional Psychologists: Suggestions for Enhancing the Profession and Individual Practices

Once central to the identity and practice of clinical psychology, psychological assessment (PA) is currently more limited in professional practice and generally less emphasized in graduate training programs than in the past. Performance-based personality tests especially are taught and used less, even though scientific evidence of their utility and validity has never been stronger. We review research on training in PA and discuss challenges that contributed to its decreased popularity. We then review continuing education requirements for ethical practice in PA and recommend that PA should be reconceptualized as a specialty best practiced by psychologists who have the resources and time to maintain competency. We offer recommendations about how professional organizations concerned with PA can promote its practice and how individual expert clinicians can assist. We conclude by describing a collaborative model for providing group consultation in PA to practicing psychologists. If implemented widely, this model could help promote PA and raise its standard of practice.     

Exploring the construct validity of the social cognition and object relations scale in a clinical sample

The Social Cognition and Object Relations Scale-Global rating method (SCORS-G; Stein, Hilsenroth, Slavin-Mulford, & Pinsker, 2011; Westen, 1995) measures the quality of object relations in narrative material. This study employed a multimethod approach to explore the structure and construct validity of the SCORS-G. The Thematic Apperception Test (TAT; Murray, 1943) was administered to 59 patients referred for psychological assessment at a large Northeastern U.S. hospital. The resulting 301 TAT narratives were rated using the SCORS-G method. The 8 SCORS variables were found to have high interrater reliability and good internal consistency. Principal components analysis revealed a 3-component solution with components tapping emotions/affect regulation in relationships, self-image, and aspects of cognition. Next, the construct validity of the SCORS-G components was explored using measures of intellectual and executive functioning, psychopathology, and normal personality. The 3 SCORS-G components showed unique and theoretically meaningful relationships across these broad and diverse psychological measures. This study demonstrates the value of using a standardized scoring method, like the SCORS-G, to reveal the rich and complex nature of narrative material.     

Assessment of attention deficit disorder using a thematic apperception technique

Attention deficit-hyperactivity disorder (AD-HD) is regarded as being relatively common among school-age children, but the literature reveals a number of confounding factors with standard assessment techniques of the disorder. Using a structured thematic apperception technique (the Tell-Me-A-Story [TEMAS] test) to measure attention to pictorial stimuli depicting characters, events, settings, and covert psychological conflicts, a study was conducted with 152 normal and 95 clinical Hispanic, Black, and White school-age children. Results reveal that the AD-HD children are significantly more likely than normal children to omit information in the stimuli about characters, events, settings, and psychological conflicts. Differences between the groups are large and persistent in the presence of structured inquiries by the test examiners. Results suggest the potential utility of structured thematic apperception techniques for the assessment of AD-HD, eventually to facilitate DSM-III-R (American Psychiatric Association, 1987) diagnosis, but also to invite closer scrutiny in carefully controlled validity studies.     

Construct validity of the Life Orientation Test

The Life Orientation Test (LOT; Scheier & Carver, 1985) is a common measure of optimism. Previous studies of the convergent and discriminant (vis-à-vis neuroticism) validity of the LOT have relied solely on questionnaires. Our Study 1 was a multitrait, multimethod investigation of the LOT incorporating Gottschalk and Gleser's (1969) method of content analysis of speech samples as a nonquestionnaire assessment method. The LOT did not correlate significantly with the Gottschalk Hope Scale (Gottschalk, 1974) derived from speech samples. Study 2 was an initial attempt to determine the nature of the differences between these alternate measures of optimism. In particular, it was hypothesized and found that the LOT was more fakable than the Hope Scale. Participants who received special instructions to "fake good" scored significantly higher on the LOT than did control participants who received no special instructions. Faking instructions did not significantly affect scores on the Hope Scale.     

A multitrait–multimethod study of assessment instruments for complex problem solving

Recently published studies on Complex Problem Solving (CPS) suggest that assessments of CPS using multiple complex systems are only moderately related to tests of classical cognitive abilities. Further, CPS assessments show incremental validity beyond tests of other cognitive abilities when predicting relevant outcomes. However, these empirical accounts have relied on single CPS assessment instruments. We do not know whether these findings will generalize to the construct level across different CPS assessment instruments. To answer this question, we tested a sample of N = 339 German university students who completed three CPS assessment instruments based on multiple complex systems (MicroDYN, the Genetics Lab, and MicroFIN) and the matrices subtest of the Intelligence Structure Test as measure of reasoning. Students further reported their school grades. Analyses including latent multitrait–multimethod models provided support for the conceptualization of CPS as a complex cognitive ability. Results indicated that different CPS assessment instruments showed sufficient convergent validity (with a consistency mostly between .50 and .60). In addition, we found evidence for the divergent validity of CPS from reasoning (reasoning predicted two CPS facets, knowledge and control, β_KNOW = .49 and β_CON = .53, respectively). In the prediction of academic achievement, CPS explained variance in natural science grades after we controlled for reasoning (β_CPS = .22), whereas social science grades were not predicted. Our findings suggest that the validity of CPS generalizes across different measurement instruments.     

Der Foto-Hand-Test

Projective psychological techniques might be useful for measuring forensically relevant constructs in the context of a multimodal assessment strategy but are often considered to be deficient concerning reliability and validity. The Photo Hand Test (FHT), a German version of the Hand Test, is a diagnostic tool aimed at measuring aggressive dispositions. It uses photographs of hand gestures which test persons are required to interpret. The answers are coded and an acting-out score is computed from diverse categories.A study on the reliability and validity of the FHT protocols from 98 offenders and a comparison group of 55 non-offenders was carried out. Internal consistency coefficients of the FHT scores were low and unsatisfactory. Offenders had higher acting-out scores than non-offenders, they did not give more aggression-prone interpretations but more interpretations coded as aggression-inhibiting. Correlations of the FHT scores with self-report scales from the Antisocial Personality Questionnaire did not show the expected patterns; however, they did suggest effects of a hostility bias: persons who interpreted many hand gestures as aggression-prone did not describe themselves as highly aggressive or impulsive but were more distrustful and paranoid. This indicates a projection mechanism different from the one postulated by the FHT authors.The results suggest that FHT measures should be interpreted with caution in forensic evaluations.