评价中心的结构效度研究

评价中心虽然具备很高的预测效度,但其结构效度指标却不太理想,如研究普遍发现其汇聚效度和区分效度较低。影响评价中心结构效度的因素众多,如评分维度因素（数量和类型）、评价者因素（培训方式和人员类型）、测评方法因素（情景导向特征、特质激活潜力、测评活动形式）以及系统的观察与评价程序。该文从上述因素出发,综述了评价中心结构效度的相关研究,总结了提高评价中心结构效度的措施,并指出了未来的研究方向     

The Construct Validity of Assessment Centre Performance Dimensions

Evidence regarding the construct validity of assessment centre performance dimensions is reviewed. The evidence strongly suggests that variance in ratings tends to reflect exercises more than individual performance dimensions, thus calling into question the construct validity and utility of these dimensions. A number of biases in the assessment centre process, as well as more general rating biases are noted that may be responsible for these pervasive exercise effects. Suggestions are made for enhancing the construct validity of performance dimensions.     

不同类型的测评维度对评价中心结构效度的影响研究

本研究将测评维度分为行为能力和心理特质两组,考察对评价中心结构效度的不同影响。A公司216名部门副经理级员工接受了本次管理素质评价中心测评,采用公文筐、角色扮演和无领导小组讨论三种方法,测查行为能力和心理特质两组维度,各自有三个维度被一种以上的方法测量。多质多法和验证性因素分析的结果表明,评价中心以行为能力比以心理特质为测评维度结构效度好;以行为能力为测评维度时,会聚效度和区分效度都较好。     

Assessor Constructs in Use as the Missing Component in Validation of Assessment Center Dimensions: A critique and directions for research

The basis for assessment center dimension ratings has been examined through the lens of various multitrait–multimethod approaches, with some researchers concluding that dimension ratings are not representative of meaningful constructs. This presents a serious challenge for those who would generalize predictor constructs, particularly those that are broadly articulated in management assessment center dimensions. This paper addresses the problems with applying such analyses to assessment center dimension ratings without first articulating expected construct relationships. Ackerman and Humphreys' analysis of constructs will be used to better frame assessment center dimensions and to articulate possible assessor constructs in use. Theories of performance held by the managers, who serve both as subject matter experts when dimensional constructs are initially defined, and as the assessors who impose these dimensions on assessees' behavior in assessment center ratings, are suggested as important bases for further construct validation. Approaches for generalization of assessor constructs in use to the managerial work domain will be discussed.     

Extending the Nomological Network of Assessment Center Construct Validity: Prediction of Cross-Situationally Consistent and Specific Aspects of Assessment Center Performance

The purpose of this study was to expand the nomological validity of assessment centers (ACs) by investigating predictors of cross-situationally consistent versus specific aspects of AC performance. Consistent with hypotheses, (a) Big Five personality factors predicted AC performance as it related to a cross-situationally consistent general performance factor but not as it related to exercise (i.e., situationally specific) factors, and (b) job knowledge predicted performance as it related to both the general performance factor and exercise-specific factors. Results are interpreted as they relate to the growing literature on AC construct validity.     

Impact of Common Rater Variance on Construct Validity of Assessment Center Dimension Judgments

In an assessment center (AC), assessors generally rate an applicant's performance on multiple dimensions in just 1 exercise. This rating procedure introduces common rater variance within exercises but not between exercises. This article hypothesizes that this phenomenon is partly responsible for the consistently reported result that the AC lacks construct validity. Therefore, in this article, the rater effect is standardized on discriminant and convergent validity via a multitrait-multimethod design in which each matrix cell is based on ratings of different assessors. Two independent studies (N = 200, N = 52) showed that, within exercises, correlations decrease when common rater variance is excluded both across exercises (by having assessors rate only 1 exercise) and within exercises (by having assessors rate only 1 dimension per exercise). Implications are discussed in the context of the recent discussion around the appropriateness of the within-exercise versus the within-dimension evaluation method.     

Assessment Center for Pilot Selection: Construct and Criterion Validity and the Impact of Assessor Type

Cette recherche a examiné la validité d'un centre d'évaluation pour la sélection de pilotes. Les scores de N = 1,036 participants ont été utilisés pour étudier la validité de construit. Un sous-échantillon de participants performants a été suivi et les évaluations des pairs ont été retenus comme mesures du critère. Les résultats démontrent une première évidence de validité de construit et de critère pour cet outil d'évaluation des compétences interpersonnelles et liées à la performance. Par ailleurs, les résultats ont aussi montré que le type d'évaluateur (psychologue vs pilote) modère la validité prédictive des scores du centre d'évaluation. Cet effet "type d'évaluateur" dépend de la sorte de variables prédictives. Les résultats sont discutés et des implications pratiques sont suggérées.
This study examined the validity of an assessment center in pilot selection as a new field of application. Assessment center ratings of N = 1,036 applicants were used to examine the construct validity. A subsample of successful applicants was followed up and peer ratings were chosen as criterion measures. The results provide first evidence of the construct and criterion validity of this assessment center approach for rating interpersonal and performance-related skills. Furthermore the type of assessor (psychologist versus pilot) moderates the predictive validity of the assessment center ratings. This type-of-assessor effect depends on the kind of predictor variables. The results are discussed and practical implications are suggested.     

评价中心的评分维度和评分效果

对近年来国内外关于评价中心的研究进行了比较系统的介绍。首先,文章讨论了评分维度的数目对于评分结果的影响,以及评价中心中的4个元维度;其次,介绍了评价中心中衡量评分效果的指标,并讨论了评分者培训的分类及其对评分效果的影响;第三,虽然评价中心具有良好的效标关联效度,但对于其结构效度的研究至今尚无统一结论。最后,文章对评价中心未来的研究趋势进行了探讨。     

Effects of Disaggregation on Managerial Assessment Center Validity

Assessment center validity data from a large, international petrochemical company were disaggregated by managerial level and business stream. As with previous research, the correlations were higher for the current job grade criterion than for performance ratings. Overall, disaggregating the data did not have the effect of increasing the assessment center—performance rating validities. However, when the validation data were disaggregated by business stream, there were differences in the pattern of the relationships between assessment factors and performance.     

中学生问题行为多视角评估的构想效度

本文以多质多法和验证性因素分析的方法对由中学生问题行为三视角评估(自我评定、教师评定和家长评定)构成的数据模型进行了构想效度的研究。研究样本为分层随机抽取的北京市292名中学生,对其三视角问题行为的施测结果进行了会聚效度和区分效度的分析,结果表明:(1)中学生问题行为三视角六因子的MTMM模型构想效度良好;(2)评定者对问题行为的评估有影响,存在评定者效应,而且评定者效应对内倾问题因子比对外倾问题因子影响大。     

面试预测效度和构想效度研究述评

面试是人才甄选中最常用的测量工具。大量研究证实,面试的预测效度比较理想,但不同类型面试的预测效度存在差异。虽然能够证实面试的预测效度较好,但对面试的测量构想却知之甚少。研究面试的构想效度,对于提高面试的递增效度有着重要的实践价值。相对人格成分而言,以往研究对面试能够测量到认知成分形成了更加一致地认识。     

The Abbreviated Dimensions of Temperament Survey: Factor Structure and Construct Validity Across Three Racial/Ethnic Groups

The factor structure, reliability, and construct validity of an abbreviated version of the Revised Dimensions of Temperament Survey (DOTS–R) were evaluated across Black, Hispanic, and White early adolescents. Primary caregivers reported on 5 dimensions of temperament for 4,701 children. Five temperament dimensions were identified via maximum likelihood exploratory factor analysis and were labeled flexibility, general activity level, positive mood, task orientation, and sleep rhythmicity. Multigroup mean and covariance structures analysis provided partial support for strong factorial invariance across these racial/ethnic groups. Mean level comparisons indicated that relative to Hispanics and Blacks, Whites had higher flexibility, greater sleep regularity, and lower activity. They also reported higher positive mood than Blacks. Blacks, relative to Hispanics, had higher flexibility and lower sleep regularity. Construct validity was supported as the 5 temperament dimensions were significantly correlated with externalizing problems and socioemotional competence. This abbreviated version of the DOTS–R could be used across racial/ethnic groups of early adolescents to assess significant dimensions of temperament risk that are associated with mental health and competent (healthy) functioning.     

Nonverbal Memory Assessment with Designs: Construct Validity and Clinical Utility

The evaluation of memory is often a principal objective in psychological testing, especially for older adults. Differentiation between subtypes of memory functioning is critical in making differential diagnoses and in predicting everyday functioning. In this paper, 23 tests using memory for designs and developed for clinical or research applications are reviewed. Overall, reliabilities are low, normative samples varied in size, and findings regarding the construct validity of design memory testing are mixed. Tests with higher reliability, large numbers of designs, and recognition formats may be more specific to nonverbal memory. A series of studies that systematically establishes construct validity for design memory tests will enhance clinical utility.     

Transparency: A Necessary Requirement for the Construct Validity of Assessment Centres

La validité de construction des centres d'évaluation et la performance perçue des candidats sont-elles sensibles au fait que ceux-ci aient ou non connaissance des dimensions jugées pertinentes? 119 étudiants de 1° cycle ont participéà une session de centre d'évaluation dans des conditions de transparence et de non transparence. II est apparu que la validité de construction était supérieure quand les participants connaissaient les dimensions recherchées et le comportement attendu de leur part. En outre, les notations des sujets de la condition "transparence" qui admettaient avoir soumis leur conduite aux dimensions attendues bénéficiaient d'une validité discriminante et convergente supérieure à celle des notations des sujets qui affirmaient n'avoir tenu aucun compte des dites dimensions. L'impact de la transparence des dimensions recherchées sur la validité de construction des centres d'évaluation est discuté.
The present study tested whether the construct validity of assessment centres and the perceived performance of candidates were influenced by whether or not the requirement dimensions were made transparent for participants. A total of 119 college students participated in an assessment centre under either transparent or nontransparent conditions. Results showed that construct validity was higher when participants knew the requirement dimensions and what behaviour was required of them. In addition, the ratings of subjects in the transparency condition who reported that they had oriented their behaviour towards the requirement dimensions had higher convergent and discriminant validity than the ratings of subjects who reported that they disregarded the requirement dimensions. The consequences of the transparency of requirement dimensions for the construct validity of assessment centres are discussed.     

Factors which Improve the Construct Validity of Assessment Centers: A Review

This article reviews 21 studies which manipulated specific variables to determine their impact on the construct validity of assessment centers. This review shows that the studies regarding the impact of different observation, evaluation, and integration procedures yielded mixed results. Conversely, dimension factors (number, conceptual distinctiveness, and transparency), assessor factors (type of assessor and type of assessor training), and exercise factors (exercise form and use of role-players) were found to moderate construct validity. On the basis of the review, practical recommendations are derived to maximize the probability that practitioners design and administer an assessment center with construct validity. Finally, new perspectives for future research are identified.     

The Construct Validity of Assertion: Contributions of Four Assessment Procedures and Norman's Personality Factors

ABSTRACT The construct validity of assertion was analyzed in three independent correlational studies The convergence of self-reported, laboratory roleplayed, self-observed, and peer-rated measures was examined, as well as their relationship to Norman's peer-rated five personality factors of extraversion, agreeableness, conscientiousness, emotional stability, and culture Twenty-three and 29 undergraduate students participated in Studies 1 and 2, respectively In general, results showed poor convergence among assertiveness measures even when increasing the representativeness and reliability of the role-played laboratory and self-observed measures, as in Study 2 In Study 3, the role-playing test was abandoned and 45 subjects served as peers for one another The results of Study 3 supported the prediction that assertiveness measures showed at least moderate convergence, in addition there were significant relationships between assertiveness and extraversion and between peer-rated assertiveness and agreeableness, conscientiousness, and culture The relative contribution of each assessment procedure to the construct validity of assertion is discussed     

The Validity of the Role Construct Repertory Test as a Measure of Sexual Identification

Although the majority of research on Therapeutic Assessment (TA) discusses the application of TA in research or private practice settings, we found that the model could be applied in a community mental health setting. We argue that when implementing this model with racially diverse, low socioeconomic status children, it is essential to integrate issues of class, privilege, and race into the assessment process. A case is presented that illuminates the specific concerns and struggles of adapting this model to a community psychology population. This analysis includes the interface with systems, placement stability, and consideration of culturally responsible treatment. We also demonstrate how the support of a treatment team helps the individual clinician process and integrate the levels of trauma and pain with which these families present.     

Validity of the Suicide Assessment Checklist in an Emergency Crisis Center

Validity of the Suicide Assessment Checklist (SAC; J. R. Rogers, 1990) was investigated in a sample of 1,969 admissions to a psychiatric emergency crisis center. Internal consistency reliability for rated items of the SAC was .87. Supporting construct‐related validity, total score differences were found in the expected directions as a function of referral reason. Convergent validity was based on observed correlations between selected SAC items and conceptually similar items on the Beck Depression Inventory (A. T. Beck, 1970). Supporting criterion‐related validity, total score differences in the expected directions were found as a function of disposition setting. Recommendations are made for the use of the SAC in the practice of counseling.     

The Importance of Construct Validity in Consumer Research

Vosgerau, Scopelliti, and Huh (2020) question the construct validity of the most commonly used measures of self‐control and argue that many studies that claim to be examining self‐control may in fact be examining something else. In this comment, we generalize this argument and propose that this criticism could be leveraged against many other areas of consumer research. Indeed, many studies fail to properly establish the construct validity of their measures. Consequently, they may not be examining the construct they purport to be studying, which can lead to unreliable and conflicting findings.