Analyzing the convergent and discriminant validity of states and traits: development and applications of multimethod latent state-trait models

The analysis of convergent and discriminant validity is an integral part of the construct validation process. Models for analyzing the convergent and discriminant validity have typically been developed for cross-sectional data. There exist, however, only a few approaches for longitudinal data that can be applied for analyzing the construct validity of fluctuating states. In this article, the authors show how models of latent state-trait theory can be combined with models of multitrait-multimethod analysis to develop a model that allows for analyzing convergent and discriminant validity in time: the multimethod latent state-trait model. The model allows for identifying different sources of variance (trait consistency, trait-method specificity, occasion-specific consistency, occasion-specific method specificity, and unreliability). It is applied to the repeated measurement of depression and anxiety in children, which was assessed by self and teacher reports (N = 375). The application shows that the proposed models fit the data well and allow a deeper understanding of method effects in clinical assessment.     

Construct Validity of ADHD/ODD Rating Scales: Recommendations for the Evaluation of Forthcoming DSM-V ADHD/ODD Scales

Exploratory structural equation modeling (SEM) was applied to a multiple indicator (26 individual symptom ratings) by multitrait (ADHD-IN, ADHD-HI and ODD factors) by multiple source (mothers, fathers and teachers) model to test the invariance, convergent and discriminant validity of the Child and Adolescent Disruptive Behavior Inventory with 872 Thai adolescents and the ADHD Rating Scale-IV and ODD scale of the Disruptive Behavior Inventory with 1,749 Spanish children. Most of the individual ADHD/ODD symptoms showed convergent and discriminant validity with the loadings and thresholds being invariant over mothers, fathers and teachers in both samples (the three latent factor means were higher for parents than teachers). The ADHD-IN, ADHD-HI and ODD latent factors demonstrated convergent and discriminant validity between mothers and fathers within the two samples. Convergent and discriminant validity between parents and teachers for the three factors was either absent (Thai sample) or only partial (Spanish sample). The application of exploratory SEM to a multiple indicator by multitrait by multisource model should prove useful for the evaluation of the construct validity of the forthcoming DSM-V ADHD/ODD rating scales.     

The Construct Validity of Acceptance: A Multitrait-Multimethod Investigation

Despite increasing clinical and empirical attention, the construct validity of acceptance has not been extensively investigated. The present study utilized a multitrait-multimethod design and a correlated trait-correlated method minus one [CT-C(M-1)] confirmatory factor analytic model to assess acceptance's convergent validity across methods and discriminant validity in comparison to cognitive reappraisal and perceived emotional control in a sample of 210 outpatients with anxiety and mood disorders. In addition, the study evaluated acceptance's concurrent validity by investigating the extent to which it was associated with variables of clinical interest over and above the two rival constructs. Results of confirmatory factor analyses supported acceptance's convergent and discriminant validity in comparison to the two neighboring constructs, and thereby provided partial support for its construct validity. However, contrary to prediction, acceptance was not significantly associated with concurrent validation measures. These results are discussed in the context of acceptance's potential therapeutic utility and functional relationships with associated constructs.     

The Association Between the Attitude of Food‐Waste‐Aversion and BMI: An Exploration in India and the United States

This research proposes the existence of a hitherto undocumented attitude related to food wastage: the attitude of food‐waste‐aversion. We develop a 6‐item scale including affective, cognitive, and conative components to measure this attitude and empirically investigate its properties in two countries using novel datasets. We test for food‐waste‐aversion scale's convergent validity by demonstrating that it is correlated in the expected direction with five theoretically related constructs—frugality, social responsibility, spendthriftness, self‐control, and materialism (Studies 1a and 1b)—and with BMI (Studies 2 and 3). We provide more indirect evidence of the scale's convergent validity by documenting that the link between food‐waste‐aversion and BMI is attenuated among those who practice refrigerating leftovers (Study 3). We also document that the food‐waste‐aversion scale is distinct from general waste aversion and external meal‐cessation rules, thus providing evidence of discriminant validity (Studies 1a, 1b, and 1c). Taken together, these results provide construct validity for the novel construct of food‐waste‐aversion. We discuss the theoretical and substantive contributions of our findings.     

A new frame for frame-of-reference training: enhancing the construct validity of assessment centers

The authors undertook a comprehensive examination of the construct validity of an assessment center in this study by (a) gathering many different types of evidence to evaluate the strength of the inference between predictor measures and constructs (e.g., reliability, accuracy, convergent and discriminant relationships), (b) introducing a theoretically relevant intervention (frame-of-reference [FOR] training) aimed at improving construct validity, and (c) examining the effect of this intervention on criterion-related validity (something heretofore unexamined in the assessment center literature). Results from 58 assessees and 122 assessors suggested that FOR training was effective at improving the reliability, accuracy, convergent and discriminant validity, and criterion-related validity of assessment center ratings. Findings are discussed in terms of implications and future directions for both FOR training and assessment center practice.     

霍式中国职业兴趣量表构想效度的验证性因素分析（Ⅰ）：量表会聚效度和区分效度的验证性因素分析

霍式中国职业兴趣量表是依据Ｈｏｌｌａｎｄ的职业兴趣理论并结合我国国情而建构的量表。采用验证性因素分析的方法,对该量表的构想效度从会聚效度和区分效度两个方面进行交叉证实检验,结果表明量表特质因素的会聚效度和区分效度良好,所体现出的理论构想形式与Ｈｏｌｌａｎｄ的六种职业兴趣类型的理论构想基本一致。     

评价中心的结构效度研究

评价中心虽然具备很高的预测效度,但其结构效度指标却不太理想,如研究普遍发现其汇聚效度和区分效度较低。影响评价中心结构效度的因素众多,如评分维度因素（数量和类型）、评价者因素（培训方式和人员类型）、测评方法因素（情景导向特征、特质激活潜力、测评活动形式）以及系统的观察与评价程序。该文从上述因素出发,综述了评价中心结构效度的相关研究,总结了提高评价中心结构效度的措施,并指出了未来的研究方向     

An examination of the construct validity of the Social Anhedonia Scale

Two studies bearing on the construct validity of the Social Anhedonia Scale (SA) are reported. Study 1 reported convergent validity coefficients between the SA scale and numerous measures of traits reflecting a generalized attitude toward others (e.g., social interest and empathy) as well as several traits reflecting maladjustment without a social component as a test of discriminant validity (e.g., pessimism and rigidity). Study 1 found impressive convergent validity for the SA scale and mixed discriminant validity. Study 2 found that the SA scale was related to two dimensions of striving for security, one of which is strongly influenced by negative interpersonal tendencies, suggesting support for the validity of the SA scale. Implications for the measurement and concept of social anhedonia are presented.     

Validation of idiosyncratic cognitive schema in cognitive case formulations: an intraindividual idiographic approach

This article describes a method for the intraindividual clinical validation of a cognitive case formulation (CCF) involving hypotheses about the patient's idiosyncratic cognitive schema (ICS). The two-stage approach begins by testing the convergent and discriminant validity of the hypothesized ICS against the individual's daily ratings of cognition items using confirmatory dynamic factor analysis. The second stage evaluates the extent to which the ICS factor scores predict daily variability in symptoms and distress and further addresses convergent and discriminant validity by evaluating intraindividual cognitive content specificity and the incremental validity of the idiographic cognition factors compared with nomothetic measures of thoughts/beliefs. This approach to validating idiographic assessment is illustrated with the CCF of a woman with comorbid mood and anxiety disorders.     

霍式中国职业兴趣量表构想效度的验证性因素分析（Ⅱ）：多特质多方法矩阵的LISREL模型比较检验

霍式中国职业兴趣量表通过验证性因素分析的交叉证实检验,显示出具备良好的会聚效度和区分效度,但同时也显示出一定的来自方法偏误的影响。在本研究中,研究者采用多特质多方法的Ｌｉｓｒｅｌ模型各成分间的比较检验法,对模型中特质因素、方法因素及方法偏误因素等各成分的作用做了进一步的区分,以考察二者各自影响作用的大小。结果表明,霍式中国职业兴趣量表构想效度良好,来自方法偏误的影响作用不大。     

Assessing the validity of multinomial models using extraneous variables: an application to prospective memory

The class of multinomial processing tree (MPT) models has been used extensively in cognitive psychology to model latent cognitive processes. Critical for the usefulness of a MPT model is its psychological validity. Generally, the validity of a MPT model is demonstrated by showing that its parameters are selectively and predictably affected by theoretically meaningful experimental manipulations. Another approach is to test the convergent validity of the model parameters and other extraneous measures intended to measure the same cognitive processes. Here, we advance the concept of construct validity (Cronbach & Meehl, 1955 ) as a criterion for model validity in MPT modelling and show how this approach can be fruitfully utilized using the example of a MPT model of event-based prospective memory. For that purpose, we investigated the convergent validity of the model parameters and established extraneous measures of prospective memory processes over a range of experimental settings, and we found a lack of convergent validity between the two indices. On a conceptual level, these results illustrate the importance of testing convergent validity. Additionally, they have implications for prospective memory research, because they demonstrate that the MPT model of event-based prospective memory is not able to differentiate between different processes contributing to prospective memory performance.     

A comparison of the psychometric properties of the psychopathic personality inventory full-length and short-form versions

The Psychopathic Personality Inventory (PPI) has shown promising construct validity as a measure of psychopathy. Because of its relative efficiency, a short-form version of the PPI (PPI-SF) was developed and has proven useful in many psychopathy studies. The validity of the PPI-SF, however, has not been thoroughly examined, and no studies have directly compared the validity of the short form with that of the full-length version. The current study was designed to compare the psychometric properties of both PPI versions, with an emphasis on convergent and discriminant validity in predicting external criteria conceptually relevant to psychopathy. We used both prison (n = 558) and college samples (n = 322) for this investigation. PPI scale scores were more reliable and more strongly correlated with the conceptually relevant criterion measures compared with the PPI-SF, particularly in the prison sample. There were no differences in relative discriminant validity. Thus, overall, the PPI full-length version showed more evidence of construct validity than did the short form, and the consequences of this psychometric difference should be considered when evaluating the clinical utility of each measure.     

A comparison of MMPI--2 measures of psychopathic deviance in a forensic setting

We examined the convergent and discriminant validity of the Minnesota Multiphasic Personality Inventory--2 (MMPI--2) measures of psychopathy, including the Clinical Scale 4, Restructured Clinical Scale 4 (RC4), Content Scale Antisocial Practices (ASP), and Personality Psychopathology Five Scale Disconstraint (DISC). Comparisons of the empirical correlates of these scales were conducted with 2 samples of participants evaluated at a criminal court clinic. The 2 samples included 59 men and 19 women and 913 men and 327 women, respectively. Two types of criteria (clinician ratings and archival record review) were utilized in the analyses. Relative to Clinical Scale 4, RC4 had significantly greater convergent validity in predicting psychopathy as measured by the Psychopathy Checklist--Screening Version (S. D. Hart, D. N. Cox, & R. D. Hare, 1995) and behavioral criteria associated with this construct. RC4 also showed substantially improved discriminant validity when compared with its Clinical Scale counterpart. Among all the MMPI-2 scales studied, RC4 was the best measure of the social deviance traits of psychopathy.     

“I Know What You Want to Know”: The Impact of Interviewees' Ability to Identify Criteria on Interview Performance and Construct-Related Validity

The current study tested whether candidates' ability to identify the targeted interview dimensions fosters their interview success as well as the interviews' convergent and discriminant validity. Ninety-two interviewees participated in a simulated structured interview developed to measure three different dimensions. In line with the hypotheses, interviewees who were more proficient at identifying the targeted dimensions received better evaluations. Furthermore, interviewees' ability to identify these evaluation criteria accounted for substantial variance in predicting their performance even after controlling for cognitive ability. Finally, the interviewer ratings showed poor discriminant and convergent validity. However, we found some support for the hypothesis that the quality of the interviewer ratings improves when one only considers ratings from questions for which interviewees had correctly identified the intended dimensions.     

Evaluation of convergent and discriminant validity with multitrait–multimethod correlations

A procedure for evaluation of convergent and discriminant validity coefficients is outlined. The method yields interval estimates of these coefficients in a construct validation study conducted via the multitrait–multimethod approach, as well as permitting examination of their population relationships. The procedure is readily employed in behavioural research using the increasingly popular latent variable modelling methodology. The method is illustrated with a numerical example.     

An Analysis of a Cross-Cultural Personality Inventory: The IPIP Big-Five Factor Markers in Croatia

We investigated the psychometric properties of the Oregon Vocational Interest Scales (ORVIS), a brief public-domain alternative to commercial inventories, in a large community sample and in a college sample. In both samples, we examined the factor structure, scale intercorrelations, and personality correlates of the ORVIS, and in the community sample, we also examined the correlations of the ORVIS scales with cognitive abilities and with the scales of a longer, proprietary interest survey. In both samples, all 8 scales—Leadership, Organization, Altruism, Creativity, Analysis, Producing, Adventuring, and Erudition—showed wide variation in scores, high internal-consistency reliabilities, and a pattern of high convergent and low discriminant correlations with the scales of the proprietary interest survey. Overall, the results support the construct validity of the scales, which are recommended for use in research on vocational interests and other individual differences.     

AN EXAMINATION OF THE EFFECTS OF USING BEHAVIOR CHECKLISTS ON THE CONSTRUCT VALIDITY OF ASSESSMENT CENTER DIMENSIONS

Although research has established the criterion-related validity of assessment centers for selection purposes, the construct validity of dimension ratings has not been demonstrated. A quasi-experimental design was used to investigate the influence of retranslated behavior checklists on the construct validity of dimension ratings for two assessment center exercises. Assessor use of behavior checklists increased the average convergent (i.e., same dimension across exercise) validity from .24 to .43 while decreasing the average discriminant (i.e., different dimension within exercise) validity (.47 to .41). Behavior checklist sums were moderately correlated with corresponding dimension ratings and demonstrated a comparable level of construct validity. It is suggested that using behavior checklists may improve dimension construct validity by reducing the cognitive demands placed on raters.     

On the validity of Raven’s matrices test: Does spatial ability contribute to performance?

The paper reports on an investigation of the convergent and discriminant validity of Raven’s matrices in considering reasoning and spatial abilities. Raven’s advanced progressive matrices (APM) and four scales representing reasoning, visualization, mental rotation and closure were applied to a sample of N = 280 university students. The data were investigated by means of structural equation modeling. The results demonstrated convergent validity of APM with respect to reasoning. In contrast, in investigating discriminant validity substantial correlations of moderate size between APM and the scales representing spatial abilities were observed. However, the investigation of the structure of the prediction of APM revealed that the scales representing spatial abilities did not improve the prediction based on the reasoning scale alone at the latent level. Consequently, it is suggested that Raven’s matrices show good convergent validity and slightly impaired discriminant validity.     

On the validity of Raven’s matrices test: Does spatial ability contribute to performance?

The paper reports on an investigation of the convergent and discriminant validity of Raven’s matrices in considering reasoning and spatial abilities. Raven’s advanced progressive matrices (APM) and four scales representing reasoning, visualization, mental rotation and closure were applied to a sample of N = 280 university students. The data were investigated by means of structural equation modeling. The results demonstrated convergent validity of APM with respect to reasoning. In contrast, in investigating discriminant validity substantial correlations of moderate size between APM and the scales representing spatial abilities were observed. However, the investigation of the structure of the prediction of APM revealed that the scales representing spatial abilities did not improve the prediction based on the reasoning scale alone at the latent level. Consequently, it is suggested that Raven’s matrices show good convergent validity and slightly impaired discriminant validity.     

Eating Disorders Inventory: Assessment of its Construct Validity in a Nonclinical French Sample of Adolescents

Recent studies have questioned the appropriateness of the original Eating Disorders Inventory (EDI) in nonclinical samples of adolescents. The main objective of the present series of studies is to systematically test the construct validity of the EDI (i.e. content, factorial, convergent, discriminant and discriminative) in a nonclinical sample of French adolescents. A total sample of 1,323 adolescents was involved in these five studies. The factorial validity and the measurement invariance of the EDI were verified through confirmatory factorial analyses. Correlation and student t-tests were also used to test the convergent and discriminative validity of the EDI. Results from the first study confirmed the unsuitability of the French original EDI for young adolescents. Items were re-worded and an adaptation for adolescents was developed (EDI-A). The following four studies provided support for the factorial validity, measurement invariance, reliability, convergent validity and discriminant validity for a short form (i.e. 24 items) of the EDI-A. The present results thus provide preliminary evidence regarding the construct validity of the 24-item EDI-A for French nonclinical adolescents. Recommendations for future uses and research activities with this instrument in French speaking adolescents are outlined.