Evaluation of convergent and discriminant validity with multitrait–multimethod correlations

A procedure for evaluation of convergent and discriminant validity coefficients is outlined. The method yields interval estimates of these coefficients in a construct validation study conducted via the multitrait–multimethod approach, as well as permitting examination of their population relationships. The procedure is readily employed in behavioural research using the increasingly popular latent variable modelling methodology. The method is illustrated with a numerical example.     

Mixture distribution latent state-trait analysis: basic ideas and applications

Extensions of latent state-trait models for continuous observed variables to mixture latent state-trait models with and without covariates of change are presented that can separate individuals differing in their occasion-specific variability. An empirical application to the repeated measurement of mood states (N=501) revealed that a model with 2 latent classes fits the data well. The larger class (76%) consists of individuals whose mood is highly variable, whose general well-being is comparatively lower, and whose mood variability is influenced by daily hassles and uplifts. The smaller class (24%) represents individuals who are rather stable and happier and whose mood is influenced only by daily uplifts but not by daily hassles. A simulation study on the model without covariates with 5 sets of sample sizes and 5 sets of number of occasions revealed that the appropriateness of the parameter estimates of this model depends on number of observations (the higher the better) and number of occasions (the higher the better). Another simulation study estimated Type I and II errors of the Lo-Mendell-Rubin test.     

The convergent and discriminant validity of subjective fit perceptions

This study examined whether employees develop perceptions about 3 different types of fit: person-organization fit, needs-supplies fit, and demands-abilities fit. Confirmatory factor analyses of data from 2 different samples strongly suggested that employees differentiate between these 3 types of fit. Furthermore, results from a longitudinal design of 187 managers supported both the convergent and discriminant validity of the different types of fit perceptions. Specifically, person-organization fit perceptions were related to organization-focused outcomes (e.g., organizational identification, citizenship behaviors, turnover decisions), whereas needs-supplies fit perceptions were related to job- and career-focused outcomes (e.g., job satisfaction, career satisfaction, occupational commitment). Although demands-abilities fit perceptions emerged as a distinct construct, they were not related to hypothesized outcomes (e.g., job performance, raises).     

The Body Esteem Scale: a convergent and discriminant validity study

This research was intended to further establish convergent and discriminant validity for a recent Body Esteem Scale (BES) which measures different dimensions of body satisfaction in young adults. One hundred and fifty-four male and 193 female undergraduates completed the BES, the Body Consciousness Questionnaire (BCQ), the Self-Esteem Scale, and several questions pertaining to exercise, food intake, and attractiveness. Good convergent and discriminant validity was demonstrated by the Male Upper Body Strength and Physical Condition subscales and by the female Weight Concern, Physical Condition, and Sexual Attractiveness subscales.     

Individual differences in sociosexuality: evidence for convergent and discriminant validity

Individual differences in willingness to engage in uncommitted sexual relations were investigated in 6 studies. In Study 1, a 5-item Sociosexual Orientation Inventory (SOI) was developed. Studies 2, 3, and 4 provided convergent validity evidence for the SOI, revealing that persons who have an unrestricted sociosexual orientation tend to (a) engage in sex at an earlier point in their relationships, (b) engage in sex with more than 1 partner at a time, and (c) be involved in relationships characterized by less investment, commitment, love, and dependency. Study 5 provided discriminant validity for the SOI, revealing that it does not covary appreciably with a good marker of sex drive. Study 6 demonstrated that the SOI correlates negligibly with measures of sexual satisfaction, anxiety, and guilt. The possible stability of, origins of, and motivational bases underlying individual differences in sociosexuality are discussed.     

The convergent and discriminant validity of the Need for Cognition Scale

This article reports two studies that examined the convergent and discriminant validity of the Need for Cognition Scale (Cacioppo & Petty, 1982). Using samples of college students and prison inmates, need for cognition scores were found to be positively associated with measures of self-esteem, masculine sex role attitudes, absorption, and private self-consciousness. Modest negative associations between need for cognition and measures of public self-consciousness and social anxiety were also uncovered. Need for cognition scores were generally unrelated to measures of feminine and androgynous sex role attitudes, shyness, sociability, and loneliness. These findings add further evidence supporting the construct validity of the Need for Cognition Scale and expand our understanding of the construct of need for cognition.     

The implications of convergent and discriminant validity data for instrument validation

It seems intuitively compelling to many investigators that measurements, on the same subjects by different methods, purportedly of the same given trait are somehow better evidenced to be mutually valid measurements of that trait to the degree that they are intercorrelated. It is similarly compelling that measurements on the same subjects of purportedly different and uncorrelated traits are somehow better evidenced to be valid measurements to the degree that they are not intercorrelated. Further, a demonstration of hetero-method mono-trait intercorrelation (convergence) jointly with one of hetero-method, or preferably mono-method, hetero-trait independence (discrimination) is more compelling than either single demonstration alone [see Campbell & Fiske, 1959]. I hope to show in what follows that this intuition is misleadingunless certain rather demanding prerequisites are satisfied. Then I hope to show that contrary demonstrations are generally too indecisive to consitute validity disconfirmations. Finally, I shall consider some issues in the practical use of the indecisive multitrait-multimethod data.An earlier version of this paper was presented at the Midwestern Society of Multivariate Experimental Psychology meetings of May 1969 in Chicago. Some explication of the concept of measuring instrument has also been presented elsewhere [Krause, 1969].     

Psychometric properties of the mobility inventory for agoraphobia: convergent, discriminant, and criterion-related validity

Aims of this study were (a) to summarize the psychometric literature on the Mobility Inventory for Agoraphobia (MIA), (b) to examine the convergent and discriminant validity of the MIA's Avoidance Alone and Avoidance Accompanied rating scales relative to clinical severity ratings of anxiety disorders from the Anxiety Disorders Interview Schedule (ADIS), and (c) to establish a cutoff score indicative of interviewers’ diagnosis of agoraphobia for the Avoidance Alone scale. A meta-analytic synthesis of 10 published studies yielded positive evidence for internal consistency and convergent and discriminant validity of the scales. Participants in the present study were 129 people with a diagnosis of panic disorder. Internal consistency was excellent for this sample, α = .95 for AAC and .96 for AAL. When the MIA scales were correlated with interviewer ratings, evidence for convergent and discriminant validity for AAL was strong (convergent r with agoraphobia severity ratings = .63 vs. discriminant rs of .10–.29 for other anxiety disorders) and more modest but still positive for AAC (.54 vs. .01–.37). Receiver operating curve analysis indicated that the optimal operating point for AAL as an indicator of ADIS agoraphobia diagnosis was 1.61, which yielded sensitivity of .87 and specificity of .73.     

Beliefs in the Inheritance of Risk Factors for Suicide Scale: development, reliability, stability, and convergent and discriminant validity

Findings from adoption, family, geographical, immigrant, molecular genetic, surname, and twin studies suggest genetic risk factors have a role in suicidal behavior. However, related mental health literacy (knowledge and beliefs) even of future health-care professionals who will be concerned with suicide lag behind this research progress. As no scale for assessing such beliefs is available, the 22-item Beliefs in the Inheritance of Risk Factors for Suicide Scale was constructed, its contents rated by experts in a validation survey and administered to samples of 70 medical and 165 psychology students. Medical students held stronger beliefs in the genetics of suicide than psychology students. Internal scale consistency and test-retest reliability were assessed as adequate. Factor analysis of the 22 items yielded a dominant first factor. Scores were positively related to knowledge about suicide (convergent validity) but for the most part unrelated to lay theories of suicide, political orientation, religiosity, and social desirability (discriminant validity). This novel scale shows potential for assessing individuals' beliefs about the genetics of suicide. Applications may include basic research, educational contexts, and evaluation of professional training.     

Reinvestment: Examining convergent,discriminant, and criterion validity using psychometric and behavioral measures

Given scarce external validity available to date concerning the reinvestment construct, the aim of this four-study research project was to further explore the validity of the Movement-Specific Reinvestment Scale and the Decision-Specific Reinvestment Scale, using psychometric and behavioral measures. Study 1 showed that deliberative participants had a higher reinvestment tendency than intuitive participants. Study 2 showed that reinvestment was linked to self-consciousness, rumination, perfectionism, and had satisfactory test–retest reliability. Moreover, it provided some potential insights on the development of reinvestment due to parental criticisms. Study 3 indicated that high decision reinvestors performed worse than their low decision reinvestor counterparts in a visual search task under pressure. Study 4 showed that older participants had a lower reinvestment score, and that reinvestment was associated with higher motor imagery ability, challenging the idea that reinvestment can only be seen as detrimental to performance.     

The redundancy of work alienation and job satisfaction: Some evidence of convergent and discriminant validity

The multitrait-multimethod matrix (MTMM) technique was used in an attempt to help clarify the ambiguities regarding concepts of work alienation, job satisfaction, and the relationships between them. Within each attitude domain considered separately, generally acceptable evidence of convergent and (to a lesser degree) discriminant validity was found for the trait measures (four dimensions of alienation and five of satisfaction). However, the discrimination between alienation and satisfaction measures was no greater than that among satisfaction and among alienation, casting doubt on the demarcation of these as two distinct domains. The measures of satisfaction and alienation also were not correlated differentially with demographic and self-esteem measures. Note was taken of the limitations in our knowledge and operationalization of these areas, as these relate to the assumptions of MTMM technique regarding independence of both underlying trait constructs and measurement methods.     

Further validation of the IDAS: evidence of convergent, discriminant, criterion, and incremental validity

The authors explicated the validity of the Inventory of Depression and Anxiety Symptoms (IDAS; D. Watson et al., 2007) in 2 samples (306 college students and 605 psychiatric patients). The IDAS scales showed strong convergent validity in relation to parallel interview-based scores on the Clinician Rating version of the IDAS; the mean convergent correlations were .51 and .62 in the student and patient samples, respectively. With the exception of the Well-Being Scale, the scales also consistently demonstrated significant discriminant validity. Furthermore, the scales displayed substantial criterion validity in relation to Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; American Psychiatric Association, 1994) mood and anxiety disorder diagnoses in the patient sample. The authors identified particularly clear and strong associations between (a) major depression and the IDAS General Depression, Dysphoria and Well-Being scales, (b) panic disorder and IDAS Panic, (c) posttraumatic stress disorder and IDAS Traumatic Intrusions, and (d) social phobia and IDAS Social Anxiety. Finally, in logistic regression analyses, the IDAS scales showed significant incremental validity in predicting several DSM-IV diagnoses when compared against the Beck Depression Inventory-II (A. T. Beck, R. A. Steer, & G. K. Brown, 1996) and the Beck Anxiety Inventory (A. T. Beck & R. A. Steer, 1990).     

Further examination of the convergent and discriminant validity of the student–teacher relationship scale

Two studies extended psychometric research on the Student–Teacher Relationship Scale (STRS) with kindergarten and preschool children (N₁ = 60–71; N₂ = 35) and their teachers. These studies used a multi‐method approach to replicate and extend previous findings concerning the convergent validity of the STRS Closeness, Conflict, and Dependency scale and to further examine the discriminant validity of the STRS. Study 1 investigated convergence between the STRS scales and child‐ and peer‐reports of the same constructs based on a multi‐trait multi‐method approach. Study 2 examined the pattern of associations between the STRS and indicators of teacher–child relationship quality rated by external observers. Support was found for the convergent validity and to a lesser extent the discriminant validity of the STRS Closeness and Conflict scale. For the STRS Dependency scale, additional research remains necessary. Copyright © 2009 John Wiley & Sons, Ltd.     

The convergent and discriminant validity of the MCMI as a measure of the DSM-III personality disorders

The personality scales of the Millon Clinical Multiaxial Inventory (MCMI) were constructed and validated to measure the typology developed by Millon (1981). The ability of the MCMI to measure the DSM-III personality disorders has not yet been empirically evaluated. The current study found better convergent validity for the DSM-III personality disorders that are consistent with Millon's typology (i.e., the avoidant and the dependent) than for the disorders that are inconsistent (i.e., the antisocial and the passive-aggressive). However, the results may reflect some advantages Millon's typology might have over the DSM-III. The discriminant validity of all four scales was limited, due in part to the overlap among the MCMI scales and the DSM-III personality disorders. We discuss implications of the results for the revision of the MCMI and the DSM-III.     

A confirmatory evaluation of the profile of mood states: Convergent and discriminant item validity

The Profile of Mood States was administered to samples of 182 college males, 179 college females, and 257 prison inmates. College males and females did not differ significantly from each other in terms of scale elevation but differed from prison inmates on all scales except Fatigue-Inertia. The college samples differed from the published normative college samples, suggesting the importance of using local norms. A confirmatory item factor analysis suggested convergent item validity with the scoring key and similarity of structure across samples. Discriminant item validity, however, suggested that a smaller number of mood scales would offer a more justifiable interpretation of this inventory.This study was supported by the Alberta Hospital Edmonton, the Solicitor General of Canada, and Social Sciences and Humanities Research Council of Canada Grant 410-80-0576-XI.     

The convergent validity of four indices of cognitive complexity in person perception A multi-index multimethod and factor analytical approach

Abstract.— An application of the Campbell and Fiske multimethod-multitrait analysis and factor analysis was used to study the convergent validity of the following indices of cognitive complexity: Bieri's Index, Vannoy's Interaction Variance Measure, Number of Factors, and First Factor Percentage. In the analysis the individual constructs elicited by Reptest were contrasted with two sets of provided constructs, namely Semantic Differential and Personality Differential scales. The subjects were 36 psychology students. The results indicated that the convergent validity of the indices over construct type was low and that the construct type had an effect upon the intercorrelations of different indices of cognitive complexity.     

Large-scale investigation of the role of trait activation theory for understanding assessment center convergent and discriminant validity

This study used trait activation theory as a theoretical framework to conduct a large-scale test of the interactionist explanation of the convergent and discriminant validity findings obtained in assessment centers. Trait activation theory specifies the conditions in which cross-situationally consistent and inconsistent candidate performances are likely to occur. Results obtained by aggregating correlations across 30 multitrait-multimethod matrices supported the propositions of trait activation theory, shedding a more positive light on the construct validity puzzle in assessment centers. Overall, convergence among assessment center ratings was better between exercises that provided an opportunity to observe behavior related to the same trait, and discrimination among ratings within exercises was generally better for dimensions that were not expressions of the same underlying traits. Implications for assessment center research and practice are discussed.     

Using item response theory to study the convergent and discriminant validity of three questionnaires measuring cigarette dependence

To determine whether the Cigarette Dependence Scale, the Fagerstr?m Test for Nicotine Dependence, and the Nicotine Dependence Syndrome Scale (NDSS) reliably and correctly assessed both weakly and severely dependent individuals, the authors collected data via Internet from 2,435 current smokers, from 2004 to 2007. They used a 2-parameter item response model to determine the difficulty and discrimination of each question and used correlations between latent scores to assess convergent and discriminant validity. The reliability of all scales was close to or exceeded .70. Both the Cigarette Dependence Scale and the Fagerstr?m Test for Nicotine Dependence had 1 misfitting item. Each NDSS scale had at least 2 misfitting items. The information curve of each of the questionnaires peaked between -2 and 2 and was low at both extremes. All questionnaires had adequate reliability and were more informative for a medium level of the underlying cigarette dependence continuum than for both extremes of this continuum. The correlations between latent scores indicated good convergent validity between questionnaires and low discriminant validity between NDSS subscales, except for Tolerance. This result suggests that nicotine dependence may not be composed of 5 dimensions but may be unidimensional and distinct from reduced sensitivity to the effects of smoking (Tolerance).     

Decomposing person and occasion-specific effects: an extension of latent state-trait (LSI) theory to hierarchical LST models

An extension of latent state-trait (LST) theory to hierarchical LST models is presented. In hierarchical LST models, the covariances between 2 or more latent traits are explained by a general 3rd-order factor, and the covariances between latent state residuals pertaining to different traits measured on the same measurement occasion are explained by 2nd-order latent occasion-specific factors. Analogous to recent developments in multitrait-multimethod methodology, all factors are interpreted in relation to factors taken as comparison standards. An empirical example from test anxiety research illustrates how estimates of additive variance components due to general trait, specific trait, occasion, state residual, method, and measurement error can be obtained using confirmatory factor analysis. Advantages and limitations of these models are discussed.