Evaluation of Revision Effect on Criterion Validity of Multiple-Component Measuring Instruments

A method for point and interval estimation of change in criterion validity of multiple-component measuring instruments as a result of revision is outlined. The procedure is developed within the framework of covariance structure modeling, which complements earlier methods for testing change in composite reliability due to addition or deletion of components from tentative scales. The approach aids the process of constructing instruments with high validity in behavioral research that is illustrated with a pair of numerical examples.     

Examining Temporal Stability of Scale Validity in Longitudinal Studies

A method for examining invariance in validity of multiple-component instruments in repeated measure designs is outlined. The approach is developed within the framework of covariance structure modeling and is applicable for purposes of ascertaining temporal stability in scale validity. In addition, the procedure provides a range of plausible values in a population under investigation for the change in composite validity across longitudinal assessment occasions. The proposed method can also be used to examine possible sources of lack of validity invariance, and is illustrated with a pair of examples.     

Scale validity evaluation with congeneric measures in hierarchical designs

A procedure for validity estimation of multi-component measuring instruments in hierarchical designs is outlined. The method is developed within the framework of the popular latent variable modelling methodology. The approach is readily applicable for examining interrelationship indices between composite sum scores and criterion variables in two-level studies. The procedure is also useful for obtaining estimates and confidence intervals of criterion validity coefficients during the process of scale construction and development with hierarchical data. The method is illustrated with an empirical example.     

Alpha if item deleted: A note on loss of criterion validity in scale development if maximizing coefficient alpha

This note is concerned with a validity‐related limitation of the widely available and routinely used index ‘alpha if item deleted’ in the process of construction and development of multiple‐component measuring instruments. Attention is drawn to the fact that this statistic can suggest dispensing with such scale components, whose removal leads to loss of criterion validity while maximizing the popular coefficient alpha. As an alternative, a latent variable modelling approach is discussed that can be used for point and interval estimation of composite criterion validity (as well as reliability) after deletion of single components. The method can also be utilized to test conventional or minimum level hypotheses about associated population change in measurement quality indices.     

Transtheoretical Model Constructs for Physical Activity Behavior are Invariant across Time among Ethnically Diverse Adults in Hawaii

OBJECTIVES: Physical activity (PA) research applying the Transtheoretical Model (TTM) to examine group differences and/or change over time requires preliminary evidence of factorial validity and invariance. The current study examined the factorial validity and longitudinal invariance of TTM constructs recently revised for PA. METHOD: Participants from an ethnically diverse sample in Hawaii (N=700) completed questionnaires capturing each TTM construct. RESULTS: Factorial validity was confirmed for each construct using confirmatory factor analysis with full-information maximum likelihood. Longitudinal invariance was evidenced across a shorter (3-month) and longer (6-month) time period via nested model comparisons. CONCLUSIONS: The questionnaires for each validated TTM construct are provided, and can now be generalized across similar subgroups and time points. Further validation of the provided measures is suggested in additional populations and across extended time points.     

Psychometric analyses of the Clinician-Administered PTSD Scale (CAPS)--Bosnian translation

Methods for assessing psychological distress in culturally diverse populations are not firmly established. This study was designed to examine the psychometric properties of the Bosnian translation of the Clinician-Administered PTSD Scale (CAPS; D. D. Blake, F. W. Weathers, L. M. Nagy, D. Kaloupek, G. Klauminzer, D. Charney, et al., 1995) in a Bosnian refugee sample. The authors interviewed 115 help-seeking Bosnian refugees with the CAPS-Bosnian translation to examine its internal consistency and convergent validity, and to provide an assessment of its factor structure. This study demonstrated optimal fit with a 2-factor model of posttraumatic stress disorder (PTSD); the authors also found high reliability with a coefficient alpha of 0.92 and strong convergent validity with instruments measuring depression, anxiety, and levels of psychosocial functioning. Future directions for the assessment of PTSD in cross-cultural populations are discussed.     

And justice for all: Revisiting the Global Belief in a Just World Scale

The Global Belief in a Just World Scale (GBJWS) has been widely used in measuring the Belief in a Just World (BJW) personality trait. Despite its widespread application across the social sciences, the validity of this scale has not been sufficiently tested in the literature. In this research, the authors examine the internal and external validity of the GBJWS using both standard correlational analyses and structural equation modeling (SEM). Specifically, the authors test the concurrent validity, internal consistency, unidimensional structure, convergent validity, and both measurement and latent mean invariance of the scale across gender and culture. The results of a pilot study suggest strong concurrent validity of the GBJWS with other BJW scales, and the findings of the two main studies support the internal and external validity of GBJWS across gender and culture. The authors’ results further show an overall greater level of BJW of Chinese individuals compared to Americans. The present research provides a much needed investigation of the validity of the GBJWS, and answers calls for research examining the scale’s utility across different populations.     

Psychometric properties of the Spanish version of the Thougth-Shape Fusion Questionnaire

The objectives of the study were to analyze the psychometric properties, factor structure and internal consistency of the Spanish version of the Thought-Shape Fusion Questionnaire (TSF-Q), as well as to determine its validity by evaluating the relationship of the TSF-Q to different instruments. Two groups were studied: one comprising 146 patients with a diagnosis of anorexia (n = 82), bulimia (n = 33) or unspecified eating disorder (n = 31), and another group of 115 undergraduates with no history of psychological disorder. All participants completed the TSF-Q, TAF-Q, EDI-2, STAI, BDI and SCL-90-R. Differences in TSF-Q scores between the diagnostic subgroups were also analyzed. Two factors were obtained which coincided with the two sections indicated by the authors of the questionnaire: conceptual and interpretative. The internal consistency of the TSF-Q and its subscales was determined by means of Cronbach's alpha, with values ranging between .93 and .96. The correlations with other instruments reflected adequate validity. There were no significant differences between the diagnostic subgroups. The Spanish version of the TSF-Q meets the psychometric requirements for measuring thought-shape fusion and shows adequate internal consistency and validity.     

Measurement Equivalence Between Applicant and Incumbent Groups: An IRT Analysis of Personality Scales

Incumbents are often used in the development and validation of a wide variety of personnel selection instruments, including noncognitive instruments such as personality tests. However, the degree to which assumed motivational factors impact the measurement equivalence and validity of tests developed using incumbents has not been adequately addressed. This study addressed this issue by examining the measurement equivalence of 6 personality scales between a group applying for jobs as sales managers in a large retail organization (N = 999) and a group of sales managers currently employed in that organization (N = 796). A graded item response theory model (Samejima, 1969) was fit to the personality scales in each group. Results indicated that moderately large differences existed in personality scale scores (approximately 1/2 standard deviation units) but only one of the six scales contained any items that evidenced differential item functioning and no scales evidenced differential test functioning. In addition, person-level analyses showed no apparent differences across groups in aberrant responding. The results suggest that personality measures used for selection retain similar psychometric properties to those used in incumbent validation studies.     

Concurrent validity of four androgyny instruments

Four sex-role instruments were administered to a group of 281 graduate and undergraduate students at an urban university. Concurrent validity was examined through the use of correlational and factor-analytic techniques. Seven problems in interpretation of the categorizations made by the several instruments were identified: (1) item homogeneity of individual scales; (2) convergence of corresponding scales and divergence of scales from their complements; (3) multifactoriality; (4) factor complexity; (5) distribution of content across instruments; (6) interaction between factor complexity and content balance; and (7) correspondence of classifications of subjects across instruments.     

Reliability and construct validity of the psychopathy checklist - revised for Latino, European American, and African American male inmates

The utility of the psychopathy construct in predicting laboratory deficits, criminal behavior, response to intervention, and recidivism has been well documented in European American populations. However, less is known about the manifestation and correlates of psychopathy in Latino and African American populations. The present study examined the reliability and construct validity of the Psychopathy Checklist - Revised (PCL-R; R. D. Hare, 2003) in 83 Latino inmates compared with matched samples of African Americans and European Americans. Results provide preliminary evidence that the PCL-R provides a reliable and valid measure of psychopathy in Latinos, with generally similar patterns emerging across external correlates; however, some ethnic group differences were noted for relationships between psychopathy indicators and some external correlates.     

Confirmatory Factor Analysis of Multitrait-Multimethod Data: The Construct Validation of Multidimensional Self-Concept Responses

ABSTRACT The multitrait-multimethod (MTMM) design is frequently used to test construct validity and is particularly appropriate for evaluating multidimensional instruments in personality research Despite its popularity there are important problems with both the traditional and confirmatory factor analysis (CFA) approaches to MTMM data In previous research (1989a, 1989b) I described a new CFA approach that I claimed to be relatively immune to many of these problems This approach is applied here to test the construct validity of preadolescent responses to three multidimensional self-concept instruments The substantive contribution of the present investigation is to demonstrate good support for the construct validity of two of the three instruments The methodological contributions are to further substantiate my claims about the new CFA approaches and to demonstrate this approach in a substantively meaningful context that has broad applicability to other personality research     

评价中心的构想效度和结构模型

采用多质多法和验证性因素分析的方法,对以无领导小组讨论、文件筐和人格测验构成的一个评价中心的构想效度和结构模型进行了研究。通过对136名被试在四个测评维度上的施测,其结果表明,在评价中心中会聚效度低于区分效度,影响评价中心测评结果的主要因素是测评方法而不是测评维度,从而得到了一个以测评方法为潜变量的评价中心结构模型。从该结构模型来看,评价中心之所以起作用是由于其多个测评方法(情景)的结果。表明测评情景对于构建评价中心有着至关重要的作用。     

The search for dimensional structure differences between normality and abnormality: a statistical review of published data on personality and psychopathology

A statistical review of published data for 37 personality and psychopathology inventories was conducted to determine whether there are dimensional structure differences between clinical and nonclinical respondents. Correlation and factor-loading matrices from multiscale inventories and from specialized measures were tested for structural invariance across populations. There was relatively consistent evidence for high levels of similarity between normal and abnormal populations both in the number of factors that exist in the data matrices and in the factor patterns. The dimensional universes of normality and abnormality are apparently the same, at least according to data derived from contemporary assessment instruments. Categorical-taxonic differences between clinical and nonclinical populations, which were not examined, may nevertheless exist within contexts of dimensional structure similarity.     

Measuring Punishment and Reward Sensitivity in children and adolescents with a parent-report version of the Bis/Bas-scales

The present study examined the validity and reliability of the parent-report version of Carver and White's BIS/BAS-scales for children, measuring Punishment and Reward Sensitivity (PS and RS). Gender and age differences in PS and RS were explored. Mothers (n = 546, 62% girls, 2–18 years) completed the BIS/BAS parent-version and other instruments assessing PS/RS, personality and psychopathology. Reliability of most (sub-)scales was at least acceptable. The BIS/BAS-scales had meaningful relationships with other instruments assessing PS/RS, personality and psychopathology. There were significant age differences on BIS/BAS-scores, but no significant gender differences. The BIS/BAS parent-version is a valid and reliable parent-report PS/RS instrument for children and adolescents.     

Personality characteristics of alcohol dependent inpatients: relationship of MCMI subtypes to self-reported drinking behavior.

Recent cluster analytic research with alcoholic inpatients has demonstrated the existence of several Millon Clinical Multiaxial Inventory (MCMI) clusters that appear to be consistent across different subject samples. The validity of these data would be strengthened by a statistical demonstration of the similarity of attained clusters across studies--a demonstration of concordance of subject classification across different clustering techniques on the same data set- and the inclusion of external, independent measures against which to evaluate the predictive validity of the cluster typology. We found a high level of concordance in subject classification across different clustering methods on the same data set and a high level of agreement with cluster typologies attained in previous studies. Subsequent multivariate analyses employing independent scales measuring various aspects of alcohol use confirmed differences among cluster members on perceived benefits of alcohol use and deleterious effects of alcohol use. The prominent differences in alcohol use along with a rationale for their development are discussed.     

Evaluating Three Models of Intimate Relationships

The purpose of this study was to evaluate three models of intimate relationships: attachment, elementary pragmatic, and selfhood. These models were evaluated with three instruments respectively constructed to evaluate the validity of each model. One convenience group of functioning individuals and three experimental groups of inpatients and outpatients were administered the three instruments. All three instruments showed relatively high internal consistency and discriminated among the four groups. Significant correlations were found among the measure of attachment with measures of the pragmatic elementary and selfhood models. These correlations indicate that all three instruments were measuring similar aspects of intimate relationships. Clinical and preventive implications and applications of these models are briefly discussed.     

Doping use in sport teams: The development and validation of measures of team-based efficacy beliefs and moral disengagement from a cross-national perspective

ObjectivesThe main goal of this research focused on the development and validation of three instruments designed to assess athletes' self-regulatory efficacy in team contexts, team collective efficacy and team moral disengagement with relevance for doping use across three European countries.DesignThe research relied on three distinct studies. A first qualitative study focused on item development. The second study assessed the factor structure and internal reliability of each of the new team instruments. The third study provided evidence for instrument validity by assessing the hypothesis that efficacy measures and moral disengagement would contribute to team athletes' doping intentions. The latter two studies also focused on the relations among measures and on measurement reliability, both within and across countries.MethodThe first study relied on focus group data collected from twenty-one team sport professionals (mean age = 34; SD = 11.65). Four hundred and fourteen adolescent athletes (mean age = 16.69; SD = 1.55) participated in the second study, whereas seven hundred forty-nine adolescent team athletes (mean age = 16.43; SD = 1.69) participated in the third study. For the latter two studies, team athletes were recruited across Italy, Germany and Greece and provided data on the new team measures. Only athletes participating in the third study provided data on doping intentions.ResultsThe findings of the three studies supported the empirical goals of the investigation and provided evidence for the factor structure, reliability and validity of the team instruments. Furthermore, multi-group findings supported the hypothesis that the new instruments would have equivalent measurement and validity characteristics across the three European countries. The conclusions focus on the conceptual and practical implications of these findings.     

Capturing the psychopathic female: a prototypicality analysis of the comprehensive assessment of psychopathic personality (CAPP) across gender

Psychopathy in women has been subject to little systematic investigation; no coherent conceptualization of the disorder in women exists. Previous research is constrained by the reliance both on a male conceptualization of psychopathy and on assessment instruments developed, and primarily validated, with men. This study utilized a newly developed, broad, personality-based, and gender-sensitive lexical model of psychopathy, the Comprehensive Assessment of Psychopathic Personality (CAPP). Prototypical analysis was used to explore gender differences in the construct and the content validity of the CAPP model across gender. Symptoms were rated by international mental health professionals (N = 132). The findings support the content validity of the CAPP across gender and suggest that – at symptom and domain levels – psychopathic men and women have key similarities, but also that important gender differences exists. This has implications for the assessment of psychopathy in women, and has relevance for the proposed revision of diagnostic criteria for personality disorders. Copyright © 2011 John Wiley & Sons, Ltd.