Invariance in Measurement and Prediction Revisited

Borsboom (Psychometrika, 71:425–440, 2006) noted that recent work on measurement invariance (MI) and predictive invariance (PI) has had little impact on the practice of measurement in psychology. To understand this contention, the definitions of MI and PI are reviewed, followed by results on the consistency between the two forms of invariance in the general case. The special parametric cases of factor analysis (strict factorial invariance) and linear regression analyses (strong regression invariance) are then described, along with findings on the inconsistency between the two forms of invariance in this context. Two numerical examples of inconsistency are reviewed in detail. The impact of violations of MI on accuracy of selection is illustrated. Finally, reasons for the slow dissemination of work on invariance are discussed, and the prospects for altering this situation are weighed. This paper is based on the Presidential Address given at the International Meeting of the Psychometric Society in Tokyo, Japan, on July 11, 2007. This research was supported by National Institute of Mental Health grants 1P30 MH 068685-01A1 and RO1 MH64707-01.     

Self-Other Differentiation Scale: Dimensionality,IRT Parameterization,and Measurement Invariance

The Self-Other Differentiation Scale (Olver, Aries, &; Batgos, 1989 Olver, R. R., Aries, E., &; Batgos, J. (1989). Self-other differentiation and the mother-child relationship: The effects of sex and birth order. The Journal of Genetic Psychology, 150, 311–321. doi:10.1080/00221325.1989.9914600[Taylor &; Francis Online] , [Google Scholar]) is a self-report instrument assessing the experience of a separate sense of self from others. The authors aimed to examine its dimensionality, reliability, and measurement invariance across gender. It was completed by 348 participants (48% men) from 17 to 30 years old in Study 1, 348 participants (40% men) from 18 to 28 years old in Study 2, and 1,068 participants (49% men) from 17 to 28 years old in Study 3. The results supported the hypothesis of just one factor underlying the scale; they also showed an appropriate internal consistency and a partial measurement invariance across gender. Results also showed evidence for a 10-item version of the scale. Globally, the Self-Other Differentiation Scale can be considered a good scale to assess individual's sense of differentiation of one's own sense of self from others.     

Statistical Power for a Simultaneous Test of Factorial and Predictive Invariance

A common finding in studies of differential prediction across groups is that although regression slopes are the same or similar across groups, group differences exist in regression intercepts. Building on earlier work by Birnbaum (1979) Birnbaum, M. H. 1979. “Procedures for the detection and correction of salary inequities.”. In Salary equity Edited by: Pezzullo, T. R. and Brittingham, B. F. 121–144. Lexington, MA: Lexington Books..  [Google Scholar], Millsap (1998) Millsap, R. E. 1998. Group differences in regression intercepts: Implications for factorial invariance. Multivariate Behavioral Research, 33: 403–424. [Taylor & Francis Online], [Web of Science ®] , [Google Scholar] presented an invariant factor model that would explain such intercept differences as arising due to unique variance or measurement error in the predictors and criterion. The model can be tested even when only a single predictor (p = 1) and criterion are present, but Millsap (1998) Millsap, R. E. 1998. Group differences in regression intercepts: Implications for factorial invariance. Multivariate Behavioral Research, 33: 403–424. [Taylor & Francis Online], [Web of Science ®] , [Google Scholar] only illustrated the use of the test in very large samples. The present study investigates the Type I error and power performance of the test of fit across a range of sample sizes and effect sizes in simulated data, using an alternative model that includes group differences in measurement intercepts. Surprising results showed little power for the chi-square test in the p = 1 case but good power for a diagnostic indicator based on negative unique variance estimates. The p = 2 case showed better power in modest samples. A real example from an application on reading disabilities in Share and Silva (2003) Share, D. L. and Silva, P. A. 2003. Gender bias in IQ-discrepancy and post-discrepancy definitions of reading disability. Journal of Learning Disabilities, 36: 4–14. [Crossref], [PubMed], [Web of Science ®] , [Google Scholar] is given. The implications of the results for studies of differential prediction involving group differences in regression intercepts are discussed.     

Perfectionism in Argentina and the United States: Measurement Structure,Invariance, and Implications for Depression

The Revised Almost Perfect Scale (APS–R; Slaney, Mobley, Trippi, Ashby, &; Johnson, 1996 Slaney, R. B., Mobley, M., Trippi, J., Ashby, J. S., &; Johnson, D. (1996). Almost Perfect Scale Revised. Psyctests. doi:10.1037/t02161-000[Crossref] , [Google Scholar]) was designed to measure perfectionistic strivings (standards), perfectionistic concerns (discrepancy), and preferences for neatness and organization (order). In this study, English and Spanish (Arana, Keegan, &; Rutsztein, 2009 Arana, F. G., Keegan, E. G., &; Rutsztein, G. (2009). Adaptación de una medida multidimensional de perfeccionismo: la Almost Perfect Scale–Revised (APS–R). Un estudio preliminar sobre sus propiedades psicométricas en una muestra de estudiantes universitarios argentinos [Adaptation of a measure of multidimensional perfectionism: The Almost Perfect Scale–Revised. A preliminary study about its psychometrical properties on a sample of Argentine university students]. Evaluar, 9, 35–53. [Google Scholar]) versions of the APS–R were compared in samples of 283 college students in Argentina and 311 U.S. students. The results offered support for cross-national scalar invariance for discrepancy and order items but standards items, with a few exceptions, raised concerns about their cross-national comparability. Tests of latent means revealed no differences between the countries for discrepancy, but the order factor mean was higher in the U.S. sample. Age and gender had negligible effects on measurement models. Discrepancy was strongly related to depressive symptoms for both samples. Unlike in the U.S. sample, there was a significant association for the Argentina sample between standards and discrepancy. Overall, results indicated that discrepancy items performed well, order items were adequate with some measurement adjustments, and standards items should be revisited for their cross-cultural utility. Translation issues and general values in Argentina and the United States (e.g., collectivism–individualism) are discussed that might affect how perfectionism is understood and measured in different cultural contexts.     

心理测量平衡性研究与实例

心理测量研究中,测量不变性(或称平衡性)是量表稳定性问题中的一个难题而且在比较研究中受到特别重视。结构方程模型因在平衡性形式捕捉方面功能强大而受到广泛应用。该研究讨论了测量平衡性的各种形式并演示了应用结构方程模型评估测量平衡性的过程。     

Gender-Based Measurement Invariance of the Emotion Regulation Questionnaire for Children and Adolescents

The aim of this study was to evaluate the measurement invariance of the Chinese version of the Emotion Regulation Questionnaire for Children and Adolescents (ERQ-CA; Gullone and Taffe (Psychological Assessment, 24(2), 409–417, 2011)) across gender. A total of 1388 Chinese adolescents (48.1% males), ranging in age from 11 to 16 years (M = 12.9, SD = 1.09) were recruited from four Chinese middle schools and completed the ERQ-CA. The Cronbach’s α values, omega coefficients, and item-total correlations for the two subscales of the ERQ-CA indicated adequate internal consistency reliability. The results of confirmatory factor analysis and multi-group confirmatory factor analysis revealed that all configural, metric, scalar, and partial strict invariance models provided acceptable fit to the data. The result of latent means analysis demonstrated statistically non-significant differences between boys and girls. In conclusion, these findings suggest that the ERQ-CA displays appropriate measurement invariance across gender for Chinese youth of ages 11 to 16.

     

Testing the Internal Consent Scale for Measurement Invariance Across Women and Men

The Internal Consent Scale (ICS) was created to measure feelings associated with a person’s willingness to engage in partnered sexual activity. Although previous studies using the ICS have assessed gender differences, evidence has not been provided to suggest that the ICS functions similarly for women and men. Using data from an online cross-sectional survey of adults (N = 874; 53.1% women), we subjected the 25-item ICS to tests of measurement invariance across gender. We found that only partial measurement invariance was tenable, which indicated that direct comparisons across gender should be interpreted with caution when using the ICS. Therefore, we created a gender-invariant short form. In support of construct validity, we found that this 15-item ICS–Short Form demonstrated similar associations with measures of sexual consent communication as the full 25-item ICS. If researchers aim to compare women and men on internal sexual consent, we recommend using the 15-item ICS–Short Form. Cognitive interviews should be conducted to further understand how women and men might differentially interpret ICS items.

     

Measurement Invariance and Informant Discrepancies of the KIDSCREEN-27 in Children with Mental Disorder

Applied Research in Quality of Life - Study objectives were to examine measurement invariance of the KIDSCREEN-27 between children with mental disorder and their parents; compare health-related...     

Testing for Measurement Invariance with Respect to an Ordinal Variable

Researchers are often interested in testing for measurement invariance with respect to an ordinal auxiliary variable such as age group, income class, or school grade. In a factor-analytic context, these tests are traditionally carried out via a likelihood ratio test statistic comparing a model where parameters differ across groups to a model where parameters are equal across groups. This test neglects the fact that the auxiliary variable is ordinal, and it is also known to be overly sensitive at large sample sizes. In this paper, we propose test statistics that explicitly account for the ordinality of the auxiliary variable, resulting in higher power against “monotonic” violations of measurement invariance and lower power against “non-monotonic” ones. The statistics are derived from a family of tests based on stochastic processes that have recently received attention in the psychometric literature. The statistics are illustrated via an application involving real data, and their performance is studied via simulation.     

Loneliness and Attitudes toward Aloneness in Belgian Adolescents: Measurement Invariance across Language,Age, and Gender Groups

In this study, the factor structure and psychometric properties of a French adaptation of the well-established Loneliness and Aloneness Scale for Children and Adolescents (LACA) was investigated in a French-speaking sample of Belgian adolescents (N?=?641; Mage?=?14.35, SD?=?2.03; 53.4% girls). In addition, measurement invariance analyses across the two main language groups in Belgium (i.e., Dutch-speaking and French-speaking) and across age and gender were conducted on a combined sample of French-speaking and Dutch-speaking Belgian adolescents (N?=?1282; Mage?=?14.36 years, SD?=?2.03; 53.6% female). Subsequently, group mean differences across language, age, and gender groups were investigated. Convincing support was found for the expected four-factor structure of the French adaptation of the LACA and the measure showed good reliability. Results further indicated that Belgian adolescents from the two principal language groups and from different age and gender groups interpreted the items of the LACA similarly. Therefore, analyses of group mean differences could be compared. Given the small effect sizes that accompanied the obtained effects, replication research is needed to further investigate these group mean differences.     

Establishing Measurement Equivalence and Invariance in Longitudinal Data With Item Response Theory

If measurement invariance does not hold over 2 or more measurement occasions, differences in observed scores are not directly interpretable. Golembiewski, Billingsley, and Yeager (1976) identified 2 types of psychometric differences over time as beta change and gamma change. Gamma change is a fundamental change in thinking about the nature of a construct over time. Beta change can be described as respondents' change in calibration of the response scale over time. Recently, researchers have had considerable success establishing measurement invariance using confirmatory factor analytic (CFA) techniques. However, the use of item response theory (IRT) techniques for assessing item parameter drift can provide additional useful information regarding the psychometric equivalence of a measure over time that is not attainable with traditional CFA techniques. This article marries the terminology commonly used in CFA and IRT techniques and illustrates real advantages for identifying beta change over time with IRT methods rather than typical CFA methods, utilizing a longitudinal assessment of job satisfaction as an example.     

Rationality,Dispositions, and the Newcomb Paradox

Philosophical Studies -     

Belief,Credence, and the Preface Paradox

Many discussions of the ‘preface paradox’ assume that it is more troubling for deductive closure constraints on rational belief if outright belief is reducible to credence. I show that this is an error: we can generate the problem without assuming such reducibility. All that we need are some very weak normative assumptions about rational relationships between belief and credence. The only view that escapes my way of formulating the problem for the deductive closure constraint is in fact itself a reductive view: namely, the view that outright belief is credence 1. However, I argue that this view is unsustainable. Moreover, my version of the problem turns on no particular theory of evidence or evidential probability, and so cannot be avoided by adopting some revisionary such theory. In sum, deductive closure is in more serious, and more general, trouble than some have thought.     

Paradox,truth and logic part I: Paradox and truth

Conclusion The discussion of the semantics of inconsistent truth theories now comes to a pause. The preceding is of course but a sketch; many interesting questions remain to be answered. The second part of this essay, however, will not seek to answer them. Rather, I will turn to the discussion of the proof theory of truth theory: the local and global logic of truth.Under the first heading, I show how to replace the inductive construction of models with an appropriate infinitary proof theory, and relate this on the one hand to the so-called dependence approach to inductive truth theories (Davis, 1979; Yablo, 1982) and on the other to van Fraassen's fact semantics for relevance logic.Under the second heading, I offer formals systems which capture the inferences valid in all approximate models. Not surprisingly, these turn out to be relevant logics.With formalism in hand, I discuss finally the extent to which the gap and/or glut approach can in fact be said to solve the paradoxes; that is, to allow us to say that the very language we are speaking is of the sort described in our theory.     

Dimensionality and Measurement Invariance of a Brief Form of the Young Schema Questionnaire for Adolescents

The current work aims to develop and examine the psychometric properties of the Brief form of the Young Schema Questionnaire for Adolescents (B-YSQ-A), so that Early Maladaptive Schemas can be accurately measured in younger populations. Early Maladaptive Schemas are self-defeating core themes underlying maladaptive cognition, affect, and behavior. A community sample of 877 adolescents, aged 12–18 years old, filled out self-report instruments on maladaptive schemas, anger management, and internalizing and externalizing symptoms. The items composing the B-YSQ-A were selected based on statistical and content analyses criteria. Its internal structure, reliability, age- and gender-based invariance, and between-gender mean differences were examined, as well as its relationship with external variables. Results indicated a satisfying fit for the 18 original schema factors through confirmatory factor analysis. These schemas also showed adequate internal consistency and test-retest reliability, with the exception of the entitlement/grandiosity and self-sacrifice schemas. The B-YSQ-A was found to be age and gender invariant. Concerning gender differences, boys scored higher on the entitlement/grandiosity, insufficient self-control/discipline, approval/recognition seeking, unrelenting standards/hyper-criticalness and punitiveness schemas, whereas girls scored higher on the abandonment/instability, mistrust/abuse, and self-sacrifice schemas. Furthermore, evidence was found for construct validity in relation to measures of internalizing and externalizing symptomatology and of anger expression. Findings offer support for the use of the B-YSQ-A with adolescents, both for research and intervention purposes.     

Protection, Paradox, and Pretending

This paper proposes that psychopathology in children can be the result of an incongruity in the hierarchical organization of the family. The parents are in a superior position to the child by the fact of being parents, and yet the problem child assumes a superior position to the parents by protecting them through symptomatic behavior that often expresses metaphorically the parents' difficulties. The paper describes three paradoxical strategies for arranging that the parents solve the presenting problem of the child and the incongruity in the family hierarchy. The therapeutic techniques described are characterized by the use of communication modalities, such as dramatizations, pretending, and make-believe, that are appropriate to children.     

Invariance,Symmetry And Rationality

Using recent work by Forster and Sober, I identify circumstances in which appeals to symmetries in physical laws are rational with respect to the aim of predictive accuracy. I then consider a Bayesian account of symmetry, and argue that such an account faces serious problems explaining when and why appeals to symmetry would be rational.     

Truthlikeness and the Lottery Paradox via the Preface Paradox

In a 2017 AJP paper, Cevolani and Schurz (C&S) propose a novel solution to the Preface Paradox that appeals to the notion of expected truthlikeness. This discussion note extends and analyses their approach by applying it to the related Lottery Paradox.     

Measurement Invariance of the Adolescent Quality of Life-Mental Health Scale (AQOL-MHS) across Gender,Age and Treatment Context

The Adolescent Quality of Life-Mental Health Scale (AQOL-MHS) was designed to measure quality of life in clinical samples of Latino adolescents aged 12–18 years, but has also been used in community samples. The original measure included three factors: Emotional Regulation (ER), Self-Concept (SC) and Social Context (SoC). The goals of this study are to replicate the factor structure using confirmatory factor analysis (CFA), shorten the instrument and test the degree of measurement invariance across gender, age, and type of sample. Participants for the analyses (N?=?354) came from two populations in the San Juan Metropolitan Area: (1) adolescents from randomly selected households, using a multi-stage probability sampling design (n?=?295), and (2) adolescents receiving treatment at mental health clinics (n?=?59). We first carried out a conceptual item analysis for item reduction purposes and then assessed dimensional, configural, metric and scalar invariance for each factor using the Mplus software system. The original 3-factor structure was replicated with comparable model fit in each treatment context. Metric invariance was attained for all three scales across groups. Either full or partial scalar invariance was also observed with DIF in a total of 6 items. Invariance testing supports the use of the abridged 21 item version of the AQOL-MHS to compare diverse individuals with little bias using observed scores, but for refined estimates the ideal scoring will be from a latent variable model.