A checklist for testing measurement invariance

The analysis of measurement invariance of latent constructs is important in research across groups, or across time. By establishing whether factor loadings, intercepts and residual variances are equivalent in a factor model that measures a latent concept, we can assure that comparisons that are made on the latent variable are valid across groups or time. Establishing measurement invariance involves running a set of increasingly constrained structural equation models, and testing whether differences between these models are significant. This paper provides a step-by-step guide to analysing measurement invariance.     

The Childhood Executive Functioning Inventory (CHEXI): Factor structure,measurement invariance,and correlates in US preschoolers

This study tests the factor structure, measurement invariance, and correlates of the Childhood Executive Functioning Inventory (CHEXI) with a large and diverse sample of 3- to 5-year-olds (n = 844). Consistent with previous studies, a two-factor model that distinguishes working memory from inhibition provides the best fit to the observed data. This two-factor model has been shown to demonstrate strong measurement invariance for different subgroups of children (boys vs. girls, high vs. low income). Whereas boys tend to have greater working memory and inhibition difficulties (Cohen’s d = 0.15 and 0.20, respectively), children from low-income households tend to have more working memory problems than their peers from high-income households (Cohen’s d = 0.25). Finally, correlations between CHEXI scores, examiner reports of child behavior, and child performance on a battery of executive function (EF) tasks were investigated. CHEXI scores were found to be more consistently related to examiner reports of child behavior than child performance on EF tasks. Tthe strengths and weaknesses of the CHEXI as a questionnaire measure of EF are discussed, and directions for future research are suggested.     

Measurement invariance of the Satisfaction With Life Scale (SWLS) by gender and age in Angola

Subjective well-being is a research arena that has grown almost exponentially: over the last 20 years, the number of publications on subjective well-being has increased approximately 16-fold (Diener, 2009). The cognitive aspect of subjective well-being or life satisfaction is referred to a conscious cognitive judgment of life (Diener, Emmons, Larsen, & Griffin 1985), in which person’s quality of life is globally assessed (Shin & Johnson, 1978). The Satisfaction With Life Scale (SWLS, Diener et al., 1985) is the most widely used instrument for its measurement. A reliable, valid and invariant measurement is critical for meaningful comparisons. The aim of this study is to examine the configural, metric and scalar invariance across age in the Portuguese version of the SWLS with a sample of 5630 Angolans. A standard measurement invariance procedure has been applied both across gender and age. Results shown that scalar invariance of the SWLS held across gender and age. This strong invariance allowed for meaningful latent mean comparisons. There were latent mean differences due to gender and age, but while gender differences were modest, the age differences were larger. Results are discussed in their relation with existing literature.     

The assessment of school engagement: examining dimensionality and measurement invariance by gender and race/ethnicity

The construct of school engagement has attracted growing interest as a way to ameliorate the decline in academic achievement and increase in dropout rates. The current study tested the fit of a second-order multidimensional factor model of school engagement, using large-scale representative data on 1103 students in middle school. In order to make valid model comparisons by group, we evaluated the extent to which the measurement structure of this model was invariant by gender and by race/ethnicity (European-American vs. African-American students). Finally, we examined differences in latent factor means by these same groups. From our confirmatory factor analyses, we concluded that school engagement was a multidimensional construct, with evidence to support the hypothesized second-order engagement factor structure with behavioral, emotional, and cognitive dimensions. In this sample, boys and girls did not substantially differ, nor did European-American and African-American students, in terms of the underlying constructs of engagement and the composition of these constructs. Finally, there were substantial differences in behavioral and emotional engagement by gender and by racial/ethnic groups in terms of second-order factor mean differences.     

The personality differentiation by intelligence hypothesis: A measurement invariance investigation

Stability in the measurement of personality is crucial to its construct and predictive validity. The personality differentiation by intelligence hypothesis suggests that intelligence may be a threat to the measurement of personality. This study builds on existing literature by applying multigroup confirmatory factor analytic measurement invariance techniques to the item-level responses of a commonly used personality inventory. In contrast to recently published findings, our results suggest that the measurement of personality is not equivalent across intelligence levels. Though these results have important implications for the use of personality in research and practice, we maintain that personality measures are still important predictors of key criteria. We discuss future research directions, and hope that the current study highlights the need to increase the sophistication of currently available measurement techniques, and will bring continued examination of the personality differentiation by intelligence hypothesis.     

Measurement invariance across gender and ethnicity on the Emotional Quotient Inventory 2.0

This study investigated the measurement invariance of the Emotional Quotient Inventory 2.0 across gender and ethnic groups in South Africa. The sample consisted of 1 144 working adults (female = 49.8%. black = 36.2%, white = 29.8%; other = 34%; age range = 18 to 74 years). Multiple group factor analysis was conducted to test for configural, metric, and scalar invariance. Results suggest that the assessment is largely invariant at configural and metric levels across gender and ethnic groups. At scalar level, full invariance was achieved for ten of the fifteen scales across gender and one scale across ethnicity. Partial scalar models were explored for all other scales. Overall, the evidence seems to suggest that the scores from the Emotional Quotient Inventory 2.0 scales had similar meanings for different gender and ethnic groups. However, for a few scales across gender, and more so for ethnicity, it seems that participant responses might also be influenced by factors other than the underlying latent constructs. Thus, while the scales of the assessment appears to be largely invariant at the configural and metric levels, a more nuanced picture emerged at scalar level with partially invariant models, especially across ethnic groups.     

On the relationship between sources of within- and between-group differences and measurement invariance in the common factor model

Investigating sources of within- and between-group differences and measurement invariance (MI) across groups is fundamental to any meaningful group comparison based on observed test scores. It is shown that by placing certain restrictions on the multigroup confirmatory factor model, it is possible to investigate the hypothesis that within- and between-group differences are due to the same factors. Moreover, the modeling approach clarifies that absence of measurement bias implies common sources of within- and between-group variation. It is shown how the influence of background variables can be incorporated in the model. The advantages of the modeling approach as compared with other commonly used methods for group comparisons is discussed and illustrated by means of an analysis of empirical data.     

Measurement invariance,factor analysis and factorial invariance

Several concepts are introduced and defined: measurement invariance, structural bias, weak measurement invariance, strong factorial invariance, and strict factorial invariance. It is shown that factorial invariance has implications for (weak) measurement invariance. Definitions of fairness in employment/admissions testing and salary equity are provided and it is argued that strict factorial invariance is required for fairness/equity to exist. Implications for item and test bias are developed and it is argued that item or test bias probably depends on the existence of latent variables that are irrelevant to the primary goal of test constructers.Presidential address delivered at the Annual Meeting of the Psychometric Society in Berkeley, California, June 18–20, 1993.     

Measurement invariance in mentoring research: A cross-cultural examination across Taiwan and the U.S.

Workplace mentoring in the international context is an emerging research area with significant potential for global integration. However, although measurement equivalence is a prerequisite for examining cross-cultural differences, this assumption has yet to be examined in mentoring research. This study contributes to the mentoring literature by assessing the measurement equivalence of the Mentoring Functions Questionnaire (MFQ-9) across two diverse cultural settings, the U.S. and Taiwan. Results of a series of multi-group confirmatory factor analyses supported full configural invariance, full metric invariance, and partial scalar invariance across the two groups. These findings suggest MFQ-9 may provide acceptable comparisons and meaningful interpretations across cultures. Implications for future international mentoring research and managerial practice are discussed.     

Socially Desirable Responding and Impression Management in the Endorsement of Love Styles

In 2 experiments, the researcher investigated the social desirability of different love styles (Eros, Ludus, Storge, Pragma, Mania, and Agape). In Experiment 1, the Marlowe-Crowne measure of social desirability (D. P. Crowne & D. Marlowe, 1960) was correlated negatively with possessive, dependent (Mania) love styles in both men and women. In men, social desirability was correlated positively with romantic, passionate love (Eros) and game-playing love (Ludus), but negatively with all-giving, selfless love (Agape). In women, social desirability was correlated positively with Agape, but negatively with Ludus. In Experiment 2, the researcher replicated these findings using an impression management manipulation (good, bad, and honest responding). The gender differences in social desirability of different love styles are explained in terms of traditional and nontraditional gender role socialization.     

Development of the Inventory of College Students' Resilience and evaluating the measurement invariance

In order to help students adapt well to stressful situations or adversity, many researchers have explored the issue of students' resilience. The main purpose of this study was to develop the Inventory of College Students' Resilience (ICSR), for assessing resilience of college students. The present study recruited 993 participants to conduct item analysis, exploratory factor analysis and reliability analysis. We then administered the questionnaire to another 1490 participants in order to conduct confirmatory factor analysis and assess criterion-related validity. The results showed the proposed model fit the data well and the ICSR had both internal consistency and criterion-related validity. The ICSR was also found to have measurement invariance across gender. We conclude that the ICSR can help students to identify strengths and weaknesses in order to reinforce their resilience and improve their response to life stress and trauma.     

Measurement invariance of personality traits from a five-factor model perspective: multi-group confirmatory factor analyses of the HP5 inventory

Associations between personality and health may be influenced by contextual factors varying across studied cohorts. Moreover, measurement bias related to contextual factors could also influence these associations. Gender and age have been discussed as important contextual factors when studying personality and health outcomes. An examination of measurement invariance across gender and age may therefore be warranted. In the present study, measurement invariance across age and gender was examined for a five-factor model of personality traits in which scales have been included to operationalise lower-order traits related to health-behaviour or health outcomes. Using multi-group confirmatory factor analysis on a population based cohort (n= 5700), all important parameters of the measurement model were found to be invariant. On the basis of this invariant model, hypotheses of mean differences across age and gender in personality traits were tested. These results were discussed in relation to meta-analyses on personality change and gender differences in personality.     

Cross-validation of the very short form of the Physical Self-Inventory (PSI-VS): invariance across genders, age groups, ethnicities and weight statuses

In a recent review of various physical self-concept instruments, Marsh and Cheng (in press) noted that the very short 12-item version of the French Physical Self-Inventory (PSI-VS) represents an important contribution to applied research but that further research was needed to investigate the robustness of its psychometric properties in new and diversified samples. The present study was designed to answer these questions based on a sample of 1103 normally achieving French adolescents. The results show that the PSI-VS measurement model is quite robust and fully invariant across subgroups of students formed according to gender, weight, age and ethnicity. The results also confirm the convergent validity and scale score reliability of the PSI-VS subscales.     

Psychometric properties of the Coach-Athlete Relationship Questionnaire (CART-Q) in seven countries

Objectives

In the sport context, the coach-athlete relationship has been viewed and studied as a universal phenomenon. It is thus important to assess the universality of the psychometric scale that is used to measure the quality of coach-athlete relationships. The present study examined the cultural equivalence or measurement invariance of the Coach-Athlete Relationship Questionnaire (11-item, athlete version CART-Q).

Design

Cross-sectional study.

Methods

A total of 1363 athletes from Belgium (N = 200), Britain (N = 382), China (N = 200), Greece (N = 115), Spain (N = 120), Sweden (N = 169), and United States of America (N = 177) completed the CART-Q.

Results

Multi-group mean and covariance structure (MACS) analyses supported the factorial validity of the CART-Q in a three-first-order factor model across the seven countries. An examination of the latent mean differences of the CART-Q revealed some variation in terms of the intensity that athletes perceive in the quality of the relationship with their coach across the different countries.

Conclusions

Overall, these results supply additional evidence of the psychometric properties of the CART-Q and highlight that it is a sound instrument that can be applied cross-culturally.     

Measurement invariance of the Mental Health Continuum-Short Form (MHC-SF) across three cultural groups

This study investigated the factorial structure and invariance of the Mental Health Continuum-Short Form (MHC-SF) across cultural groups from three nations, namely, the Netherlands, South Africa, and Iran (N = 1120). The three-dimensional structure of mental well-being was supported in all the groups. The results of measurement invariance testing confirmed the full metric and partial scalar invariance of the MHC-SF. The study also compared the latent means for the aspects of mental well-being in the three cultural groups, and found significant differences. The significance and implications of the results are discussed.     

Assessing conspiracist beliefs across societies: Validation of the Serbian adaptation of the Generic Conspiracist Beliefs Scale (GCBS)

Initial validation of the Generic Conspiracist Beliefs Scale (GCBS) identified five correlated factors: Government Malfeasance, Malevolent Global Conspiracies, Extraterrestrial Cover-up, Personal Well-being and Control of Information. Despite replications of this structure, ensuing research, mostly using translated versions of the GCBS (i.e., Persian and Japanese), has reported factorial alternatives. Concomitantly, researchers postulated that belief in conspiracies was higher in historically troubled societies (e.g., Serbia). To further assess the cultural stability of the five correlated factor solution and test the assumption that GCBS scores are affected by social-political instability, a Serbian adaptation of the GCBS was assessed alongside the standard English language version. Data were collected from Serbian (n = 259, 44.40% males) and UK samples (n = 402, 52.25% males). Within samples, confirmatory factor analysis replicated the five-factor solution. Additionally, national and gender invariance was confirmed. National comparisons (Serbia vs. UK) revealed that the Serbian sample scored higher on Government Malfeasance, Malevolent Global Conspiracies, Personal Well-being and Control of Information, whereas the UK sample scored higher on Extraterrestrial Cover-up. There were no gender differences on GCBS factors. Findings indicated that while the Serbian GCBS was interpreted similarly to the GCBS, societal factors influenced endorsement of belief facets.     

Personal assessment of intimacy in relationships: Validity and measurement invariance across gender

IntroductionIntimacy is central in close relationships and a variety of definitions and instruments exist in scientific literature, making this concept complex to investigate. Furthermore, gender differences were identified in the definition and experience of intimacy.ObjectivesThis study aims to confirm the three-factor structure of the personal assessment of intimacy in relationships (PAIR, Schaefer & Olson, 1981), commonly used in research and marital therapy, developed by Moore et al. (1998) and to examine the measurement equivalence of the scale across gender.MethodA confirmatory factor analysis (CFA) was performed on the responses of 313 women and 251 men in committed relationships in a French population. Measurement equivalence of the scale across gender was then conducted with the best-fitting model.ResultsThe results of CFA and post-hoc analyses revealed that the three-factor model revised with 18 items was the best fit to the data: (1) engagement (10 items), (2) communication (5 items), (3) shared friends (3 items). PAIR-18 displayed a lack of metric and scalar invariance across gender but estimated means of latent variables is not very different between the full invariant model and the unconstraint models.ConclusionNevertheless, consideration of gender differences in the construct of intimacy is very important for theoretical research on intimacy in romantic relationships and for the interventions of marital therapists based on intimacy.     

Measurement invariance of the Emotion Regulation Questionnaire (ERQ). A cross-national validity study

The goal of this research was to examine the measurement invariance of the Emotion Regulation Questionnaire (ERQ; Gross & John, 2003) across two European nations. Participants were Italian and German undergraduate students. First, confirmatory factor analysis was used to test for the two-factor structure of the ERQ; subsequently, measurement invariance was analysed. The results showed acceptable fit indices for the German and Italian sample; however, results with regard to the Italian sample showed lower fit indices than results regarding the German one. Measurement invariance of the ERQ across Italy and Germany was confirmed. Despite the fact that we assumed measurement invariance of the ERQ, future research is needed to deal with the lower fit regarding the Italian version.     

Measuring individualism and collectivism: The importance of considering differential components,reference groups,and measurement invariance

A new instrument of individualism and collectivism (I/C) was developed and three key issues in I/C measurement were addressed: differentiating components of I/C, understanding the impact of reference groups, and testing of measurement invariance. Three components of I/C were assessed in China and the U.S.: independence, competitiveness, and uniqueness for individualism; considering of one’s decisions on others, sharing of positive outcomes, and sharing of negative outcomes for collectivism. Collectivism was measured with respect to parents, friends, and general others. Results indicate that Chinese participants are less unique but more independent and competitive than their counterparts in the U.S. The expected cultural difference is found for parent collectivism across all three components and for sharing negative outcome with respect to all three reference groups. These results suggest that individualism is a multidimensional construct, whereas the dimensionality of collectivism appears to be a function of social distance. Measurement invariance was tested at configural, factor loading, and intercept levels for all components of I/C.     

Measurement invariance and age-related differences of trait anger across the adult lifespan

The present study examined measurement invariance of trait anger with respect to age. In addition, age-related differences in latent variables (factor variances, factor covariances, and factor means) were investigated. A sample of 1216 participants, divided into six age groups, ranging from under 30 to over 70 years, was tested using the angry temperament and angry reaction scales from the State-Trait Anger Expression Inventory (STAXI; Spielberger, 1988). Results show that strong measurement invariance held across the six age groups. For both angry temperament and angry reaction, factor variances tended to decrease into middle age and then increase again into old age. Factor correlations decreased into old age, implying a differentiation of trait anger. Finally, regarding factor means, older adults showed less anger than younger adults, suggesting higher levels of self-regulation or less exposure to anger-provoking contexts in later life.