Evaluation on types of invariance in studying extreme response bias with an IRTree approach

In recent years, item response tree (IRTree) approaches have received increasing attention in the response style literature for their ability to partial out response style latent variables as well as associated item parameters. When an IRTree approach is adopted to measure extreme response styles, directional and content invariance could be assumed at the latent variable and item parameter levels. In this study, we propose to evaluate the empirical validity of these invariance assumptions by employing a general IRTree model with relaxed invariance assumptions. This would allow us to examine extreme response biases, beyond extreme response styles. With three empirical applications of the proposed evaluation, we find that relaxing some of the invariance assumptions improves the model fit, which suggests that not all assumed invariances are empirically supported. Specifically, at the latent variable level, we find reasonable evidence for directional invariance but mixed evidence for content invariance, although we also find that estimated correlations between content-specific extreme response latent variables are high, hinting at the potential presence of a general extreme response tendency. At the item parameter level, we find no directional or content invariance for thresholds and no content invariance for slopes. We discuss how the variant item parameter estimates obtained from a general IRTree model can offer useful insight to help us understand response bias related to extreme responding measured within the IRTree framework.     

Weak measurement invariance with respect to unmeasured variables: An implication of strict factorial invariance

Measurement bias refers to systematic differences across subpopulations in the relation between observed test scores and the latent variant underlying the test scores. Comparisons of subpopulations with the same score on the latent variable can be expected to have the same observed test score. Measurement invariance is therefore one of the key issues in psychological testing. It has been established that strict factorial invariance (SFI) with respect to a selection variable V almost certainly implies weak measurement invariance with respect to V: given SFI, means and variances of observed scores do not depend on V. It is shown that this result can be extended. SFI in groups derived by selection on V has implications not only for V but also for potentially biasing variables W, if W and the selection variable V and/or if W and the factor underlying the observed test scores are statistically dependent. Given SFI with respect to V and prior knowledge concerning these dependencies, it is not necessary to measure and model variables W in order to exclude them as potentially biasing variables if the investigation focuses on groups selected on V.     

Invariance in Measurement and Prediction Revisited

Borsboom (Psychometrika, 71:425–440, 2006) noted that recent work on measurement invariance (MI) and predictive invariance (PI) has had little impact on the practice of measurement in psychology. To understand this contention, the definitions of MI and PI are reviewed, followed by results on the consistency between the two forms of invariance in the general case. The special parametric cases of factor analysis (strict factorial invariance) and linear regression analyses (strong regression invariance) are then described, along with findings on the inconsistency between the two forms of invariance in this context. Two numerical examples of inconsistency are reviewed in detail. The impact of violations of MI on accuracy of selection is illustrated. Finally, reasons for the slow dissemination of work on invariance are discussed, and the prospects for altering this situation are weighed. This paper is based on the Presidential Address given at the International Meeting of the Psychometric Society in Tokyo, Japan, on July 11, 2007. This research was supported by National Institute of Mental Health grants 1P30 MH 068685-01A1 and RO1 MH64707-01.     

Factorial invariance,differential item functioning,and norms of the Orgasm Rating Scale

Background/Objective: Orgasm Rating Scale (ORS) assess the subjective orgasm experience in context of sexual relationship. It is composed of four dimensions attributed to the orgasm (Affective, Sensory, Intimacy, and Rewards). The purpose is to analyse the factorial invariance of the ORS across groups, to examine the metric equivalence across sex, and to present the standard scores. Method: A total of 1,472 Spanish adults (715 men and 757 women) were evaluated. They were distributed across age groups (18-34, 35-49 and 50 years old and older). Factorial invariance across different groups and the differential functioning of the items across sex were analyzed, internal consistency was examined, and the standard scores were developed. Results: The structure of the ORS showed strict measurement invariance across sex, relationship status, sexual orientation and education level. It also reached a scalar measurement invariance across age range and duration of the relationship. Some items showed a differential functioning between sexes. Conclusions: The Spanish version of the ORS is invariant across different groups at a factorial level, and it shows equivalence across sex in most of its items at a metric level. The standard scores allow a more accurate assessment of the subjective orgasm experience in context of sexual relationship.     

Bayesian tests of measurement invariance

Random item effects models provide a natural framework for the exploration of violations of measurement invariance without the need for anchor items. Within the random item effects modelling framework, Bayesian tests (Bayes factor, deviance information criterion) are proposed which enable multiple marginal invariance hypotheses to be tested simultaneously. The performance of the tests is evaluated with a simulation study which shows that the tests have high power and low Type I error rate. Data from the European Social Survey are used to test for measurement invariance of attitude towards immigrant items and to show that background information can be used to explain cross‐national variation in item functioning.     

Achievement Emotions Questionnaire-Mathematics (AEQ-M) in adolescents: Factorial structure,measurement invariance and convergent validity with personality

ABSTRACT

The Achievement Emotions Questionnaire-Mathematics (AEQ-M) is a self-report measure of emotions experienced in class, when self-studying, and during tests for the domain of mathematics. Our aim was to present a Portuguese version of this instrument for use with adolescents and to test its reliability, factorial structure, measurement invariance, and construct validity with personality dimensions. Our sample comprised 1,387 Portuguese students from the 7th, 8th and 9th grades (mean age = 13.2 years). Student responses to the AEQ-M were found to be reliable. Confirmatory factor analysis validated a seven-emotion × three-setting factorial structure. This model demonstrated measurement invariance across gender and grade. As a demonstration of construct validity, the emotions measured by the AEQ-M showed a pattern of associations with psychobiological personality dimensions that were in line with theoretical predictions. These results validate the AEQ-M as a suitable instrument for assessing adolescents’ mathematics-related achievement emotions and their associations with personality.     

Measuring attachment security via the Security Scale: Latent structure,invariance across mothers and fathers and convergent validity

The Security Scale (SS) is a widely used questionnaire measuring attachment towards mother and father in school-aged children. Whilst existing evidence supports concurrent and discriminant validity of the SS, its factorial structure remains largely underexplored. The current study examined the factorial structure of the SS, explored its measurement invariance across mother and father, and evaluated its convergent validity with the Coping Strategies Questionnaire in a sample of 149 Italian children aged 8–10 years. In statistical analyses, we accounted for the ordinal nature of the data and adopted an ad hoc bootstrap procedure for the estimation of measurement invariance. Results supported the factorial validity of the SS and the structural invariance of its underlying construct across parents. Evidence for convergent validity was also found. Overall, the Italian version of the SS is a promising tool to assess attachment in school-aged children.     

Measurement invariance of the Mental Health Continuum-Short Form (MHC-SF) across three cultural groups

This study investigated the factorial structure and invariance of the Mental Health Continuum-Short Form (MHC-SF) across cultural groups from three nations, namely, the Netherlands, South Africa, and Iran (N = 1120). The three-dimensional structure of mental well-being was supported in all the groups. The results of measurement invariance testing confirmed the full metric and partial scalar invariance of the MHC-SF. The study also compared the latent means for the aspects of mental well-being in the three cultural groups, and found significant differences. The significance and implications of the results are discussed.     

网络测验和纸笔测验的测量不变性研究：以生活满意度量表为例

以生活满意度量表为例,运用实证性因素分析,考察在中国文化下网络测验和传统纸笔测验之间的测量不变性。结果显示,网络测验和纸笔测验之间存在弱不变性,即网络测验和纸笔测验有着相同的测量单位;但网络测验和纸笔测验只存在部分的强不变性和部分的严格不变性,测验实施环境对结果的影响不可忽视。该研究表明,恰当设计的网络测验是可靠的,同时还提示,当一个测验在不同情境下运用时,检验测量不变性十分必要     

A gender-based measurement invariance study of the Sociocultural Attitudes Toward Appearance Questionnaire-3

The current study sought to explore the measurement invariance of the SATAQ-3 across gender using a single mixed gender sample consisting of 122 men and 268 women. Participants' age ranged from 18 to 36 years (M=19.6, SD=1.9). Preliminary results indicate that the 28 item scale was a poor fit for either gender in the current sample. Reverse scored items were deleted as they formed a unique method factor with low factor loadings. The resulting 21 items were a good fit to the hypothesized four factor model for both males and females and established evidence of both strict factorial invariance and population heterogeneity across groups. Coefficient alpha estimates of internal consistency reliability ranged from .79 to .94. These findings support use of the SATAQ-3 in mixed gender samples and validate previous research that reported analysis of gender-based mean differences.     

Age and gender invariance of self‐concept factor structure: An investigation of a newly developed Chinese self‐concept instrument

A new instrument, the Chinese Adolescent Self‐Esteem Scales (CASES), was developed to measure the self‐concepts of the young people in Hong Kong in seven aspects: social, academic, appearance, moral, family, physical/sport, and general self‐esteem. LISREL procedures were utilized to test the extent of factorial invariance for age and gender based on the responses to CASES of 551 Hong Kong adolescents. It was found that CASES possesses the necessary invariance properties for between‐group measurement in terms of the number and pattern of the underlying factors, item factor loadings, and inter‐factor relations, but not in terms of item uniqueness. Implications of these findings are discussed in terms of the use of CASES and of empirical support for the equivalence of self‐concept factor structure for age and gender groups for both Western and non‐Western adolescents.     

On the relationship between sources of within- and between-group differences and measurement invariance in the common factor model

Investigating sources of within- and between-group differences and measurement invariance (MI) across groups is fundamental to any meaningful group comparison based on observed test scores. It is shown that by placing certain restrictions on the multigroup confirmatory factor model, it is possible to investigate the hypothesis that within- and between-group differences are due to the same factors. Moreover, the modeling approach clarifies that absence of measurement bias implies common sources of within- and between-group variation. It is shown how the influence of background variables can be incorporated in the model. The advantages of the modeling approach as compared with other commonly used methods for group comparisons is discussed and illustrated by means of an analysis of empirical data.     

The Italian version of the Empathy Questionnaire for 18- to 36-months-old children: psychometric properties and measurement invariance across gender of the EmQue-I13

Empathy is the basic ability to respond affectively to the emotions of others and is observed early in human development. This study (N = 304) tested the Italian version of the Empathy Questionnaire (EmQue), which is a parent scale assessing empathy-related behaviors in toddlers. For this study, it was completed by participants’ mothers only. The EmQue measurement model and its factorial invariance across children’s gender were tested via confirmatory factor analysis. Highly satisfactory goodness-of-fit indexes were found for a three factor-structure (contagion, attention to the feelings of others and prosocial actions) with 13 loading items. Furthermore, the data supported invariance of measurement across gender groups. These results suggest that the EmQue-I13 is a reliable instrument for investigating empathy and prosocial behavior in 18- to 36-month-old children in the Italian context.     

A checklist for testing measurement invariance

The analysis of measurement invariance of latent constructs is important in research across groups, or across time. By establishing whether factor loadings, intercepts and residual variances are equivalent in a factor model that measures a latent concept, we can assure that comparisons that are made on the latent variable are valid across groups or time. Establishing measurement invariance involves running a set of increasingly constrained structural equation models, and testing whether differences between these models are significant. This paper provides a step-by-step guide to analysing measurement invariance.     

Factorial invariance and latent mean differences of scores on trait emotional intelligence across gender and age

Little is known about the factorial invariance and latent mean difference across sex and age in trait emotional intelligence (EI). The purpose of this study was to examine whether the measurement structure underlying trait EI is equivalent across sex and age groups. The sample consisted of 2919 teenagers, youths and adults. In order to investigate the above hypothesis the measurement and structural equivalence as well as the equality of latent means of scores across sex and age were tested. The multi-group confirmatory factor analysis results revealed that configural, metric, scalar and structural invariance exist across sex and age samples. Findings regarding the latent mean differences across sex and age groups are discussed with reference to recent and past findings.     

Factor structure and measurement invariance of a short measure of the Big Five personality traits

The main purpose of this study is to assess the factor structure and the measurement invariance of the Mini-International Personality Item Pool (Mini-IPIP; Donnellan, Oswald, Baird, & Lucas, 2006). The Mini-IPIP is a brief instrument evaluating personality traits according to the Big Five models. Two samples were collected comprising nearly 800 participants. Confirmatory factor analyzes revealed a five-factor solution consistent with the Big Five model. Measurement invariance analyses showed that the Mini-IPIP was reasonably invariant across samples, genders and age groups. Overall, results pointed to a satisfactory factorial structure and an adequate invariance of the measure.     

Dimensional structure and measurement invariance of the BRIEF-2 across gender in a socially vulnerable sample of primary school-aged children

The present study aims to analyze, on the one hand, the dimensional structure of the BRIEF-2 and, on the other, its measurement invariance across gender in a sample of primary school-aged children. Participants were 168 students (51.8% girls) in conditions of social risk and in foster homes, with ages ranging from 6 to 15 years old (M = 10.08; SD = 2.09). The children’s executive functioning level was obtained from their parents or caregivers. The analysis of the internal structure using confirmatory factor analysis reveals that the model with three indexes (Behavioral, Emotional and Cognitive) and nine scales (Inhibit, Shift, Self-Monitor, Emotional Control, Initiate, Working Memory, Plan/Organize, Organization of Materials and Task-Monitor) showed a good fit to the sample. Likewise, the results support the measurement invariance of the BRIEF-2 across genders. The coefficient alpha values for the index scores ranged from .71 to .91, with coefficients for the individual scales ranging from .66 to .88. The findings support the factorial validity of the BRIEF-2 scores, suggesting that it is an adequate instrument to evaluate executive functioning reported by parents and caregivers. Future studies must continue to examine the structure and invariance of the BRIEF-2 in different populations and cultures.     

Cross‐cultural invariance of NPI‐13: Entitlement as culturally specific,leadership and grandiosity as culturally universal

The current study explores the problem with the lack of measurement invariance for the Narcissistic Personality Inventory (NPI) by addressing two issues: conceptual heterogeneity of narcissism and methodological issues related to the binary character of data. We examine the measurement invariance of the 13‐item version of the NPI in three populations in Japan, Poland and the UK. Analyses revealed that leadership/authority and grandiose exhibitionism dimensions of the NPI were cross‐culturally invariant, while entitlement/exploitativeness was culturally specific. Therefore, we proposed NPI‐9 as indicating scalar invariance, and we examined the pattern of correlations between NPI‐9 and other variables across three countries. The results suggest that NPI‐9 is valid brief scale measuring general levels of narcissism in cross‐cultural studies, while the NPI‐13 remains suitable for research within specific countries.     

Factorial invariance in multilevel confirmatory factor analysis

This paper presents a procedure to test factorial invariance in multilevel confirmatory factor analysis. When the group membership is at level 2, multilevel factorial invariance can be tested by a simple extension of the standard procedure. However level‐1 group membership raises problems which cannot be appropriately handled by the standard procedure, because the dependency between members of different level‐1 groups is not appropriately taken into account. The procedure presented in this article provides a solution to this problem. This paper also shows Muthén's maximum likelihood (MUML) estimation for testing multilevel factorial invariance across level‐1 groups as a viable alternative to maximum likelihood estimation. Testing multilevel factorial invariance across level‐2 groups and testing multilevel factorial invariance across level‐1 groups are illustrated using empirical examples. SAS macro and Mplus syntax are provided.     

Factor structure and measurement invariance of the multidimensional driving style inventory across gender and age: An ESEM approach

The Multidimensional Driving Style Inventory (MDSI) is the most comprehensive measure of typical driving behavior to date and has been frequently used to compare driving styles across different groups of drivers, particularly between gender- and age-related groups. However, the factor structure of MDSI has not been clearly established and its measurement invariance has not been demonstrated. The goal of the present study was to examine the internal structure and measurement invariance of the MDSI across gender and age. A sample of 1277 drivers from Argentina responded to the Argentinian version of the MDSI. Exploratory structural equation modeling (ESEM) was used to test the factor structure and measurement invariance across females (n = 602) and males (n = 675), and across young (18–29, n = 558), adult (30–49, n = 395) and older (50 and older, n = 317) drivers. The results showed that a 36-item six-factor ESEM model represented by risky, angry, dissociative, anxious, distress-reduction and careful and patient driving styles was the best model based on fit indices and interpretability. Configural, weak and strong invariance of the six-factor ESEM model across gender and age was also supported. The MDSI in its Argentinian version is equivalent across gender and age, supporting the validity of previous research findings examining gender and age differences in driving styles. Future studies should examine the measurement invariance of the MDSI across other relevant driving-related variables.