A checklist for testing measurement invariance

The analysis of measurement invariance of latent constructs is important in research across groups, or across time. By establishing whether factor loadings, intercepts and residual variances are equivalent in a factor model that measures a latent concept, we can assure that comparisons that are made on the latent variable are valid across groups or time. Establishing measurement invariance involves running a set of increasingly constrained structural equation models, and testing whether differences between these models are significant. This paper provides a step-by-step guide to analysing measurement invariance.     

Estimation for the multiple factor model when data are missing

A maximum likelihood method of estimating the parameters of the multiple factor model when data are missing from the sample is presented. A Monte Carlo study compares the method with 5 heuristic methods of dealing with the problem. The present method shows some advantage in accuracy of estimation over the heuristic methods but is considerably more costly computationally.This paper is based on the author's doctoral dissertation at the Department of Psychology, University of Illinois at Urbana-Champaign. The author gratefully acknowledges the aid of Drs. Robert Bohrer, Charles Lewis, Robert Linn, Maurice Tatsuoka, and Ledyard Tucker.     

题目因素分析：基于SEM和基于IRT的两类方法

心理和教育测量一般只能达到顺序量表的水平,其测量数据与被测因子间并非简单线性关系。题目因素分析是用来描述测量题目与因子间非线性关系的统计模型。题目因素分析主要有基于结构方程模型和基于项目反应理论两类方法,两类方法之间存在紧密的联系,甚至可以看作是同一模型的两种表现形式。本文详细阐述了该关系,同时对两类方法在参数估计、模型拟合指标、测量一致性检验和支撑软件等方面的特点进行了分析和比较,以便研究者选择最为适合其研究的方法。     

The assessment of school engagement: examining dimensionality and measurement invariance by gender and race/ethnicity

The construct of school engagement has attracted growing interest as a way to ameliorate the decline in academic achievement and increase in dropout rates. The current study tested the fit of a second-order multidimensional factor model of school engagement, using large-scale representative data on 1103 students in middle school. In order to make valid model comparisons by group, we evaluated the extent to which the measurement structure of this model was invariant by gender and by race/ethnicity (European-American vs. African-American students). Finally, we examined differences in latent factor means by these same groups. From our confirmatory factor analyses, we concluded that school engagement was a multidimensional construct, with evidence to support the hypothesized second-order engagement factor structure with behavioral, emotional, and cognitive dimensions. In this sample, boys and girls did not substantially differ, nor did European-American and African-American students, in terms of the underlying constructs of engagement and the composition of these constructs. Finally, there were substantial differences in behavioral and emotional engagement by gender and by racial/ethnic groups in terms of second-order factor mean differences.     

Measuring current achievement motivation with the QCM: Short form development and investigation of measurement invariance

This article explores the measurement properties of the Questionnaire on Current Motivation (QCM; Rheinberg, Vollmeyer, & Burns, 2001), which measures four factors of current achievement motivation (anxiety, challenge, interest, and probability of success) in the context of taking an abstract reasoning test. Two studies were conducted. In study 1, the measurement model of the QCM was tested, and a short form consisting of 12 of the original 18 QCM items was derived. Data were obtained from N = 350 students completing the QCM in anticipation of working on a test of Latin Square Tasks. In study 2, the measurement properties of the short form were replicated with data from a sample of N = 509 students. Measurement invariance with respect to gender was investigated using multigroup CFA models. The results showed that the assumptions of equal numbers of factors, factor loadings, and residual variances could be confirmed, but evidence for full intercept invariance was not obtained. There were significant differences in the latent means for anxiety and probability of success between female and male test takers, but none of these two factors had an influence on actual test performance, which was instead significantly predicted by interest in the task.     

A unified approach to exploratory factor analysis with missing data,nonnormal data,and in the presence of outliers

Factor analysis is regularly used for analyzing survey data. Missing data, data with outliers and consequently nonnormal data are very common for data obtained through questionnaires. Based on covariance matrix estimates for such nonstandard samples, a unified approach for factor analysis is developed. By generalizing the approach of maximum likelihood under constraints, statistical properties of the estimates for factor loadings and error variances are obtained. A rescaled Bartlett-corrected statistic is proposed for evaluating the number of factors. Equivariance and invariance of parameter estimates and their standard errors for canonical, varimax, and normalized varimax rotations are discussed. Numerical results illustrate the sensitivity of classical methods and advantages of the proposed procedures.This project was supported by a University of North Texas Faculty Research Grant, Grant #R49/CCR610528 for Disease Control and Prevention from the National Center for Injury Prevention and Control, and Grant DA01070 from the National Institute on Drug Abuse. The results do not necessarily represent the official view of the funding agencies. The authors are grateful to three reviewers for suggestions that improved the presentation of this paper.     

The personality differentiation by intelligence hypothesis: A measurement invariance investigation

Stability in the measurement of personality is crucial to its construct and predictive validity. The personality differentiation by intelligence hypothesis suggests that intelligence may be a threat to the measurement of personality. This study builds on existing literature by applying multigroup confirmatory factor analytic measurement invariance techniques to the item-level responses of a commonly used personality inventory. In contrast to recently published findings, our results suggest that the measurement of personality is not equivalent across intelligence levels. Though these results have important implications for the use of personality in research and practice, we maintain that personality measures are still important predictors of key criteria. We discuss future research directions, and hope that the current study highlights the need to increase the sophistication of currently available measurement techniques, and will bring continued examination of the personality differentiation by intelligence hypothesis.     

等级数据的测量不变性检验及影响因素模拟研究

研究介绍了针对等级数据的模型建构（LRV,潜在反应变量模型）和参数估计（WLSMV）方法,以及在此基础上的测量不变性检验（DIFFTEST）方法,同时采用蒙特卡洛模拟研究方法,考察样本总量大小、组间样本量对比情况、阈值差异程度、量表长度等因素,对DIFTEST进行针对等级数据的测量不变性检验效果的影响情况,以及WLSMV估计方法下的参数复原情况。研究结果发现WLSMV估计方法参数的复原效果很好;DIFFTEST的一类错误概率达到可接受水平,在大样本情况下、组间样本量基本相等、阈值差异程度较大时,DIFFTEST检测力较好。在控制测量不变性遭受破坏的项目个数情况下,随着测验长度的增加,DIFFTEST的检测力下降。     

Measuring individualism and collectivism: The importance of considering differential components,reference groups,and measurement invariance

A new instrument of individualism and collectivism (I/C) was developed and three key issues in I/C measurement were addressed: differentiating components of I/C, understanding the impact of reference groups, and testing of measurement invariance. Three components of I/C were assessed in China and the U.S.: independence, competitiveness, and uniqueness for individualism; considering of one’s decisions on others, sharing of positive outcomes, and sharing of negative outcomes for collectivism. Collectivism was measured with respect to parents, friends, and general others. Results indicate that Chinese participants are less unique but more independent and competitive than their counterparts in the U.S. The expected cultural difference is found for parent collectivism across all three components and for sharing negative outcome with respect to all three reference groups. These results suggest that individualism is a multidimensional construct, whereas the dimensionality of collectivism appears to be a function of social distance. Measurement invariance was tested at configural, factor loading, and intercept levels for all components of I/C.     

Measurement invariance tests of the Impression Management sub-scale of the Balanced Inventory of Desirable Responding

The Impression Management (IM) subscale of the Balanced Inventory of Desirable Responding (Paulhus, 1988) has been used as a proxy for common method variance in anonymous settings and as a cause of faking on personality testing in confidential settings. This study uses confirmatory factor analysis to conduct measurement invariance/equivalence tests (also known as multiple group analysis) on the IM subscale in a quasi-experiment in anonymous and confidential data collection settings. Using Brown’s (2006) bottom-up approach to Cheung and Rensvold’s (1999) model testing steps and Cheung and Rensvold’s (2002) statistical tests, the IM sub-scale was determined to have equal form, equal factor loadings, equal indicator error variances, equal factor variance, and an equal factor covariance in both 1-factor and 2-factor models in both data collection settings. Mean scale scores were significantly higher in the confidential group than in the anonymous group. These results suggest that using the IM sub-scale as a cause of faking and as a proxy for common method variance (CMV) is likely to be acceptable because the psychometric properties of the instrument are invariant across testing conditions.     

An estimate of the covariance between variables which are not jointly observed

Situations sometimes arise in which variables collected in a study are not jointly observed. This typically occurs because of study design. An example is an equating study where distinct groups of subjects are administered different sections of a test. In the normal maximum likelihood function to estimate the covariance matrix among all variables, elements corresponding to those that are not jointly observed are unidentified. If a factor analysis model holds for the variables, however, then all sections of the matrix can be accurately estimated, using the fact that the covariances are a function of the factor loadings. Standard errors of the estimated covariances can be obtained by the delta method. In addition to estimating the covariance matrix in this design, the method can be applied to other problems such as regression factor analysis. Two examples are presented to illustrate the method. This research was partially supported by NIMH grant MH5-4576     

Maximum likelihood estimation of the polychoric correlation coefficient

The polychoric correlation is discussed as a generalization of the tetrachoric correlation coefficient to more than two classes. Two estimation methods are discussed: Maximum likelihood estimation, and what may be called two-step maximum likelihood estimation. For the latter method, the thresholds are estimated in the first step. For both methods, asymptotic covariance matrices for estimates are derived, and the methods are illustrated and compared with artificial and real data.This paper was read at the 1978 European Meeting on Psychometrics and Mathematical Psychology in Uppsala, Sweden, June 1978.Research reported in this paper has been supported by the Bank of Sweden Tercentenary Foundation under project Structural Equation Models in the Social Sciences, project director Karl G. Jöreskog.     

Multigroup confirmatory factor analysis and structural invariance with age of the Behavior Rating Inventory of Executive Function (BRIEF)—French version

The parent and teacher forms of the French version of the Behavioral Rating Inventory of Executive Function (BRIEF) were used to evaluate executive function in everyday life in a large sample of healthy children (N = 951) aged between 5 and 18.

Several psychometric methods were applied, with a view to providing clinicians with tools for score interpretation. The parent and teacher forms of the BRIEF were acceptably reliable. Demographic variables (such as age and gender) were found to influence the BRIEF scores. Confirmatory factor analysis was then used to test five competing models of the BRIEF's latent structure. Two of these models (a three-factor model and a two-factor model, both based on a nine-scale structure) had a good fit. However, structural invariance with age was only obtained with the two-factor model.

The French version of the BRIEF provides a useful measure of everyday executive function and can be recommended for use in clinical research and practice.     

Psychometric properties of the Coach-Athlete Relationship Questionnaire (CART-Q) in seven countries

Objectives

In the sport context, the coach-athlete relationship has been viewed and studied as a universal phenomenon. It is thus important to assess the universality of the psychometric scale that is used to measure the quality of coach-athlete relationships. The present study examined the cultural equivalence or measurement invariance of the Coach-Athlete Relationship Questionnaire (11-item, athlete version CART-Q).

Design

Cross-sectional study.

Methods

A total of 1363 athletes from Belgium (N = 200), Britain (N = 382), China (N = 200), Greece (N = 115), Spain (N = 120), Sweden (N = 169), and United States of America (N = 177) completed the CART-Q.

Results

Multi-group mean and covariance structure (MACS) analyses supported the factorial validity of the CART-Q in a three-first-order factor model across the seven countries. An examination of the latent mean differences of the CART-Q revealed some variation in terms of the intensity that athletes perceive in the quality of the relationship with their coach across the different countries.

Conclusions

Overall, these results supply additional evidence of the psychometric properties of the CART-Q and highlight that it is a sound instrument that can be applied cross-culturally.     

Self-Other Differentiation Scale: Dimensionality,IRT Parameterization,and Measurement Invariance

The Self-Other Differentiation Scale (Olver, Aries, &; Batgos, 1989 Olver, R. R., Aries, E., &; Batgos, J. (1989). Self-other differentiation and the mother-child relationship: The effects of sex and birth order. The Journal of Genetic Psychology, 150, 311–321. doi:10.1080/00221325.1989.9914600[Taylor &; Francis Online] , [Google Scholar]) is a self-report instrument assessing the experience of a separate sense of self from others. The authors aimed to examine its dimensionality, reliability, and measurement invariance across gender. It was completed by 348 participants (48% men) from 17 to 30 years old in Study 1, 348 participants (40% men) from 18 to 28 years old in Study 2, and 1,068 participants (49% men) from 17 to 28 years old in Study 3. The results supported the hypothesis of just one factor underlying the scale; they also showed an appropriate internal consistency and a partial measurement invariance across gender. Results also showed evidence for a 10-item version of the scale. Globally, the Self-Other Differentiation Scale can be considered a good scale to assess individual's sense of differentiation of one's own sense of self from others.     

定序变量回归模型在心理数据分析中的应用

定序变量在心理现象和心理数据中随处可见, 采用综合的定序变量回归分析模型可以对“镜像模式”和“漏斗模型”的心理现象做出合理的解释和预测。首先通过非参数检验对影响因素进行初步降维, 其次用Probit定序回归对降维后的影响因素贡献率进行判别, 从而进一步筛选具有显著性判断水平的有效指标, 最后用Logistic回归模型对某种特定的心理现象发生与否进行信息量足够大的解释和预测。大学毕业生工作生活质量满意度的预测对这种综合定序变量回归分析模型的实例拟合, 证实了综合定序变量回归分析模型在心理现象和心理数据分析中的应用价值。     

追踪研究中缺失数据处理方法及应用现状分析

追踪研究中普遍存在缺失数据, 缺失数据处理方法的选择影响统计推断的精度及研究结果的有效性。首先, 阐述缺失机制及判断方法, 比较追踪研究中主要的缺失数据处理方法的特点、及实际应用中的缺失处理方法的选择和软件实现。其次, 对国内心理学中92篇追踪研究文献进行分析, 发现有59篇(64.13%)报告不同程度缺失, 其中仅39篇报告了处理方法且均为删除法。未来研究应深入探讨现有缺失数据处理方法的有效性, 进一步规范应用研究中缺失数据的处理。