Testing for Measurement Invariance with Respect to an Ordinal Variable

Researchers are often interested in testing for measurement invariance with respect to an ordinal auxiliary variable such as age group, income class, or school grade. In a factor-analytic context, these tests are traditionally carried out via a likelihood ratio test statistic comparing a model where parameters differ across groups to a model where parameters are equal across groups. This test neglects the fact that the auxiliary variable is ordinal, and it is also known to be overly sensitive at large sample sizes. In this paper, we propose test statistics that explicitly account for the ordinality of the auxiliary variable, resulting in higher power against “monotonic” violations of measurement invariance and lower power against “non-monotonic” ones. The statistics are derived from a family of tests based on stochastic processes that have recently received attention in the psychometric literature. The statistics are illustrated via an application involving real data, and their performance is studied via simulation.     

心理测量平衡性研究与实例

心理测量研究中,测量不变性(或称平衡性)是量表稳定性问题中的一个难题而且在比较研究中受到特别重视。结构方程模型因在平衡性形式捕捉方面功能强大而受到广泛应用。该研究讨论了测量平衡性的各种形式并演示了应用结构方程模型评估测量平衡性的过程。     

Invariance in Measurement and Prediction Revisited

Borsboom (Psychometrika, 71:425–440, 2006) noted that recent work on measurement invariance (MI) and predictive invariance (PI) has had little impact on the practice of measurement in psychology. To understand this contention, the definitions of MI and PI are reviewed, followed by results on the consistency between the two forms of invariance in the general case. The special parametric cases of factor analysis (strict factorial invariance) and linear regression analyses (strong regression invariance) are then described, along with findings on the inconsistency between the two forms of invariance in this context. Two numerical examples of inconsistency are reviewed in detail. The impact of violations of MI on accuracy of selection is illustrated. Finally, reasons for the slow dissemination of work on invariance are discussed, and the prospects for altering this situation are weighed. This paper is based on the Presidential Address given at the International Meeting of the Psychometric Society in Tokyo, Japan, on July 11, 2007. This research was supported by National Institute of Mental Health grants 1P30 MH 068685-01A1 and RO1 MH64707-01.     

Testing the Internal Consent Scale for Measurement Invariance Across Women and Men

The Internal Consent Scale (ICS) was created to measure feelings associated with a person’s willingness to engage in partnered sexual activity. Although previous studies using the ICS have assessed gender differences, evidence has not been provided to suggest that the ICS functions similarly for women and men. Using data from an online cross-sectional survey of adults (N = 874; 53.1% women), we subjected the 25-item ICS to tests of measurement invariance across gender. We found that only partial measurement invariance was tenable, which indicated that direct comparisons across gender should be interpreted with caution when using the ICS. Therefore, we created a gender-invariant short form. In support of construct validity, we found that this 15-item ICS–Short Form demonstrated similar associations with measures of sexual consent communication as the full 25-item ICS. If researchers aim to compare women and men on internal sexual consent, we recommend using the 15-item ICS–Short Form. Cognitive interviews should be conducted to further understand how women and men might differentially interpret ICS items.

     

A Bayesian Semiparametric Latent Variable Model for Mixed Responses

In this paper we introduce a latent variable model (LVM) for mixed ordinal and continuous responses, where covariate effects on the continuous latent variables are modelled through a flexible semiparametric Gaussian regression model. We extend existing LVMs with the usual linear covariate effects by including nonparametric components for nonlinear effects of continuous covariates and interactions with other covariates as well as spatial effects. Full Bayesian modelling is based on penalized spline and Markov random field priors and is performed by computationally efficient Markov chain Monte Carlo (MCMC) methods. We apply our approach to a German social science survey which motivated our methodological development. We thank the editor and the referees for their constructive and helpful comments, leading to substantial improvements of a first version, and Sven Steinert for computational assistance. Partial financial support from the SFB 386 “Statistical Analysis of Discrete Structures” is also acknowledged.     

Assessing the reliability of Curriculum-Based Measurement: an application of Latent Growth Modeling

The purpose of this study was to demonstrate the use of Latent Growth Modeling (LGM) as a method for estimating reliability of Curriculum-Based Measurement (CBM) progress-monitoring data. The LGM approach permits the error associated with each measure to differ at each time point, thus providing an alternative method for examining of the reliability of CBM reading aloud data over repeated measurements. The analysis revealed that the reliability of CBM data was not a fixed property of the measure, but it changed with time. The study demonstrates the need to consider reliability in new ways with respect to the use of CBM data as repeated measures.     

A Probabilistic Latent Class Model for Assessing Inter-Judge Reliability

Increasingly behavioral researchers are soliciting cognitive responses in addition to standard attitudinal measures when attempting to assess the effects of persuasive communications. The coding of the elicited cognitive responses generally involves some sort of categorization, typically undertaken by independent judges, and the quality of the data is, to a large degree, evaluated in terms of some reliability coefficient which reflects the extent to which the independent judges agreed. The purpose of this paper is to present and illustrate a probabilistic model for assessing inter-judge reliability. The proposed probabilistic model allows one to (a) use formal test statistics to evaluate the extent and character of inter-judge reliability, (b) estimate the assignment error rates and their standard errors, and (c) test for simultaneous agreement for more than two judges. The probabilistic model is operationalized in terms of restricted latent class models.     

Null Results in Assessing Survey Score Comparability: Illustrating Measurement Invariance Using Item Response Theory

In using organizational surveys for decision-making, it is essential to consider measurement equivalence/invariance (ME/I), which addresses the questions of whether score differences are attributable to differences in the latent variable we intend to measure, or attributable to confounding differences in measurement properties. Due to the tendency for null results to remain unpublished, most articles have focused on findings of, and reasons for violations of ME/I. On the other hand, little is available to practitioners and researchers concerning situations where ME/I can be expected to uphold. This is especially disconcerting due to the fact that the null is the desired result in such analyses, and allows for unfettered observed-score comparisons. This special issue presents a unique opportunity to provide such a discussion using real-world examples from an organizational culture survey. In doing so we hope to clear up confusion surrounding the concept of ME/I, when it can be expected, and how it relates to actual differences in scores. First, we review the basic tenets and past findings focusing on ME/I, and discuss the item response theory differential item functioning framework used here. Next, we show ME/I being upheld using organizational survey data wherein violations of ME/I would reasonably not be expected (i.e., the null hypothesis was predicted and supported), and simulate the consequences of ignoring ME/I. Finally, we suggest a set of conditions wherein ME/I is likely to be upheld.     

A Latent Variable Framework for Power Estimation Within Intervention Contexts

This paper presents a latent growth SEM approach for the estimation of treatment effects, and power to detect such effects, within a true experimental design setting in which subjects are randomly assigned to treatment and control conditions. Power estimation is a critical component of intervention experiment design and the testing of their results. Although researchers have become increasingly sophisticated in applying tests for statistical significance in intervention contexts, few are aware of the power of these tests. The issues raised in this paper are not new; however, reminding researchers to consider these points is important. Exactly how the researcher handles these issues will depend on the questions asked and the resources available, as well as other considerations. Discussion underscores the relationship between the reliability of a study's measures and concomitant increases in power obtained within the SEM framework.     

A Two-Step Bayesian Approach for Propensity Score Analysis: Simulations and Case Study

A two-step Bayesian propensity score approach is introduced that incorporates prior information in the propensity score equation and outcome equation without the problems associated with simultaneous Bayesian propensity score approaches. The corresponding variance estimators are also provided. The two-step Bayesian propensity score is provided for three methods of implementation: propensity score stratification, weighting, and optimal full matching. Three simulation studies and one case study are presented to elaborate the proposed two-step Bayesian propensity score approach. Results of the simulation studies reveal that greater precision in the propensity score equation yields better recovery of the frequentist-based treatment effect. A slight advantage is shown for the Bayesian approach in small samples. Results also reveal that greater precision around the wrong treatment effect can lead to seriously distorted results. However, greater precision around the correct treatment effect parameter yields quite good results, with slight improvement seen with greater precision in the propensity score equation. A comparison of coverage rates for the conventional frequentist approach and proposed Bayesian approach is also provided. The case study reveals that credible intervals are wider than frequentist confidence intervals when priors are non-informative.     

Social Network Mediation Analysis: A Latent Space Approach

Psychometrika - A social network comprises both actors and the social connections among them. Such connections reflect the dependence among social actors, which is essential for individuals’...     

Assessing Advocacies for Forcibly Displaced People: A Comprehensive Approach

This article assesses various advocacy practices for forcibly displaced people (FDP) through the analysis of advocacy networks, the examination of the goals that they pursue, and their ways of working. Three basic approaches, the welfare-based, the legal-based, and the capability-based approaches, are assessed. From this assessment, this study suggests the recognition of shared humanity as an entry point for advocacy, which offers a cosmopolitan understanding of rights and duties, and the most comprehensive protection for FDP. The main argument of this study is that if the demand for recognition is not heard, relief for refugees and other displaced people will lack an essential dimension. It is the demand to be recognized as human beings that engenders responsibility for forced migrants. Instead of prescribing a list of what to do, or not to do, this reflection has rather suggested a way of being and dealing with the forcibly displaced. This stance goes beyond the facility of typical responses that are known in advance.     

Assessing Measurement Invariance of the Children's Depression Inventory in Chinese and Italian Primary School Student Samples

This study tested the measurement invariance of Children's Depression Inventory (CDI) and compared its factorial variance/covariance and latent means among Chinese and Italian children. Multigroup confirmatory factor analysis of the original five factors identified by Kovacs revealed that full measurement invariance did not hold. Further analysis showed that 4 of 21 factor loadings, 14 of 26 intercepts, and 12 of 26 item errors were noninvariant. Factor variance and covariance invariant tests revealed significant differences between Chinese and Italian samples. The latent factor mean comparison suggested no significant difference across the two groups. Nevertheless, the finding of partial metric and scalar invariance suggested that observed mean differences on the CDI items cannot be fully explained by the mean differences in the latent factor. These results suggest that researchers and practitioners exercise caution when gauging the size of the true national population differences in depressive symptoms among Italian and Chinese children when assessed via CDI. In addition to providing needed evidence on the use of the CDI in Italian and Chinese children specifically, the methods used in this research can serve more generally as an example for other cross-cultural assessment research to test structural equivalence and measurement invariance of scales and to determine why it is important to do so.     

Long-Term Development of Employee Well-Being: A Latent Transition Approach

The long-term development of employee well-being is still poorly understood. Consequently, in this three-wave 10-year longitudinal study among Finnish managers (n = 402) the development of employee well-being was examined in in detail. Specifically, the long-term development of job-related affective well-being was investigated at the intra-individual level, simultaneously taking into account positive and negative indicators of well-being, the level of well-being, and the direction of change. Further, the issue how (changes in) job resources and employee well-being were related across time was examined. By applying a novel person-centered methodology, factor mixture modeling and latent transition analysis, the results revealed that the development of favorable job-related affective well-being was eight times more probable than that of unfavorable development across the 10-year study period. Job resources predicted a high level of job-related well-being and, also, job resources increased along with favorable changes in well-being. Overall, the findings contribute to knowledge in the area of positive occupational health psychology by offering a detailed picture of the level of job-related affective well-being and its development over time.     

A Composite Likelihood Inference in Latent Variable Models for Ordinal Longitudinal Responses

The paper proposes a composite likelihood estimation approach that uses bivariate instead of multivariate marginal probabilities for ordinal longitudinal responses using a latent variable model. The model considers time-dependent latent variables and item-specific random effects to be accountable for the interdependencies of the multivariate ordinal items. Time-dependent latent variables are linked with an autoregressive model. Simulation results have shown composite likelihood estimators to have a small amount of bias and mean square error and as such they are feasible alternatives to full maximum likelihood. Model selection criteria developed for composite likelihood estimation are used in the applications. Furthermore, lower-order residuals are used as measures-of-fit for the selected models.     

Assessing Specific Discipline Techniques: A Mixed-Methods Approach

This study explored, in a community sample of mothers of toddlers, parenting beliefs and values, to gain insight into the parent–child relationship. Acceptance of specific discipline techniques (DTs), and their actual use in daily life were examined. A mixed-methods approach comprising three different methods was used: (1) parenting beliefs and values were explored with Q-methodology; (2) acceptance of the DTs was assessed with the questionnaire Dimensions of Discipline Inventory; and (3) actual use of those DTs in daily-life incidents of discipline was documented using ecological momentary assessment for ten consecutive days. The results showed the mothers’ parenting beliefs and values reflected a warm parent–child relationship. The mothers rated explaining rules, timeout, removal of privileges, and social reinforcement as moderately to highly acceptable. However, planned ignoring received a low acceptance rating. Mothers’ high acceptability ratings of the DTs contrasted with moderate use when they were faced with their misbehaving child, with the exception of explaining rules, which was always manifested. Yelling and spanking received the lowest acceptance ratings. Nonetheless, in daily life, yelling was employed as often as timeout. These findings suggest the need for more attention to be paid to both acceptance and daily use of specific DTs in order to highlight DTs which parents may have difficulty implementing.     

Tests of Measurement Invariance Without Subgroups: A Generalization of Classical Methods

The issue of measurement invariance commonly arises in factor-analytic contexts, with methods for assessment including likelihood ratio tests, Lagrange multiplier tests, and Wald tests. These tests all require advance definition of the number of groups, group membership, and offending model parameters. In this paper, we study tests of measurement invariance based on stochastic processes of casewise derivatives of the likelihood function. These tests can be viewed as generalizations of the Lagrange multiplier test, and they are especially useful for: (i) identifying subgroups of individuals that violate measurement invariance along a continuous auxiliary variable without prespecified thresholds, and (ii) identifying specific parameters impacted by measurement invariance violations. The tests are presented and illustrated in detail, including an application to a study of stereotype threat and simulations examining the tests’ abilities in controlled conditions.