Estimating Latent Variable Interactions With Nonnormal Observed Data: A Comparison of Four Approaches

A Monte Carlo simulation was conducted to investigate the robustness of 4 latent variable interaction modeling approaches (Constrained Product Indicator [CPI], Generalized Appended Product Indicator [GAPI], Unconstrained Product Indicator [UPI], and Latent Moderated Structural Equations [LMS]) under high degrees of nonnormality of the observed exogenous variables. Results showed that the CPI and LMS approaches yielded biased estimates of the interaction effect when the exogenous variables were highly nonnormal. When the violation of nonnormality was not severe (normal; symmetric with excess kurtosis < 1), the LMS approach yielded the most efficient estimates of the latent interaction effect with the highest statistical power. In highly nonnormal conditions, the GAPI and UPI approaches with maximum likelihood (ML) estimation yielded unbiased latent interaction effect estimates, with acceptable actual Type I error rates for both the Wald and likelihood ratio tests of interaction effect at N ≥ 500. An empirical example illustrated the use of the 4 approaches in testing a latent variable interaction between academic self-efficacy and positive family role models in the prediction of academic performance.     

Empirically Corrected Rescaled Statistics for SEM with Small N and Large p

Survey data often contain many variables. Structural equation modeling (SEM) is commonly used in analyzing such data. With typical nonnormally distributed data in practice, a rescaled statistic T_rml proposed by Satorra and Bentler was recommended in the literature of SEM. However, T_rml has been shown to be problematic when the sample size N is small and/or the number of variables p is large. There does not exist a reliable test statistic for SEM with small N or large p, especially with nonnormally distributed data. Following the principle of Bartlett correction, this article develops empirical corrections to T_rml so that the mean of the empirically corrected statistics approximately equals the degrees of freedom of the nominal chi-square distribution. Results show that empirically corrected statistics control type I errors reasonably well even when N is smaller than 2p, where T_rml may reject the correct model 100% even for normally distributed data. The application of the empirically corrected statistics is illustrated via a real data example.     

Abstract: Inference and Interval Estimation for Indirect Effects With Latent Variable Models

Models specifying indirect effects (or mediation) and structural equation modeling are both popular in the social sciences. Yet relatively little research has compared methods that test for indirect effects among latent variables and provided precise estimates of the effectiveness of different methods.

This simulation study provides an extensive comparison of methods for constructing confidence intervals and for making inferences about indirect effects with latent variables. We compared the percentile (PC) bootstrap, bias-corrected (BC) bootstrap, bias-corrected accelerated (BC _a ) bootstrap, likelihood-based confidence intervals (Neale & Miller, 1997), partial posterior predictive (Biesanz, Falk, and Savalei, 2010), and joint significance tests based on Wald tests or likelihood ratio tests. All models included three reflective latent variables representing the independent, dependent, and mediating variables. The design included the following fully crossed conditions: (a) sample size: 100, 200, and 500; (b) number of indicators per latent variable: 3 versus 5; (c) reliability per set of indicators: .7 versus .9; (d) and 16 different path combinations for the indirect effect (α = 0, .14, .39, or .59; and β = 0, .14, .39, or .59). Simulations were performed using a WestGrid cluster of 1680 3.06GHz Intel Xeon processors running R and OpenMx.

Results based on 1,000 replications per cell and 2,000 resamples per bootstrap method indicated that the BC and BC _a bootstrap methods have inflated Type I error rates. Likelihood-based confidence intervals and the PC bootstrap emerged as methods that adequately control Type I error and have good coverage rates.     

A general class of latent variable models for ordinal manifest variables with covariate effects on the manifest and latent variables

Previous work on a general class of multidimensional latent variable models for analysing ordinal manifest variables is extended here to allow for direct covariate effects on the manifest ordinal variables and covariate effects on the latent variables. A full maximum likelihood estimation method is used to estimate all the model parameters simultaneously. Goodness‐of‐fit statistics and standard errors are discussed. Two examples from the 1996 British Social Attitudes Survey are used to illustrate the methodology.     

What Causes the Mean Bias of the Likelihood Ratio Statistic with Many Variables?

Abstract

Survey data often contain many variables. Structural equation modeling (SEM) is commonly used in analyzing such data. However, conventional SEM methods are not crafted to handle data with a large number of variables (p). A large p can cause T_ml, the most widely used likelihood ratio statistic, to depart drastically from the assumed chi-square distribution even with normally distributed data and a relatively large sample size N. A key element affecting this behavior of T_ml is its mean bias. The focus of this article is to determine the cause of the bias. To this end, empirical means of T_ml via Monte Carlo simulation are used to obtain the empirical bias. The most effective predictors of the mean bias are subsequently identified and their predictive utility examined. The results are further used to predict type I errors of T_ml. The article also illustrates how to use the obtained results to determine the required sample size for T_ml to behave reasonably well. A real data example is presented to show the effect of the mean bias on model inference as well as how to correct the bias in practice.     

A Composite Likelihood Inference in Latent Variable Models for Ordinal Longitudinal Responses

The paper proposes a composite likelihood estimation approach that uses bivariate instead of multivariate marginal probabilities for ordinal longitudinal responses using a latent variable model. The model considers time-dependent latent variables and item-specific random effects to be accountable for the interdependencies of the multivariate ordinal items. Time-dependent latent variables are linked with an autoregressive model. Simulation results have shown composite likelihood estimators to have a small amount of bias and mean square error and as such they are feasible alternatives to full maximum likelihood. Model selection criteria developed for composite likelihood estimation are used in the applications. Furthermore, lower-order residuals are used as measures-of-fit for the selected models.     

A scaled difference chi-square test statistic for moment structure analysis

A family of scaling corrections aimed to improve the chi-square approximation of goodness-of-fit test statistics in small samples, large models, and nonnormal data was proposed in Satorra and Bentler (1994). For structural equations models, Satorra-Bentler's (SB) scaling corrections are available in standard computer software. Often, however, the interest is not on the overall fit of a model, but on a test of the restrictions that a null model sayM ₀ implies on a less restricted oneM ₁. IfT ₀ andT ₁ denote the goodness-of-fit test statistics associated toM ₀ andM ₁, respectively, then typically the differenceT _d =T ₀−T ₁ is used as a chi-square test statistic with degrees of freedom equal to the difference on the number of independent parameters estimated under the modelsM ₀ andM ₁. As in the case of the goodness-of-fit test, it is of interest to scale the statisticT _d in order to improve its chi-square approximation in realistic, that is, nonasymptotic and nonormal, applications. In a recent paper, Satorra (2000) shows that the difference between two SB scaled test statistics for overall model fit does not yield the correct SB scaled difference test statistic. Satorra developed an expression that permits scaling the difference test statistic, but his formula has some practical limitations, since it requires heavy computations that are not available in standard computer software. The purpose of the present paper is to provide an easy way to compute the scaled difference chi-square statistic from the scaled goodness-of-fit test statistics of modelsM ₀ andM ₁. A Monte Carlo study is provided to illustrate the performance of the competing statistics. This research was supported by the Spanish grants PB96-0300 and BEC2000-0983, and USPHS grants DA00017 and DA01070.     

Identifying the Source of Misfit in Item Response Theory Models

When an item response theory model fails to fit adequately, the items for which the model provides a good fit and those for which it does not must be determined. To this end, we compare the performance of several fit statistics for item pairs with known asymptotic distributions under maximum likelihood estimation of the item parameters: (a) a mean and variance adjustment to bivariate Pearson's X², (b) a bivariate subtable analog to Reiser's (1996) overall goodness-of-fit test, (c) a z statistic for the bivariate residual cross product, and (d) Maydeu-Olivares and Joe's (2006) M₂ statistic applied to bivariate subtables. The unadjusted Pearson's X² with heuristically determined degrees of freedom is also included in the comparison. For binary and ordinal data, our simulation results suggest that the z statistic has the best Type I error and power behavior among all the statistics under investigation when the observed information matrix is used in its computation. However, if one has to use the cross-product information, the mean and variance adjusted X² is recommended. We illustrate the use of pairwise fit statistics in 2 real-data examples and discuss possible extensions of the current research in various directions.     

The use of item scores and response times to detect examinees who may have benefited from item preknowledge

According to Wollack and Schoenig (2018, The Sage encyclopedia of educational research, measurement, and evaluation. Thousand Oaks, CA: Sage, 260), benefiting from item preknowledge is one of the three broad types of test fraud that occur in educational assessments. We use tools from constrained statistical inference to suggest a new statistic that is based on item scores and response times and can be used to detect examinees who may have benefited from item preknowledge for the case when the set of compromised items is known. The asymptotic distribution of the new statistic under no preknowledge is proved to be a simple mixture of two χ² distributions. We perform a detailed simulation study to show that the Type I error rate of the new statistic is very close to the nominal level and that the power of the new statistic is satisfactory in comparison to that of the existing statistics for detecting item preknowledge based on both item scores and response times. We also include a real data example to demonstrate the usefulness of the suggested statistic.     

A generalized rasch model for manifest predictors

A logistic regression model is suggested for estimating the relation between a set of manifest predictors and a latent trait assumed to be measured by a set ofk dichotomous items. Usually the estimated subject parameters of latent trait models are biased, especially for short tests. Therefore, the relation between a latent trait and a set of predictors should not be estimated with a regression model in which the estimated subject parameters are used as a dependent variable. Direct estimation of the relation between the latent trait and one or more independent variables is suggested instead. Estimation methods and test statistics for the Rasch model are discussed and the model is illustrated with simulated and empirical data.     

Reliability measures in item response theory: Manifest versus latent correlation functions

For item response theory (IRT) models, which belong to the class of generalized linear or non‐linear mixed models, reliability at the scale of observed scores (i.e., manifest correlation) is more difficult to calculate than latent correlation based reliability, but usually of greater scientific interest. This is not least because it cannot be calculated explicitly when the logit link is used in conjunction with normal random effects. As such, approximations such as Fisher's information coefficient, Cronbach's α, or the latent correlation are calculated, allegedly because it is easy to do so. Cronbach's α has well‐known and serious drawbacks, Fisher's information is not meaningful under certain circumstances, and there is an important but often overlooked difference between latent and manifest correlations. Here, manifest correlation refers to correlation between observed scores, while latent correlation refers to correlation between scores at the latent (e.g., logit or probit) scale. Thus, using one in place of the other can lead to erroneous conclusions. Taylor series based reliability measures, which are based on manifest correlation functions, are derived and a careful comparison of reliability measures based on latent correlations, Fisher's information, and exact reliability is carried out. The latent correlations are virtually always considerably higher than their manifest counterparts, Fisher's information measure shows no coherent behaviour (it is even negative in some cases), while the newly introduced Taylor series based approximations reflect the exact reliability very closely. Comparisons among the various types of correlations, for various IRT models, are made using algebraic expressions, Monte Carlo simulations, and data analysis. Given the light computational burden and the performance of Taylor series based reliability measures, their use is recommended.     

认知诊断测验中基于信息矩阵的多群组DIF检验

基于改进的Wald统计量,将适用于两群组的DIF检测方法拓展至多群组的项目功能差异（DIF）检验;改进的Wald统计量将分别通过计算观察信息矩阵（Obs）和经验交叉相乘信息矩阵（XPD）而得到。模拟研究探讨了此二者与传统计算方法在多个群组下的DIF检验情况,结果表明：（1）Obs和XPD的一类错误率明显低于传统方法,DINA模型估计下Obs和XPD的一类错误率接近理论水平;（2）样本量和DIF量较大时,Obs和XPD具有与传统Wald统计量大体相同的统计检验力。     

A note on moran's measure of multiple rank correlation

A measure of multiple rank correlation,T _y.12 ², is proposed for the situation with no tied observations in the variables. The measure is a weighted average of two squared Kendall taus. It is shown thatT _y.12 ² is equivalent to a statistic previously proposed by Moran and thus a new interpretation is given to Moran's statistic.The author wishes to thank Nancy Anderson, Willard Larkin, and Kent Norman for their helpful comments.     

A note on monotone likelihood ratio of the total score variable in unidimensional item response theory

This note provides a direct, elementary proof of the fundamental result on monotone likelihood ratio of the total score variable in unidimensional item response theory (IRT). This result is very important for practical measurement in IRT, because it justifies the use of the total score variable to order participants on the latent trait. The proof relies on a basic inequality for elementary symmetric functions which is proved by means of few purely algebraic, straightforward transformations. In particular, flaws in a proof of this result by Huynh [(1994) . A new proof for monotone likelihood ratio for the sum of independent Bernoulli random variables. Psychometrika, 59, 77–79] are pointed out and corrected, and a natural generalization of the fundamental result to non‐linear (quasi‐ordered) latent trait spaces is presented. This may be useful for multidimensional IRT or knowledge space theory, in which the latent ‘ability’ spaces are partially ordered with respect to, for instance, coordinate‐wise vector‐ordering or set‐inclusion, respectively.     

A Propensity Score Adjustment for Multiple Group Structural Equation Modeling

In the behavioral and social sciences, quasi-experimental and observational studies are used due to the difficulty achieving a random assignment. However, the estimation of differences between groups in observational studies frequently suffers from bias due to differences in the distributions of covariates. To estimate average treatment effects when the treatment variable is binary, Rosenbaum and Rubin (1983a) proposed adjustment methods for pretreatment variables using the propensity score. However, these studies were interested only in estimating the average causal effect and/or marginal means. In the behavioral and social sciences, a general estimation method is required to estimate parameters in multiple group structural equation modeling where the differences of covariates are adjusted. We show that a Horvitz–Thompson-type estimator, propensity score weighted M estimator (PWME) is consistent, even when we use estimated propensity scores, and the asymptotic variance of the PWME is shown to be less than that with true propensity scores. Furthermore, we show that the asymptotic distribution of the propensity score weighted statistic under a null hypothesis is a weighted sum of independent χ² ₁ variables. We show the method can compare latent variable means with covariates adjusted using propensity scores, which was not feasible by previous methods. We also apply the proposed method for correlated longitudinal binary responses with informative dropout using data from the Longitudinal Study of Aging (LSOA). The results of a simulation study indicate that the proposed estimation method is more robust than the maximum likelihood (ML) estimation method, in that PWME does not require the knowledge of the relationships among dependent variables and covariates.     

Correcting Too Much or Too Little? The Performance of Three Chi-Square Corrections

This simulation study investigates the performance of three test statistics, T₁, T₂, and T₃, used to evaluate structural equation model fit under non normal data conditions. T₁ is the well-known mean-adjusted statistic of Satorra and Bentler. T₂ is the mean-and-variance adjusted statistic of Sattertwaithe type where the degrees of freedom is manipulated. T₃ is a recently proposed version of T₂ that does not manipulate degrees of freedom. Discrepancies between these statistics and their nominal chi-square distribution in terms of errors of Type I and Type II are investigated. All statistics are shown to be sensitive to increasing kurtosis in the data, with Type I error rates often far off the nominal level. Under excess kurtosis true models are generally over-rejected by T₁ and under-rejected by T₂ and T₃, which have similar performance in all conditions. Under misspecification there is a loss of power with increasing kurtosis, especially for T₂ and T₃. The coefficient of variation of the nonzero eigenvalues of a certain matrix is shown to be a reliable indicator for the adequacy of these statistics.     

Evaluating Structural Equation Models for Categorical Outcomes: A New Test Statistic and a Practical Challenge of Interpretation

This research is concerned with two topics in assessing model fit for categorical data analysis. The first topic involves the application of a limited-information overall test, introduced in the item response theory literature, to structural equation modeling (SEM) of categorical outcome variables. Most popular SEM test statistics assess how well the model reproduces estimated polychoric correlations. In contrast, limited-information test statistics assess how well the underlying categorical data are reproduced. Here, the recently introduced C₂ statistic of Cai and Monroe (2014) is applied. The second topic concerns how the root mean square error of approximation (RMSEA) fit index can be affected by the number of categories in the outcome variable. This relationship creates challenges for interpreting RMSEA. While the two topics initially appear unrelated, they may conveniently be studied in tandem since RMSEA is based on an overall test statistic, such as C₂. The results are illustrated with an empirical application to data from a large-scale educational survey.     

Testing for Measurement Invariance with Respect to an Ordinal Variable

Researchers are often interested in testing for measurement invariance with respect to an ordinal auxiliary variable such as age group, income class, or school grade. In a factor-analytic context, these tests are traditionally carried out via a likelihood ratio test statistic comparing a model where parameters differ across groups to a model where parameters are equal across groups. This test neglects the fact that the auxiliary variable is ordinal, and it is also known to be overly sensitive at large sample sizes. In this paper, we propose test statistics that explicitly account for the ordinality of the auxiliary variable, resulting in higher power against “monotonic” violations of measurement invariance and lower power against “non-monotonic” ones. The statistics are derived from a family of tests based on stochastic processes that have recently received attention in the psychometric literature. The statistics are illustrated via an application involving real data, and their performance is studied via simulation.     

Assessing Approximate Fit in Categorical Data Analysis

A family of Root Mean Square Error of Approximation (RMSEA) statistics is proposed for assessing the goodness of approximation in discrete multivariate analysis with applications to item response theory (IRT) models. The family includes RMSEAs to assess the approximation up to any level of association of the discrete variables. Two members of this family are RMSEA₂, which uses up to bivariate moments, and the full information RMSEA_n. The RMSEA₂ is estimated using the M₂ statistic of Maydeu-Olivares and Joe (2005, 2006), whereas for maximum likelihood estimation, RMSEA_n is estimated using Pearson's X² statistic. Using IRT models, we provide cutoff criteria of adequate, good, and excellent fit using the RMSEA₂. When the data are ordinal, we find a strong linear relationship between the RMSEA₂ and the Standardized Root Mean Squared Residual goodness-of-fit index. We are unable to offer cutoff criteria for the RMSEA_n as its population values decrease as the number of variables and categories increase.