Abstract: Evaluation of Test Statistics for Robust Structural Equation Modeling With Nonnormal Missing Data

Traditional structural equation modeling (SEM) techniques have trouble dealing with incomplete and/or nonnormal data that are often encountered in practice. Yuan and Zhang (2011a) developed a two-stage procedure for SEM to handle nonnormal missing data and proposed four test statistics for overall model evaluation. Although these statistics have been shown to work well with complete data, their performance for incomplete data has not been investigated in the context of robust statistics.

Focusing on a linear growth curve model, a systematic simulation study is conducted to evaluate the accuracy of the parameter estimates and the performance of five test statistics including the naive statistic derived from normal distribution based maximum likelihood (ML), the Satorra-Bentler scaled chi-square statistic (RML), the mean- and variance-adjusted chi-square statistic (AML), Yuan-Bentler residual-based test statistic (CRADF), and Yuan-Bentler residual-based F statistic (RF). Data are generated and analyzed in R using the package rsem (Yuan & Zhang, 2011b).

Based on the simulation study, we can observe the following: (a) The traditional normal distribution-based method cannot yield accurate parameter estimates for nonnormal data, whereas the robust method obtains much more accurate model parameter estimates for nonnormal data and performs almost as well as the normal distribution based method for normal distributed data. (b) With the increase of sample size, or the decrease of missing rate or the number of outliers, the parameter estimates are less biased and the empirical distributions of test statistics are closer to their nominal distributions. (c) The ML test statistic does not work well for nonnormal or missing data. (d) For nonnormal complete data, CRADF and RF work relatively better than RML and AML. (e) For missing completely at random (MCAR) missing data, in almost all the cases, RML and AML work better than CRADF and RF. (f) For nonnormal missing at random (MAR) missing data, CRADF and RF work better than AML. (g) The performance of the robust method does not seem to be influenced by the symmetry of outliers.     

A scaled difference chi-square test statistic for moment structure analysis

A family of scaling corrections aimed to improve the chi-square approximation of goodness-of-fit test statistics in small samples, large models, and nonnormal data was proposed in Satorra and Bentler (1994). For structural equations models, Satorra-Bentler's (SB) scaling corrections are available in standard computer software. Often, however, the interest is not on the overall fit of a model, but on a test of the restrictions that a null model sayM ₀ implies on a less restricted oneM ₁. IfT ₀ andT ₁ denote the goodness-of-fit test statistics associated toM ₀ andM ₁, respectively, then typically the differenceT _d =T ₀−T ₁ is used as a chi-square test statistic with degrees of freedom equal to the difference on the number of independent parameters estimated under the modelsM ₀ andM ₁. As in the case of the goodness-of-fit test, it is of interest to scale the statisticT _d in order to improve its chi-square approximation in realistic, that is, nonasymptotic and nonormal, applications. In a recent paper, Satorra (2000) shows that the difference between two SB scaled test statistics for overall model fit does not yield the correct SB scaled difference test statistic. Satorra developed an expression that permits scaling the difference test statistic, but his formula has some practical limitations, since it requires heavy computations that are not available in standard computer software. The purpose of the present paper is to provide an easy way to compute the scaled difference chi-square statistic from the scaled goodness-of-fit test statistics of modelsM ₀ andM ₁. A Monte Carlo study is provided to illustrate the performance of the competing statistics. This research was supported by the Spanish grants PB96-0300 and BEC2000-0983, and USPHS grants DA00017 and DA01070.     

On robusiness of the normal-theory based asymptotic distributions of three reliability coefficient estimates

This paper studies the asymptotic distributions of three reliability coefficient estimates: Sample coefficient alpha, the reliability estimate of a composite score following a factor analysis, and the estimate of the maximal reliability of a linear combination of item scores following a factor analysis. Results indicate that the asymptotic distribution for each of the coefficient estimates, obtained based on a normal sampling distribution, is still valid within a large class of nonnormal distributions. Therefore, a formula for calculating the standard error of the sample coefficient alpha, recently obtained by van Zyl, Neudecker and Nel, applies to other reliability coefficients and can still be used even with skewed and kurtotic data such as are typical in the social and behavioral sciences.This research was supported by grants DA01070 and DA00017 from the National Institute on Drug Abuse and a University of North Texas faculty research grant. We would like to thank the Associate Editor and two reviewers for suggestions that helped to improve the paper.     

Robustness of Parameter Estimation to Assumptions of Normality in the Multidimensional Graded Response Model

A central assumption that is implicit in estimating item parameters in item response theory (IRT) models is the normality of the latent trait distribution, whereas a similar assumption made in categorical confirmatory factor analysis (CCFA) models is the multivariate normality of the latent response variables. Violation of the normality assumption can lead to biased parameter estimates. Although previous studies have focused primarily on unidimensional IRT models, this study extended the literature by considering a multidimensional IRT model for polytomous responses, namely the multidimensional graded response model. Moreover, this study is one of few studies that specifically compared the performance of full-information maximum likelihood (FIML) estimation versus robust weighted least squares (WLS) estimation when the normality assumption is violated. The research also manipulated the number of nonnormal latent trait dimensions. Results showed that FIML consistently outperformed WLS when there were one or multiple skewed latent trait distributions. More interestingly, the bias of the discrimination parameters was non-ignorable only when the corresponding factor was skewed. Having other skewed factors did not further exacerbate the bias, whereas biases of boundary parameters increased as more nonnormal factors were added. The item parameter standard errors recovered well with both estimation algorithms regardless of the number of nonnormal dimensions.     

Model-Implied Instrumental Variable—Generalized Method of Moments (MIIV-GMM) Estimators for Latent Variable Models

The common maximum likelihood (ML) estimator for structural equation models (SEMs) has optimal asymptotic properties under ideal conditions (e.g., correct structure, no excess kurtosis, etc.) that are rarely met in practice. This paper proposes model-implied instrumental variable – generalized method of moments (MIIV-GMM) estimators for latent variable SEMs that are more robust than ML to violations of both the model structure and distributional assumptions. Under less demanding assumptions, the MIIV-GMM estimators are consistent, asymptotically unbiased, asymptotically normal, and have an asymptotic covariance matrix. They are “distribution-free,” robust to heteroscedasticity, and have overidentification goodness-of-fit J-tests with asymptotic chi-square distributions. In addition, MIIV-GMM estimators are “scalable” in that they can estimate and test the full model or any subset of equations, and hence allow better pinpointing of those parts of the model that fit and do not fit the data. An empirical example illustrates MIIV-GMM estimators. Two simulation studies explore their finite sample properties and find that they perform well across a range of sample sizes.     

Tests of Homoscedasticity,Normality, and Missing Completely at Random for Incomplete Multivariate Data

Test of homogeneity of covariances (or homoscedasticity) among several groups has many applications in statistical analysis. In the context of incomplete data analysis, tests of homoscedasticity among groups of cases with identical missing data patterns have been proposed to test whether data are missing completely at random (MCAR). These tests of MCAR require large sample sizes n and/or large group sample sizes n _i , and they usually fail when applied to nonnormal data. Hawkins (Technometrics 23:105–110, 1981) proposed a test of multivariate normality and homoscedasticity that is an exact test for complete data when n _i are small. This paper proposes a modification of this test for complete data to improve its performance, and extends its application to test of homoscedasticity and MCAR when data are multivariate normal and incomplete. Moreover, it is shown that the statistic used in the Hawkins test in conjunction with a nonparametric k-sample test can be used to obtain a nonparametric test of homoscedasticity that works well for both normal and nonnormal data. It is explained how a combination of the proposed normal-theory Hawkins test and the nonparametric test can be employed to test for homoscedasticity, MCAR, and multivariate normality. Simulation studies show that the newly proposed tests generally outperform their existing competitors in terms of Type I error rejection rates. Also, a power study of the proposed tests indicates good power. The proposed methods use appropriate missing data imputations to impute missing data. Methods of multiple imputation are described and one of the methods is employed to confirm the result of our single imputation methods. Examples are provided where multiple imputation enables one to identify a group or groups whose covariance matrices differ from the majority of other groups.     

A high-level language program to obtain the bootstrap corrected ADF test statistic

A high-level language program to obtain the bootstrap-corrected asymptotic distribution-free (ADF) test statistic proposed by Yung and Bentler (1994) is reviewed. The program uses the Gauss-Newton algorithm, first to obtain the ADF test statistic from the raw data, and second, to achieve the corrected test statistic from 500 independent bootstrap samples. A generator of nonnormal random samples was also implemented, according to the algorithms of Fleishman (1978) and Vale and Maurelli (1983), which permits the realization of Monte Carlo simulations. Furthermore, the open nature of the program facilitates the inclusion of new procedures as well as the possibility of increased control of the procedures, variables, and equations.     

A Cautionary Note on Using G2(dif) to Assess Relative Model Fit in Categorical Data Analysis

The likelihood ratio test statistic G²(dif) is widely used for comparing the fit of nested models in categorical data analysis. In large samples, this statistic is distributed as a chi-square with degrees of freedom equal to the difference in degrees of freedom between the tested models, but only if the least restrictive model is correctly specified. Yet, this statistic is often used in applications without assessing the adequacy of the least restrictive model. This may result in incorrect substantive conclusions as the above large sample reference distribution for G²(dif) is no longer appropriate. Rather, its large sample distribution will depend on the degree of model misspecification of the least restrictive model. To illustrate this, a simulation study is performed where this statistic is used to compare nested item response theory models under various degrees of misspecification of the least restrictive model. G²(dif) was found to be robust only under small model misspecification of the least restrictive model. Consequently, we argue that some indication of the absolute goodness of fit of the least restrictive model is needed before employing G²(dif) to assess relative model fit.     

Apparatus for Smoking Kymograph Drum Papers

In a computer simulation study, random samples from a uniform density were substituted for each of two independent samples from normal and various nonnormal densities. This procedure was compared with conventional ranking and with Bell and Doksum's (1965) procedure, which substituted random normal deviates for initial sample values. After performing the Student t test, the program transformed the initial scores and performed additional t tests on ranks, random uniform scores, and random normal scores. For several distributions, the test on random normal scores was more powerful than the others, consistent with known asymptotic results. The probabilities of Type I and Type II errors of the test on random uniform scores were nearly the same as those of the Mann-Whitney-Wilcoxon test, for all distributions examined.     

On the asymptotic distributions of two statistics for two-level covariance structure models within the class of elliptical distributions

Since data in social and behavioral sciences are often hierarchically organized, special statistical procedures for covariance structure models have been developed to reflect such hierarchical structures. Most of these developments are based on a multivariate normality distribution assumption, which may not be realistic for practical data. It is of interest to know whether normal theory-based inference can still be valid with violations of the distribution condition. Various interesting results have been obtained for conventional covariance structure analysis based on the class of elliptical distributions. This paper shows that similar results still hold for 2-level covariance structure models. Specifically, when both the level-1 (within cluster) and level-2 (between cluster) random components follow the same elliptical distribution, the rescaled statistic recently developed by Yuan and Bentler asymptotically follows a chi-square distribution. When level-1 and level-2 have different elliptical distributions, an additional rescaled statistic can be constructed that also asymptotically follows a chi-square distribution. Our results provide a rationale for applying these rescaled statistics to general non-normal distributions, and also provide insight into issues related to level-1 and level-2 sample sizes. The authors thank an associate editor and three referees for their constructive comments, which led to an improved version of the paper. This research was supported by grants DA01070 and DA00017 from the National Institute on Drug Abuse and a University of Notre Dame faculty research grant.     

Asymptotic biases in exploratory factor analysis and structural equation modeling

Formulas for the asymptotic biases of the parameter estimates in structural equation models are provided in the case of the Wishart maximum likelihood estimation for normally and nonnormally distributed variables. When multivariate normality is satisfied, considerable simplification is obtained for the models of unstandardized variables. Formulas for the models of standardized variables are also provided. Numerical examples with Monte Carlo simulations in factor analysis show the accuracy of the formulas and suggest the asymptotic robustness of the asymptotic biases with normality assumption against nonnormal data. Some relationships between the asymptotic biases and other asymptotic values are discussed.The author is indebted to the editor and anonymous reviewers for their comments, corrections, and suggestions on this paper, and to Yutaka Kano for discussion on biases.     

A one-way random effects model for trimmed means

The random effects ANOVA model plays an important role in many psychological studies, but the usual model suffers from at least two serious problems. The first is that even under normality, violating the assumption of equal variances can have serious consequences in terms of Type I errors or significance levels, and it can affect power as well. The second and perhaps more serious concern is that even slight departures from normality can result in a substantial loss of power when testing hypotheses. Jeyaratnam and Othman (1985) proposed a method for handling unequal variances, under the assumption of normality, but no results were given on how their procedure performs when distributions are nonnormal. A secondary goal in this paper is to address this issue via simulations. As will be seen, problems arise with both Type I errors and power. Another secondary goal is to provide new simulation results on the Rust-Fligner modification of the Kruskal-Wallis test. The primary goal is to propose a generalization of the usual random effects model based on trimmed means. The resulting test of no differences among J randomly sampled groups has certain advantages in terms of Type I errors, and it can yield substantial gains in power when distributions have heavy tails and outliers. This last feature is very important in applied work because recent investigations indicate that heavy-tailed distributions are common. Included is a suggestion for a heteroscedastic Winsorized analog of the usual intraclass correlation coefficient.     

Testing latent longitudinal models of social ties and depression among the elderly: a comparison of distribution-free and maximum likelihood estimates with nonnormal data.

Using a latent-variable modeling approach, relationships between social ties and depression were studied in a sample of 201 older adults. Both positive and negative ties were related to concurrent depression, whereas only negative ties predicted future depression. Nonnormally distributed scores were observed for several variables, and results based on maximum likelihood (ML), which assumes multivariate normality, were compared with those obtained using Browne's (1982, 1984) arbitrary distribution function (ADF) estimator for nonnormal variables. Inappropriate use of ML with nonnormal data yielded model chi-square values that were too large and standard errors that were too small. ML also failed to detect the over-time effect of negative ties on depression. The results suggest that the negative functions of social networks may causally influence depression and illustrate the need to test distributional assumptions when estimating latent-variable models.     

Improved statistics for contrasting means of two samples under non‐normality

This paper presents the asymptotic expansions of the distributions of the two‐sample t‐statistic and the Welch statistic, for testing the equality of the means of two independent populations under non‐normality. Unlike other approaches, we obtain the null distributions in terms of the distribution and density functions of the standard normal variable up to n^?1, where n is the pooled sample size. Based on these expansions, monotone transformations are employed to remove the higher‐order cumulant effect. We show that the new statistics can improve the precision of statistical inference to the level of o (n^?1). Numerical studies are carried out to demonstrate the performance of the improved statistics. Some general rules for practitioners are also recommended.     

结构方程模型常用拟合指数检验的实质

拟合指数检验是评价结构方程模型(SEM)的重要环节。从协方差结构分析的角度将SEM与传统的回归模型比较,容易理解为什么SEM需要拟合指数。揭示了目前几种流行的拟合指数检验的实质:基于卡方的绝对拟合指数(如RMSEA)检验的实质是重新设定卡方检验的显著性水平(不同于通常的.05),相对拟合指数(如NNFI和CFI)检验的实质是基于虚模型设定均方(卡方与自由度之比)降低到的比例;在NNFI大于临界值后,报告和检验CFI是不必要的。根据研究结果提出了一些方便实用的拟合检验建议。     

Applicability of the parallel-clock model to duration discrimination

The parallel-clock model assumes that when an observer is presented with two durations in succession, the total subjective duration (corresponding to the sum of first and second durations) and the second duration are each accumulated in a separate sensory register. In dealing with some relation between the two durations, the observer compares the difference between the contents of the two registers with the contents of the second register. With the further assumption that the psychophysical power law is valid for the continuum of time, the model has previously been shown to account well for duration scaling data. After adapting the model in the vein of Thurstone for duration discrimination data, it was tried out on such data gathered by Allan (1977) from 13 observers. With chi-square as the examined statistic, (1) five distributions were compared and the normal one chosen, (2) one categorization model, three choice models, and two linear regression models were compared with the 3-parameter version of the parallel-lock model, which proved superior, and (3) a 4–5-parameter version of the parallel-clock model, assuming a discontinuity in the psychophysical function, was shown to yield an excellent fit in terms of the Kolmogorov-Smirnov statistic.     

Asymptotic expansion of the distributions of the estimators in factor analysis under non‐normality

Equations for the Edgeworth expansion of the distributions of the estimators in exploratory factor analysis and structural equation modeling are given. The equations cover the cases of non‐normal data, as well as normal ones with and without known first‐order asymptotic standard errors. When the standard errors are unknown, the distributions of the Studentized statistics are expanded. Methods of constructing confidence intervals of population parameters with arbitrary asymptotic confidence coefficients are given using the Cornish‐Fisher expansion. Simulations are performed to see the usefulness of the asymptotic expansions in exploratory factor analysis with rotated solutions and confirmatory factor analysis. The results show that asymptotic expansion gives substantial improvement of approximation to the exact distribution constructed by simulations over the usual normal approximation.     

Some contributions to efficient statistics in structural models: Specification and estimation of moment structures

Current practice in structural modeling of observed continuous random variables is limited to representation systems for first and second moments (e.g., means and covariances), and to distribution theory based on multivariate normality. In psychometrics the multinormality assumption is often incorrect, so that statistical tests on parameters, or model goodness of fit, will frequently be incorrect as well. It is shown that higher order product moments yield important structural information when the distribution of variables is arbitrary. Structural representations are developed for generalizations of the Bentler-Weeks, Jöreskog-Keesling-Wiley, and factor analytic models. Some asymptotically distribution-free efficient estimators for such arbitrary structural models are developed. Limited information estimators are obtained as well. The special case of elliptical distributions that allow nonzero but equal kurtoses for variables is discussed in some detail. The argument is made that multivariate normal theory for covariance structure models should be abandoned in favor of elliptical theory, which is only slightly more difficult to apply in practice but specializes to the traditional case when normality holds. Many open research areas are described.