An EM algorithm for fitting two-level structural equation models

Maximum likelihood is an important approach to analysis of two-level structural equation models. Different algorithms for this purpose have been available in the literature. In this paper, we present a new formulation of two-level structural equation models and develop an EM algorithm for fitting this formulation. This new formulation covers a variety of two-level structural equation models. As a result, the proposed EM algorithm is widely applicable in practice. A practical example illustrates the performance of the EM algorithm and the maximum likelihood statistic.We are thankful to the reviewers for their constructive comments that have led to significant improvement on the first version of this paper. Special thanks are due to the reviewer who suggested a comparison with the LISREL program in the saturated means model, and provided its setup and output. This work was supported by National Institute on Drug Abuse grants DA01070, DA00017, and a UNH 2002 Summer Faculty Fellowship.     

Tests for stability in linear structural equation systems

Interpretations regarding the effects of exogenous and endogenous variables on endogenous variables in linear structural equation systems depend upon the convergence of a matrix power series. Convergence depends upon the eigenvalues of the structural coefficient matrix. The test for convergence developed by Jöreskog and Sörbom is shown to be only sufficient, not necessary and sufficient.     

Bayesian estimation and testing of structural equation models

The Gibbs sampler can be used to obtain samples of arbitrary size from the posterior distribution over the parameters of a structural equation model (SEM) given covariance data and a prior distribution over the parameters. Point estimates, standard deviations and interval estimates for the parameters can be computed from these samples. If the prior distribution over the parameters is uninformative, the posterior is proportional to the likelihood, and asymptotically the inferences based on the Gibbs sample are the same as those based on the maximum likelihood solution, for example, output from LISREL or EQS. In small samples, however, the likelihood surface is not Gaussian and in some cases contains local maxima. Nevertheless, the Gibbs sample comes from the correct posterior distribution over the parameters regardless of the sample size and the shape of the likelihood surface. With an informative prior distribution over the parameters, the posterior can be used to make inferences about the parameters underidentified models, as we illustrate on a simple errors-in-variables model.We thank David Spiegelhalter for suggesting applying the Gibbs sampler to structural equation models to the first author at a 1994 workshop in Wiesbaden. We thank Ulf Böckenholt, Chris Meek, Marijtje van Duijn, Clark Glymour, Ivo Molenaar, Steve Klepper, Thomas Richardson, Teddy Seidenfeld, and Tom Snijders for helpful discussions, mathematical advice, and critiques of earlier drafts of this paper.     

The analysis of structural equation models by means of derivative free nonlinear least squares

It is shown that the PAR Derivative-Free Nonlinear Regression program in BMDP can be used to fit structural equation models, producing generalized least squares estimates, standard errors, and goodness-of-fit test statistics. Covariance structure models more general than LISREL can be analyzed. The approach is particularly useful for dealing with new non-standard models and experimenting with alternate methods of estimation. The research of the second author was supported by the NSF grant MCS 83-01587. We wish to thank our referees for some very valuable suggestions.     

Parameterizing inequality constraints on unique variances in linear structural models

Current computer programs for analyzing linear structural models will apparently handle only two types of constraints: fixed parameters, and equality of parameters. An important constraint not handled is inequality; this is particularly crucial for preventing negative variance estimates. In this paper, a method is described for imposing several kinds of inequality constraints in models, without the necessity for having computer programs which explicitly allow such constraints. The examples discussed include the prevention of Heywood cases, extension to inequalities of parameters to be greater than a specified value, and imposing ordered inequalities. Work on this project was aided by the City University of New York—Professional Staff Congress Research Award Program Grant Number 13631.     

Exploratory structural equation modeling,bifactor models,and standard confirmatory factor analysis models: Application to the BASC-2 Behavioral and Emotional Screening System Teacher Form

Several psychological assessment instruments are based on the assumption of a general construct that is composed of multiple interrelated domains. Standard confirmatory factor analysis is often not well suited for examining the factor structure of such scales. This study used data from 1885 elementary school students (mean age = 8.77 years, SD = 1.47 years) to examine the factor structure of the Behavioral Assessment System for Children, Second Edition (BASC-2) Behavioral and Emotional Screening System (BESS) Teacher Form that was designed to assess general risk for emotional/behavioral difficulty among children. The modeling sequence included the relatively new exploratory structural equation modeling (ESEM) approach and bifactor models in addition to more standard techniques. Findings revealed that the factor structure of the BASC-2 BESS Teacher Form is multidimensional. Both ESEM and bifactor models showed good fit to the data. Bifactor models were preferred on conceptual grounds. Findings illuminate the hypothesis-generating power of ESEM and suggest that it might not be optimal for instruments designed to assess a predominant general factor underlying the data.     

Fitting direct covariance structures by the MSTRUCT modeling language of the CALIS procedure

This paper demonstrates the usefulness and flexibility of the general structural equation modelling (SEM) approach to fitting direct covariance patterns or structures (as opposed to fitting implied covariance structures from functional relationships among variables). In particular, the MSTRUCT modelling language (or syntax) of the CALIS procedure (SAS/STAT version 9.22 or later: SAS Institute, 2010) is used to illustrate the SEM approach. The MSTRUCT modelling language supports a direct covariance pattern specification of each covariance element. It also supports the input of additional independent and dependent parameters. Model tests, fit statistics, estimates, and their standard errors are then produced under the general SEM framework. By using numerical and computational examples, the following tests of basic covariance patterns are illustrated: sphericity, compound symmetry, and multiple‐group covariance patterns. Specification and testing of two complex correlation structures, the circumplex pattern and the composite direct product models with or without composite errors and scales, are also illustrated by the MSTRUCT syntax. It is concluded that the SEM approach offers a general and flexible modelling of direct covariance and correlation patterns. In conjunction with the use of SAS macros, the MSTRUCT syntax provides an easy‐to‐use interface for specifying and fitting complex covariance and correlation structures, even when the number of variables or parameters becomes large.     

Asymptotic biases in exploratory factor analysis and structural equation modeling

Formulas for the asymptotic biases of the parameter estimates in structural equation models are provided in the case of the Wishart maximum likelihood estimation for normally and nonnormally distributed variables. When multivariate normality is satisfied, considerable simplification is obtained for the models of unstandardized variables. Formulas for the models of standardized variables are also provided. Numerical examples with Monte Carlo simulations in factor analysis show the accuracy of the formulas and suggest the asymptotic robustness of the asymptotic biases with normality assumption against nonnormal data. Some relationships between the asymptotic biases and other asymptotic values are discussed.The author is indebted to the editor and anonymous reviewers for their comments, corrections, and suggestions on this paper, and to Yutaka Kano for discussion on biases.     

On normal theory based inference for multilevel models with distributional violations

Data in social and behavioral sciences are often hierarchically organized though seldom normal, yet normal theory based inference procedures are routinely used for analyzing multilevel models. Based on this observation, simple adjustments to normal theory based results are proposed to minimize the consequences of violating normality assumptions. For characterizing the distribution of parameter estimates, sandwich-type covariance matrices are derived. Standard errors based on these covariance matrices remain consistent under distributional violations. Implications of various covariance estimators are also discussed. For evaluating the quality of a multilevel model, a rescaled statistic is given for both the hierarchical linear model and the hierarchical structural equation model. The rescaled statistic, improving the likelihood ratio statistic by estimating one extra parameter, approaches the same mean as its reference distribution. A simulation study with a 2-level factor model implies that the rescaled statistic is preferable.This research was supported by grants DA01070 and DA00017 from the National Institute on Drug Abuse and a University of North Texas faculty research grant. We would like to thank the Associate Editor and two reviewers for suggestions that helped to improve the paper.     

On structural models for analyzing change

The recommendation to base the analysis of multi-wave data upon explicit models for change is advocated. Several univariate and multivariate models are described, which emerge from an interaction between the classical test theory and the structural equation modeling approach. The resulting structural models for analyzing change reflect in some of their parameters substantively interesting aspects of intra- and interindividual change in follow-up studies. The models are viewed as an alternative to an ANOVA-based analysis of longitudinal data, and are illustrated on data from a cognitive intervention study of old adults (Bakes et al , 1986). The approach presents a useful means of analyzing change over time, and is applicable for purposes of (latent) growth curve analysis when analysis of variance assumptions are violated (e.g., Schaie & Hertzog, 1982; Morrison, 1976).     

Generalized multilevel structural equation modeling

A unifying framework for generalized multilevel structural equation modeling is introduced. The models in the framework, called generalized linear latent and mixed models (GLLAMM), combine features of generalized linear mixed models (GLMM) and structural equation models (SEM) and consist of a response model and a structural model for the latent variables. The response model generalizes GLMMs to incorporate factor structures in addition to random intercepts and coefficients. As in GLMMs, the data can have an arbitrary number of levels and can be highly unbalanced with different numbers of lower-level units in the higher-level units and missing data. A wide range of response processes can be modeled including ordered and unordered categorical responses, counts, and responses of mixed types. The structural model is similar to the structural part of a SEM except that it may include latent and observed variables varying at different levels. For example, unit-level latent variables (factors or random coefficients) can be regressed on cluster-level latent variables. Special cases of this framework are explored and data from the British Social Attitudes Survey are used for illustration. Maximum likelihood estimation and empirical Bayes latent score prediction within the GLLAMM framework can be performed using adaptive quadrature in gllamm, a freely available program running in Stata.gllamm can be downloaded from http://www.gllamm.org. The paper was written while Sophia Rabe-Hesketh was employed at and Anders Skrondal was visiting the Department of Biostatistics and Computing, Institute of Psychiatry, King's College London.     

结构方程模型统计检验力分析：原理与方法

结构方程模型是心理学、管理学、社会学等学科中重要的统计工具之一。然而, 大量使用结构方程模型的研究忽视了对该方法的统计检验力进行必要的分析和报告, 在一定程度上降低了这些研究的结果的证明效力。结构方程模型的统计检验力分析方法主要有Satorra-Saris法、MacCallum法与Monte Carlo法三类。其中Satorra-Saris法适用于备择模型清晰、检验对象相对简单、检验方法基于χ²分布的情形; MacCallum法适用于基于χ²分布的模型拟合检验且备择模型不明的情形; Monte Carlo法适用于检验对象相对复杂、采用模拟或重抽样方法进行检验的情形。在实际应用中, 研究者应当首先判断检验的目的、方法以及是否有明确的备择模型, 并根据这些信息选择具体的分析方法。     

Influence analysis of structural equation models with polytomous variables

The estimation of model parameters in structural equation models with polytomous variables can be handled by several computationally efficient procedures. However, sensitivity or influence analysis of the model is not well studied. We demonstrate that the existing influence analysis methods for contingency tables or for normal theory structural equation models cannot be applied directly to structural equation models with polytomous variables; and we develop appropriate procedures based on the local influence approach of Cook (1986). The proposed procedures are computationally efficient, the necessary bits of the proposed diagnostic measures are readily available following an usual fit of the model. We consider the influence of an individual cell frequency with respect to three cases: when all parameters in an unstructured model are of interest, when the unstructured polychoric correlations are of interest, and when the structural parameters are of interest. We also consider the sensitivity of the parameters estimates. Two examples based on real data are presented for illustration.The work described in this paper was partially supported by a Chinese University of Hong Kong Postdoctoral Fellows Scheme and a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (RGC Ref. No. CUHK4186/98P). We are indebted to P.M. Bentler and M.D. Newcomb for providing the data set, Michael Leung for his assistance, and the Editor and the referees for some very valuable comments.     

Structural equation models with continuous and polytomous variables

A two-stage procedure is developed for analyzing structural equation models with continuous and polytomous variables. At the first stage, the maximum likelihood estimates of the thresholds, polychoric covariances and variances, and polyserial covariances are simultaneously obtained with the help of an appropriate transformation that significantly simplifies the computation. An asymptotic covariance matrix of the estiates is also computed. At the second stage, the parameters in the structural covariance model are obtained via the generalized least squares approach. Basic statistical properties of the estimates are derived and some illustrative examples and a small simulation study are reported.This research was supported in part by a research grant DA01070 from the U. S. Public Health Service. We are indebted to several referees and the editor for very valuable comments and suggestions for improvement of this paper. The computing assistance of King-Hong Leung and Man-Lai Tang is also gratefully acknowledged.     

Processing speed, intelligence, creativity, and school performance: Testing of causal hypotheses using structural equation models

According to mental speed theory of intelligence, the speed of information processing constitutes an important basis for cognitive abilities. However, the question, how mental speed relates to real world criteria, like school, academic, or job performance, is still unanswered. The aim of the study is to test an indirect speed-factor model in comparison to rivaling models explaining the relationships between different mental abilities and performance. In this speed-factor model, basic cognitive processing is assumed to influence higher mental abilities (IQ and creativity). Intelligence and creativity themselves should be valid predictors of school performance. We computed bivariate correlations and structural equation models to test this hypothesis, using indicators of processing speed [Zahlen-Verbindungs-Test (ZVT) and Coding Test], psychometric intelligence [Kognitiver Fähigkeits-Test (KFT) and Raven's Advanced Progressive Matrices (APM)], creativity [Verbaler Kreativitäts-Test (VKT) and Verwendungs-Test (VWT)] and school performance (grades). In a sample of 271 students from German gymnasiums (Class Levels 9 to 11) the speed-factor model can reproduce at best the empirical relationships between processing speed, intelligence, creativity, and school performance: It assumes that processing speed influences higher mental abilities (intelligence and creativity), which, in the sequel, influence school performance. Therefore, processing speed seems to have no direct effect on school performance; the effect is indirect as it operates via mediation through higher cognitive abilities.     

The robustness of estimates of total indirect effects in covariance structure models estimated by maximum

The large sample distribution of total indirect effects in covariance structure models in well known. Using Monte Carlo methods, this study examines the applicability of the large sample theory to maximum likelihood estimates oftotal indirect effects in sample sizes of 50, 100, 200, 400, and 800. Two models are studied. Model 1 is a recursive model with observable variables and Model 2 is a nonrecursive model with latent variables. For the large sample theory to apply, the results suggest that sample szes of 200 or more and 400 or more are required for models such as Model 1 and Model 2, respectively.For helpful comments on a previous draft of this paper, we are grateful to Gerhard Arminger, Clifford C. Clogg, and several anonymous reviewers.     

Model comparison of nonlinear structural equation models with fixed covariates

Recently, it has been recognized that the commonly used linear structural equation model is inadequate to deal with some complicated substantive theory. A new nonlinear structural equation model with fixed covariates is proposed in this article. A procedure, which utilizes the powerful path sampling for computing the Bayes factor, is developed for model comparison. In the implementation, the required random observations are simulated via a hybrid algorithm that combines the Gibbs sampler and the Metropolis-Hastings algorithm. It is shown that the proposed procedure is efficient and flexible; and it produces Bayesian estimates of the parameters, latent variables, and their highest posterior density intervals as by-products. Empirical performances of the proposed procedure such as sensitivity to prior inputs are illustrated by a simulation study and a real example.This research is fully supported by a grant from the Research Grant Council of the Hong Kong Special Administrative Region, China (Project No. CUHK 4346/01H). The authors are thankful to the Editor, the Associate Editor, and anonymous reviewers for valuable comments which improve the paper significantly, and grateful to ICPSR and the relevant funding agency for allowing use of the data in the example. The assistance of Michael K.H. Leung and Esther L.S. Tam is gratefully acknowledged.     

Equivalent models in covariance structure analysis

Defining equivalent models as those that reproduce the same set of covariance matrices, necessary and sufficient conditions are stated for the local equivalence of two expanded identified modelsM ₁ andM ₂ when fitting the more restricted modelM ₀. Assuming several regularity conditions, the rank deficiency of the Jacobian matrix, composed of derivatives of the covariance elements with respect to the union of the free parameters ofM ₁ andM ₂ (which characterizes modelM ₁₂), is a necessary and sufficient condition for the local equivalence ofM ₁ andM ₂. This condition is satisfied, in practice, when the analysis dealing with the fitting ofM ₀, predicts that the decreases in the chi-square goodness-of-fit statistic for the fitting ofM ₁ orM ₂, orM ₁₂ are all equal for any set of sample data, except on differences due to rounding errors.This research was supported by the Foundation of Social-Cultural Sciences which is subsidized by the Dutch Scientific Organization (N.W.O.) under project number 500-278-003. The author wishes to thank Anne Boomsma, Ivo Molenaar, Albert Satorra, and Tom Snijders for their stimulating and crucial comments during the research, and the Editor, Paul Bekker, Henk Broer, and anonymous reviewers for their helpful suggestions.