Estimation for structural equation models with missing data

A direct method in handling incomplete data in general covariance structural models is investigated. Asymptotic statistical properties of the generalized least squares method are developed. It is shown that this approach has very close relationships with the maximum likelihood approach. Iterative procedures for obtaining the generalized least squares estimates, the maximum likelihood estimates, as well as their standard error estimates are derived. Computer programs for the confirmatory factor analysis model are implemented. A longitudinal type data set is used as an example to illustrate the results.This research was supported in part by Research Grant DAD1070 from the U.S. Public Health Service. The author is indebted to anonymous reviewers for some very valuable suggestions. Computer funding is provided by the Computer Services Centre, The Chinese University of Hong Kong.     

A feasible method for standard errors of estimate in maximum likelihood factor analysis

A jackknife-like procedure is developed for producing standard errors of estimate in maximum likelihood factor analysis. Unlike earlier methods based on information theory, the procedure developed is computationally feasible on larger problems. Unlike earlier methods based on the jackknife, the present procedure is not plagued by the factor alignment problem, the Heywood case problem, or the necessity to jackknife by groups. Standard errors may be produced for rotated and unrotated loading estimates using either orthogonal or oblique rotation as well as for estimates of unique factor variances and common factor correlations. The total cost for larger problems is a small multiple of the square of the number of variables times the number of observations used in the analysis. Examples are given to demonstrate the feasibility of the method.The research done by R. I. Jennrich was supported in part by NSF Grant MCS 77-02121. The research done by D. B. Clarkson was supported in part by NSERC Grant A3109.     

A study of algorithms for covariance structure analysis with specific comparisons using factor analysis

Several algorithms for covariance structure analysis are considered in addition to the Fletcher-Powell algorithm. These include the Gauss-Newton, Newton-Raphson, Fisher Scoring, and Fletcher-Reeves algorithms. Two methods of estimation are considered, maximum likelihood and weighted least squares. It is shown that the Gauss-Newton algorithm which in standard form produces weighted least squares estimates can, in iteratively reweighted form, produce maximum likelihood estimates as well. Previously unavailable standard error estimates to be used in conjunction with the Fletcher-Reeves algorithm are derived. Finally all the algorithms are applied to a number of maximum likelihood and weighted least squares factor analysis problems to compare the estimates and the standard errors produced. The algorithms appear to give satisfactory estimates but there are serious discrepancies in the standard errors. Because it is robust to poor starting values, converges rapidly and conveniently produces consistent standard errors for both maximum likelihood and weighted least squares problems, the Gauss-Newton algorithm represents an attractive alternative for at least some covariance structure analyses.Work by the first author has been supported in part by Grant No. Da01070 from the U. S. Public Health Service. Work by the second author has been supported in part by Grant No. MCS 77-02121 from the National Science Foundation.     

Exploratory Bi-factor Analysis: The Oblique Case

Bi-factor analysis is a form of confirmatory factor analysis originally introduced by Holzinger and Swineford (Psychometrika 47:41?C54, 1937). The bi-factor model has a general factor, a number of group factors, and an explicit bi-factor structure. Jennrich and Bentler (Psychometrika 76:537?C549, 2011) introduced an exploratory form of bi-factor analysis that does not require one to provide an explicit bi-factor structure a priori. They use exploratory factor analysis and a bifactor rotation criterion designed to produce a rotated loading matrix that has an approximate bi-factor structure. Among other things this can be used as an aid in finding an explicit bi-factor structure for use in a confirmatory bi-factor analysis. They considered only orthogonal rotation. The purpose of this paper is to consider oblique rotation and to compare it to orthogonal rotation. Because there are many more oblique rotations of an initial loading matrix than orthogonal rotations, one expects the oblique results to approximate a bi-factor structure better than orthogonal rotations and this is indeed the case. A surprising result arises when oblique bi-factor rotation methods are applied to ideal data.     

Asymptotic sampling variances for factor analytic models identified by specified zero parameters

Formulas are derived for the asymptotic variances and covariances of the maximum likelihood estimators for oblique simple structure models which are identified by prior specification of zero elements in the factor loading matrix. The formulas are expressed in terms of the various submatrices of the inverse of the required variance-covariance matrix. A numerical example using artificial data is given and problems in the application of the formulas discussed.Now at The Pennsylvania State University.     

Determinants of Standard Errors of MLEs in Confirmatory Factor Analysis

This paper studies changes of standard errors (SE) of the normal-distribution-based maximum likelihood estimates (MLE) for confirmatory factor models as model parameters vary. Using logical analysis, simplified formulas and numerical verification, monotonic relationships between SEs and factor loadings as well as unique variances are found. Conditions under which monotonic relationships do not exist are also identified. Such functional relationships allow researchers to better understand the problem when significant factor loading estimates are expected but not obtained, and vice versa. What will affect the likelihood for Heywood cases (negative unique variance estimates) is also explicit through these relationships. Empirical findings in the literature are discussed using the obtained results.     

Approximated Penalized Maximum Likelihood for Exploratory Factor Analysis: An Orthogonal Case

The problem of penalized maximum likelihood (PML) for an exploratory factor analysis (EFA) model is studied in this paper. An EFA model is typically estimated using maximum likelihood and then the estimated loading matrix is rotated to obtain a sparse representation. Penalized maximum likelihood simultaneously fits the EFA model and produces a sparse loading matrix. To overcome some of the computational drawbacks of PML, an approximation to PML is proposed in this paper. It is further applied to an empirical dataset for illustration. A simulation study shows that the approximation naturally produces a sparse loading matrix and more accurately estimates the factor loadings and the covariance matrix, in the sense of having a lower mean squared error than factor rotations, under various conditions.     

Rotation to Simple Loadings Using Component Loss Functions: The Oblique Case

Component loss functions (CLFs) similar to those used in orthogonal rotation are introduced to define criteria for oblique rotation in factor analysis. It is shown how the shape of the CLF affects the performance of the criterion it defines. For example, it is shown that monotone concave CLFs give criteria that are minimized by loadings with perfect simple structure when such loadings exist. Moreover, if the CLFs are strictly concave, minimizing must produce perfect simple structure whenever it exists. Examples show that methods defined by concave CLFs perform well much more generally. While it appears important to use a concave CLF, the specific CLF used is less important. For example, the very simple linear CLF gives a rotation method that can easily outperform the most popular oblique rotation methods promax and quartimin and is competitive with the more complex simplimax and geomin methods. The author would like to thank the editor and three reviewers for helpful suggestions and for identifying numerous errors.     

Standard errors for rotated factor loadings

Beginning with the results of Girshick on the asymptotic distribution of principal component loadings and those of Lawley on the distribution of unrotated maximum likelihood factor loadings, the asymptotic distribution of the corresponding analytically rotated loadings is obtained. The principal difficulty is the fact that the transformation matrix which produces the rotation is usually itself a function of the data. The approach is to use implicit differentiation to find the partial derivatives of an arbitrary orthogonal rotation algorithm. Specific details are given for the orthomax algorithms and an example involving maximum likelihood estimation and varimax rotation is presented.This research was supported in part by NIH Grant RR-3. The authors are grateful to Dorothy T. Thayer who implemented the algorithms discussed here as well as those of Lawley and Maxwell. We are particularly indebted to Michael Browne for convincing us of the significance of this work and for helping to guide its development and to Harry H. Harman who many years ago pointed out the need for standard errors of estimate.     

Standard errors for the class of orthomax-rotated factor loadings: Some matrix results

The partial derivative matrices of the class of orthomax-rotated factor loadings with respect to the unrotated maximum likelihood factor loadings are derived. The reported results are useful for obtaining standard errors of the orthomax-rotated factor loadings, with or without row normalization (standardization) of the initial factor loading matrix for rotation. Using a numerical example, we verify our analytic formulas by comparing the obtained standard error estimates with that from some existing methods. Some advantages of the current approach are discussed.Authorship is determined by alphabetical order. The authors contributed equally to the research. Kentaro Hayashi is now at the Department of Mathematics, Bucknell University, Lewisburg, PA 17837 (email: khayashi@Bucknell.edu). Yiu-Fai Yung is now at the SAS Institute, Inc., SAS Campus Drive, Cary, NC 27513 (email: yiyung@wnt.sas.com).Part of the research was completed while Yiu-Fai Yung was a visiting scholar at the Department of Psychology, the Ohio State University. The visit was supported in part by grant N4856118101 from the NIMH and the Mason and Linda Stephenson Travel Award from the Department of Psychology, University of North Carolina at Chapel Hill. The authors are grateful to Michael Browne who suggested some relevant references and provided valuable comments on the research, and to Robert Cudeck who provided the FAS program for the numerical comparison. The expert comments by the reviewers are deeply appreciated.     

Oblique factors and components with independent clusters

Relationships between the results of factor analysis and component analysis are derived when oblique factors have independent clusters with equal variances of unique factors. The factor loadings are analytically shown to be smaller than the corresponding component loadings while the factor correlations are shown to be greater than the corresponding component correlations. The condition for the inequality of the factor/component contributions is derived in the case with different variances for unique factors. Further, the asymptotic standard errors of parameter estimates are obtained for a simplified model with the assumption of multivariate normality, which shows that the component loading estimate is more stable than the corresponding factor loading estimate.     

On the estimation of parameters in latent structure analysis

In this note, we describe the iterative procedure introduced earlier by Goodman to calculate the maximum likelihood estimates of the parameters in latent structure analysis, and we provide here a simple and direct proof of the fact that the parameter estimates obtained with the iterative procedure cannot lie outside the allowed interval. Formann recently stated that Goodman's algorithm can yield parameter estimates that lie outside the allowed interval, and we prove in the present note that Formann's contention is incorrect.This research was supported in part by Research Contract No. NSF SOC 76-80389 from the Division of the Social Sciences of the National Science Foundation. The author is indebted to C. C. Clogg for helpful comments and for the numerical results reported here (see, e.g., Table 1).     

A gauss-newton algorithm for exploratory factor analysis

It is shown that the scoring algorithm for maximum likelihood estimation in exploratory factor analysis can be developed in a way that is many times more efficient than a direct development based on information matrices and score vectors. The algorithm offers a simple alternative to current algorithms and when used in one-step mode provides the simplest and fastest method presently available for moving from consistent to efficient estimates. Perhaps of greater importance is its potential for extension to the confirmatory model. The algorithm is developed as a Gauss-Newton algorithm to facilitate its application to generalized least squares and to maximum likelihood estimation.This research was supported by NSF Grant MCS-8301587.     

On equivariance and invariance of standard errors in three exploratory factor models

Current practice in factor analysis typically involves analysis of correlation rather than covariance matrices. We study whether the standardz-statistic that evaluates whether a factor loading is statistically necessary is correctly applied in such situations and more generally when the variables being analyzed are arbitrarily rescaled. Effects of rescaling on estimated standard errors of factor loading estimates, and the consequent effect onz-statistics, are studied in three variants of the classical exploratory factor model under canonical, raw varimax, and normal varimax solutions. For models with analytical solutions we find that some of the standard errors as well as their estimates are scale equivariant, while others are invariant. For a model in which an analytical solution does not exist, we use an example to illustrate that neither the factor loading estimates nor the standard error estimates possess scale equivariance or invariance, implying that different conclusions could be obtained with different scalings. Together with the prior findings on parameter estimates, these results provide new guidance for a key statistical aspect of factor analysis.We gratefully acknowledge the help of the Associate Editor and three referees whose constructive comments lead to an improved version of the paper. This work was supported by National Institute on Drug Abuse Grants DA01070 and DA00017 and by the University of North Texas Faculty Research Grant Program.     

Properties of the maximum likelihood solution in factor analysis regression

Algebraic properties of the normal theory maximum likelihood solution in factor analysis regression are investigated. Two commonly employed measures of the within sample predictive accuracy of the factor analysis regression function are considered: the variance of the regression residuals and the squared correlation coefficient between the criterion variable and the regression function. It is shown that this within sample residual variance and within sample squared correlation may be obtained directly from the factor loading and unique variance estimates, without use of the original observations or the sample covariance matrix.     

Estimation of covariance structure models with parameters subject to functional restraints

This paper demonstrates the feasibility of using the penalty function method to estimate parameters that are subject to a set of functional constraints in covariance structure analysis. Both types of inequality and equality constraints are studied. The approaches of maximum likelihood and generalized least squares estimation are considered. A modified Scoring algorithm and a modified Gauss-Newton algorithm are implemented to produce the appropriate constrained estimates. The methodology is illustrated by its applications to Heywood cases in confirmatory factor analysis, quasi-Weiner simplex model, and multitrait-multimethod matrix analysis.The author is indebted to several anonymous reviewers for creative suggestions for improvement of this paper. Computer funding is provided by the Computer Services Centre, The Chinese University of Hong Kong.     

A general approach to confirmatory maximum likelihood factor analysis

We describe a general procedure by which any number of parameters of the factor analytic model can be held fixed at any values and the remaining free parameters estimated by the maximum likelihood method. The generality of the approach makes it possible to deal with all kinds of solutions: orthogonal, oblique and various mixtures of these. By choosing the fixed parameters appropriately, factors can be defined to have desired properties and make subsequent rotation unnecessary. The goodness of fit of the maximum likelihood solution under the hypothesis represented by the fixed parameters is tested by a large samplex ² test based on the likelihood ratio technique. A by-product of the procedure is an estimate of the variance-covariance matrix of the estimated parameters. From this, approximate confidence intervals for the parameters can be obtained. Several examples illustrating the usefulness of the procedure are given.This work was supported by a grant (NSF-GB 1985) from the National Science Foundation to Educational Testing Service.     

Techniques for rotating two or more loading matrices to optimal agreement and simple structure: A comparison and some technical details

Matrices of factor loadings are often rotated to simple structure. When more than one loading matrix is available for the same variables, the loading matrices can be compared after rotating them all (separately) to simple structure. An alternative procedure is to rotate them to optimal agreement, and then compare them. In the present paper techniques are described that combine these two procedures. Specifically, five techniques that combine the ideals of rotation to optimal agreement and rotation to simple structure are compared on the basis of contrived and empirical data. For the contrived data, it is assessed to what extent the rotations recover the underlying common structure. For both the contrived and the empirical data it is studied to what extent the techniques give well matching rotated matrices, to what extent these have a simple structure, and to what extent the most prominent parts of the different loading matrices agree. It was found that the simple procedure of combining a Generalized Procrustes Analysis (GPA) with Varimax on the mean of the matched loading matrices performs very well on all criteria, and, for most purposes, offers an attractive compromise of rotation to agreement and simple structure. In addition to this comparison, some technical improvements are proposed for Bloxom's rotation to simple structure and maximum similarity.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the author. The author is obliged to René van der Heijden for assistance in programming the procedures in the simulation study reported in this paper, and to Jos ten Berge, three anonymous reviewers and an associate editor for helpful comments on an earlier version of this paper.     

A Structural Equation Analysis of the Wide Range Assessment of Memory and Learning in a Clinical Sample

A maximum likelihood confirmatory factor analysis was performed by applying LISREL VII to the Wide Range Assessment of Memory and Learning (WRAML) in a clinical sample (N = 271). Analyses were designed to determine which of 10 hypothesized oblique factor solutions could best explain memory as measured by the WRAML. Competing latent variable models were identified in previous studies and monographs on memory. Findings supported a three-factor model including Verbal Memory, Visual Memory, and Attention/Concentration factors. Our results are consistent with previous studies suggesting that attention is an important component of memory as measured by the WRAML. This study replicates previous research in which the Learning Index was not empirically supported.     

A note on identification in the oblique and orthogonal factor analysis models

Conditions for removing the indeterminancy due to rotation are given for both the oblique and orthogonal factor analysis models. The conditions indicate why published counterexamples to conditions discussed by Jöreskog are not identifiable.The author would like to thank Gordon Bechtel and the reviewers for their comments and suggestions.