A general approach to confirmatory maximum likelihood factor analysis

We describe a general procedure by which any number of parameters of the factor analytic model can be held fixed at any values and the remaining free parameters estimated by the maximum likelihood method. The generality of the approach makes it possible to deal with all kinds of solutions: orthogonal, oblique and various mixtures of these. By choosing the fixed parameters appropriately, factors can be defined to have desired properties and make subsequent rotation unnecessary. The goodness of fit of the maximum likelihood solution under the hypothesis represented by the fixed parameters is tested by a large samplex ² test based on the likelihood ratio technique. A by-product of the procedure is an estimate of the variance-covariance matrix of the estimated parameters. From this, approximate confidence intervals for the parameters can be obtained. Several examples illustrating the usefulness of the procedure are given.This work was supported by a grant (NSF-GB 1985) from the National Science Foundation to Educational Testing Service.     

A reliability coefficient for maximum likelihood factor analysis

Maximum likelihood factor analysis provides an effective method for estimation of factor matrices and a useful test statistic in the likelihood ratio for rejection of overly simple factor models. A reliability coefficient is proposed to indicate quality of representation of interrelations among attributes in a battery by a maximum likelihood factor analysis. Usually, for a large sample of individuals or objects, the likelihood ratio statistic could indicate that an otherwise acceptable factor model does not exactly represent the interrelations among the attributes for a population. The reliability coefficient could indicate a very close representation in this case and be a better indication as to whether to accept or reject the factor solution. This research was supported by the Personnel and Training Research Programs Office of the Office of Naval Research under contract US NAVY/00014-67-A-0305-0003. Critical review of the development and suggestions by Richard Montanelli were most helpful.     

A Newton-Raphson algorithm for maximum likelihood factor analysis

This paper demonstrates the feasibility of using a Newton-Raphson algorithm to solve the likelihood equations which arise in maximum likelihood factor analysis. The algorithm leads to clean easily identifiable convergence and provides a means of verifying that the solution obtained is at least a local maximum of the likelihood function. It is shown that a popular iteration algorithm is numerically unstable under conditions which are encountered in practice and that, as a result, inaccurate solutions have been presented in the literature. The key result is a computationally feasible formula for the second differential of a partially maximized form of the likelihood function. In addition to implementing the Newton-Raphson algorithm, this formula provides a means for estimating the asymptotic variances and covariances of the maximum likelihood estimators. This research was supported by the Air Force Office of Scientific Research, Grant No. AF-AFOSR-4.59-66 and by National Institutes of Health, Grant No. FR-3.     

Properties of the maximum likelihood solution in factor analysis regression

Algebraic properties of the normal theory maximum likelihood solution in factor analysis regression are investigated. Two commonly employed measures of the within sample predictive accuracy of the factor analysis regression function are considered: the variance of the regression residuals and the squared correlation coefficient between the criterion variable and the regression function. It is shown that this within sample residual variance and within sample squared correlation may be obtained directly from the factor loading and unique variance estimates, without use of the original observations or the sample covariance matrix.     

An SAS/IML procedure for maximum likelihood factor analysis

Maximum likelihood factor analysis (MLFA), originally introduced by Lawley (1940), is based on a firm mathematical foundation that allows hypothesis testing when normality is assumed with large sample sizes. MLFA has gained in popularity since Jöreskog (1967) implemented an iterative algorithm to estimate parameters. This article presents a concise program using matrix language SAS/LML with the optimization subroutine NLPQN to obtain MLFA solutions. The program is pedagogically useful because it shows the step-by-step computational processes for MLFA, whereas almost all other statistical packages for MLFA are in “black boxes.” It is also demonstrated that this approach can be extended to other multivariate methods requiring numerical optimizations, such as the widely used structural equation modeling. Researchers may find this program useful in conducting Monte Carlo simulation studies to investigate the properties of multivariate methods that involve numerical optimizations.     

High-dimensional maximum marginal likelihood item factor analysis by adaptive quadrature

Although the Bock–Aitkin likelihood-based estimation method for factor analysis of dichotomous item response data has important advantages over classical analysis of item tetrachoric correlations, a serious limitation of the method is its reliance on fixed-point Gauss-Hermite (G-H) quadrature in the solution of the likelihood equations and likelihood-ratio tests. When the number of latent dimensions is large, computational considerations require that the number of quadrature points per dimension be few. But with large numbers of items, the dispersion of the likelihood, given the response pattern, becomes so small that the likelihood cannot be accurately evaluated with the sparse fixed points in the latent space. In this paper, we demonstrate that substantial improvement in accuracy can be obtained by adapting the quadrature points to the location and dispersion of the likelihood surfaces corresponding to each distinct pattern in the data. In particular, we show that adaptive G-H quadrature, combined with mean and covariance adjustments at each iteration of an EM algorithm, produces an accurate fast-converging solution with as few as two points per dimension. Evaluations of this method with simulated data are shown to yield accurate recovery of the generating factor loadings for models of upto eight dimensions. Unlike an earlier application of adaptive Gibbs sampling to this problem by Meng and Schilling, the simulations also confirm the validity of the present method in calculating likelihood-ratio chi-square statistics for determining the number of factors required in the model. Finally, we apply the method to a sample of real data from a test of teacher qualifications.     

On various causes of improper solutions in maximum likelihood factor analysis

In the applications of maximum likelihood factor analysis the occurrence of boundary minima instead of proper minima is no exception at all. In the past the causes of such improper solutions could not be detected. This was impossible because the matrices containing the parameters of the factor analysis model were kept positive definite. By dropping these constraints, it becomes possible to distinguish between the different causes of improper solutions. In this paper some of the most important causes are discussed and illustrated by means of artificial and empirical data.The author is indebted to H. J. Prins for stimulating and encouraging discussions.     

A note on Lawley's formulas for standard errors in maximum likelihood factor analysis

Evidence is given to indicate that Lawley's formulas for the standard errors of maximum likelihood loading estimates do not produce exact asymptotic results. A small modification is derived which appears to eliminate this difficulty.The authors are indebted to Walter Kristof and Thomas Stroud for their helpful reviews of an earlier version of this paper and particularly to D. N. Lawley for his review, comments, and encouragement.     

A feasible method for standard errors of estimate in maximum likelihood factor analysis

A jackknife-like procedure is developed for producing standard errors of estimate in maximum likelihood factor analysis. Unlike earlier methods based on information theory, the procedure developed is computationally feasible on larger problems. Unlike earlier methods based on the jackknife, the present procedure is not plagued by the factor alignment problem, the Heywood case problem, or the necessity to jackknife by groups. Standard errors may be produced for rotated and unrotated loading estimates using either orthogonal or oblique rotation as well as for estimates of unique factor variances and common factor correlations. The total cost for larger problems is a small multiple of the square of the number of variables times the number of observations used in the analysis. Examples are given to demonstrate the feasibility of the method.The research done by R. I. Jennrich was supported in part by NSF Grant MCS 77-02121. The research done by D. B. Clarkson was supported in part by NSERC Grant A3109.     

Some small sample results for maximum likelihood estimation in multidimensional scaling

Some aspects of the small sample behavior of maximum likelihood estimates in multidimensional scaling are investigated by Monte Carlo. An investigation of Model M2 in the MULTISCALE program package shows that the chi-square test of dimensionality requires a correction of tabled chi-square values to be unbiased. A formula for this correction in the case of two dimensions is estimated. The power of the test of dimensionality is acceptable with as few as two replications for 15 stimuli and as few as five replications for 10 stimuli. The biases in the exponent and standard error estimates in this model are also investigated.The research reported here was supported by grant number APA 320 to the author by the National Science and Engineering Research Council of Canada.     

Heteroscedastic one‐factor models and marginal maximum likelihood estimation

In the present paper, a general class of heteroscedastic one‐factor models is considered. In these models, the residual variances of the observed scores are explicitly modelled as parametric functions of the one‐dimensional factor score. A marginal maximum likelihood procedure for parameter estimation is proposed under both the assumption of multivariate normality of the observed scores conditional on the single common factor score and the assumption of normality of the common factor score. A likelihood ratio test is derived, which can be used to test the usual homoscedastic one‐factor model against one of the proposed heteroscedastic models. Simulation studies are carried out to investigate the robustness and the power of this likelihood ratio test. Results show that the asymptotic properties of the test statistic hold under both small test length conditions and small sample size conditions. Results also show under what conditions the power to detect different heteroscedasticity parameter values is either small, medium, or large. Finally, for illustrative purposes, the marginal maximum likelihood estimation procedure and the likelihood ratio test are applied to real data.     

Maximum likelihood estimation and factor analysis

Fisher's method of maximum likelihood is applied to the problem of estimation in factor analysis, as initiated by Lawley, and found to lead to a generalization of the Eckart matrix approximation problem. The solution of this in a special case is applied to show how test fallability enters into factor determination, it being noted that the method of communalities underestimates the number of factors.Dr. George Brown of Princeton University has independently made the same suggestion in some unpublished work.     

Maximum likelihood solution to factor analysis when some factors are completely specified

In the factor-analytic model, let some of the factors be known (i.e., the factor loadings are given in advance; they maye.g. be obtained from some previous analyses). However, their covariance matrix may, or may not, be known. The remaining factors (if any) are assumed to be uncorrelated among themselves and to the first set. For this model, the maximum likelihood equations are obtained and an iterative method for the solution is proposed.The work was done while the author was at Deutsches Rechenzentrum, Darmstadt, Germany.     

Assessing local influence for specific restricted likelihood: Application to factor analysis

In restricted statistical models, since the first derivatives of the likelihood displacement are often nonzero, the commonly adopted formulation for local influence analysis is not appropriate. However, there are two kinds of model restrictions in which the first derivatives of the likelihood displacement are still zero. General formulas for assessing local influence under these restrictions are derived and applied to factor analysis as the usually used restriction in factor analysis satisfies the conditions. Various influence schemes are introduced and a comparison to the influence function approach is discussed. It is also shown that local influence for factor analysis is invariant to the scale of the data and is independent of the rotation of the factor loadings. The authors are most grateful to the referees, the Associate Editor, and the Editor for helpful suggestions for improving the clarity of the paper.     

The effect of sampling error on convergence,improper solutions,and goodness-of-fit indices for maximum likelihood confirmatory factor analysis

A Monte Carlo study assessed the effect of sampling error and model characteristics on the occurrence of nonconvergent solutions, improper solutions and the distribution of goodness-of-fit indices in maximum likelihood confirmatory factor analysis. Nonconvergent and improper solutions occurred more frequently for smaller sample sizes and for models with fewer indicators of each factor. Effects of practical significance due to sample size, the number of indicators per factor and the number of factors were found for GFI, AGFI, and RMR, whereas no practical effects were found for the probability values associated with the chi-square likelihood ratio test.James Anderson is now at the J. L. Kellogg Graduate School of Management, Northwestern University. The authors gratefully acknowledge the comments and suggestions of Kenneth Land and the reviewers, and the assistance of A. Narayanan with the analysis. Support for this research was provided by the Graduate School of Business and the University Research Institute of the University of Texas at Austin.