Estimating the standard errors of rotated factor loadings by jackknifing

The jackknife by groups and modifications of the jackknife by groups are used to estimate standard errors of rotated factor loadings for selected populations in common factor model maximum likelihood factor analysis. Simulations are performed in whicht-statistics based upon these jackknife estimates of the standard errors are computed. The validity of thet-statistics and their associated confidence intervals is assessed. Methods are given through which the computational efficiency of the jackknife may be greatly enhanced in the factor analysis model.Computing assistance was obtained from the Health Sciences Computing Facility, UCLA, sponsored by NIH Special Research Resources Grant RR-3.The author wishes to thank his doctoral committee co-chairmen, Drs James W. Frane and Robert I. Jennrich, UCLA, for their contributions to this research.     

Properties of the maximum likelihood solution in factor analysis regression

Algebraic properties of the normal theory maximum likelihood solution in factor analysis regression are investigated. Two commonly employed measures of the within sample predictive accuracy of the factor analysis regression function are considered: the variance of the regression residuals and the squared correlation coefficient between the criterion variable and the regression function. It is shown that this within sample residual variance and within sample squared correlation may be obtained directly from the factor loading and unique variance estimates, without use of the original observations or the sample covariance matrix.     

EM algorithms for ML factor analysis

The details of EM algorithms for maximum likelihood factor analysis are presented for both the exploratory and confirmatory models. The algorithm is essentially the same for both cases and involves only simple least squares regression operations; the largest matrix inversion required is for aq ×q symmetric matrix whereq is the matrix of factors. The example that is used demonstrates that the likelihood for the factor analysis model may have multiple modes that are not simply rotations of each other; such behavior should concern users of maximum likelihood factor analysis and certainly should cast doubt on the general utility of second derivatives of the log likelihood as measures of precision of estimation.     

A note on Lawley's formulas for standard errors in maximum likelihood factor analysis

Evidence is given to indicate that Lawley's formulas for the standard errors of maximum likelihood loading estimates do not produce exact asymptotic results. A small modification is derived which appears to eliminate this difficulty.The authors are indebted to Walter Kristof and Thomas Stroud for their helpful reviews of an earlier version of this paper and particularly to D. N. Lawley for his review, comments, and encouragement.     

On various causes of improper solutions in maximum likelihood factor analysis

In the applications of maximum likelihood factor analysis the occurrence of boundary minima instead of proper minima is no exception at all. In the past the causes of such improper solutions could not be detected. This was impossible because the matrices containing the parameters of the factor analysis model were kept positive definite. By dropping these constraints, it becomes possible to distinguish between the different causes of improper solutions. In this paper some of the most important causes are discussed and illustrated by means of artificial and empirical data.The author is indebted to H. J. Prins for stimulating and encouraging discussions.     

A loss function for alpha factor analysis

Most of the factor solutions can be got by minimizing a corresponding loss function. However, up to now, a loss function for the alpha factor analysis (AFA) has not been known. The present paper establishes such a loss function for the AFA. Some analogies to the maximum likelihood factor analysis are discussed.The author is greatly indebted to Prof. Henry F. Kaiser (University of California, Berkeley) for his kind encouragement. He is also indebted to an anonymous referee ofPsychometrika for having confronted him with the problem in 1977. Financial support by the Wiener Hochschuljubiläumsstiftung is gratefully acknowledged.     

A maximum likelihood method for latent class regression involving a censored dependent variable

The standard tobit or censored regression model is typically utilized for regression analysis when the dependent variable is censored. This model is generalized by developing a conditional mixture, maximum likelihood method for latent class censored regression. The proposed method simultaneously estimates separate regression functions and subject membership in K latent classes or groups given a censored dependent variable for a cross-section of subjects. Maximum likelihood estimates are obtained using an EM algorithm. The proposed method is illustrated via a consumer psychology application.     

More on EM for ML factor analysis

We address several issues that are raised by Bentler and Tanaka's [1983] discussion of Rubin and Thayer [1982]. Our conclusions are: standard methods do not completely monitor the possible existence of multiple local maxima; summarizing inferential precision by the standard output based on second derivatives of the log likelihood at a maximum can be inappropriate, even if there exists a unique local maximum; EM and LISREL can be viewed as complementary, albeit not entirely adequate, tools for factor analysis.This work was partially supported by the Program Statistics Research Project at Educational Testing Service.     

Standard errors for the class of orthomax-rotated factor loadings: Some matrix results

The partial derivative matrices of the class of orthomax-rotated factor loadings with respect to the unrotated maximum likelihood factor loadings are derived. The reported results are useful for obtaining standard errors of the orthomax-rotated factor loadings, with or without row normalization (standardization) of the initial factor loading matrix for rotation. Using a numerical example, we verify our analytic formulas by comparing the obtained standard error estimates with that from some existing methods. Some advantages of the current approach are discussed.Authorship is determined by alphabetical order. The authors contributed equally to the research. Kentaro Hayashi is now at the Department of Mathematics, Bucknell University, Lewisburg, PA 17837 (email: khayashi@Bucknell.edu). Yiu-Fai Yung is now at the SAS Institute, Inc., SAS Campus Drive, Cary, NC 27513 (email: yiyung@wnt.sas.com).Part of the research was completed while Yiu-Fai Yung was a visiting scholar at the Department of Psychology, the Ohio State University. The visit was supported in part by grant N4856118101 from the NIMH and the Mason and Linda Stephenson Travel Award from the Department of Psychology, University of North Carolina at Chapel Hill. The authors are grateful to Michael Browne who suggested some relevant references and provided valuable comments on the research, and to Robert Cudeck who provided the FAS program for the numerical comparison. The expert comments by the reviewers are deeply appreciated.     

Marginal maximum likelihood estimation of item response theory (IRT) equating coefficients for the common-examinee design

A method of estimating item response theory (IRT) equating coefficients by the common-examinee design with the assumption of the two-parameter logistic model is provided. The method uses the marginal maximum likelihood estimation, in which individual ability parameters in a common-examinee group are numerically integrated out. The abilities of the common examinees are assumed to follow a normal distribution but with an unknown mean and standard deviation on one of the two tests to be equated. The distribution parameters are jointly estimated with the equating coefficients. Further, the asymptotic standard errors of the estimates of the equating coefficients and the parameters for the ability distribution are given. Numerical examples are provided to show the accuracy of the method.     

On equivariance and invariance of standard errors in three exploratory factor models

Current practice in factor analysis typically involves analysis of correlation rather than covariance matrices. We study whether the standardz-statistic that evaluates whether a factor loading is statistically necessary is correctly applied in such situations and more generally when the variables being analyzed are arbitrarily rescaled. Effects of rescaling on estimated standard errors of factor loading estimates, and the consequent effect onz-statistics, are studied in three variants of the classical exploratory factor model under canonical, raw varimax, and normal varimax solutions. For models with analytical solutions we find that some of the standard errors as well as their estimates are scale equivariant, while others are invariant. For a model in which an analytical solution does not exist, we use an example to illustrate that neither the factor loading estimates nor the standard error estimates possess scale equivariance or invariance, implying that different conclusions could be obtained with different scalings. Together with the prior findings on parameter estimates, these results provide new guidance for a key statistical aspect of factor analysis.We gratefully acknowledge the help of the Associate Editor and three referees whose constructive comments lead to an improved version of the paper. This work was supported by National Institute on Drug Abuse Grants DA01070 and DA00017 and by the University of North Texas Faculty Research Grant Program.     

A new estimator of the uniqueness in factor analysis

A closed form estimator of the uniqueness (unique variance) in factor analysis is proposed. It has analytically desirable properties—consistency, asymptotic normality and scale invariance. The estimation procedure is given through the application to the two sets of Emmett's data and Holzinger and Swineford's data. The new estimator is shown to lead to values rather close to the maximum likelihood estimator.     

The effect of sampling error on convergence,improper solutions,and goodness-of-fit indices for maximum likelihood confirmatory factor analysis

A Monte Carlo study assessed the effect of sampling error and model characteristics on the occurrence of nonconvergent solutions, improper solutions and the distribution of goodness-of-fit indices in maximum likelihood confirmatory factor analysis. Nonconvergent and improper solutions occurred more frequently for smaller sample sizes and for models with fewer indicators of each factor. Effects of practical significance due to sample size, the number of indicators per factor and the number of factors were found for GFI, AGFI, and RMR, whereas no practical effects were found for the probability values associated with the chi-square likelihood ratio test.James Anderson is now at the J. L. Kellogg Graduate School of Management, Northwestern University. The authors gratefully acknowledge the comments and suggestions of Kenneth Land and the reviewers, and the assistance of A. Narayanan with the analysis. Support for this research was provided by the Graduate School of Business and the University Research Institute of the University of Texas at Austin.     

A penalized likelihood method for multi-group structural equation modelling

In the past two decades, statistical modelling with sparsity has become an active research topic in the fields of statistics and machine learning. Recently, Huang, Chen and Weng (2017, Psychometrika, 82, 329) and Jacobucci, Grimm, and McArdle (2016, Structural Equation Modeling: A Multidisciplinary Journal, 23, 555) both proposed sparse estimation methods for structural equation modelling (SEM). These methods, however, are restricted to performing single-group analysis. The aim of the present work is to establish a penalized likelihood (PL) method for multi-group SEM. Our proposed method decomposes each group model parameter into a common reference component and a group-specific increment component. By penalizing the increment components, the heterogeneity of parameter values across the population can be explored since the null group-specific effects are expected to diminish. We developed an expectation-conditional maximization algorithm to optimize the PL criteria. A numerical experiment and a real data example are presented to demonstrate the potential utility of the proposed method.     

Application of the bootstrap methods in factor analysis

A Monte Carlo experiment is conducted to investigate the performance of the bootstrap methods in normal theory maximum likelihood factor analysis both when the distributional assumption is satisfied and unsatisfied. The parameters and their functions of interest include unrotated loadings, analytically rotated loadings, and unique variances. The results reveal that (a) bootstrap bias estimation performs sometimes poorly for factor loadings and nonstandardized unique variances; (b) bootstrap variance estimation performs well even when the distributional assumption is violated; (c) bootstrap confidence intervals based on the Studentized statistics are recommended; (d) if structural hypothesis about the population covariance matrix is taken into account then the bootstrap distribution of the normal theory likelihood ratio test statistic is close to the corresponding sampling distribution with slightly heavier right tail.This study was carried out in part under the ISM cooperative research program (91-ISM · CRP-85, 92-ISM · CRP-102). The authors would like to thank the editor and three reviewers for their helpful comments and suggestions which improved the quality of this paper considerably.     

Asymptotic distributions of the estimators of communalities in factor analysis

Asymptotic distributions of the estimators of communalities are derived for the maximum likelihood method in factor analysis. It is shown that the common practice of equating the asymptotic standard error of the communality estimate to the unique variance estimate is correct for standardized communality but not correct for unstandardized communality. In a Monte Carlo simulation the accuracy of the normal approximation to the distributions of the estimators are assessed when the sample size is 150 or 300. This study was carried out in part under the ISM Cooperative Research Program (90-ISM-CRP-9).     

A study of algorithms for covariance structure analysis with specific comparisons using factor analysis

Several algorithms for covariance structure analysis are considered in addition to the Fletcher-Powell algorithm. These include the Gauss-Newton, Newton-Raphson, Fisher Scoring, and Fletcher-Reeves algorithms. Two methods of estimation are considered, maximum likelihood and weighted least squares. It is shown that the Gauss-Newton algorithm which in standard form produces weighted least squares estimates can, in iteratively reweighted form, produce maximum likelihood estimates as well. Previously unavailable standard error estimates to be used in conjunction with the Fletcher-Reeves algorithm are derived. Finally all the algorithms are applied to a number of maximum likelihood and weighted least squares factor analysis problems to compare the estimates and the standard errors produced. The algorithms appear to give satisfactory estimates but there are serious discrepancies in the standard errors. Because it is robust to poor starting values, converges rapidly and conveniently produces consistent standard errors for both maximum likelihood and weighted least squares problems, the Gauss-Newton algorithm represents an attractive alternative for at least some covariance structure analyses.Work by the first author has been supported in part by Grant No. Da01070 from the U. S. Public Health Service. Work by the second author has been supported in part by Grant No. MCS 77-02121 from the National Science Foundation.     

A comparison of factor analysis programs in SPSS,BMDP, and SAS

Factor analysis programs in SAS, BMDP, and SPSS are discussed and compared in terms of documentation, methods and options available, internal logic, computational accuracy, and results provided. Some problems with respect to logic and output are described. Based on these comparisons, recommendations are offered which include a clear overall preference for SAS, and advice against general use of SPSS for factor analysis.     

Noniterative estimation and the choice of the number of factors in exploratory factor analysis

Based on the usual factor analysis model, this paper investigates the relationship between improper solutions and the number of factors, and discusses the properties of the noniterative estimation method of Ihara and Kano in exploratory factor analysis. The consistency of the Ihara and Kano estimator is shown to hold even for an overestimated number of factors, which provides a theoretical basis for the rare occurrence of improper solutions and for a new method of choosing the number of factors. The comparative study of their estimator and that based on maximum likelihood is carried out by a Monte Carlo experiment.The author would like to express his thanks to Masashi Okamoto and Masamori Ihara for helpful comments and to the editor and referees for critically reading the earlier versions and making many valuable suggestions. He also thanks Shigeo Aki for his comments on physical random numbers.