Conditions for factor (in)determinacy in factor analysis

The subject of factor indeterminacy has a vast history in factor analysis (Guttman, 1955; Lederman, 1938; Wilson, 1928). It has lead to strong differences in opinion (Steiger, 1979). The current paper gives necessary and sufficient conditions for observability of factors in terms of the parameter matrices and a finite number of variables. Five conditions are given which rigorously define indeterminacy. It is shown that (un)observable factors are (in)determinate. Specifically, the indeterminacy proof by Guttman is extended to Heywood cases. The results are illustrated by two examples and implications for indeterminacy are discussed.     

Bayesian estimation in unrestricted factor analysis: A treatment for heywood cases

A Bayesian procedure is given for estimation in unrestricted common factor analysis. A choice of the form of the prior distribution is justified. It is shown empirically that the procedure achieves its objective of avoiding inadmissible estimates of unique variances, and is reasonably insensitive to certain variations in the shape of the prior distribution.     

On various causes of improper solutions in maximum likelihood factor analysis

In the applications of maximum likelihood factor analysis the occurrence of boundary minima instead of proper minima is no exception at all. In the past the causes of such improper solutions could not be detected. This was impossible because the matrices containing the parameters of the factor analysis model were kept positive definite. By dropping these constraints, it becomes possible to distinguish between the different causes of improper solutions. In this paper some of the most important causes are discussed and illustrated by means of artificial and empirical data.The author is indebted to H. J. Prins for stimulating and encouraging discussions.     

Statistical inference of minimum rank factor analysis

For any given number of factors, Minimum Rank Factor Analysis yields optimal communalities for an observed covariance matrix in the sense that the unexplained common variance with that number of factors is minimized, subject to the constraint that both the diagonal matrix of unique variances and the observed covariance matrix minus that diagonal matrix are positive semidefinite. As a result, it becomes possible to distinguish the explained common variance from the total common variance. The percentage of explained common variance is similar in meaning to the percentage of explained observed variance in Principal Component Analysis, but typically the former is much closer to 100 than the latter. So far, no statistical theory of MRFA has been developed. The present paper is a first start. It yields closed-form expressions for the asymptotic bias of the explained common variance, or, more precisely, of the unexplained common variance, under the assumption of multivariate normality. Also, the asymptotic variance of this bias is derived, and also the asymptotic covariance matrix of the unique variances that define a MRFA solution. The presented asymptotic statistical inference is based on a recently developed perturbation theory of semidefinite programming. A numerical example is also offered to demonstrate the accuracy of the expressions.This work was supported, in part, by grant DMS-0073770 from the National Science Foundation.     

Bayesian estimation in the two-parameter logistic model

A Bayesian procedure is developed for the estimation of parameters in the two-parameter logistic item response model. Joint modal estimates of the parameters are obtained and procedures for the specification of prior information are described. Through simulation studies it is shown that Bayesian estimates of the parameters are superior to maximum likelihood estimates in the sense that they are (a) more meaningful since they do not drift out of range, and (b) more accurate in that they result in smaller mean squared differences between estimates and true values.The research reported here was performed pursuant to Grant No. N0014-79-C-0039 with the Office of Naval Research.     

On the relations among regular, equal unique variances, and image factor analysis models

We investigate under what conditions the matrix of factor loadings from the factor analysis model with equal unique variances will give a good approximation to the matrix of factor loadings from the regular factor analysis model. We show that the two models will give similar matrices of factor loadings if Schneeweiss' condition, that the difference between the largest and the smallest value of unique variances is small relative to the sizes of the column sums of squared factor loadings, holds. Furthermore, we generalize our results and discus the conditions under which the matrix of factor loadings from the regular factor analysis model will be well approximated by the matrix of factor loadings from Jöreskog's image factor analysis model. Especially, we discuss Guttman's condition (i.e., the number of variables increases without limit) for the two models to agree, in relation with the condition we have shown, and conclude that Schneeweiss' condition is a generalization of Guttman's condition. Some implications for practice are discussed.Kentaro Hayashi is a visiting Assistant Professor, Department of Mathematics, Bucknell University, Lewisburg PA 17837, and Peter M. Bentler is Professor, Departments of Psychology and Statistics, University of California, Los Angeles CA 90095-1563. (Emails: Khayashi@bucknell.edu, bentler@ucla.edu) Parts of this paper were discussed in a session on Factor Analysis (J. ten Berge, Chair) at the IFCS-98 International Conference, Rome, July, 1998. This work was supported by National Institute on Drug Abuse grant DA 01070. The authors thank Professors Hans Schneeweiss and Ke-Hai Yuan, and four anonymous referees, for their invaluable comments which led to an improved version of this paper.     

An improvement of the Harman-Fukuda-method for the minres solution in factor analysis

The method proposed by Harman and Fukuda to treat the so-called Heywood case in the minres method in factor analysis i.e., the case where the resulting communalities are greater than one, involves the frequent solution of eigenvalue problems. A simple method to treat this problem requiring less computing time and enjoying higher numerical stability is described in this paper.     

On the estimation of parameters in latent structure analysis

In this note, we describe the iterative procedure introduced earlier by Goodman to calculate the maximum likelihood estimates of the parameters in latent structure analysis, and we provide here a simple and direct proof of the fact that the parameter estimates obtained with the iterative procedure cannot lie outside the allowed interval. Formann recently stated that Goodman's algorithm can yield parameter estimates that lie outside the allowed interval, and we prove in the present note that Formann's contention is incorrect.This research was supported in part by Research Contract No. NSF SOC 76-80389 from the Division of the Social Sciences of the National Science Foundation. The author is indebted to C. C. Clogg for helpful comments and for the numerical results reported here (see, e.g., Table 1).     

Bayesian factor analysis for multilevel binary observations

Multilevel covariance structure models have become increasingly popular in the psychometric literature in the past few years to account for population heterogeneity and complex study designs. We develop practical simulation based procedures for Bayesian inference of multilevel binary factor analysis models. We illustrate how Markov Chain Monte Carlo procedures such as Gibbs sampling and Metropolis-Hastings methods can be used to perform Bayesian inference, model checking and model comparison without the need for multidimensional numerical integration. We illustrate the proposed estimation methods using three simulation studies and an application involving student's achievement results in different areas of mathematics. The authors thank Ian Westbury, University of Illinois at Urbana Champaign for kindly providing the SIMS data for the application.     

The role of collateral information about examinees in item parameter estimation

Standard procedures for estimating item parameters in item response theory (IRT) ignore collateral information that may be available about examinees, such as their standing on demographic and educational variables. This paper describes circumstances under which collateral information about examineesmay be used to make inferences about item parameters more precise, and circumstances under which itmust be used to obtain correct inferences.This work was supported by Contract No. N00014-85-K-0683, project designation NR 150-539, from the Cognitive Science Program, Cognitive and Neural Sciences Division, Office of Naval Research. Reproduction in whole or in part is permitted for any purpose of the United States Government. We are indebted to Tim Davey, Eugene Johnson, and three anonymous referees for their comments on earlier versions of the paper.     

A unified approach to exploratory factor analysis with missing data,nonnormal data,and in the presence of outliers

Factor analysis is regularly used for analyzing survey data. Missing data, data with outliers and consequently nonnormal data are very common for data obtained through questionnaires. Based on covariance matrix estimates for such nonstandard samples, a unified approach for factor analysis is developed. By generalizing the approach of maximum likelihood under constraints, statistical properties of the estimates for factor loadings and error variances are obtained. A rescaled Bartlett-corrected statistic is proposed for evaluating the number of factors. Equivariance and invariance of parameter estimates and their standard errors for canonical, varimax, and normalized varimax rotations are discussed. Numerical results illustrate the sensitivity of classical methods and advantages of the proposed procedures.This project was supported by a University of North Texas Faculty Research Grant, Grant #R49/CCR610528 for Disease Control and Prevention from the National Center for Injury Prevention and Control, and Grant DA01070 from the National Institute on Drug Abuse. The results do not necessarily represent the official view of the funding agencies. The authors are grateful to three reviewers for suggestions that improved the presentation of this paper.     

Regression estimation and post-stratification in factor analysis

Regression estimation and poststratification are methods used in survey sampling to estimate a population mean, when additional information is available for some auxiliary variables. The extension of these methods to factor analysis is considered. These methods may be used either to improve the efficiency of estimation or to adjust for the effects of nonrandom selection. The estimation procedure may be formulated in a LISREL framework.The author is grateful to the referees for their comments.     

A feasible method for standard errors of estimate in maximum likelihood factor analysis

A jackknife-like procedure is developed for producing standard errors of estimate in maximum likelihood factor analysis. Unlike earlier methods based on information theory, the procedure developed is computationally feasible on larger problems. Unlike earlier methods based on the jackknife, the present procedure is not plagued by the factor alignment problem, the Heywood case problem, or the necessity to jackknife by groups. Standard errors may be produced for rotated and unrotated loading estimates using either orthogonal or oblique rotation as well as for estimates of unique factor variances and common factor correlations. The total cost for larger problems is a small multiple of the square of the number of variables times the number of observations used in the analysis. Examples are given to demonstrate the feasibility of the method.The research done by R. I. Jennrich was supported in part by NSF Grant MCS 77-02121. The research done by D. B. Clarkson was supported in part by NSERC Grant A3109.     

Scale freeness in factor analysis

The notion of scale freeness does not seem to have been well understood in the factor analytic literature. It has been believed that if the loss function that is minimized to obtain estimates of the parameters in the factor model is scale invariant, then the estimates are scale free. It is shown that scale invariance of the loss function is neither a necessary nor a sufficient condition for scale freeness. A theorem that ensures scale freeness in the orthogonal factor model is given in this paper.The authors are grateful for the suggestions of the referees.     

On the relationship between item response theory and factor analysis of discretized variables

Equivalence of marginal likelihood of the two-parameter normal ogive model in item response theory (IRT) and factor analysis of dichotomized variables (FA) was formally proved. The basic result on the dichotomous variables was extended to multicategory cases, both ordered and unordered categorical data. Pair comparison data arising from multiple-judgment sampling were discussed as a special case of the unordered categorical data. A taxonomy of data for the IRT and FA models was also attempted.The work reported in this paper has been supported by Grant A6394 to the first author from the Natural Sciences and Engineering Research Council of Canada.     

The choice of constraints in correspondence analysis

A discussion of alternative constraint systems has been lacking in the literature on correspondence analysis and related techniques. This paper reiterates earlier results that an explicit choice of constraints has to be made which can have important effects on the resulting scores. The paper also presents new results on dealing with missing data and probabilistic category assignment.I am most grateful to the following for their helpful comments. Arto Demirjian, Michael Greenacre, Michael Healy, Shizuhiko Nishisato, Roderick Mcdonald, and several anonymous referees.     

A bayesian estimation of factor score in confirmatory factor model with polytomous, censored or truncated data

The posterior analysis in estimating factor score in a confirmatory factor analysis model with polytomous, censored or truncated data is investigated in this paper. For the above three types of data, posterior distributions of the factor score are studied, and the estimators of the factor score are obtained to be the location parameters of the posterior distributions. The accuracy of Bayesian estimates is studied via simulation studies.This research was supported by a Hong Kong UGC grant.     

Noniterative estimation and the choice of the number of factors in exploratory factor analysis

Based on the usual factor analysis model, this paper investigates the relationship between improper solutions and the number of factors, and discusses the properties of the noniterative estimation method of Ihara and Kano in exploratory factor analysis. The consistency of the Ihara and Kano estimator is shown to hold even for an overestimated number of factors, which provides a theoretical basis for the rare occurrence of improper solutions and for a new method of choosing the number of factors. The comparative study of their estimator and that based on maximum likelihood is carried out by a Monte Carlo experiment.The author would like to express his thanks to Masashi Okamoto and Masamori Ihara for helpful comments and to the editor and referees for critically reading the earlier versions and making many valuable suggestions. He also thanks Shigeo Aki for his comments on physical random numbers.     

Asymptotic biases in exploratory factor analysis and structural equation modeling

Formulas for the asymptotic biases of the parameter estimates in structural equation models are provided in the case of the Wishart maximum likelihood estimation for normally and nonnormally distributed variables. When multivariate normality is satisfied, considerable simplification is obtained for the models of unstandardized variables. Formulas for the models of standardized variables are also provided. Numerical examples with Monte Carlo simulations in factor analysis show the accuracy of the formulas and suggest the asymptotic robustness of the asymptotic biases with normality assumption against nonnormal data. Some relationships between the asymptotic biases and other asymptotic values are discussed.The author is indebted to the editor and anonymous reviewers for their comments, corrections, and suggestions on this paper, and to Yutaka Kano for discussion on biases.