Estimating Standard Errors in Exploratory Factor Analysis

The article describes 6 issues influencing standard errors in exploratory factor analysis and reviews 7 methods of computing standard errors for rotated factor loadings and factor correlations. These 7 methods are the augmented information method, the nonparametric bootstrap method, the infinitesimal jackknife method, the method using the asymptotic distributions of unrotated factor loadings, the sandwich method, the parametric bootstrap method, and the jackknife method. Standard error estimates are illustrated using a personality study with 537 men and an intelligence study with 145 children.     

平衡秤任务复杂性的事前与事后分析

任务复杂性的分析和评估是心理测量学和认知心理学都非常关注的重要主题。以264名小学四、五、六年级儿童为被试,平衡秤任务为研究材料,考察任务在未旋转时第一个因素上的载荷(事后分析)能否作为评价任务复杂性的一个指标,以及关系-表征复杂性模型对平衡秤任务复杂性分析(事前分析)的有效性两个问题。结果表明:平衡秤任务施测后所得所有项目的因素载荷与其难度之间没有显著正相关,即因素载荷的高低没有反映平衡秤任务复杂性的大小;而基于关系-表征复杂性模型对任务的事前分析所确定的任务等级复杂性和知识经验对任务难度的解释率为95.0%,可见,关系-表征复杂性模型提供的分析任务复杂性的思路和方法是较为合理的。     

The Infinitesimal Jackknife with Exploratory Factor Analysis

The infinitesimal jackknife, a nonparametric method for estimating standard errors, has been used to obtain standard error estimates in covariance structure analysis. In this article, we adapt it for obtaining standard errors for rotated factor loadings and factor correlations in exploratory factor analysis with sample correlation matrices. Both maximum likelihood estimation and ordinary least squares estimation are considered.     

Exploratory second-order analyses for components and factors

Abstract: Exploratory methods using second‐order components and second‐order common factors were proposed. The second‐order components were obtained from the resolution of the correlation matrix of obliquely rotated first‐order principal components. The standard errors of the estimates of the second‐order component loadings were derived from an augmented information matrix with restrictions for the loadings and associated parameters. The second‐order factor analysis proposed was similar to the classical method in that the factor correlations among the first‐order factors were further resolved by the exploratory method of factor analysis. However, in this paper the second‐order factor loadings were estimated by the generalized least squares using the asymptotic variance‐covariance matrix for the first‐order factor correlations. The asymptotic standard errors for the estimates of the second‐order factor loadings were also derived. A numerical example was presented with simulated results.     

On the relations among regular, equal unique variances, and image factor analysis models

We investigate under what conditions the matrix of factor loadings from the factor analysis model with equal unique variances will give a good approximation to the matrix of factor loadings from the regular factor analysis model. We show that the two models will give similar matrices of factor loadings if Schneeweiss' condition, that the difference between the largest and the smallest value of unique variances is small relative to the sizes of the column sums of squared factor loadings, holds. Furthermore, we generalize our results and discus the conditions under which the matrix of factor loadings from the regular factor analysis model will be well approximated by the matrix of factor loadings from Jöreskog's image factor analysis model. Especially, we discuss Guttman's condition (i.e., the number of variables increases without limit) for the two models to agree, in relation with the condition we have shown, and conclude that Schneeweiss' condition is a generalization of Guttman's condition. Some implications for practice are discussed.Kentaro Hayashi is a visiting Assistant Professor, Department of Mathematics, Bucknell University, Lewisburg PA 17837, and Peter M. Bentler is Professor, Departments of Psychology and Statistics, University of California, Los Angeles CA 90095-1563. (Emails: Khayashi@bucknell.edu, bentler@ucla.edu) Parts of this paper were discussed in a session on Factor Analysis (J. ten Berge, Chair) at the IFCS-98 International Conference, Rome, July, 1998. This work was supported by National Institute on Drug Abuse grant DA 01070. The authors thank Professors Hans Schneeweiss and Ke-Hai Yuan, and four anonymous referees, for their invaluable comments which led to an improved version of this paper.     

Independent component analysis as a rotation method: A very different solution to Thurstone's box problem

In this paper we consider the well‐known Thurstone box problem in exploratory factor analysis. Initial loadings and components are extracted using principal component analysis. Rotating the components towards independence rather than rotating the loadings towards simplicity allows one to accurately recover the dimensions of each box and also produce simple loadings. It is shown how this may be done using an appropriate rotation criterion and a general rotation algorithm. Methods from independent component analysis are used, and this paper may be viewed as an introduction to independent component analysis from the perspective of factor analysis.     

The relation between factor loadings and biserial correlations in item analysis

The first centroid factor loadings obtained from various interitem relations are compared with item discrimination indices commonly used in item analysis. Depending upon what type of matrix is factored, the factor loadings are shown to be related to point biserial and biserial correlations.     

Exploratory Factor Analysis With Small Samples and Missing Data

Exploratory factor analysis (EFA) is an extremely popular method for determining the underlying factor structure for a set of variables. Due to its exploratory nature, EFA is notorious for being conducted with small sample sizes, and recent reviews of psychological research have reported that between 40% and 60% of applied studies have 200 or fewer observations. Recent methodological studies have addressed small size requirements for EFA models; however, these models have only considered complete data, which are the exception rather than the rule in psychology. Furthermore, the extant literature on missing data techniques with small samples is scant, and nearly all existing studies focus on topics that are not of primary interest to EFA models. Therefore, this article presents a simulation to assess the performance of various missing data techniques for EFA models with both small samples and missing data. Results show that deletion methods do not extract the proper number of factors and estimate the factor loadings with severe bias, even when data are missing completely at random. Predictive mean matching is the best method overall when considering extracting the correct number of factors and estimating factor loadings without bias, although 2-stage estimation was a close second.     

Parceling Cannot Reduce Factor Indeterminacy in Factor Analysis: A Research Note

Parceling—using composites of observed variables as indicators for a common factor—strengthens loadings, but reduces the number of indicators. Factor indeterminacy is reduced when there are many observed variables per factor, and when loadings and factor correlations are strong. It is proven that parceling cannot reduce factor indeterminacy. In special cases where the ratio of loading to residual variance is the same for all items included in each parcel, factor indeterminacy is unaffected by parceling. Otherwise, parceling worsens factor indeterminacy. While factor indeterminacy does not affect the parameter estimates, standard errors, or fit indices associated with a factor model, it does create uncertainty, which endangers valid inference.

     

A method of factor analysis by means of which all coordinates of the factor matrix are given simultaneously

In general, the methods of factor analysis developed during the past five years are based on the reduction of the correlational matrix by successive steps. The first factor loadings are determined and eliminated from the correlational matrix, giving a residual matrix. This process is continued for successive factor loadings until the elements of the last obtained residual matrix may be regarded as due to chance. The method outlined in this paper assumes the maximum number of factorsm in the correlational matrix. Them factor vectors are solved for simultaneously. Once them factor vectors are found, any vectors having only negligible factor loadings may be discarded.     

Influence curves for factor loadings

Influence curves for the initial and rotated loadings are derived for the maximum likelihood factor analysis (MLFA) model. Cook's distances based on the empirical influence curves of factor loadings are proposed for the identification of influential observations. The distances are shown to be invariant under scale transformation and factor rotation. We find that an observation with a very large Cook's distance based on the sample influence curve may not necessarily exert an excessive influence on the factor loadings pattern but may change the ordering of the factors. The issue of the switching of factors is also studied by means of the empirical influence curve and factor scores.     

Obtaining maximum-likelihood estimates of factor loadings and communalities using an easily implemented iterative computer procedure

This paper considers the problem of computing estimates of factor loadings, specific variances, and communalities for a factor analytic model. The equations for maximum-likelihood estimators are discussed. Iterative formulas are developed to solve the maximum-likelihood equations and a simple and efficient method of implementation on a digital computer is described. Use of the iterative formulas and computing techniques for other estimators of factor loadings and communalities is also considered to provide a very general approach for this aspect of factor analysis.     

THE RELATIONSHIP BETWEEN DIFFICULTY AND FACTOR LOADINGS OF SOME VISUAL-PERCEPTUAL TESTS

WERDELIN, I. & STJERNBERG, G. The relationship between difficulty and factor loadings of some visual-perceptual tests. Scand. J. Psychol. , 1971, 12, 21–28. – The study aimed at investigating whether it is possible to change factor loadings by varying the difficulty and complexity of the same visual-perceptual tests. 171 sixth grade pupils were given 27 tests. Some of these defined reference factors, others were differently difficult versions of four tests from separate parts of the visual-perceptual field. Data were treated by factor analysis, yielding the four factors R, S, N and P. It was found that the more difficult the test the higher its loadings on the S and R factors, and the easier the test the higher its loadings on the N and particularly the P factor.     

Oblique factors and components with independent clusters

Relationships between the results of factor analysis and component analysis are derived when oblique factors have independent clusters with equal variances of unique factors. The factor loadings are analytically shown to be smaller than the corresponding component loadings while the factor correlations are shown to be greater than the corresponding component correlations. The condition for the inequality of the factor/component contributions is derived in the case with different variances for unique factors. Further, the asymptotic standard errors of parameter estimates are obtained for a simplified model with the assumption of multivariate normality, which shows that the component loading estimate is more stable than the corresponding factor loading estimate.     

Exploratory Factor Analysis With Small Sample Sizes

Exploratory factor analysis (EFA) is generally regarded as a technique for large sample sizes (N), with N = 50 as a reasonable absolute minimum. This study offers a comprehensive overview of the conditions in which EFA can yield good quality results for N below 50. Simulations were carried out to estimate the minimum required N for different levels of loadings (λ), number of factors (f), and number of variables (p) and to examine the extent to which a small N solution can sustain the presence of small distortions such as interfactor correlations, model error, secondary loadings, unequal loadings, and unequal p/f. Factor recovery was assessed in terms of pattern congruence coefficients, factor score correlations, Heywood cases, and the gap size between eigenvalues. A subsampling study was also conducted on a psychological dataset of individuals who filled in a Big Five Inventory via the Internet. Results showed that when data are well conditioned (i.e., high λ, low f, high p), EFA can yield reliable results for N well below 50, even in the presence of small distortions. Such conditions may be uncommon but should certainly not be ruled out in behavioral research data.     

Exploratory Factor Analysis of the Pain Outcomes Profile

The Pain Outcomes Profile (POP) is a brief, multidimensional measure intended to assess pain intensity, functioning, and affect. It is presented as a practical measure with clinical utility. Results of studies support its concurrent, construct and predictive validity at the scale level. However, there have been no published studies of the measure at the item level. The present study was intended to assess the construct validity of the POP by way of factor analysis. A sample of 447 assessments of patients at a chronic non-cancer pain outpatient treatment center was employed. The 20 substantive items comprising the POP were entered into a factor analysis with oblique rotation. Five salient factors were obtained. Item-inclusion was generally consistent with factor loadings although noteworthy exceptions were observed in the Fear, Mobility and Vitality scales. Recommendations for further study and limitations of the current project are delineated.     

Variance estimation in factor analysis: An application of the bootstrap

Sampling variability of the estimates of factor loadings is neglected in modern factor analysis. Such investigations are generally normal theory based and asymptotic in nature. The bootstrap, a computer-based methodology, is described and then applied to demonstrate how the sampling variability of the estimates of factor loadings can be estimated for a given set of data. The issue of the number of factors to be retained in a factor model is also addressed. The bootstrap is shown to be an effective data-analytic tool for computing various statistics of interest which are otherwise intractable.     

Determinants of Standard Errors of MLEs in Confirmatory Factor Analysis

This paper studies changes of standard errors (SE) of the normal-distribution-based maximum likelihood estimates (MLE) for confirmatory factor models as model parameters vary. Using logical analysis, simplified formulas and numerical verification, monotonic relationships between SEs and factor loadings as well as unique variances are found. Conditions under which monotonic relationships do not exist are also identified. Such functional relationships allow researchers to better understand the problem when significant factor loading estimates are expected but not obtained, and vice versa. What will affect the likelihood for Heywood cases (negative unique variance estimates) is also explicit through these relationships. Empirical findings in the literature are discussed using the obtained results.     

Ordinary Least Squares Estimation of Parameters in Exploratory Factor Analysis With Ordinal Data

Exploratory factor analysis (EFA) is often conducted with ordinal data (e.g., items with 5-point responses) in the social and behavioral sciences. These ordinal variables are often treated as if they were continuous in practice. An alternative strategy is to assume that a normally distributed continuous variable underlies each ordinal variable. The EFA model is specified for these underlying continuous variables rather than the observed ordinal variables. Although these underlying continuous variables are not observed directly, their correlations can be estimated from the ordinal variables. These correlations are referred to as polychoric correlations. This article is concerned with ordinary least squares (OLS) estimation of parameters in EFA with polychoric correlations. Standard errors and confidence intervals for rotated factor loadings and factor correlations are presented. OLS estimates and the associated standard error estimates and confidence intervals are illustrated using personality trait ratings from 228 college students. Statistical properties of the proposed procedure are explored using a Monte Carlo study. The empirical illustration and the Monte Carlo study showed that (a) OLS estimation of EFA is feasible with large models, (b) point estimates of rotated factor loadings are unbiased, (c) point estimates of factor correlations are slightly negatively biased with small samples, and (d) standard error estimates and confidence intervals perform satisfactorily at moderately large samples.