Newton Algorithms for Analytic Rotation: an Implicit Function Approach

In this paper implicit function-based parameterizations for orthogonal and oblique rotation matrices are proposed. The parameterizations are used to construct Newton algorithms for minimizing differentiable rotation criteria applied to m factors and p variables. The speed of the new algorithms is compared to that of existing algorithms and to that of Newton algorithms based on alternative parameterizations. Several rotation criteria were examined and the algorithms were evaluated over a range of values for m. Initial guesses for Newton algorithms were improved by subconvergence iterations of the gradient projection algorithm. Simulation results suggest that no one algorithm is fastest for minimizing all criteria for all values of m. Among competing algorithms, the gradient projection algorithm alone was faster than the implicit function algorithm for minimizing a quartic criterion over oblique rotation matrices when m is large. In all other conditions, however, the implicit function algorithms were competitive with or faster than the fastest existing algorithms. The new algorithms showed the greatest advantage over other algorithms when minimizing a nonquartic component loss criterion.     

Rotation to Simple Loadings Using Component Loss Functions: The Oblique Case

Component loss functions (CLFs) similar to those used in orthogonal rotation are introduced to define criteria for oblique rotation in factor analysis. It is shown how the shape of the CLF affects the performance of the criterion it defines. For example, it is shown that monotone concave CLFs give criteria that are minimized by loadings with perfect simple structure when such loadings exist. Moreover, if the CLFs are strictly concave, minimizing must produce perfect simple structure whenever it exists. Examples show that methods defined by concave CLFs perform well much more generally. While it appears important to use a concave CLF, the specific CLF used is less important. For example, the very simple linear CLF gives a rotation method that can easily outperform the most popular oblique rotation methods promax and quartimin and is competitive with the more complex simplimax and geomin methods. The author would like to thank the editor and three reviewers for helpful suggestions and for identifying numerous errors.     

Standard errors for obliquely rotated factor loadings

In a manner similar to that used in the orthogonal case, formulas for the aymptotic standard errors of analytically rotated oblique factor loading estimates are obtained. This is done by finding expressions for the partial derivatives of an oblique rotation algorithm and using previously derived results for unrotated loadings. These include the results of Lawley for maximum likelihood factor analysis and those of Girshick for principal components analysis. Details are given in cases including direct oblimin and direct Crawford-Ferguson rotation. Numerical results for an example involving maximum likelihood estimation with direct quartimin rotation are presented. They include simultaneous tests for significant loading estimates.This research was supported in part by NIH Grant RR-3. The author is indebted to Dorothy Thayer who implemented the algorithms required for the example and to Gunnar Gruvaeus and Allen Yates for reviewing an earlier version of this paper. Special thanks are extended to Michael Browne for many conversations devoted to clarifying the thoughts of the author.     

Standard errors for rotated factor loadings

Beginning with the results of Girshick on the asymptotic distribution of principal component loadings and those of Lawley on the distribution of unrotated maximum likelihood factor loadings, the asymptotic distribution of the corresponding analytically rotated loadings is obtained. The principal difficulty is the fact that the transformation matrix which produces the rotation is usually itself a function of the data. The approach is to use implicit differentiation to find the partial derivatives of an arbitrary orthogonal rotation algorithm. Specific details are given for the orthomax algorithms and an example involving maximum likelihood estimation and varimax rotation is presented.This research was supported in part by NIH Grant RR-3. The authors are grateful to Dorothy T. Thayer who implemented the algorithms discussed here as well as those of Lawley and Maxwell. We are particularly indebted to Michael Browne for convincing us of the significance of this work and for helping to guide its development and to Harry H. Harman who many years ago pointed out the need for standard errors of estimate.     

A monotonically convergent algorithm for orthogonal congruence rotation

Brokken has proposed a method for orthogonal rotation of one matrix such that its columns have a maximal sum of congruences with the columns of a target matrix. This method employs an algorithm for which convergence from every starting point is not guaranteed. In the present paper, an iterative majorization algorithm is proposed which is guaranteed to converge from every starting point. Specifically, it is proven that the function value converges monotonically, and that the difference between subsequent iterates converges to zero. In addition to the better convergence properties, another advantage of the present algorithm over Brokken's one is that it is easier to program. The algorithms are compared on 80 simulated data sets, and it turned out that the new algorithm performed well in all cases, whereas Brokken's algorithm failed in almost half the cases. The derivation of the algorithm is given in full detail because it involves a series of inequalities that can be of use to derive similar algorithms in different contexts.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the first author. The authors are obliged to Willem J. Heiser and Jos M. F. ten Berge for useful comments on an earlier version of this paper.     

Standard errors for procrustes solutions

The asymptotic standard errors for the procrustes solutions are derived for orthogonal rotation, direct oblique rotation and indirect oblique rotation. The standard errors for the first two rotations are obtained using the augmented information matrices. For the indirect oblique solution, the standard errors of rotated parameters are derived from the information matrix of unrotated loadings using the chain rule for information matrices. For all three types of rotation, the standard errors of rotated parameters are presented for unstandardized and standardized manifest variables. Numerical examples show the similarity of theoretical and simulated values.     

Quartic rotation criteria and algorithms

Most of the currently used analytic rotation criteria for simple structure in factor analysis are summarized and identified as members of a general symmetric family of quartic criteria. A unified development of algorithms for orthogonal and direct oblique rotation using arbitrary criteria from this family is given. These algorithms represent fairly straightforward extensions of present methodology, and appear to be the best methods currently available.The research done by R. I. Jennrich was supported by NSF Grant MCS-8301587.     

A cluster‐based factor rotation

A new oblique factor rotation method is proposed, the aim of which is to identify a simple and well‐clustered structure in a factor loading matrix. A criterion consisting of the complexity of a factor loading matrix and a between‐cluster dissimilarity is optimized using the gradient projection algorithm and the k‐means algorithm. It is shown that if there is an oblique rotation of an initial loading matrix that has a perfect simple structure, then the proposed method with Kaiser's normalization will produce the perfect simple structure. Although many rotation methods can also recover a perfect simple structure, they perform poorly when a perfect simple structure is not possible. In this case, the new method tends to perform better because it clusters the loadings without requiring the clusters to be perfect. Artificial and real data analyses demonstrate that the proposed method can give a simple structure, which the other methods cannot produce, and provides a more interpretable result than those of widely known rotation techniques.     

Hierarchical factor solutions without rotation

A method is presented for securing a hierarchical factor solution which achieves simple structure at each hierarchical level without rotation or even preliminary arbitrary orthogonal or oblique solutions. The method is based upon the assumption that if overlap is removed from clusters the remaining specifics will achieve simple structure automatically. The problem presented earlier by Schmid and Leiman, using oblique simple structural rotation as a basis, is reworked by this new approach.     

Assessing stress in cancer patients: a second-order factor analysis model for the Perceived Stress Scale

Using the Perceived Stress Scale (PSS), perceptions of global stress were assessed in 111 women following breast cancer surgery and at 12 and 24 months later This is the first study to factor analyze the PSS. The PSS data were factor analyzed each time using exploratory factor analysis with oblique direct quartimin rotation. Goodness-of-fit indices (root mean square error of approximation [RMSEA]), magnitude and pattern of factor loadings, and confidence interval data revealed a two-factor solution of positive versus negative stress items. The findings, replicated across time, also indicate factor stability. Hierarchical factor analyses supported a second-order factor of "perceived stress." This alternative factor model of the PSS is presented along with observations regarding the measure's use in cancer research.     

A General Approach for Fitting Pure Exploratory Bifactor Models

This article proposes a procedure for fitting a pure exploratory bifactor solution in which the general factor is orthogonal to the group factors, but the loadings on the group factors can satisfy any orthogonal or oblique rotation criterion. The proposal combines orthogonal Procrustes rotations with analytical rotations and consists of a sequence of four steps. The basic input is a semispecified target matrix that can be (a) defined by the user, (b) obtained by using Schmid-Leiman orthogonalization, or (c) automatically built from a conventional unrestricted solution based on a prescribed number of factors. The relevance of the proposal and its advantages over existing procedures is discussed and assessed via simulation. Its feasibility in practice is illustrated with two empirical examples in the personality domain.     

A simple general method for oblique rotation

A simple and very general algorithm for oblique rotation is identified. While motivated by the rotation problem in factor analysis, it may be used to minimize almost any function of a not necessarily square matrix whose columns are restricted to have unit length. The algorithm has two steps. The first is to compute the gradient of the rotation criterion and the second is to project this onto a manifold of matrices with unit length columns. For this reason it is called a gradient projection algorithm. Because the projection step is very simple, implementation of the algorithm involves little more than computing the gradient of the rotation criterion which for many applications is very simple. It is proven that the algorithm is strictly monotone, that is as long as it is not already at a stationary point, each step will decrease the value of the criterion. Examples from a variety of areas are used to demonstrate the algorithm, including oblimin rotation, target rotation, simplimax rotation, and rotation to similarity and simplicity simultaneously. While it may be, the algorithm is not intended for use as a standard algorithm for well established problems, but rather as a tool for investigating new methods where its generality and simplicity may save an investigator substantial effort.The author would like to thank the review team for their insights and recommendations.     

Exploratory Bi-factor Analysis: The Oblique Case

Bi-factor analysis is a form of confirmatory factor analysis originally introduced by Holzinger and Swineford (Psychometrika 47:41?C54, 1937). The bi-factor model has a general factor, a number of group factors, and an explicit bi-factor structure. Jennrich and Bentler (Psychometrika 76:537?C549, 2011) introduced an exploratory form of bi-factor analysis that does not require one to provide an explicit bi-factor structure a priori. They use exploratory factor analysis and a bifactor rotation criterion designed to produce a rotated loading matrix that has an approximate bi-factor structure. Among other things this can be used as an aid in finding an explicit bi-factor structure for use in a confirmatory bi-factor analysis. They considered only orthogonal rotation. The purpose of this paper is to consider oblique rotation and to compare it to orthogonal rotation. Because there are many more oblique rotations of an initial loading matrix than orthogonal rotations, one expects the oblique results to approximate a bi-factor structure better than orthogonal rotations and this is indeed the case. A surprising result arises when oblique bi-factor rotation methods are applied to ideal data.     

Independent component analysis as a rotation method: A very different solution to Thurstone's box problem

In this paper we consider the well‐known Thurstone box problem in exploratory factor analysis. Initial loadings and components are extracted using principal component analysis. Rotating the components towards independence rather than rotating the loadings towards simplicity allows one to accurately recover the dimensions of each box and also produce simple loadings. It is shown how this may be done using an appropriate rotation criterion and a general rotation algorithm. Methods from independent component analysis are used, and this paper may be viewed as an introduction to independent component analysis from the perspective of factor analysis.     

Derivative free gradient projection algorithms for rotation

A simple modification substantially simplifies the use of the gradient projection (GP) rotation algorithms of Jennrich (2001, 2002). These algorithms require subroutines to compute the value and gradient of any specific rotation criterion of interest. The gradient can be difficult to derive and program. It is shown that using numerical gradients gives almost precisely the same results as using exact gradients. The resulting algorithm is very easy to use because the only problem specific code required is that needed to define the rotation criterion. The computing time is increased when using numerical gradients, but it is still very modest for most purposes. While used extensively elsewhere, numerical derivatives seem to be underutilized in statistics.     

Comparison Of Oblique And Orthogonal Simple Structure Solutions For Personality And Interest Factors

The study was directed to the need to structure in a few variables the domain measured by personality and interest measures commonly employed in educational counseling: Strong, Kuder, EPPS and the Study of Values. Despite initial uncertainty regarding number of factors to be employed, effects of ipsative scores and of mixing test formats, both an oblique and orthogonal rotation yielded nearly identical results. Of the twenty factors identified by both the biquartimin and varimax solutions, Seven linked vocational interest clusters with personality. Two of the remaining factors had only interest loadings, while of the eleven personality factors, only four were scale specific. Definition of the 16 common factors required that extraction proceed beyond the unit latent root criterion. The results offer evidence that over- extracting factors does not confuse the results of rotation. Further, psycho- metric differences between tests had essentially no effect on the factors found. Of three oblimin rotations attempted, only the biquartimin was successful, yielding results essentially like those of the varimax solution. Because of the vast difference in computation time for these two solutions (computer time 20 times greater for biquartimin), however, the orthogonal varimax remains the method of choice.     

Characteristics factor structures of the Japanese version of the State-Trait Anxiety Inventory: coexistence of positive-negative and state-trait factor structures

Previous factor studies of the State-Trait Anxiety Inventory (STAI; Spielberger, Gorsuch, & Lushene, 1970) have reported certain typical factors that are state-trait (S-T) 2-factor solutions and positively-negatively (P-N) worded item 2-factor solutions in addition to 4-factor solutions (positively and negatively worded state factors, positively and negatively worded trait factors). We explored the possibility that these factor structures are included in a factor space. Responses to the Japanese version of the STAI in a sample of 848 male workers were factor analyzed. The first-order factors obtained from principal-component analysis were almost equal to the previous 4 factors, except for a minor factor, and their second-order factors were the P-N factors. However, the S-T factors were also obtained from the same first-order factors by the oblique Procrustes rotation. Moreover, coexistence of these two 2-factor structures was determined in the same factor space by the orthogonal Procrustes rotation.     

Characteristic Factor Structures of the Japanese Version of the State-Trait Anxiety Inventory: Coexistence of Positive-Negative and State-Trait Factor Structures

Previous factor studies of the State-Trait Anxiety Inventory (STAI; Spielberger, Gorsuch, & Lushene, 1970) have reported certain typical factors that are state-trait (S-T) 2-factor solutions and positively-negatively (P-N) worded item 2-factor solutions in addition to 4-factor solutions (positively and negatively worded state factors, positively and negatively worded trait factors). We explored the possibility that these factor structures are included in a factor space. Responses to the Japanese version of the STAI in a sample of 848 male workers were factor analyzed. The first-order factors obtained from principal-component analysis were almost equal to the previous 4 factors, except for a minor factor, and their second-order factors were the P-N factors. However, the S-T factors were also obtained from the same first-order factors by the oblique Procrustes rotation. Moreover, coexistence of these two 2-factor structures was determined in the same factor space by the orthogonal Procrustes rotation.     

A general rotation criterion and its use in orthogonal rotation

Measures of test parsimony and factor parsimony are defined. Minimizing their weighted sum produces a general rotation criterion for either oblique or orthogonal rotation. The quartimax, varimax and equamax criteria are special cases of the expression. Two new criteria are developed. One of these, the parsimax criterion, apparently gives excellent results. It is argued that one of the most important factors bearing on the choice of a rotation criterion for a particular problem is the amount of information available on the number of factors that should be rotated. This research was supported by the National Research Council of Canada research grant 291-13 to Dr. G. A. Ferguson.