Simplimax: Oblique rotation to an optimal target with simple structure

Factor analysis and principal component analysis are usually followed by simple structure rotations of the loadings. These rotations optimize a certain criterion (e.g., varimax, oblimin), designed to measure the degree of simple structure of the pattern matrix. Simple structure can be considered optimal if a (usually large) number of pattern elements is exactly zero. In the present paper, a class of oblique rotation procedures is proposed to rotate a pattern matrix such that it optimally resembles a matrix which has an exact simple pattern. It is demonstrated that this method can recover relatively complex simple structures where other well-known simple structure rotation techniques fail.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences. The author is obliged to Jos ten Berge for helpful comments on an earlier version.     

Oblique rotation in correspondence analysis: A step forward in the search for the simplest interpretation

Correspondence analysis (CA) is a popular method that can be used to analyse relationships between categorical variables. It is closely related to several popular multivariate analysis methods such as canonical correlation analysis and principal component analysis. Like principal component analysis, CA solutions can be rotated orthogonally as well as obliquely into a simple structure without affecting the total amount of explained inertia. However, some specific aspects of CA prevent standard rotation procedures from being applied in a straightforward fashion. In particular, the role played by weights assigned to points and dimensions and the duality of CA solutions are unique to CA. For orthogonal simple structure rotation, procedures recently have been proposed. In this paper, we construct oblique rotation methods for CA that take into account these specific difficulties. We illustrate the benefits of our oblique rotation procedure by means of two illustrative examples.     

Independent component analysis as a rotation method: A very different solution to Thurstone's box problem

In this paper we consider the well‐known Thurstone box problem in exploratory factor analysis. Initial loadings and components are extracted using principal component analysis. Rotating the components towards independence rather than rotating the loadings towards simplicity allows one to accurately recover the dimensions of each box and also produce simple loadings. It is shown how this may be done using an appropriate rotation criterion and a general rotation algorithm. Methods from independent component analysis are used, and this paper may be viewed as an introduction to independent component analysis from the perspective of factor analysis.     

A cluster‐based factor rotation

A new oblique factor rotation method is proposed, the aim of which is to identify a simple and well‐clustered structure in a factor loading matrix. A criterion consisting of the complexity of a factor loading matrix and a between‐cluster dissimilarity is optimized using the gradient projection algorithm and the k‐means algorithm. It is shown that if there is an oblique rotation of an initial loading matrix that has a perfect simple structure, then the proposed method with Kaiser's normalization will produce the perfect simple structure. Although many rotation methods can also recover a perfect simple structure, they perform poorly when a perfect simple structure is not possible. In this case, the new method tends to perform better because it clusters the loadings without requiring the clusters to be perfect. Artificial and real data analyses demonstrate that the proposed method can give a simple structure, which the other methods cannot produce, and provides a more interpretable result than those of widely known rotation techniques.     

Three-mode orthomax rotation

Factor analysis and principal components analysis (PCA) are often followed by an orthomax rotation to rotate a loading matrix to simple structure. The simple structure is usually defined in terms of the simplicity of the columns of the loading matrix. In Three-mode PCA, rotational freedom of the so called core (a three-way array relating components for the three different modes) can be used similarly to find a simple structure of the core. Simple structure of the core can be defined with respect to all three modes simultaneously, possibly with different emphases on the different modes. The present paper provides a fully flexible approach for orthomax rotation of the core to simple structure with respect to three modes simultaneously. Computationally, this approach relies on repeated (two-way) orthomax applied to supermatrices containing the frontal, lateral or horizontal slabs, respectively. The procedure is illustrated by means of a number of exemplary analyses. As a by-product, application of the Three-mode Orthomax procedures to two-way arrays is shown to reveal interesting relations with and interpretations of existing two-way simple structure rotation techniques.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the author. The author is obliged to Jos ten Berge and two anonymous reviewers for useful comments on an earlier version of this paper.     

Coefficients alpha and reliabilities of unrotated and rotated components

It has been shown by Kaiser that the sum of coefficients alpha of a set of principal components does not change when the components are transformed by an orthogonal rotation. In this paper, Kaiser's result is generalized. First, the invariance property is shown to hold for any set of orthogonal components. Next, a similar invariance property is derived for the reliability of any set of components. Both generalizations are established by considering simultaneously optimal weights for components with maximum alpha and with maximum reliability, respectively. A short-cut formula is offered to evaluate the coefficients alpha for orthogonally rotated principal components from rotation weights and eigenvalues of the correlation matrix. Finally, the greatest lower bound to reliability and a weighted version are discussed.Comments by Henk A.L. Kiers and by anonymous referees are gratefully acknowledged.     

A simple general method for oblique rotation

A simple and very general algorithm for oblique rotation is identified. While motivated by the rotation problem in factor analysis, it may be used to minimize almost any function of a not necessarily square matrix whose columns are restricted to have unit length. The algorithm has two steps. The first is to compute the gradient of the rotation criterion and the second is to project this onto a manifold of matrices with unit length columns. For this reason it is called a gradient projection algorithm. Because the projection step is very simple, implementation of the algorithm involves little more than computing the gradient of the rotation criterion which for many applications is very simple. It is proven that the algorithm is strictly monotone, that is as long as it is not already at a stationary point, each step will decrease the value of the criterion. Examples from a variety of areas are used to demonstrate the algorithm, including oblimin rotation, target rotation, simplimax rotation, and rotation to similarity and simplicity simultaneously. While it may be, the algorithm is not intended for use as a standard algorithm for well established problems, but rather as a tool for investigating new methods where its generality and simplicity may save an investigator substantial effort.The author would like to thank the review team for their insights and recommendations.     

Hierarchical factor solutions without rotation

A method is presented for securing a hierarchical factor solution which achieves simple structure at each hierarchical level without rotation or even preliminary arbitrary orthogonal or oblique solutions. The method is based upon the assumption that if overlap is removed from clusters the remaining specifics will achieve simple structure automatically. The problem presented earlier by Schmid and Leiman, using oblique simple structural rotation as a basis, is reworked by this new approach.     

On weighted procrustes and hyperplane fitting in factor analytic rotation

A weighted collinearity criterion for Procrustean rotation is developed, and it is shown that special cases with respect to the choice of weights and a vector norm are forms of hyperplane fitting, classical oblique Procrustes, etc.; a family of Procrustean transformation procedures is thereby generated. Numerical illustrations utilizing the Holzinger-Swineford data are presented.     

The weighted oblimin rotation

Cureton & Mulaik (1975) proposed the Weighted Varimax rotation so that Varimax (Kaiser, 1958) could reach simple solutions when the complexities of the variables in the solution are larger than one. In the present paper the weighting procedure proposed by Cureton & Mulaik (1975) is applied to Direct Oblimin (Clarkson & Jennrich, 1988), and the rotation method obtained is called Weighted Oblimin. It has been tested on artificial complex data and real data, and the results seem to indicate that, even though Direct Oblimin rotation fails when applied to complex data, Weighted Oblimin gives good results if a variable with complexity one can be found for each factor in the pattern. Although the weighting procedure proposed by Cureton & Mulaik is based on Landahl's (1938) expression for orthogonal factors, Weighted Oblimin seems to be adequate even with highly oblique factors. The new rotation method was compared to other rotation methods based on the same weighting procedure and, whenever a variable with complexity one could be found for each factor in the pattern, Weighted Oblimin gave the best results. When rotating a simple empirical loading matrix, Weighted Oblimin seemed to slightly increase the performance of Direct Oblimin.The author is obliged to Henk A. L. Kiers and three anonymous reviewers for helpful comments on an earlier version of this paper.     

A general approach to procrustes pattern rotation

A method is introduced for oblique rotation to a pattern target matrix specified in advance. The target matrix may have all or only some of its elements specified. Values are estimated by means of a general procedure for minimization with equality constraints. Results are shown using data from Harman and Browne.The author is indebted to K. G. Jöreskog and M. W. Browne for many helpful suggestions and comments throughout the course of this study. Thanks are also due P. H. Schönemann and H. Weiner.     

Factor simplicity index and transformations

A scale-invariant index of factorial simplicity is proposed as a summary statistic for principal components and factor analysis. The index ranges from zero to one, and attains its maximum when all variables are simple rather than factorially complex. A factor scale-free oblique transformation method is developed to maximize the index. In addition, a new orthogonal rotation procedure is developed. These factor transformation methods are implemented using rapidly convergent computer programs. Observed results indicate that the procedures produce meaningfully simple factor pattern solutions.This investigation was supported in part by a Research Scientist Development Award (K02-DA00017) and research grants (MH24149 and DA01070) from the U. S. Public Health Service. The assistance of Andrew L. Comrey, Henry F. Kaiser, Bonnie Barron, Marion Hee, and several anonymous reviewers is gratefully acknowledged.     

On the equivalence of two oblique congruence rotation methods,and orthogonal approximations

Tucker's method of oblique congruence rotation is shown to be equivalent to a procedure by Meredith. This implies that Monte Carlo studies on congruence by Nesselroade, Baltes and Labouvie and by Korth and Tucker are highly comparable. The problem of rotating two matrices orthogonally to maximal congruence has not yet been solved. An approximate solution to this problem can be derived from Tucker's method. Even better results can be obtained from a Procrustes rotation followed by rotation to simple structure.     

Rotation to Simple Loadings Using Component Loss Functions: The Oblique Case

Component loss functions (CLFs) similar to those used in orthogonal rotation are introduced to define criteria for oblique rotation in factor analysis. It is shown how the shape of the CLF affects the performance of the criterion it defines. For example, it is shown that monotone concave CLFs give criteria that are minimized by loadings with perfect simple structure when such loadings exist. Moreover, if the CLFs are strictly concave, minimizing must produce perfect simple structure whenever it exists. Examples show that methods defined by concave CLFs perform well much more generally. While it appears important to use a concave CLF, the specific CLF used is less important. For example, the very simple linear CLF gives a rotation method that can easily outperform the most popular oblique rotation methods promax and quartimin and is competitive with the more complex simplimax and geomin methods. The author would like to thank the editor and three reviewers for helpful suggestions and for identifying numerous errors.     

Riemannian Newton and trust-region algorithms for analytic rotation in exploratory factor analysis

In exploratory factor analysis, latent factors and factor loadings are seldom interpretable until analytic rotation is performed. Typically, the rotation problem is solved by numerically searching for an element in the manifold of orthogonal or oblique rotation matrices such that the rotated factor loadings minimize a pre-specified complexity function. The widely used gradient projection (GP) algorithm, although simple to program and able to deal with both orthogonal and oblique rotation, is found to suffer from slow convergence when the number of manifest variables and/or the number of latent factors is large. The present work examines the effectiveness of two Riemannian second-order algorithms, which respectively generalize the well-established truncated Newton and trust-region strategies for unconstrained optimization in Euclidean spaces, in solving the rotation problem. When approaching a local minimum, the second-order algorithms usually converge superlinearly or even quadratically, better than first-order algorithms that only converge linearly. It is further observed in Monte Carlo studies that, compared to the GP algorithm, the Riemannian truncated Newton and trust-region algorithms require not only much fewer iterations but also much less processing time to meet the same convergence criterion, especially in the case of oblique rotation.     

The weighted varimax rotation and the promax rotation

Kaiser's iterative algorithm for the varimax rotation fails when (a) there is a substantial cluster of test vectors near the middle of each bounding hyperplane, leading to non-bounding hyperplanes more heavily overdetermined than those at the boundaries of the configuration of test vectors, and/or (b) there are appreciably more thanm (m factors) tests whose loadings on one of the factors of the initialF-matrix, usually the first, are near-zero, leading to overdetermination of the hyperplane orthogonal to this initialF-axis before rotation. These difficulties are overcome by weighting the test vectors, giving maximum weights to those likely to be near the primary axes, intermediate weights to those likely to be near hyperplanes but not near primary axes, and near-zero weights to those almost collinear with or almost orthogonal to the first initialF-axis. Applications to the Promax rotation are discussed, and it is shown that these procedures solve Thurstone's hitherto intractable “invariant” box problem as well as other more common problems based on real data.     

Orthomax rotation and perfect simple structure

A loading matrix has perfect simple structure if each row has at most one nonzero element. It is shown that if there is an orthogonal rotation of an initial loading matrix that has perfect simple structure, then orthomax rotation with 0 1 of the initial loading matrix will produce the perfect simple structure. In particular, varimax and quartimax will produce rotations with perfect simple structure whenever they exist.     

Two-matrix orthogonal rotation procedures

Examples are presented in which it is either desirable or necessary to transform two sets of orthogonal axes to simple structure positions by means of the same transformation matrix. A solution is then outlined which represents a two-matrix extension of the general orthomax orthogonal rotation criterion. In certain circumstances, oblique two-matrix solutions are possible using the procedure outlined and the Harris-Kaiser [1964] logic. Finally, an illustrative example is presented in which the preceding technique is applied in the context of an inter-battery factor analysis.The work reported herein was supported by Grant S72-1886 from the Canada Council. The author acknowledges the helpful contributions of Nancy Reid and Lawrence Ward to parts of this paper.     

Some relations between guttman's principal components of scale analysis and other psychometric theory

Guttman's principal components for the weighting system are the item scoring weights that maximize the generalized Kuder-Richardson reliability coefficient. The principal component for any item is effectively the same as the factor loading of the item divided by the item standard deviation, the factor loadings being obtained from an ordinary factor analysis of the item intercorrelation matrix.