Orthogonal procrustes rotation for two or more matrices

Necessary and sufficient conditions for rotating matrices to maximal agreement in the least-squares sense are discussed. A theorem by Fischer and Roppert, which solves the case of two matrices, is given a more straightforward proof. A sufficient condition for a best least-squares fit for more than two matrices is formulated and shown to be not necessary. In addition, necessary conditions suggested by Kristof and Wingersky are shown to be not sufficient. A rotation procedure that is an alternative to the one by Kristof and Wingersky is presented. Upper bounds are derived for determining the extent to which the procedure falls short of attaining the best least-squares fit. The problem of scaling matrices to maximal agreement is discussed. Modifications of Gower's method of generalized Procrustes analysis are suggested.     

A monotonically convergent algorithm for orthogonal congruence rotation

Brokken has proposed a method for orthogonal rotation of one matrix such that its columns have a maximal sum of congruences with the columns of a target matrix. This method employs an algorithm for which convergence from every starting point is not guaranteed. In the present paper, an iterative majorization algorithm is proposed which is guaranteed to converge from every starting point. Specifically, it is proven that the function value converges monotonically, and that the difference between subsequent iterates converges to zero. In addition to the better convergence properties, another advantage of the present algorithm over Brokken's one is that it is easier to program. The algorithms are compared on 80 simulated data sets, and it turned out that the new algorithm performed well in all cases, whereas Brokken's algorithm failed in almost half the cases. The derivation of the algorithm is given in full detail because it involves a series of inequalities that can be of use to derive similar algorithms in different contexts.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the first author. The authors are obliged to Willem J. Heiser and Jos M. F. ten Berge for useful comments on an earlier version of this paper.     

Alternating least squares algorithms for simultaneous components analysis with equal component weight matrices in two or more populations

Millsap and Meredith (1988) have developed a generalization of principal components analysis for the simultaneous analysis of a number of variables observed in several populations or on several occasions. The algorithm they provide has some disadvantages. The present paper offers two alternating least squares algorithms for their method, suitable for small and large data sets, respectively. Lower and upper bounds are given for the loss function to be minimized in the Millsap and Meredith method. These can serve to indicate whether or not a global optimum for the simultaneous components analysis problem has been attained.Financial support by the Netherlands organization for scientific research (NWO) is gratefully acknowledged.     

On the equivalence of two oblique congruence rotation methods,and orthogonal approximations

Tucker's method of oblique congruence rotation is shown to be equivalent to a procedure by Meredith. This implies that Monte Carlo studies on congruence by Nesselroade, Baltes and Labouvie and by Korth and Tucker are highly comparable. The problem of rotating two matrices orthogonally to maximal congruence has not yet been solved. An approximate solution to this problem can be derived from Tucker's method. Even better results can be obtained from a Procrustes rotation followed by rotation to simple structure.     

Orthogonal rotations to maximal agreement for two or more matrices of different column orders

Methods for orthogonal Procrustes rotation and orthogonal rotation to a maximal sum of inner products are examined for the case when the matrices involved have different numbers of columns. An inner product solution offered by Cliff is generalized to the case of more than two matrices. A nonrandom start for a Procrustes solution suggested by Green and Gower is shown to give better results than a random start. The Green-Gower Procrustes solution (with nonrandom start) is generalized to the case of more than two matrices. Simulation studies indicate that both the generalized inner product solution and the generalized Procrustes solution tend to attain their global optima within acceptable computation times. A simple procedure is offered for approximating simple structure for the rotated matrices without affecting either the Procrustes or the inner product criterion.The authors are obliged to Charles Lewis for helpful comments on a previous draft of this paper and to Frank Brokken for preparing a computer program that was used in this study.     

The weighted oblimin rotation

Cureton & Mulaik (1975) proposed the Weighted Varimax rotation so that Varimax (Kaiser, 1958) could reach simple solutions when the complexities of the variables in the solution are larger than one. In the present paper the weighting procedure proposed by Cureton & Mulaik (1975) is applied to Direct Oblimin (Clarkson & Jennrich, 1988), and the rotation method obtained is called Weighted Oblimin. It has been tested on artificial complex data and real data, and the results seem to indicate that, even though Direct Oblimin rotation fails when applied to complex data, Weighted Oblimin gives good results if a variable with complexity one can be found for each factor in the pattern. Although the weighting procedure proposed by Cureton & Mulaik is based on Landahl's (1938) expression for orthogonal factors, Weighted Oblimin seems to be adequate even with highly oblique factors. The new rotation method was compared to other rotation methods based on the same weighting procedure and, whenever a variable with complexity one could be found for each factor in the pattern, Weighted Oblimin gave the best results. When rotating a simple empirical loading matrix, Weighted Oblimin seemed to slightly increase the performance of Direct Oblimin.The author is obliged to Henk A. L. Kiers and three anonymous reviewers for helpful comments on an earlier version of this paper.     

Approximate uniqueness estimates for singular correlation matrices

The residual variance (one minus the squared multiple correlation) is often used as an approximation to the uniqueness in factor analysis. An upper bound approximation to the residual variance is given for the case when the correlation matrix is singular. The approximation is computationally simpler than the exact solution, especially since it can be applied routinely without prior knowledge as to the singularity or nonsingularity of the correlation matrix.     

Pairwise comparisons of trimmed means for two or more groups

The paper takes up the problem of performing all pairwise comparisons amongJ independent groups based on 20% trimmed means. Currently, a method that stands out is the percentile-t bootstrap method where the bootstrap is used to estimate the quantiles of a Studentized maximum modulus distribution when all pairs of population trimmed means are equal. However, a concern is that in simulations, the actual probability of one or more Type I errors can drop well below the nominal level when sample sizes are small. A practical issue is whether a method can be found that corrects this problem while maintaining the positive features of the percentile-t bootstrap. Three new methods are considered here, one of which achieves the desired goal. Another method, which takes advantage of theoretical results by Singh (1998), performs almost as well but is not recommended when the smallest sample size drops below 15. In some situations, however, it gives substantially shorter confidence intervals.     

The extension of component analysis to four-mode matrices

A model for four-mode component analysis is developed and presented. The developed model, which is an extension of Tucker's three-mode factor analytic model, allows for the simultaneous analysis of all modes of a four-mode data matrix and the consideration of relationships among the modes. An empirical example based upon viewer perceptions of repetitive advertising shows the four-mode model applicable to real data.This research was supported by the University of Kansas School of Business Research Fund provided by the Fourth National Bank & Trust Company, Wichita. The ideas and opinions expressed herein are solely those of the author.     

Techniques for rotating two or more loading matrices to optimal agreement and simple structure: A comparison and some technical details

Matrices of factor loadings are often rotated to simple structure. When more than one loading matrix is available for the same variables, the loading matrices can be compared after rotating them all (separately) to simple structure. An alternative procedure is to rotate them to optimal agreement, and then compare them. In the present paper techniques are described that combine these two procedures. Specifically, five techniques that combine the ideals of rotation to optimal agreement and rotation to simple structure are compared on the basis of contrived and empirical data. For the contrived data, it is assessed to what extent the rotations recover the underlying common structure. For both the contrived and the empirical data it is studied to what extent the techniques give well matching rotated matrices, to what extent these have a simple structure, and to what extent the most prominent parts of the different loading matrices agree. It was found that the simple procedure of combining a Generalized Procrustes Analysis (GPA) with Varimax on the mean of the matched loading matrices performs very well on all criteria, and, for most purposes, offers an attractive compromise of rotation to agreement and simple structure. In addition to this comparison, some technical improvements are proposed for Bloxom's rotation to simple structure and maximum similarity.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the author. The author is obliged to René van der Heijden for assistance in programming the procedures in the simulation study reported in this paper, and to Jos ten Berge, three anonymous reviewers and an associate editor for helpful comments on an earlier version of this paper.     

Circumplex models for correlation matrices

Structural models that yield circumplex inequality patterns for the elements of correlation matrices are reviewed. Particular attention is given to a stochastic process defined on the circle proposed by T. W. Anderson. It is shown that the Anderson circumplex contains the Markov Process model for a simplex as a limiting case when a parameter tends to infinity.Anderson's model is intended for correlation matrices with positive elements. A replacement for Anderson's correlation function that permits negative correlations is suggested. It is shown that the resulting model may be reparametrzed as a factor analysis model with nonlinear constraints on the factor loadings. An unrestricted factor analysis, followed by an appropriate rotation, is employed to obtain parameter estimates. These estimates may be used as initial approximations in an iterative procedure to obtain minimum discrepancy estimates.Practical applications are reported.Presented as the 1992 Psychometric Society Presidential Address. I am greatly indebted to Stephen Du Toit for help in the development of the computer program employed here. Part of this research was carried out at the University of South Africa and at the Institute for Statistical Research of the South African Human Sciences Research Council.     

Best univocal estimates of orthogonal common factors

A general formulation is presented for obtaining conditionally unbiased, univocal common-factor score estimates that have maximum validity for the true orthogonal factor scores. We note that although this expression is formally different from both Bartlett's formulation and Heermann's approximate expression, all three, while developed from very different rationales, yield identical results given that the common-factor model holds for the data. Although the true factor score validities can be raised by a different non-orthogonal transformation of orthogonalized regression estimates—as described by Mulaik—the resulting estimates lose their univocality.     

The rank of reduced dispersion matrices

Psychometricians working in factor analysis and econometricians working in regression with measurement error in all variables are both interested in the rank of dispersion matrices under variation of the diagonal elements. Psychometricians concentrate on cases in which low rank can be attained, preferably rank one, the Spearman case. Econometricians cocentrate on cases in which the rank cannot be reduced below the number of variables minus one, the Frisch case. In this paper we give an extensive historial discussion of both fields, we prove the two key results in a more satisfactory and uniform way, we point out various small errors and misunderstandings, and we present a methodological comparison of factor analysis and regression on the basis of our results.Financial support by the Netherlands Organization for the Advancement of Pure Research (ZWO) is gratefully acknowledged.     

The analysis of multitrait-multimethod matrices via constrained components analysis

Multitrait-Multimethod (MTMM) matrices are often analyzed by means of confirmatory factor analysis (CFA). However, fitting MTMM models often leads to improper solutions, or non-convergence. In an attempt to overcome these problems, various alternative CFA models have been proposed, but with none of these the problem of finding improper solutions was solved completely. In the present paper, an approach is proposed where improper solutions are ruled out altogether and convergence is guaranteed. The approach is based on constrained variants of components analysis (CA). Besides the fact that these methods do not give improper solutions, they have the advantage that they provide component scores which can later on be used to relate the components to external variables. The new methods are illustrated by means of simulated data, as well as empirical data sets.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the first author. The authors are obliged to three anonymous reviewers and an associate editor for constructive suggestions on the first version of this paper.     

A note on the factor analysis of partial covariance matrices

A relationship is given between the joint common factor structure of two sets of variables, and the factor structure of the partial covariance matrix of one of the sets with the other partialled out.     

Structural analysis of covariance and correlation matrices

A general approach to the analysis of covariance structures is considered, in which the variances and covariances or correlations of the observed variables are directly expressed in terms of the parameters of interest. The statistical problems of identification, estimation and testing of such covariance or correlation structures are discussed.Several different types of covariance structures are considered as special cases of the general model. These include models for sets of congeneric tests, models for confirmatory and exploratory factor analysis, models for estimation of variance and covariance components, regression models with measurement errors, path analysis models, simplex and circumplex models. Many of the different types of covariance structures are illustrated by means of real data.1978 Psychometric Society Presidential Address.This research has been supported by the Bank of Sweden Tercentenary Foundation under the project entitledStructural Equation Models in the Social Sciences, Karl G. Jöreskog, project director.     

A simple general method for oblique rotation

A simple and very general algorithm for oblique rotation is identified. While motivated by the rotation problem in factor analysis, it may be used to minimize almost any function of a not necessarily square matrix whose columns are restricted to have unit length. The algorithm has two steps. The first is to compute the gradient of the rotation criterion and the second is to project this onto a manifold of matrices with unit length columns. For this reason it is called a gradient projection algorithm. Because the projection step is very simple, implementation of the algorithm involves little more than computing the gradient of the rotation criterion which for many applications is very simple. It is proven that the algorithm is strictly monotone, that is as long as it is not already at a stationary point, each step will decrease the value of the criterion. Examples from a variety of areas are used to demonstrate the algorithm, including oblimin rotation, target rotation, simplimax rotation, and rotation to similarity and simplicity simultaneously. While it may be, the algorithm is not intended for use as a standard algorithm for well established problems, but rather as a tool for investigating new methods where its generality and simplicity may save an investigator substantial effort.The author would like to thank the review team for their insights and recommendations.