Three-mode orthomax rotation

Factor analysis and principal components analysis (PCA) are often followed by an orthomax rotation to rotate a loading matrix to simple structure. The simple structure is usually defined in terms of the simplicity of the columns of the loading matrix. In Three-mode PCA, rotational freedom of the so called core (a three-way array relating components for the three different modes) can be used similarly to find a simple structure of the core. Simple structure of the core can be defined with respect to all three modes simultaneously, possibly with different emphases on the different modes. The present paper provides a fully flexible approach for orthomax rotation of the core to simple structure with respect to three modes simultaneously. Computationally, this approach relies on repeated (two-way) orthomax applied to supermatrices containing the frontal, lateral or horizontal slabs, respectively. The procedure is illustrated by means of a number of exemplary analyses. As a by-product, application of the Three-mode Orthomax procedures to two-way arrays is shown to reveal interesting relations with and interpretations of existing two-way simple structure rotation techniques.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the author. The author is obliged to Jos ten Berge and two anonymous reviewers for useful comments on an earlier version of this paper.     

Constructing bootstrap confidence intervals for principal component loadings in the presence of missing data: a multiple-imputation approach

Earlier research has shown that bootstrap confidence intervals from principal component loadings give a good coverage of the population loadings. However, this only applies to complete data. When data are incomplete, missing data have to be handled before analysing the data. Multiple imputation may be used for this purpose. The question is how bootstrap confidence intervals for principal component loadings should be corrected for multiply imputed data. In this paper, several solutions are proposed. Simulations show that the proposed corrections for multiply imputed data give a good coverage of the population loadings in various situations.     

Retrieving the correlation matrix from a truncated PCA solution: The inverse principal component problem

Whenr Principal Components are available fork variables, the correlation matrix is approximated in the least squares sense by the loading matrix times its transpose. The approximation is generally not perfect unlessr =k. In the present paper it is shown that, whenr is at or above the Ledermann bound,r principal components are enough to perfectly reconstruct the correlation matrix, albeit in a way more involved than taking the loading matrix times its transpose. In certain cases just below the Ledermann bound, recovery of the correlation matrix is still possible when the set of all eigenvalues of the correlation matrix is available as additional information.     

A case of extreme simplicity of the core matrix in three-mode principal components analysis

In three-mode Principal Components Analysis, theP ×Q ×R core matrixG can be transformed to simple structure before it is interpreted. It is well-known that, whenP=QR,G can be transformed to the identity matrix, which implies that all elements become equal to values specified a priori. In the present paper it is shown that, whenP=QR − 1,G can be transformed to have nearly all elements equal to values spectified a priori. A cllsed-form solution for this transformation is offered. Theoretical and practical implications of this simple structure transformation ofG are discussed. Constructive comments from anonymous reviewers are gratefully acknowledged.     

A numerical approach to the approximate and the exact minimum rank of a covariance matrix

A concept of approximate minimum rank for a covariance matrix is defined, which contains the (exact) minimum rank as a special case. A computational procedure to evaluate the approximate minimum rank is offered. The procedure yields those proper communalities for which the unexplained common variance, ignored in low-rank factor analysis, is minimized. The procedure also permits a numerical determination of the exact minimum rank of a covariance matrix, within limits of computational accuracy. A set of 180 covariance matrices with known or bounded minimum rank was analyzed. The procedure was successful throughout in recovering the desired rank.The authors are obliged to Paul Bekker for stimulating and helpful comments.     

Explicit candecomp/parafac solutions for a contrived 2 × 2 × 2 array of rank three

Kruskal, Harshman and Lundy have contrived a special 2 × 2 × 2 array to examine formal properties of degenerate Candecomp/Parafac solutions. It is shown that for this array the Candecomp/Parafac loss has an infimum of 1. In addition, the array will be used to challenge the tradition of fitting Indscal and related models by means of the Candecomp/Parafac process.     

Simplimax: Oblique rotation to an optimal target with simple structure

Factor analysis and principal component analysis are usually followed by simple structure rotations of the loadings. These rotations optimize a certain criterion (e.g., varimax, oblimin), designed to measure the degree of simple structure of the pattern matrix. Simple structure can be considered optimal if a (usually large) number of pattern elements is exactly zero. In the present paper, a class of oblique rotation procedures is proposed to rotate a pattern matrix such that it optimally resembles a matrix which has an exact simple pattern. It is demonstrated that this method can recover relatively complex simple structures where other well-known simple structure rotation techniques fail.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences. The author is obliged to Jos ten Berge for helpful comments on an earlier version.     

Discriminating between strong and weak structures in three‐mode principal component analysis

Recently, a number of model selection heuristics (i.e. DIFFIT, CORCONDIA, the numerical convex hull based heuristic) have been proposed for choosing among Parafac and/or Tucker3 solutions of different complexity for a given three‐way three‐mode data set. Such heuristics are often validated by means of extensive simulation studies. However, these simulation studies are unrealistic in that it is assumed that the variance in real three‐way data can be split into two parts: structural variance, due to a true underlying Parafac or Tucker3 model of low complexity, and random noise. In this paper, we start from the much more reasonable assumption that the variance in any real three‐way data set is due to three different sources: (1) a strong Parafac or Tucker3 structure of low complexity, accounting for a considerable amount of variance, (2) a weak Tucker3 structure, capturing less prominent data aspects, and (3) random noise. As such, Parafac and Tucker3 simulation studies are run in which the data are generated by adding a weak Tucker3 structure to a strong Parafac or Tucker3 one and perturbing the resulting data with random noise. The design of these studies is based on the reanalysis of real data sets. In these studies, the performance of the numerical convex hull based model selection method is evaluated with respect to its capability of discriminating strong from weak underlying structures. The results show that in about two‐thirds of the simulated cases, the hull heuristic yields a model of the same complexity as the strong underlying structure and thus succeeds in disentangling strong and weak underlying structures. In the vast majority of the remaining third, this heuristic selects a solution that combines the strong structure and (part of) the weak structure.     

Maximization of sums of quotients of quadratic forms and some generalizations

Monotonically convergent algorithms are described for maximizing six (constrained) functions of vectors x, or matricesX with columns x₁, ..., x _r . These functions are h₁(x)= _k (xA _kx)(xC _kx)^–1, H₁(X)= _k tr (XA _k X)(XC _k X)^–1, h₁(X)= _{k l} (x _l A _kx _l ) (x _l C _kx _l )^–1 withX constrained to be columnwise orthonormal, h₂(x)= _k (xA _kx)²(xC _kx)^–1 subject to xx=1, H₂(X)= _k tr(XA _kX)(XA_kX)(XC_kX)^–1 subject toXX=I, and h₂(X)= _{k l} (x _l A _kx _l )² (x _l C _kX _l )^–1 subject toXX=I. In these functions the matricesC _k are assumed to be positive definite. The matricesA _k can be arbitrary square matrices. The general formulation of the functions and the algorithms allows for application of the algorithms in various problems that arise in multivariate analysis. Several applications of the general algorithms are given. Specifically, algorithms are given for reciprocal principal components analysis, binormamin rotation, generalized discriminant analysis, variants of generalized principal components analysis, simple structure rotation for one of the latter variants, and set component analysis. For most of these methods the algorithms appear to be new, for the others the existing algorithms turn out to be special cases of the newly derived general algorithms.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the author. The author is obliged to Jos ten Berge for stimulating this research and for helpful comments on an earlier version of this paper.     

An alternating least squares algorithm for fitting the two- and three-way dedicom model and the idioscal model

The DEDICOM model is a model for representing asymmetric relations among a set of objects by means of a set of coordinates for the objects on a limited number of dimensions. The present paper offers an alternating least squares algorithm for fitting the DEDICOM model. The model can be generalized to represent any number of sets of relations among the same set of objects. An algorithm for fitting this three-way DEDICOM model is provided as well. Based on the algorithm for the three-way DEDICOM model an algorithm is developed for fitting the IDIOSCAL model in the least squares sense.The author is obliged to Jos ten Berge and Richard Harshman.