Alternating least squares algorithms for simultaneous components analysis with equal component weight matrices in two or more populations

Millsap and Meredith (1988) have developed a generalization of principal components analysis for the simultaneous analysis of a number of variables observed in several populations or on several occasions. The algorithm they provide has some disadvantages. The present paper offers two alternating least squares algorithms for their method, suitable for small and large data sets, respectively. Lower and upper bounds are given for the loss function to be minimized in the Millsap and Meredith method. These can serve to indicate whether or not a global optimum for the simultaneous components analysis problem has been attained.Financial support by the Netherlands organization for scientific research (NWO) is gratefully acknowledged.     

Penalized optimal scaling for ordinal variables with an application to international classification of functioning core sets

Ordinal data occur frequently in the social sciences. When applying principal component analysis (PCA), however, those data are often treated as numeric, implying linear relationships between the variables at hand; alternatively, non-linear PCA is applied where the obtained quantifications are sometimes hard to interpret. Non-linear PCA for categorical data, also called optimal scoring/scaling, constructs new variables by assigning numerical values to categories such that the proportion of variance in those new variables that is explained by a predefined number of principal components (PCs) is maximized. We propose a penalized version of non-linear PCA for ordinal variables that is a smoothed intermediate between standard PCA on category labels and non-linear PCA as used so far. The new approach is by no means limited to monotonic effects and offers both better interpretability of the non-linear transformation of the category labels and better performance on validation data than unpenalized non-linear PCA and/or standard linear PCA. In particular, an application of penalized optimal scaling to ordinal data as given with the International Classification of Functioning, Disability and Health (ICF) is provided.     

An alternating least squares method for the weighted approximation of a symmetric matrix

Bailey and Gower examined the least squares approximationC to a symmetric matrixB, when the squared discrepancies for diagonal elements receive specific nonunit weights. They focussed on mathematical properties of the optimalC, in constrained and unconstrained cases, rather than on how to obtainC for any givenB. In the present paper a computational solution is given for the case whereC is constrained to be positive semidefinite and of a fixed rankr or less. The solution is based on weakly constrained linear regression analysis.The authors are obliged to John C. Gower for stimulating this research.     

Some uniqueness results for PARAFAC2

Whereas the unique axes properties of PARAFAC1 have been examined extensively, little is known about uniqueness properties for the PARAFAC2 model for covariance matrices. This paper is concerned with uniqueness in the rank two case of PARAFAC2. For this case, Harshman and Lundy have recently shown, subject to mild assumptions, that PARAFAC2 is unique when five (covariance) matrices are analyzed. In the present paper, this result is sharpened. PARAFAC2 is shown to be usually unique with four matrices. With three matrices it is not unique unless a certain additional assumption is introduced. If, for instance, the diagonal matrices of weights are constrained to be non-negative, three matrices are enough to have uniqueness in the rank two case of PARAFAC2. The authors are obliged to Richard Harshman for stimulating this research, and to the Associate Editor and reviewers for suggesting major improvements in the presentation.     

Some clarifications of the CANDECOMP algorithm applied to INDSCAL

Carroll and Chang have claimed that CANDECOMP applied to symmetric matrices yields equivalent coordinate matrices, as needed for INDSCAL. Although this claim has appeared to be valid for all practical purposes, it has gone without a rigorous mathematical footing. The purpose of the present paper is to clarify CANDECOMP in this respect. It is shown that equivalent coordinate matrices are not granted at global minima when the symmetric matrices are not Gramian, or when these matrices are Gramian but the solution not globally optimal.Part of this research has been supported by The Netherlands Organization for Scientific Research (NWO), PSYCHON-grant (560-267-011).     

Construct Validity of Concepts of Attention in Healthy Controls and Patients with CHI

The present study aimed to examine the construct validity of three aspects of attention, namely focused, divided, and supervisory control of attention. Factor-analytic techniques were applied to scores of healthy subjects on a series of neuropsychological tests tapping these aspects of attention. The two components found did not match the hypothesized aspects and were labeled as Memory-driven Action and Stimulus-driven Reaction. The second question was whether the same components could be found in a group of patients with CHI. The pattern of attentional functions found in healthy subjects had changed qualitatively in patients with CHI. A possible explanation for this result in terms of a shift from automatic to controlled processing is discussed.     

Maximization of sums of quotients of quadratic forms and some generalizations

Monotonically convergent algorithms are described for maximizing six (constrained) functions of vectors x, or matricesX with columns x₁, ..., x _r . These functions are h₁(x)= _k (xA _kx)(xC _kx)^–1, H₁(X)= _k tr (XA _k X)(XC _k X)^–1, h₁(X)= _{k l} (x _l A _kx _l ) (x _l C _kx _l )^–1 withX constrained to be columnwise orthonormal, h₂(x)= _k (xA _kx)²(xC _kx)^–1 subject to xx=1, H₂(X)= _k tr(XA _kX)(XA_kX)(XC_kX)^–1 subject toXX=I, and h₂(X)= _{k l} (x _l A _kx _l )² (x _l C _kX _l )^–1 subject toXX=I. In these functions the matricesC _k are assumed to be positive definite. The matricesA _k can be arbitrary square matrices. The general formulation of the functions and the algorithms allows for application of the algorithms in various problems that arise in multivariate analysis. Several applications of the general algorithms are given. Specifically, algorithms are given for reciprocal principal components analysis, binormamin rotation, generalized discriminant analysis, variants of generalized principal components analysis, simple structure rotation for one of the latter variants, and set component analysis. For most of these methods the algorithms appear to be new, for the others the existing algorithms turn out to be special cases of the newly derived general algorithms.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the author. The author is obliged to Jos ten Berge for stimulating this research and for helpful comments on an earlier version of this paper.     

Techniques for rotating two or more loading matrices to optimal agreement and simple structure: A comparison and some technical details

Matrices of factor loadings are often rotated to simple structure. When more than one loading matrix is available for the same variables, the loading matrices can be compared after rotating them all (separately) to simple structure. An alternative procedure is to rotate them to optimal agreement, and then compare them. In the present paper techniques are described that combine these two procedures. Specifically, five techniques that combine the ideals of rotation to optimal agreement and rotation to simple structure are compared on the basis of contrived and empirical data. For the contrived data, it is assessed to what extent the rotations recover the underlying common structure. For both the contrived and the empirical data it is studied to what extent the techniques give well matching rotated matrices, to what extent these have a simple structure, and to what extent the most prominent parts of the different loading matrices agree. It was found that the simple procedure of combining a Generalized Procrustes Analysis (GPA) with Varimax on the mean of the matched loading matrices performs very well on all criteria, and, for most purposes, offers an attractive compromise of rotation to agreement and simple structure. In addition to this comparison, some technical improvements are proposed for Bloxom's rotation to simple structure and maximum similarity.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the author. The author is obliged to René van der Heijden for assistance in programming the procedures in the simulation study reported in this paper, and to Jos ten Berge, three anonymous reviewers and an associate editor for helpful comments on an earlier version of this paper.