Generalized approaches to the maxbet problem and the maxdiff problem,with applications to canonical correlations

Van de Geer has reviewed various criteria for transforming two or more matrices to maximal agreement, subject to orthogonality constraints. The criteria have applications in the context of matching factor or configuration matrices and in the context of canonical correlation analysis for two or more matrices. The present paper summarizes and gives a unified treatment of fully general computational solutions for two of these criteria, Maxbet and Maxdiff. These solutions will be shown to encompass various well-known methods as special cases. It will be argued that the Maxdiff solution should be preferred to the Maxbet solution whenever the two criteria coincide. Horst's Maxcor method will be shown to lack the property of monotone convergence. Finally, simultaneous and successive versions of the Maxbet and Maxdiff solutions will be treated as special cases of a fully flexible approach where the columns of the rotation matrices are obtained in successive blocks.The author is obliged to Henk Kiers for computational assistance and helpful comments.     

Alternating least squares algorithms for simultaneous components analysis with equal component weight matrices in two or more populations

Millsap and Meredith (1988) have developed a generalization of principal components analysis for the simultaneous analysis of a number of variables observed in several populations or on several occasions. The algorithm they provide has some disadvantages. The present paper offers two alternating least squares algorithms for their method, suitable for small and large data sets, respectively. Lower and upper bounds are given for the loss function to be minimized in the Millsap and Meredith method. These can serve to indicate whether or not a global optimum for the simultaneous components analysis problem has been attained.Financial support by the Netherlands organization for scientific research (NWO) is gratefully acknowledged.     

Least-squares approximation of an improper correlation matrix by a proper one

An algorithm is presented for the best least-squares fitting correlation matrix approximating a given missing value or improper correlation matrix. The proposed algorithm is based upon a solution for Mosier's oblique Procrustes rotation problem offered by ten Berge and Nevels. A necessary and sufficient condition is given for a solution to yield the unique global minimum of the least-squares function. Empirical verification of the condition indicates that the occurrence of non-optimal solutions with the proposed algorithm is very unlikely. A possible drawback of the optimal solution is that it is a singular matrix of necessity. In cases where singularity is undesirable, one may impose the additional nonsingularity constraint that the smallest eigenvalue of the solution be , where is an arbitrary small positive constant. Finally, it may be desirable to weight the squared errors of estimation differentially. A generalized solution is derived which satisfies the additional nonsingularity constraint and also allows for weighting. The generalized solution can readily be obtained from the standard unweighted singular solution by transforming the observed improper correlation matrix in a suitable way.     

An alternating least squares method for the weighted approximation of a symmetric matrix

Bailey and Gower examined the least squares approximationC to a symmetric matrixB, when the squared discrepancies for diagonal elements receive specific nonunit weights. They focussed on mathematical properties of the optimalC, in constrained and unconstrained cases, rather than on how to obtainC for any givenB. In the present paper a computational solution is given for the case whereC is constrained to be positive semidefinite and of a fixed rankr or less. The solution is based on weakly constrained linear regression analysis.The authors are obliged to John C. Gower for stimulating this research.     

A simplification of a result by zellini on the maximal rank of symmetric three-way arrays

Zellini (1979, Theorem 3.1) has shown how to decompose an arbitrary symmetric matrix of ordern ×n as a linear combination of 1/2n(n+1) fixed rank one matrices, thus constructing an explicit tensor basis for the set of symmetricn ×n matrices. Zellini's decomposition is based on properties of persymmetric matrices. In the present paper, a simplified tensor basis is given, by showing that a symmetric matrix can also be decomposed in terms of 1/2n(n+1) fixed binary matrices of rank one. The decomposition implies that ann ×n ×p array consisting ofp symmetricn ×n slabs has maximal rank 1/2n(n+1). Likewise, an unconstrained INDSCAL (symmetric CANDECOMP/PARAFAC) decomposition of such an array will yield a perfect fit in 1/2n(n+1) dimensions. When the fitting only pertains to the off-diagonal elements of the symmetric matrices, as is the case in a version of PARAFAC where communalities are involved, the maximal number of dimensions can be further reduced to 1/2n(n–1). However, when the saliences in INDSCAL are constrained to be nonnegative, the tensor basis result does not apply. In fact, it is shown that in this case the number of dimensions needed can be as large asp, the number of matrices analyzed.     

Some uniqueness results for PARAFAC2

Whereas the unique axes properties of PARAFAC1 have been examined extensively, little is known about uniqueness properties for the PARAFAC2 model for covariance matrices. This paper is concerned with uniqueness in the rank two case of PARAFAC2. For this case, Harshman and Lundy have recently shown, subject to mild assumptions, that PARAFAC2 is unique when five (covariance) matrices are analyzed. In the present paper, this result is sharpened. PARAFAC2 is shown to be usually unique with four matrices. With three matrices it is not unique unless a certain additional assumption is introduced. If, for instance, the diagonal matrices of weights are constrained to be non-negative, three matrices are enough to have uniqueness in the rank two case of PARAFAC2. The authors are obliged to Richard Harshman for stimulating this research, and to the Associate Editor and reviewers for suggesting major improvements in the presentation.     

A generalization of Kristof's theorem on the trace of certain matrix products

Kristof has derived a theorem on the maximum and minimum of the trace of matrix products of the form \(X_1 \hat \Gamma _1 X_2 \hat \Gamma _2 \cdots X_n \hat \Gamma _n\) where the matrices \(\hat \Gamma _i\) are diagonal and fixed and theX _i vary unrestrictedly and independently over the set of orthonormal matrices. The theorem is a useful tool in deriving maxima and minima of matrix trace functions subject to orthogonality constraints. The present paper contains a generalization of Kristof's theorem to the case where theX _i are merely required to be submatrices of orthonormal matrices and to have a specified maximum rank. The generalized theorem contains the Schwarz inequality as a special case. Various examples from the psychometric literature, illustrating the practical use of the generalized theorem, are discussed.     

Some clarifications of the CANDECOMP algorithm applied to INDSCAL

Carroll and Chang have claimed that CANDECOMP applied to symmetric matrices yields equivalent coordinate matrices, as needed for INDSCAL. Although this claim has appeared to be valid for all practical purposes, it has gone without a rigorous mathematical footing. The purpose of the present paper is to clarify CANDECOMP in this respect. It is shown that equivalent coordinate matrices are not granted at global minima when the symmetric matrices are not Gramian, or when these matrices are Gramian but the solution not globally optimal.Part of this research has been supported by The Netherlands Organization for Scientific Research (NWO), PSYCHON-grant (560-267-011).