A numerical approach to the approximate and the exact minimum rank of a covariance matrix

A concept of approximate minimum rank for a covariance matrix is defined, which contains the (exact) minimum rank as a special case. A computational procedure to evaluate the approximate minimum rank is offered. The procedure yields those proper communalities for which the unexplained common variance, ignored in low-rank factor analysis, is minimized. The procedure also permits a numerical determination of the exact minimum rank of a covariance matrix, within limits of computational accuracy. A set of 180 covariance matrices with known or bounded minimum rank was analyzed. The procedure was successful throughout in recovering the desired rank.The authors are obliged to Paul Bekker for stimulating and helpful comments.     

The typical rank of tall three-way arrays

The rank of a three-way array refers to the smallest number of rank-one arrays (outer products of three vectors) that generate the array as their sum. It is also the number of components required for a full decomposition of a three-way array by CANDECOMP/PARAFAC. The typical rank of a three-way array refers to the rank a three-way array has almost surely. The present paper deals with typical rank, and generalizes existing results on the typical rank ofI × J × K arrays withK = 2 to a particular class of arrays withK ≥ 2. It is shown that the typical rank isI when the array is tall in the sense thatJK − J < I < JK. In addition, typical rank results are given for the case whereI equalsJK − J. The author is obliged to Henk Kiers, Tom Snijders, and Philip Thijsse for helpful comments.     

Sufficient conditions for uniqueness in Candecomp/Parafac and Indscal with random component matrices

A key feature of the analysis of three-way arrays by Candecomp/Parafac is the essential uniqueness of the trilinear decomposition. We examine the uniqueness of the Candecomp/Parafac and Indscal decompositions. In the latter, the array to be decomposed has symmetric slices. We consider the case where two component matrices are randomly sampled from a continuous distribution, and the third component matrix has full column rank. In this context, we obtain almost sure sufficient uniqueness conditions for the Candecomp/Parafac and Indscal models separately, involving only the order of the three-way array and the number of components in the decomposition. Both uniqueness conditions are closer to necessity than the classical uniqueness condition by Kruskal. Part of this research was supported by (1) the Flemish Government: (a) Research Council K.U. Leuven: GOA-MEFISTO-666, GOA-Ambiorics, (b) F.W.O. project G.0240.99, (c) F.W.O. Research Communities ICCoS and ANMMM, (d) Tournesol project T2004.13; and (2) the Belgian Federal Science Policy Office: IUAP P5/22. Lieven De Lathauwer holds a permanent research position with the French Centre National de la Recherche Scientifique (C.N.R.S.). He also holds an honorary research position with the K.U. Leuven, Leuven, Belgium.     

Statistical inference of minimum rank factor analysis

For any given number of factors, Minimum Rank Factor Analysis yields optimal communalities for an observed covariance matrix in the sense that the unexplained common variance with that number of factors is minimized, subject to the constraint that both the diagonal matrix of unique variances and the observed covariance matrix minus that diagonal matrix are positive semidefinite. As a result, it becomes possible to distinguish the explained common variance from the total common variance. The percentage of explained common variance is similar in meaning to the percentage of explained observed variance in Principal Component Analysis, but typically the former is much closer to 100 than the latter. So far, no statistical theory of MRFA has been developed. The present paper is a first start. It yields closed-form expressions for the asymptotic bias of the explained common variance, or, more precisely, of the unexplained common variance, under the assumption of multivariate normality. Also, the asymptotic variance of this bias is derived, and also the asymptotic covariance matrix of the unique variances that define a MRFA solution. The presented asymptotic statistical inference is based on a recently developed perturbation theory of semidefinite programming. A numerical example is also offered to demonstrate the accuracy of the expressions.This work was supported, in part, by grant DMS-0073770 from the National Science Foundation.     

A generalization of Verhelst's solution for a constrained regression problem in alscal and related MDS-algorithms

Verhelst derived a solution for a constrained regression problem which occurs in the interval measurement application of ALSCAL and related MDS-algorithms. In the present paper it is shown that Verhelst's solution is based on an implicit nonsingularity assumption. A general solution, which contains Verhelst's solution as a special case, is derived by a simple completing-the-squares type approach instead of partial differentiation with a Lagrange multiplier. In addition, this approach permits the identification of a small interval which uniquely contains the optimal value of a parameter needed to solve the special case where Verhelst's solution is valid.The author is obliged to Dirk Knol and Klaas Nevels for helpful comments.     

Computational aspects of the greatest lower bound to the reliability and constrained minimum trace factor analysis

In the last decade several algorithms for computing the greatest lower bound to reliability or the constrained minimum-trace communality solution in factor analysis have been developed. In this paper convergence properties of these methods are examined. Instead of using Lagrange multipliers a new theorem is applied that gives a sufficient condition for a symmetric matrix to be Gramian. Whereas computational pitfalls for two methods suggested by Woodhouse and Jackson can be constructed it is shown that a slightly modified version of one method suggested by Bentler and Woodward can safely be applied to any set of data. A uniqueness proof for the solution desired is offered.The authors are obliged to Charles Lewis and Dirk Knol for helpful comments, and to Frank Brokken and Henk Camstra for developing computer programs.     

A general solution for a class of weakly constrained linear regression problems

This paper contains a globally optimal solution for a class of functions composed of a linear regression function and a penalty function for the sum of squared regression weights. Global optimality is obtained from inequalities rather than from partial derivatives of a Lagrangian function. Applications arise in multidimensional scaling of symmetric or rectangular matrices of squared distances, in Procrustes analysis, and in ridge regression analysis. The similarity of existing solutions for these applications is explained by considering them as special cases of the general class of functions addressed.The author is obliged to Henk Kiers and Willem Heiser for helpful comments.     

Correlation coefficients for more than one scale type: An alternative to the Janson and Vegelius approach

Janson and Vegelius have recently suggested a family of correlations for variables of mixed scale types, including nominal scales. The resulting correlations areE-coefficients, which means that they are unity if the variables involved are identical up to permissible transformations, and that they can be considered as inner products in a Euclidian space. Some of the coefficients of the correlation family suggested by Janson and Vegelius are generalized squared product-moment correlations and some are not. In the present paper, a family of correlations for variables of mixed scale types is advocated all members of which are generalized squared product-moment correlations. Some practical advantages of the latter family are explained. The authors are obliged to Klaas Nevels and Henk Kiers for helpful comments.     

On the equivalence of two oblique congruence rotation methods,and orthogonal approximations

Tucker's method of oblique congruence rotation is shown to be equivalent to a procedure by Meredith. This implies that Monte Carlo studies on congruence by Nesselroade, Baltes and Labouvie and by Korth and Tucker are highly comparable. The problem of rotating two matrices orthogonally to maximal congruence has not yet been solved. An approximate solution to this problem can be derived from Tucker's method. Even better results can be obtained from a Procrustes rotation followed by rotation to simple structure.