On uniqueness in candecomp/parafac

One of the basic issues in the analysis of three-way arrays by CANDECOMP/PARAFAC (CP) has been the question of uniqueness of the decomposition. Kruskal (1977) has proved that uniqueness is guaranteed when the sum of thek-ranks of the three component matrices involved is at least twice the rank of the solution plus 2. Since then, little has been achieved that might further qualify Kruskal's sufficient condition. Attempts to prove that it is also necessary for uniqueness (except for rank 1 or 2) have failed, but counterexamples to necessity have not been detected. The present paper gives a method for generating the class of all solutions (or at least a subset of that class), given a CP solution that satisfies certain conditions. This offers the possibility to examine uniqueness for a great variety of specific CP solutions. It will be shown that Kruskal's condition is necessary and sufficient when the rank of the solution is three, but that uniqueness may hold even if the condition is not satisfied, when the rank is four or higher. The authors are obliged to Henk Kiers for commenting on a previous draft, and to Tom Snijders for suggesting a proof mentioned in the appendix.     

Uniqueness of three-mode factor models with sparse cores: The 3 × 3 × 3 case

Three-Mode Factor Analysis (3MFA) and PARAFAC are methods to describe three-way data. Both methods employ models with components for the three modes of a three-way array; the 3MFA model also uses a three-way core array for linking all components to each other. The use of the core array makes the 3MFA model more general than the PARAFAC model (thus allowing a better fit), but also more complicated. Moreover, in the 3MFA model the components are not uniquely determined, and it seems hard to choose among all possible solutions. A particularly interesting feature of the PARAFAC model is that it does give unique components. The present paper introduces a class of 3MFA models in between 3MFA and PARAFAC that share the good properties of the 3MFA model and the PARAFAC model: They fit (almost) as well as the 3MFA model, they are relatively simple and they have the same uniqueness properties as the PARAFAC model.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the first author. Part of this research has been presented at the first conference on ThRee-way methods In Chemistry (TRIC), a meeting of Psychometrics and Chemometrics, Epe, The Netherlands, August 1993. The authors are obliged to Age Smilde for stimulating this research, and two anonymous reviewers for many helpful suggestions.     

Some additional results on principal components analysis of three-mode data by means of alternating least squares algorithms

Kroonenberg and de Leeuw (1980) have developed an alternating least-squares method TUCKALS-3 as a solution for Tucker's three-way principal components model. The present paper offers some additional features of their method. Starting from a reanalysis of Tucker's problem in terms of a rank-constrained regression problem, it is shown that the fitted sum of squares in TUCKALS-3 can be partitioned according to elements of each mode of the three-way data matrix. An upper bound to the total fitted sum of squares is derived. Finally, a special case of TUCKALS-3 is related to the Carroll/Harshman CANDECOMP/PARAFAC model.     

A simplification of a result by zellini on the maximal rank of symmetric three-way arrays

Zellini (1979, Theorem 3.1) has shown how to decompose an arbitrary symmetric matrix of ordern ×n as a linear combination of 1/2n(n+1) fixed rank one matrices, thus constructing an explicit tensor basis for the set of symmetricn ×n matrices. Zellini's decomposition is based on properties of persymmetric matrices. In the present paper, a simplified tensor basis is given, by showing that a symmetric matrix can also be decomposed in terms of 1/2n(n+1) fixed binary matrices of rank one. The decomposition implies that ann ×n ×p array consisting ofp symmetricn ×n slabs has maximal rank 1/2n(n+1). Likewise, an unconstrained INDSCAL (symmetric CANDECOMP/PARAFAC) decomposition of such an array will yield a perfect fit in 1/2n(n+1) dimensions. When the fitting only pertains to the off-diagonal elements of the symmetric matrices, as is the case in a version of PARAFAC where communalities are involved, the maximal number of dimensions can be further reduced to 1/2n(n–1). However, when the saliences in INDSCAL are constrained to be nonnegative, the tensor basis result does not apply. In fact, it is shown that in this case the number of dimensions needed can be as large asp, the number of matrices analyzed.     

Some clarifications of the TUCKALS2 algorithm applied to the IDIOSCAL problem

Kroonenberg and de Leeuw have suggested fitting the IDIOSCAL model by the TUCKALS2 algorithm for three-way components analysis. In theory, this is problematic because TUCKALS2 produces two possibly different coordinate matrices, that are useless for IDIOSCAL unless they are equal. Kroonenberg has claimed that, when IDIOSCAL is fitted by TUCKALS2, the resulting coordinate matrices will be identical. In the present paper, this claim is proven valid when the data matrices are semidefinite. However, counterexamples for indefinite matrices are also constructed, by examining the global minimum in the case where the data matrices have the same eigenvectors. Similar counterexamples have been considered by ten Berge and Kiers in the related context of CANDECOMP/PARAFAC to fit the INDSCAL model.     

A note on the identification of restricted factor loading matrices

It is shown that problems of rotational equivalence of restricted factor loading matrices in orthogonal factor analysis are equivalent to problems of identification in simultaneous equations systems with covariance restrictions. A necessary (under a regularity assumption) and sufficient condition for local uniqueness is given and a counterexample is provided to a theorem by J. Algina concerning necessary and sufficient conditions for global uniqueness.     

A numerical approach to the approximate and the exact minimum rank of a covariance matrix

A concept of approximate minimum rank for a covariance matrix is defined, which contains the (exact) minimum rank as a special case. A computational procedure to evaluate the approximate minimum rank is offered. The procedure yields those proper communalities for which the unexplained common variance, ignored in low-rank factor analysis, is minimized. The procedure also permits a numerical determination of the exact minimum rank of a covariance matrix, within limits of computational accuracy. A set of 180 covariance matrices with known or bounded minimum rank was analyzed. The procedure was successful throughout in recovering the desired rank.The authors are obliged to Paul Bekker for stimulating and helpful comments.     

The typical rank of tall three-way arrays

The rank of a three-way array refers to the smallest number of rank-one arrays (outer products of three vectors) that generate the array as their sum. It is also the number of components required for a full decomposition of a three-way array by CANDECOMP/PARAFAC. The typical rank of a three-way array refers to the rank a three-way array has almost surely. The present paper deals with typical rank, and generalizes existing results on the typical rank ofI × J × K arrays withK = 2 to a particular class of arrays withK ≥ 2. It is shown that the typical rank isI when the array is tall in the sense thatJK − J < I < JK. In addition, typical rank results are given for the case whereI equalsJK − J. The author is obliged to Henk Kiers, Tom Snijders, and Philip Thijsse for helpful comments.     

Sensitivity analysis in factor analysis: Difference between using covariance and correlation matrices

Influence curves of some parameters under various methods of factor analysis have been given in the literature. These influence curves depend on the influence curves for either the covariance or the correlation matrix used in the analysis. The differences between the influence curves based on the covariance and the correlation matrices are derived in this paper. Simple formulas for the differences of the influence curves, based on the two matrices, for the unique variance matrix, factor loadings and some other parameter are obtained under scale-invariant estimation methods, though the influence curves themselves are in complex forms.The authors are most grateful to the referees, the Associate Editor, the Editor and Raymond Lam for helpful suggestions for improving the clarity of the paper.     

Explicit candecomp/parafac solutions for a contrived 2 × 2 × 2 array of rank three

Kruskal, Harshman and Lundy have contrived a special 2 × 2 × 2 array to examine formal properties of degenerate Candecomp/Parafac solutions. It is shown that for this array the Candecomp/Parafac loss has an infimum of 1. In addition, the array will be used to challenge the tradition of fitting Indscal and related models by means of the Candecomp/Parafac process.     

Kruskal's polynomial for 2×2×2 arrays and a generalization to 2×n×n arrays

A remarkable difference between the concept of rank for matrices and that for three-way arrays has to do with the occurrence of non-maximal rank. The set ofn×n matrices that have a rank less thann has zero volume. Kruskal pointed out that a 2×2×2 array has rank three or less, and that the subsets of those 2×2×2 arrays for which the rank is two or three both have positive volume. These subsets can be distinguished by the roots of a certain polynomial. The present paper generalizes Kruskal's results to 2×n×n arrays. Incidentally, it is shown that twon ×n matrices can be diagonalized simultaneously with positive probability.The author is obliged to Joe Kruskal and Henk Kiers for commenting on an earlier draft, and to Tom Wansbeek for raising stimulating questions.     

On Green's best linear composites with a specified structure,and oblique estimates of factor scores

Green solved the problem of least-squares estimation of several criteria subject to the constraint that the estimates have an arbitrary fixed covariance or correlation matrix. In the present paper an omission in Green's proof is discussed and resolved. Furthermore, it is shown that some recently published solutions for estimating oblique factor scores are special cases of Green's solution for the case of fixed covariance matrices.     

Rank-reducibility of a symmetric matrix and sampling theory of minimum trace factor analysis

One of the intriguing questions of factor analysis is the extent to which one can reduce the rank of a symmetric matrix by only changing its diagonal entries. We show in this paper that the set of matrices, which can be reduced to rankr, has positive (Lebesgue) measure if and only ifr is greater or equal to the Ledermann bound. In other words the Ledermann bound is shown to bealmost surely the greatest lower bound to a reduced rank of the sample covariance matrix. Afterwards an asymptotic sampling theory of so-called minimum trace factor analysis (MTFA) is proposed. The theory is based on continuous and differential properties of functions involved in the MTFA. Convex analysis techniques are utilized to obtain conditions for differentiability of these functions.     

Three‐mode analysis of multimode covariance matrices

Multimode covariance matrices, such as multitrait‐multimethod matrices, contain the covariances of subject scores on variables for different occasions or conditions. This paper presents a comparison of three‐mode component analysis and three‐mode factor analysis applied to such covariance matrices. The differences and similarities between the non‐stochastic and stochastic approaches are demonstrated by two examples, one of which has a longitudinal design. The empirical comparison is facilitated by deriving, as a heuristic device, a statistic based on the maximum likelihood function for three‐mode factor analysis and its associated degrees of freedom for the three‐mode component models. Furthermore, within the present context a case is made for interpreting the core array as second‐order components.     

Identification with deficient rank loading matrices in confirmatory factor analysis: Multitrait-multimethod models

This paper presents some results on identification in multitrait-multimethod (MTMM) confirmatory factor analysis (CFA) models. Some MTMM models are not identified when the (factorial-patterned) loadings matrix is of deficient column rank. For at least one other MTMM model, identification does exist despite such deficiency. It is also shown that for some MTMM CFA models, Howe's (1955) conditions sufficient for rotational uniqueness can fail, yet the model may well be identified and rotationally unique. Implications of these results for CFA models in general are discussed.     

The rank of reduced dispersion matrices

Psychometricians working in factor analysis and econometricians working in regression with measurement error in all variables are both interested in the rank of dispersion matrices under variation of the diagonal elements. Psychometricians concentrate on cases in which low rank can be attained, preferably rank one, the Spearman case. Econometricians cocentrate on cases in which the rank cannot be reduced below the number of variables minus one, the Frisch case. In this paper we give an extensive historial discussion of both fields, we prove the two key results in a more satisfactory and uniform way, we point out various small errors and misunderstandings, and we present a methodological comparison of factor analysis and regression on the basis of our results.Financial support by the Netherlands Organization for the Advancement of Pure Research (ZWO) is gratefully acknowledged.     

Sufficient conditions for uniqueness in Candecomp/Parafac and Indscal with random component matrices

A key feature of the analysis of three-way arrays by Candecomp/Parafac is the essential uniqueness of the trilinear decomposition. We examine the uniqueness of the Candecomp/Parafac and Indscal decompositions. In the latter, the array to be decomposed has symmetric slices. We consider the case where two component matrices are randomly sampled from a continuous distribution, and the third component matrix has full column rank. In this context, we obtain almost sure sufficient uniqueness conditions for the Candecomp/Parafac and Indscal models separately, involving only the order of the three-way array and the number of components in the decomposition. Both uniqueness conditions are closer to necessity than the classical uniqueness condition by Kruskal. Part of this research was supported by (1) the Flemish Government: (a) Research Council K.U. Leuven: GOA-MEFISTO-666, GOA-Ambiorics, (b) F.W.O. project G.0240.99, (c) F.W.O. Research Communities ICCoS and ANMMM, (d) Tournesol project T2004.13; and (2) the Belgian Federal Science Policy Office: IUAP P5/22. Lieven De Lathauwer holds a permanent research position with the French Centre National de la Recherche Scientifique (C.N.R.S.). He also holds an honorary research position with the K.U. Leuven, Leuven, Belgium.     

Some Mathematical Properties of the Matrix Decomposition Solution in Factor Analysis

A new factor analysis (FA) procedure has recently been proposed which can be called matrix decomposition FA (MDFA). All FA model parameters (common and unique factors, loadings, and unique variances) are treated as fixed unknown matrices. Then, the MDFA model simply becomes a specific data matrix decomposition. The MDFA parameters are found by minimizing the discrepancy between the data and the MDFA model. Several algorithms have been developed and some properties have been discussed in the literature (notably by Stegeman in Comput Stat Data Anal 99:189–203, 2016), but, as a whole, MDFA has not been studied fully yet. A number of new properties are discovered in this paper, and some existing ones are derived more explicitly. The properties provided concern the uniqueness of results, covariances among common factors, unique factors, and residuals, and assessment of the degree of indeterminacy of common and unique factor scores. The properties are illustrated using a real data example.     

A Class of Population Covariance Matrices in the Bootstrap Approach to Covariance Structure Analysis

Model evaluation in covariance structure analysis is critical before the results can be trusted. Due to finite sample sizes and unknown distributions of real data, existing conclusions regarding a particular statistic may not be applicable in practice. The bootstrap procedure automatically takes care of the unknown distribution and, for a given sample size, also provides more accurate results than those based on standard asymptotics. But the procedure needs a matrix to play the role of the population covariance matrix. The closer the matrix is to the true population covariance matrix, the more valid the bootstrap inference is. The current paper proposes a class of covariance matrices by combining theory and data. Thus, a proper matrix from this class is closer to the true population covariance matrix than those constructed by any existing methods. Each of the covariance matrices is easy to generate and also satisfies several desired properties. An example with nine cognitive variables and a confirmatory factor model illustrates the details for creating population covariance matrices with different misspecifications. When evaluating the substantive model, bootstrap or simulation procedures based on these matrices will lead to more accurate conclusion than that based on artificial covariance matrices.     

Constrained least squares estimators of oblique common factors

An expression is given for weighted least squares estimators of oblique common factors, constrained to have the same covariance matrix as the factors they estimate. It is shown that if as in exploratory factor analysis, the common factors are obtained by oblique transformation from the Lawley-Rao basis, the constrained estimators are given by the same transformation. Finally a proof of uniqueness is given.The research reported in this paper was partly supported by Natural Sciences and Engineering Research Council Grant No. A6346.