Explicit candecomp/parafac solutions for a contrived 2 × 2 × 2 array of rank three

Kruskal, Harshman and Lundy have contrived a special 2 × 2 × 2 array to examine formal properties of degenerate Candecomp/Parafac solutions. It is shown that for this array the Candecomp/Parafac loss has an infimum of 1. In addition, the array will be used to challenge the tradition of fitting Indscal and related models by means of the Candecomp/Parafac process.     

Degeneracy in Candecomp/Parafac and Indscal Explained For Several Three-Sliced Arrays With A Two-Valued Typical Rank

The Candecomp/Parafac (CP) method decomposes a three-way array into a prespecified number R of rank-1 arrays, by minimizing the sum of squares of the residual array. The practical use of CP is sometimes complicated by the occurrence of so-called degenerate sequences of solutions, in which several rank-1 arrays become highly correlated in all three modes and some elements of the rank-1 arrays become arbitrarily large. We consider the real-valued CP decomposition of all known three-sliced arrays, i.e., of size p×q×3, with a two-valued typical rank. These are the 5×3×3 and 8×4×3 arrays, and the 3×3×4 and 3×3×5 arrays with symmetric 3×3 slices. In the latter two cases, CP is equivalent to the Indscal model. For a typical rank of {m,m+1}, we consider the CP decomposition with R=m of an array of rank m+1. We show that (in most cases) the CP objective function does not have a minimum but an infimum. Moreover, any sequence of feasible CP solutions in which the objective value approaches the infimum will become degenerate. We use the tools developed in Stegeman (2006), who considers p×p×2 arrays, and present a framework of analysis which is of use to the future study of CP degeneracy related to a two-valued typical rank. Moreover, our examples show that CP uniqueness is not necessary for degenerate solutions to occur. The author is supported by the Dutch Organisation for Scientific Research (NWO), VENI grant 451-04-102.     

Degeneracy in Candecomp/Parafac explained for p × p × 2 arrays of rank p + 1 or higher

The Candecomp/Parafac (CP) model decomposes a three-way array into a prespecified number R of rank-1 arrays and a residual array, in which the sum of squares of the residual array is minimized. The practical use of CP is sometimes complicated by the occurrence of so-called degenerate solutions, in which some components are highly correlated in all three modes and the elements of these components become arbitrarily large. We consider the real-valued CP model in which p × p × 2 arrays of rank p + 1 or higher are decomposed into p rank-1 arrays and a residual array. It is shown that the CP objective function does not have a minimum in these cases, but an infimum. Moreover, any sequence of CP approximations, of which the objective value approaches the infimum, will become degenerate. This result extends Ten Berge, Kiers, & De Leeuw (1988), who consider a particular 2 × 2 × 2 array of rank 3. Request for reprints should be sent to Alwin Stegeman, Heijmans Institute of Psychological Research, University of Groningen, Grote Kruisstraat 2/1, 9712 TS Groningen, The Netherlands. The author is obliged to Jos ten Berge and Henk Kiers for helpful comments. Also, the author would like to thank the Associate Editor and the anonymous reviewers for many suggestions on how to improve the contents and the presentation of the paper.     

Constrained Candecomp/Parafac via the Lasso

The Candecomp/Parafac (CP) model is a well-known tool for summarizing a three-way array by extracting a limited number of components. Unfortunately, in some cases, the model suffers from the so-called degeneracy, that is a solution with diverging and uninterpretable components. To avoid degeneracy, orthogonality constraints are usually applied to one of the component matrices. This solves the problem only from a technical point of view because the existence of orthogonal components underlying the data is not guaranteed. For this purpose, we consider some variants of the CP model where the orthogonality constraints are relaxed either by constraining only a pair, or a subset, of components or by stimulating the CP solution to be possibly orthogonal. We theoretically clarify that only the latter approach, based on the least absolute shrinkage and selection operator and named the CP-Lasso, is helpful in solving the degeneracy problem. The results of the application of CP-Lasso on simulated and real life data show its effectiveness.     

Selecting among three‐mode principal component models of different types and complexities: A numerical convex hull based method

Several three‐mode principal component models can be considered for the modelling of three‐way, three‐mode data, including the Candecomp/Parafac, Tucker3, Tucker2, and Tucker1 models. The following question then may be raised: given a specific data set, which of these models should be selected, and at what complexity (i.e. with how many components)? We address this question by proposing a numerical model selection heuristic based on a convex hull. Simulation results show that this heuristic performs almost perfectly, except for Tucker3 data arrays with at least one small mode and a relatively large amount of error.     

The CHIC Model: A Global Model for Coupled Binary Data

Often problems result in the collection of coupled data, which consist of different N-way N-mode data blocks that have one or more modes in common. To reveal the structure underlying such data, an integrated modeling strategy, with a single set of parameters for the common mode(s), that is estimated based on the information in all data blocks, may be most appropriate. Such a strategy implies a global model, consisting of different N-way N-mode submodels, and a global loss function that is a (weighted) sum of the partial loss functions associated with the different submodels. In this paper, such a global model for an integrated analysis of a three-way three-mode binary data array and a two-way two-mode binary data matrix that have one mode in common is presented. A simulated annealing algorithm to estimate the model parameters is described and evaluated in a simulation study. An application of the model to real psychological data is discussed. T. Wilderjans is a Research Assistant of the Fund for Scientific Research—Flanders (Belgium). The research reported in this paper was partially supported by the Research Council of K.U. Leuven (GOA/2005/04). We are grateful to Kristof Vansteelandt for providing us with an interesting data set. We also thank three anonymous reviewers for their useful comments.     

A taxonomy of latent structure assumptions for probability matrix decomposition models

A taxonomy of latent structure assumptions (LSAs) for probability matrix decomposition (PMD) models is proposed which includes the original PMD model (Maris, De Boeck, & Van Mechelen, 1996) as well as a three-way extension of the multiple classification latent class model (Maris, 1999). It is shown that PMD models involving different LSAs are actually restricted latent class models with latent variables that depend on some external variables. For parameter estimation a combined approach is proposed that uses both a mode-finding algorithm (EM) and a sampling-based approach (Gibbs sampling). A simulation study is conducted to investigate the extent to which information criteria, specific model checks, and checks for global goodness of fit may help to specify the basic assumptions of the different PMD models. Finally, an application is described with models involving different latent structure assumptions for data on hostile behavior in frustrating situations.Note: The research reported in this paper was partially supported by the Fund for Scientific Research-Flanders (Belgium) (project G.0207.97 awarded to Paul De Boeck and Iven Van Mechelen), and the Research Fund of K.U. Leuven (F/96/6 fellowship to Andrew Gelman, OT/96/10 project awarded to Iven Van Mechelen and GOA/2000/02 awarded to Paul De Boeck and Iven Van Mechelen). We thank Marcel Croon and Kristof Vansteelandt for commenting on an earlier draft of this paper.     

Convergence properties of an iterative procedure of ipsatizing and standardizing a data matrix,with applications to parafac/candecomp preprocessing

Centering a matrix row-wise and rescaling it column-wise to a unit sum of squares requires an iterative procedure. It is shown that this procedure converges to a stable solution. This solution need not be centered row-wise if the limiting point of the interations is a matrix of rank one. The results of the present paper bear directly on several types of preprocessing methods in Parafac/Candecomp.     

Tucker3 hierarchical classes analysis

This paper presents a new model for binary three-way three-mode data, called Tucker3 hierarchical classes model (Tucker3-HICLAS). This new model generalizes Leenen, Van Mechelen, De Boeck, and Rosenberg's (1999) individual differences hierarchical classes model (INDCLAS). Like the INDCLAS model, the Tucker3-HICLAS model includes a hierarchical classification of the elements of each mode, and a linking structure among the three hierarchies. Unlike INDCLAS, Tucker3-HICLAS (a) does not restrict the hierarchical classifications of the three modes to have the same rank, and (b) allows for more complex linking structures among the three hierarchies. An algorithm to fit the Tucker3-HICLAS model is described and evaluated in an extensive simulation study. An application of the model to hostility data is discussed.The first author is a Research Assistant of the Fund for Scientific Research-Flanders (Belgium). The research reported in this paper was partially supported by the Research Council of K.U. Leuven (GOA/2000/02). We are grateful to Kristof Vansteelandt for providing us with an interesting data set.     

Three-Mode Factor Analysis by Means of Candecomp/Parafac

A three-mode covariance matrix contains covariances of N observations (e.g., subject scores) on J variables for K different occasions or conditions. We model such an JK×JK covariance matrix as the sum of a (common) covariance matrix having Candecomp/Parafac form, and a diagonal matrix of unique variances. The Candecomp/Parafac form is a generalization of the two-mode case under the assumption of parallel factors. We estimate the unique variances by Minimum Rank Factor Analysis. The factors can be chosen oblique or orthogonal. Our approach yields a model that is easy to estimate and easy to interpret. Moreover, the unique variances, the factor covariance matrix, and the communalities are guaranteed to be proper, a percentage of explained common variance can be obtained for each variable-condition combination, and the estimated model is rotationally unique under mild conditions. We apply our model to several datasets in the literature, and demonstrate our estimation procedure in a simulation study.     

Tucker2 hierarchical classes analysis

This paper presents a new hierarchical classes model, called Tucker2-HICLAS, for binary three-way three-mode data. As any three-way hierarchical classes model, the Tucker2-HICLAS model includes a representation of the association relation among the three modes and a hierarchical classification of the elements of each mode. A distinctive feature of the Tucker2-HICLAS model, being closely related to the Tucker3-HICLAS model (Ceulemans, Van Mechelen & Leenen, 2003), is that one of the three modes is minimally reduced and, hence, that the differences among the association patterns of the elements of this mode are maximally retained in the model. Moreover, as compared to Tucker3-HICLAS, Tucker2-HICLAS implies three rather than four different types of parameters and as such is simpler to interpret. Two types of Tucker2-HICLAS models are distinguished: a disjunctive and a conjunctive type. An algorithm for fitting the Tucker2-HICLAS model is described and evaluated in a simulation study. The model is illustrated with longitudinal data on interpersonal emotions. The first author is a Researcher of the Fund for Scientific Research—Flanders (Belgium). The research reported in this paper was partially supported by the Research Council of K.U. Leuven (GOA/2000/02). The authors are grateful to Iwin Leenen for the fruitful discussions.     

Uniqueness of three-mode factor models with sparse cores: The 3 × 3 × 3 case

Three-Mode Factor Analysis (3MFA) and PARAFAC are methods to describe three-way data. Both methods employ models with components for the three modes of a three-way array; the 3MFA model also uses a three-way core array for linking all components to each other. The use of the core array makes the 3MFA model more general than the PARAFAC model (thus allowing a better fit), but also more complicated. Moreover, in the 3MFA model the components are not uniquely determined, and it seems hard to choose among all possible solutions. A particularly interesting feature of the PARAFAC model is that it does give unique components. The present paper introduces a class of 3MFA models in between 3MFA and PARAFAC that share the good properties of the 3MFA model and the PARAFAC model: They fit (almost) as well as the 3MFA model, they are relatively simple and they have the same uniqueness properties as the PARAFAC model.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the first author. Part of this research has been presented at the first conference on ThRee-way methods In Chemistry (TRIC), a meeting of Psychometrics and Chemometrics, Epe, The Netherlands, August 1993. The authors are obliged to Age Smilde for stimulating this research, and two anonymous reviewers for many helpful suggestions.     

The societal and psycho-sociological background of football hooliganism

On October 11, 1985, the Belgian Council of Ministers ordered scientific research on violence on the football terraces. This research was conducted in the years 1986 and 1987 by the “Onderzoeksgroep Jeugdcriminologie” (Research group on Juvenile Criminology) of the K.U. Leuven in close relationship with the general police of the home office. (The report was submitted on December 23, 1987 [VAN LIMBERGEN, COLAERS en WALGRAVE, 1987].)     

The typical rank of tall three-way arrays

The rank of a three-way array refers to the smallest number of rank-one arrays (outer products of three vectors) that generate the array as their sum. It is also the number of components required for a full decomposition of a three-way array by CANDECOMP/PARAFAC. The typical rank of a three-way array refers to the rank a three-way array has almost surely. The present paper deals with typical rank, and generalizes existing results on the typical rank ofI × J × K arrays withK = 2 to a particular class of arrays withK ≥ 2. It is shown that the typical rank isI when the array is tall in the sense thatJK − J < I < JK. In addition, typical rank results are given for the case whereI equalsJK − J. The author is obliged to Henk Kiers, Tom Snijders, and Philip Thijsse for helpful comments.     

The Local Minima Problem in Hierarchical Classes Analysis: An Evaluation of a Simulated Annealing Algorithm and Various Multistart Procedures

Hierarchical classes models are quasi-order retaining Boolean decomposition models for N-way N-mode binary data. To fit these models to data, rationally started alternating least squares (or, equivalently, alternating least absolute deviations) algorithms have been proposed. Extensive simulation studies showed that these algorithms succeed quite well in recovering the underlying truth but frequently end in a local minimum. In this paper we evaluate whether or not this local minimum problem can be mitigated by means of two common strategies for avoiding local minima in combinatorial data analysis: simulated annealing (SA) and use of a multistart procedure. In particular, we propose a generic SA algorithm for hierarchical classes analysis and three different types of random starts. The effectiveness of the SA algorithm and the random starts is evaluated by reanalyzing data sets of previous simulation studies. The reported results support the use of the proposed SA algorithm in combination with a random multistart procedure, regardless of the properties of the data set under study. Eva Ceulemans is a post-doctoral fellow of the Fund for Scientific Research Flanders (Belgium). Iwin Leenen is a post-doctoral researcher of the Spanish Ministerio de Educación y Ciencia (programa Ramón y Cajal). The research reported in this paper was partially supported by the Research Council of K.U. Leuven (GOA/05/04).     

The Assumption of Proportional Components when Candecomp is Applied to Symmetric Matrices in the Context of Indscal

The use of Candecomp to fit scalar products in the context of INDSCAL is based on the assumption that the symmetry of the data matrices involved causes the component matrices to be equal when Candecomp converges. Ten Berge and Kiers gave examples where this assumption is violated for Gramian data matrices. These examples are believed to be local minima. It is now shown that, in the single-component case, the assumption can only be violated at saddle points. Chances of Candecomp converging to a saddle point are small but still nonzero.     

On uniqueness in candecomp/parafac

One of the basic issues in the analysis of three-way arrays by CANDECOMP/PARAFAC (CP) has been the question of uniqueness of the decomposition. Kruskal (1977) has proved that uniqueness is guaranteed when the sum of thek-ranks of the three component matrices involved is at least twice the rank of the solution plus 2. Since then, little has been achieved that might further qualify Kruskal's sufficient condition. Attempts to prove that it is also necessary for uniqueness (except for rank 1 or 2) have failed, but counterexamples to necessity have not been detected. The present paper gives a method for generating the class of all solutions (or at least a subset of that class), given a CP solution that satisfies certain conditions. This offers the possibility to examine uniqueness for a great variety of specific CP solutions. It will be shown that Kruskal's condition is necessary and sufficient when the rank of the solution is three, but that uniqueness may hold even if the condition is not satisfied, when the rank is four or higher. The authors are obliged to Henk Kiers for commenting on a previous draft, and to Tom Snijders for suggesting a proof mentioned in the appendix.     

Hierarchical Classes Modeling of Rating Data

Hierarchical classes (HICLAS) models constitute a distinct family of structural models for N-way N-mode data. All members of the family include N simultaneous and linked classifications of the elements of the N modes implied by the data; those classifications are organized in terms of hierarchical, if–then-type relations. Moreover, the models are accompanied by comprehensive, insightful graphical representations. Up to now, the hierarchical classes family has been limited to dichotomous or dichotomized data. In the present paper we propose a novel extension of the family to two-way two-mode rating data (HICLAS-R). The HICLAS-R model preserves the representation of simultaneous and linked classifications as well as of generalized if–then-type relations, and keeps being accompanied by a comprehensive graphical representation. It is shown to bear interesting relationships with classical real-valued two-way component analysis and with methods of optimal scaling. The research reported in this paper was supported by the Research Fund of the University of Leuven (GOA/00/02 and GOA/05/04) and by the Fund for Scientific Research-Flanders (project G.0146.06). Eva Ceulemans is a Post-doctoral Researcher supported by the Fund for Scientific Research, Flanders. The authors gratefully acknowledge the help of Gert Quintiens and Kaatje Bollaerts in collecting the data used in Section 4 and of Jan Schepers in additional analyses of these data.     

Book Review Section

Books reviewed in this article:
K err , C lark , D unlop , J ohn T., H arbison , F rederick H., and M yers , C harles A. Industrialism and Industrial Man: The Problems of Labor and Management in Economic Growth.
B ennett , W illard E. Manager Selection, Education and Training.
L ivingston , R obert T eviot , and W aite , W illiam W. (Editors) The Manager's Job.
R emitz , U no . Professional Satisfaction Among Swedish Bank Employees.
D ahl , R obert A., H aire , M ason , and L azarsfeld , P aul F. Social Science Research on Business: Product and Potential.
M arcson , S imon . The Scientist in American Industry: Some Organizational Determinants in Manpower Utilization.
C artwright , D orwin , and Z ander , A lvin (Editors) Group Dynamics: Research and Theory.
N euschel , R ichard F. Management by System.
S iegel , S idney , and F ouraker , L awrence E. Bargaining and Group Decision Making: Experiments in Bilateral Monopoly.
S eeman , M elvin . Social Status and Leadership: The Case of the School Executive.