Clusterwise simultaneous component analysis for analyzing structural differences in multivariate multiblock data

Many studies yield multivariate multiblock data, that is, multiple data blocks that all involve the same set of variables (e.g., the scores of different groups of subjects on the same set of variables). The question then rises whether the same processes underlie the different data blocks. To explore the structure of such multivariate multiblock data, component analysis can be very useful. Specifically, 2 approaches are often applied: principal component analysis (PCA) on each data block separately and different variants of simultaneous component analysis (SCA) on all data blocks simultaneously. The PCA approach yields a different loading matrix for each data block and is thus not useful for discovering structural similarities. The SCA approach may fail to yield insight into structural differences, since the obtained loading matrix is identical for all data blocks. We introduce a new generic modeling strategy, called clusterwise SCA, that comprises the separate PCA approach and SCA as special cases. The key idea behind clusterwise SCA is that the data blocks form a few clusters, where data blocks that belong to the same cluster are modeled with SCA and thus have the same structure, and different clusters have different underlying structures. In this article, we use the SCA variant that imposes equal average cross-products constraints (ECP). An algorithm for fitting clusterwise SCA-ECP solutions is proposed and evaluated in a simulation study. Finally, the usefulness of clusterwise SCA is illustrated by empirical examples from eating disorder research and social psychology.     

A clusterwise simultaneous component method for capturing within‐cluster differences in component variances and correlations

This paper presents a clusterwise simultaneous component analysis for tracing structural differences and similarities between data of different groups of subjects. This model partitions the groups into a number of clusters according to the covariance structure of the data of each group and performs a simultaneous component analysis with invariant pattern restrictions (SCA‐P) for each cluster. These restrictions imply that the model allows for between‐group differences in the variances and the correlations of the cluster‐specific components. As such, clusterwise SCA‐P is more flexible than the earlier proposed clusterwise SCA‐ECP model, which imposed equal average cross‐products constraints on the component scores of the groups that belong to the same cluster. Using clusterwise SCA‐P, a finer‐grained, yet parsimonious picture of the group differences and similarities can be obtained. An algorithm for fitting clusterwise SCA‐P solutions is presented and its performance is evaluated by means of a simulation study. The value of the model for empirical research is illustrated with data from psychiatric diagnosis research.     

Modeling Differences in the Dimensionality of Multiblock Data by Means of Clusterwise Simultaneous Component Analysis

Given multivariate multiblock data (e.g., subjects nested in groups are measured on multiple variables), one may be interested in the nature and number of dimensions that underlie the variables, and in differences in dimensional structure across data blocks. To this end, clusterwise simultaneous component analysis (SCA) was proposed which simultaneously clusters blocks with a similar structure and performs an SCA per cluster. However, the number of components was restricted to be the same across clusters, which is often unrealistic. In this paper, this restriction is removed. The resulting challenges with respect to model estimation and selection are resolved.     

Simultaneous analysis of coupled data matrices subject to different amounts of noise

In many areas of science, research questions imply the analysis of a set of coupled data blocks, with, for instance, each block being an experimental unit by variable matrix, and the variables being the same in all matrices. To obtain an overall picture of the mechanisms that play a role in the different data matrices, the information in these matrices needs to be integrated. This may be achieved by applying a data‐analytic strategy in which a global model is fitted to all data matrices simultaneously, as in some forms of simultaneous component analysis (SCA). Since such a strategy implies that all data entries, regardless the matrix they belong to, contribute equally to the analysis, it may obfuscate the overall picture of the mechanisms underlying the data when the different data matrices are subject to different amounts of noise. One way out is to downweight entries from noisy data matrices in favour of entries from less noisy matrices. Information regarding the amount of noise that is present in each matrix, however, is, in most cases, not available. To deal with these problems, in this paper a novel maximum‐likelihood‐based simultaneous component analysis method, referred to as MxLSCA, is proposed. Being a stochastic extension of SCA, in MxLSCA the amount of noise in each data matrix is estimated and entries from noisy data matrices are downweighted. Both in an extensive simulation study and in an application to data stemming from cross‐cultural emotion psychology, it is shown that the novel MxLSCA strategy outperforms the SCA strategy with respect to disclosing the mechanisms underlying the coupled data.     

The Special Sign Indeterminacy of the Direct-Fitting Parafac2 Model: Some Implications, Cautions, and Recommendations for Simultaneous Component Analysis

Parafac2 is the most flexible Simultaneous Component Analysis (SCA) model that produces an essentially unique solution. In this paper, we discuss how Parafac2’s special sign indeterminacy affects applications of SCA, and we reveal how an external criterion variable can be used to ensure that estimated Parafac2 weights are meaningfully signed across the levels of the nesting mode. We present an example with real data from clinical psychology that illustrates the importance of Parafac2’s special sign indeterminacy, as well as the effectiveness of our proposed solution. We also discuss the implications of our results for general applications of SCA.     

The LMPCA program: A graphical user interface for fitting the linked-mode PARAFAC-PCA model to coupled real-valued data

In behavioral research, PARAFAC analysis, a three-mode generalization of standard principal component analysis (PCA), is often used to disclose the structure of three-way three-mode data. To get insight into the underlying mechanisms, one often wants to relate the component matrices resulting from such a PARAFAC analysis to external (two-way two-mode) information, regarding one of the modes of the three-way data. To this end, linked-mode PARAFAC-PCA analysis can be used, in which the three-way and the two-way data set, which have one mode in common, are simultaneously analyzed. More specifically, a PARAFAC and a PCA model are fitted to the three-way and the two-way data, respectively, restricting the component matrix for the common mode to be equal in both models. Until now, however, no software program has been publicly available to perform such an analysis. Therefore, in this article, the LMPCA program, a free and easy-to-use MATLAB graphical user interface, is presented to perform a linked-mode PARAFAC-PCA analysis. The LMPCA software can be obtained from the authors at http://ppw.kuleuven.be/okp/software/LMPCA. For users who do not have access to MATLAB, a stand-alone version is provided.     

Accounting for Graded Performance within a Discrete Search Framework

This article presents a process account of some typicality effects and related similarity-dependent accuracy and response time phenomena that arise in the context of supervised concept acquisition. We describe Symbolic Concept Acquisition (SCA), a computational system that acquires and activates category prediction rules. In contrast to gradient representations, SCA performs by probing for prediction rules in a series of discrete steps. For learning new rules, it acquires general rules but then incrementally learns more specific ones. In describing SCA, we emphasize its functionality in terms of accuracy and efficiency and motivate its design within the set of symbolic mechanisms and memory structures defined by the Soar architecture (Laird, Newell & Rosenbloom, 1987). For replicating human behavior, we first show how SCA exhibits some typicality effects in the course of learning responding faster and more accurately to more typical test examples. Then, using data from human experiments, we evaluate SCA's qualitative predictions on accuracy and response time on individual dataset instances. We show how SCA's predictions correlate with human data across three experimental conditions concerning the effect of instruction on learning strategy.     

谱聚类算法在不同属性层级结构诊断评估中的应用

聚类分析已成功用于认知诊断评估(CDA)中,使用广泛的聚类分析方法为K-means算法,有研究已证明K-means在CDA中具有较好的聚类效果。而谱聚类算法通常比K-means分类效果更佳,本研究将谱聚类算法引进CDA,探讨了属性层级结构、属性个数、样本量和失误率对该方法的影响。研究发现：（1）谱聚类算法要比K-means提供更好的聚类结果,尤其在实验条件较苛刻时,谱聚类算法更加稳健;（2）线型结构聚类效果最好,收敛型和发散型相近,独立型结构表现较差;（3）属性个数和失误率增加后,聚类效果会下降;（4）样本量增加后,聚类效果有所提升,但K-means方法有时会有反向结果出现。     

Multilevel component analysis

A general framework for the exploratory component analysis of multilevel data (MLCA) is proposed. In this framework, a separate component model is specified for each group of objects at a certain level. The similarities between the groups of objects at a given level can be expressed by imposing constraints on component models of the groups using the approach adopted in simultaneous component analysis. The constraints used are based on the loading matrices and on the covariances of the component scores of each group. MLCA is related to three‐way component analysis and to currently available multilevel structural equation models. It is shown that the latter are less flexible than MLCA. The use of MLCA is illustrated by means of an empirical example.     

Semi-sparse PCA

It is well known that the classical exploratory factor analysis (EFA) of data with more observations than variables has several types of indeterminacy. We study the factor indeterminacy and show some new aspects of this problem by considering EFA as a specific data matrix decomposition. We adopt a new approach to the EFA estimation and achieve a new characterization of the factor indeterminacy problem. A new alternative model is proposed, which gives determinate factors and can be seen as a semi-sparse principal component analysis (PCA). An alternating algorithm is developed, where in each step a Procrustes problem is solved. It is demonstrated that the new model/algorithm can act as a specific sparse PCA and as a low-rank-plus-sparse matrix decomposition. Numerical examples with several large data sets illustrate the versatility of the new model, and the performance and behaviour of its algorithmic implementation.

     

Estimation of a Latent Variable Regression Growth Curve Model for Individuals Cross-Classified by Clusters

The cross-classified multiple membership latent variable regression (CCMM-LVR) model is a recent extension to the three-level latent variable regression (HM3-LVR) model which can be utilized for longitudinal data that contains individuals who changed clusters over time (for instance, student mobility across schools). The HM3-LVR model can include the initial status on growth effect as varying across those clusters and allows testing of more flexible hypotheses about the influence of initial status on growth and of factors that might impact that relationship, but only in the presence of pure clustering of participants within higher-level units. This Monte Carlo study was conducted to evaluate model estimation under a variety of conditions and to measure the impact of ignoring cross-classified data when estimating the incorrectly specified HM3-LVR model in a scenario in which true values for parameters are known. Furthermore, results from a real-data analysis were used to inform the design of the simulation. Overall, it would be recommended for researchers to utilize the CCMM-LVR model over the HM3-LVR model when individuals are cross-classified, and to use a bare minimum of more than 100 clustering units in order to avoid overestimation of the level-3 variance component estimates.     

Testing the Rasch model by means of the mixture fit index

Rudas, Clogg, and Lindsay (RCL) proposed a new index of fit for contingency table analysis. Using the overparametrized two‐component mixture, where the first component with weight 1?w represents the model to be tested and the second component with weight w is unstructured, the mixture index of fit was defined to be the smallest w compatible with the saturated two‐component mixture. This index of fit, which is insensitive to sample size, is applied to the problem of assessing the fit of the Rasch model. In this application, use is made of the equivalence of the semi‐parametric version of the Rasch model to specifically restricted latent class models. Therefore, the Rasch model can be represented by the structured component of the RCL mixture, with this component itself consisting of two or more subcomponents corresponding to the classes, and the unstructured component capturing the discrepancies between the data and the model. An empirical example demonstrates the application of this approach. Based on four‐item data, the one‐ and two‐class unrestricted latent class models and the one‐ to three‐class models restricted according to the Rasch model are considered, with respect to both their chi‐squared statistics and their mixture fit indices.     

Dimensions of interpersonal orientation: Cross-cultural studies regarding the structure of the ‘DOI Kit’

The DOI Kit is a four-version instrument (Junior Self and Other-Report, and Adult Self-and Other-Report) developed in Spain to measure ‘Dimensions of Interpersonal Orientation’ (Spanish: ‘Dimensiones de Orientación Interpersonal’ or DOI), which are defined as prevalent postures in interpersonal behaviour. The DOI Kit has six first-level scales that collapse into two broad oblique dimensions: Prosocial versus Antisocial Behaviour, and Sociability versus Unsociability.The aim of this article is to find out whether the first- and second-level structure of the Spanish DOI Kit can also be applied to data gathered in Chile and in Germany. A total of eight groups of data were studied. In order to find the best common structure for Spanish, Chilean, and German samples, Simultaneous Component Analysis (SCA) was applied. SCA shows an excellent replication of the original DOI Kit structure. This is also found when separate factor analyses (varimax and oblimin rotations) are performed.     

Hanging in the balance: the role of self-construal abstractness in navigating self-relevant uncertainty

People inevitably face moments of uncertainty as they await feedback regarding self-relevant life outcomes, but they react to this uncertainty with varying amounts of anxiety. Self-construal abstractness (SCA) may be one key predictor of anxiety in the face of uncertain outcomes. SCA refers to a broad self-concept based on generalizations rather than a detailed, low-level self-concept that is based on specific behaviors or events. The current studies examined SCA and anxiety over self-relevant uncertainty. Studies 1 and 2 measured naturally occurring levels of SCA and found that reflecting on an abstract self-construal buffered people from anxiety when upcoming evaluative feedback was highly self-relevant (Study 1) and immediate (Study 2). Study 3 revealed that SCA is equally effective as a buffer against anxiety when manipulated with a subtle prime. The potential for SCA to serve as the target for anxiety-reduction interventions in uncertain situations is discussed.     

Properties of and Algorithms for Fitting Three-Way Component Models with Offset Terms

Prior to a three-way component analysis of a three-way data set, it is customary to preprocess the data by centering and/or rescaling them. Harshman and Lundy (1984) considered that three-way data actually consist of a three-way model part, which in fact pertains to ratio scale measurements, as well as additive “offset” terms that turn the ratio scale measurements into interval scale measurements. They mentioned that such offset terms might be estimated by incorporating additional components in the model, but discarded this idea in favor of an approach to remove such terms from the model by means of centering. Then estimates for the three-way component model parameters are obtained by analyzing the centered data. In the present paper, the possibility of actually estimating the offset terms is taken up again. First, it is mentioned in which cases such offset terms can be estimated uniquely. Next, procedures are offered for estimating model parameters and offset parameters simultaneously, as well as successively (i.e., providing offset term estimates after the three-way model parameters have been estimated in the traditional way on the basis of the centered data). These procedures are provided for both the CANDECOMP/PARAFAC model and the Tucker3 model extended with offset terms. The successive and the simultaneous approaches for estimating model and offset parameters have been compared on the basis of simulated data. It was found that both procedures perform well when the fitted model captures at least all offset terms actually underlying the data. The simultaneous procedures performed slightly better than the successive procedures. If fewer offset terms are fitted than there are underlying the model, the results are considerably poorer, but in these cases the successive procedures performed better than the simultaneous ones. All in all, it can be concluded that the traditional approach for estimating model parameters can hardly be improved upon, and that offset terms can sufficiently well be estimated by the proposed successive approach, which is a simple extension of the traditional approach. The author is obliged to Jos M.F. ten Berge and Marieke Timmerman for helpful comments on an earlier version of this paper. The author is obliged to Iven van Mechelen for making available the data set used in Section 6.     

Principal component analysis of three-mode data by means of alternating least squares algorithms

A new method to estimate the parameters of Tucker's three-mode principal component model is discussed, and the convergence properties of the alternating least squares algorithm to solve the estimation problem are considered. A special case of the general Tucker model, in which the principal component analysis is only performed over two of the three modes is briefly outlined as well. The Miller & Nicely data on the confusion of English consonants are used to illustrate the programs TUCKALS3 and TUCKALS2 which incorporate the algorithms for the two models described.     

On the Measurement of Stability in Over-Time Data

ABSTRACT In this article, autoregressive models and growth curve models are compared Autoregressive models are useful because they allow for random change, permit scores to increase or decrease, and do not require strong assumptions about the level of measurement Three previously presented designs for estimating stability are described (a) time-series, (b) simplex, and (c) two-wave, one-factor methods A two-wave, multiple-factor model also is presented, in which the variables are assumed to be caused by a set of latent variables The factor structure does not change over time and so the synchronous relationships are temporally invariant The factors do not cause each other and have the same stability The parameters of the model are the factor loading structure, each variable's reliability, and the stability of the factors We apply the model to two data sets For eight cognitive skill variables measured at four times, the 2-year stability is estimated to be 92 and the 6-year stability is 83 For nine personality variables, the 3-year stability is 68 We speculate that for many variables there are two components one component that changes very slowly (the trait component) and another that changes very rapidly (the state component), thus each variable is a mixture of trait and state Circumstantial evidence supporting this view is presented     

A comparison of computational driver models using naturalistic and test-track data from cyclist-overtaking manoeuvres

The improvement of advanced driver assistance systems (ADAS) and their safety assessment rely on the understanding of scenario-dependent driving behaviours, such as steering to avoid collisions.This study compares driver models that predict when a driver starts steering away to overtake a cyclist on rural roads. The comparison is among four models: a threshold model, an accumulator model, and two models inspired by a proportional-integral and proportional-integral-derivative controller. These models were tested and cross-applied using two different datasets: one from a naturalistic driving (ND) study and one from a test-track (TT) experiment. Two perceptual variables, expansion rate (the horizontal angular expansion rate of the image of the lead road user on the driver’s retina) and inverse tau (the ratio between the image’s expansion rate and its horizontal optical size), were tested as input to the models. A linear cost function is proposed that can obtain the optimal parameters of the models by computationally efficient linear programming.The results show that the models based on inverse tau fitted the data better than the models that included expansion rate. In general, the models fitted the ND data reasonably well, but not as well the TT data. For the ND data, the models including an accumulative component outperformed the threshold model. For the TT data, due to the poorer fit of the models, more analysis is required to determine the merit of the models. The models fitted to TT data captured the overall pattern of steering onsets in the ND data rather well, but with a persistent bias, probably due to the drivers employing a more cautious strategy in TT.The models compared in this paper may support the virtual safety assessment of ADAS so that driver behaviour may be considered in the design and evaluation of new safety systems.     

Hierarchical classes models for three-way three-mode binary data: interrelations and model selection

Several hierarchical classes models can be considered for the modeling of three-way three-mode binary data, including the INDCLAS model (Leenen, Van Mechelen, De Boeck, and Rosenberg, 1999), the Tucker3-HICLAS model (Ceulemans, Van Mechelen, and Leenen, 2003), the Tucker2-HICLAS model (Ceulemans and Van Mechelen, 2004), and the Tucker1-HICLAS model that is introduced in this paper. Two questions then may be raised: (1) how are these models interrelated, and (2) given a specific data set, which of these models should be selected, and in which rank? In the present paper, we deal with these questions by (1) showing that the distinct hierarchical classes models for three-way three-mode binary data can be organized into a partially ordered hierarchy, and (2) by presenting model selection strategies based on extensions of the well-known scree test and on the Akaike information criterion. The latter strategies are evaluated by means of an extensive simulation study and are illustrated with an application to interpersonal emotion data. Finally, the presented hierarchy and model selection strategies are related to corresponding work by Kiers (1991) for principal component models for three-way three-mode real-valued data.     

Three-Way Tucker2 Component Analysis Solutions of Stimuli × Responses × Individuals Data with Simple Structure and the Fewest Core Differences

Multivariate stimulus-response designs can be described by a three-way array of stimuli by responses by individuals. Its underlying structure can be represented by a network based on the Tucker2 component model in which stimulus components are connected with response components by means of the links that differ between individuals. For each individual such links are represented in a slice of the extended core array. For a proper understanding of these links, it is desirable that [1] the individual core slices as well as the component matrices have simple structures and [2] the differences of core slices between individuals are as few as possible. For attaining [1] and [2] we propose a method in which both the component matrices and the core slices of a Tucker2 solution are transformed simultaneously in order that the component matrices match simple target matrices and the core slices are summarized by a simple target slice. The proposed method is evaluated in a simulation study and illustrated with a three-way data array of semantic differential ratings.