Clusterwise simultaneous component analysis for analyzing structural differences in multivariate multiblock data

Many studies yield multivariate multiblock data, that is, multiple data blocks that all involve the same set of variables (e.g., the scores of different groups of subjects on the same set of variables). The question then rises whether the same processes underlie the different data blocks. To explore the structure of such multivariate multiblock data, component analysis can be very useful. Specifically, 2 approaches are often applied: principal component analysis (PCA) on each data block separately and different variants of simultaneous component analysis (SCA) on all data blocks simultaneously. The PCA approach yields a different loading matrix for each data block and is thus not useful for discovering structural similarities. The SCA approach may fail to yield insight into structural differences, since the obtained loading matrix is identical for all data blocks. We introduce a new generic modeling strategy, called clusterwise SCA, that comprises the separate PCA approach and SCA as special cases. The key idea behind clusterwise SCA is that the data blocks form a few clusters, where data blocks that belong to the same cluster are modeled with SCA and thus have the same structure, and different clusters have different underlying structures. In this article, we use the SCA variant that imposes equal average cross-products constraints (ECP). An algorithm for fitting clusterwise SCA-ECP solutions is proposed and evaluated in a simulation study. Finally, the usefulness of clusterwise SCA is illustrated by empirical examples from eating disorder research and social psychology.     

A clusterwise simultaneous component method for capturing within‐cluster differences in component variances and correlations

This paper presents a clusterwise simultaneous component analysis for tracing structural differences and similarities between data of different groups of subjects. This model partitions the groups into a number of clusters according to the covariance structure of the data of each group and performs a simultaneous component analysis with invariant pattern restrictions (SCA‐P) for each cluster. These restrictions imply that the model allows for between‐group differences in the variances and the correlations of the cluster‐specific components. As such, clusterwise SCA‐P is more flexible than the earlier proposed clusterwise SCA‐ECP model, which imposed equal average cross‐products constraints on the component scores of the groups that belong to the same cluster. Using clusterwise SCA‐P, a finer‐grained, yet parsimonious picture of the group differences and similarities can be obtained. An algorithm for fitting clusterwise SCA‐P solutions is presented and its performance is evaluated by means of a simulation study. The value of the model for empirical research is illustrated with data from psychiatric diagnosis research.     

Simultaneous Component Analysis by Means of Tucker3

A new model for simultaneous component analysis (SCA) is introduced that contains the existing SCA models with common loading matrix as special cases. The new SCA-T3 model is a multi-set generalization of the Tucker3 model for component analysis of three-way data. For each mode (observational units, variables, sets) a different number of components can be chosen and the obtained solution can be rotated without loss of fit to facilitate interpretation. SCA-T3 can be fitted on centered multi-set data and also on the corresponding covariance matrices. For this purpose, alternating least squares algorithms are derived. SCA-T3 is evaluated in a simulation study, and its practical merits are demonstrated for several benchmark datasets.     

A Tabu-Search Heuristic for Deterministic Two-Mode Blockmodeling of Binary Network Matrices

Two-mode binary data matrices arise in a variety of social network contexts, such as the attendance or non-attendance of individuals at events, the participation or lack of participation of groups in projects, and the votes of judges on cases. A popular method for analyzing such data is two-mode blockmodeling based on structural equivalence, where the goal is to identify partitions for the row and column objects such that the clusters of the row and column objects form blocks that are either complete (all 1s) or null (all 0s) to the greatest extent possible. Multiple restarts of an object relocation heuristic that seeks to minimize the number of inconsistencies (i.e., 1s in null blocks and 0s in complete blocks) with ideal block structure is the predominant approach for tackling this problem. As an alternative, we propose a fast and effective implementation of tabu search. Computational comparisons across a set of 48 large network matrices revealed that the new tabu-search heuristic always provided objective function values that were better than those of the relocation heuristic when the two methods were constrained to the same amount of computation time.     

Simultaneous analysis of coupled data matrices subject to different amounts of noise

In many areas of science, research questions imply the analysis of a set of coupled data blocks, with, for instance, each block being an experimental unit by variable matrix, and the variables being the same in all matrices. To obtain an overall picture of the mechanisms that play a role in the different data matrices, the information in these matrices needs to be integrated. This may be achieved by applying a data‐analytic strategy in which a global model is fitted to all data matrices simultaneously, as in some forms of simultaneous component analysis (SCA). Since such a strategy implies that all data entries, regardless the matrix they belong to, contribute equally to the analysis, it may obfuscate the overall picture of the mechanisms underlying the data when the different data matrices are subject to different amounts of noise. One way out is to downweight entries from noisy data matrices in favour of entries from less noisy matrices. Information regarding the amount of noise that is present in each matrix, however, is, in most cases, not available. To deal with these problems, in this paper a novel maximum‐likelihood‐based simultaneous component analysis method, referred to as MxLSCA, is proposed. Being a stochastic extension of SCA, in MxLSCA the amount of noise in each data matrix is estimated and entries from noisy data matrices are downweighted. Both in an extensive simulation study and in an application to data stemming from cross‐cultural emotion psychology, it is shown that the novel MxLSCA strategy outperforms the SCA strategy with respect to disclosing the mechanisms underlying the coupled data.     

Clusterwise HICLAS: A generic modeling strategy to trace similarities and differences in multiblock binary data

In many areas of the behavioral sciences, different groups of objects are measured on the same set of binary variables, resulting in coupled binary object × variable data blocks. Take, as an example, success/failure scores for different samples of testees, with each sample belonging to a different country, regarding a set of test items. When dealing with such data, a key challenge consists of uncovering the differences and similarities between the structural mechanisms that underlie the different blocks. To tackle this challenge for the case of a single data block, one may rely on HICLAS, in which the variables are reduced to a limited set of binary bundles that represent the underlying structural mechanisms, and the objects are given scores for these bundles. In the case of multiple binary data blocks, one may perform HICLAS on each data block separately. However, such an analysis strategy obscures the similarities and, in the case of many data blocks, also the differences between the blocks. To resolve this problem, we proposed the new Clusterwise HICLAS generic modeling strategy. In this strategy, the different data blocks are assumed to form a set of mutually exclusive clusters. For each cluster, different bundles are derived. As such, blocks belonging to the same cluster have the same bundles, whereas blocks of different clusters are modeled with different bundles. Furthermore, we evaluated the performance of Clusterwise HICLAS by means of an extensive simulation study and by applying the strategy to coupled binary data regarding emotion differentiation and regulation.     

The Special Sign Indeterminacy of the Direct-Fitting Parafac2 Model: Some Implications, Cautions, and Recommendations for Simultaneous Component Analysis

Parafac2 is the most flexible Simultaneous Component Analysis (SCA) model that produces an essentially unique solution. In this paper, we discuss how Parafac2’s special sign indeterminacy affects applications of SCA, and we reveal how an external criterion variable can be used to ensure that estimated Parafac2 weights are meaningfully signed across the levels of the nesting mode. We present an example with real data from clinical psychology that illustrates the importance of Parafac2’s special sign indeterminacy, as well as the effectiveness of our proposed solution. We also discuss the implications of our results for general applications of SCA.     

A Network Approach for Evaluating Coherence in Multivariate Systems: An Application to Psychophysiological Emotion Data

We present an approach for evaluating coherence in multivariate systems that considers all the variables simultaneously. We operationalize the multivariate system as a network and define coherence as the efficiency with which a signal is transmitted throughout the network. We illustrate this approach with time series data from 15 psychophysiological signals representing individuals’ moment-by-moment emotional reactions to emotional films. First, we summarize the time series through nonparametric Receiver Operating Characteristic (ROC) curves. Second, we use Spearman rank correlations to calculate relationships between each pair of variables. Third, based on the obtained associations, we construct a network using the variables as nodes. Finally, we examine signal transmission through all the nodes in the network. Our results indicate that the network consisting of the 15 psychophysiological signals has a small-world structure, with three clusters of variables and strong within-cluster connections. This structure supports an effective signal transmission across the entire network. When compared across experimental conditions, our results indicate that coherence is relatively stronger for intense emotional stimuli than for neutral stimuli. These findings are discussed in relation to multivariate methods and emotion theories.     

Quantification of three-dimensional structures

A mathematical model, based on additive subcomponents of grouping, subitizing and adding, was derived to account for quantification latencies of three-dimensional block arrangements. Subitizing is the process that people use to directly quantify a small number of objects without counting. It was found that most people consistently subitized up to four blocks. With more than four blocks, people resorted to grouping and adding, and the model was able to account for these data. The structural variables of compactness, symmetry, linearity, and planarity were shown to have small effects on quantification latencies relative to the large effect of number of blocks. Of these structural variables, compactness had the largest effect, and in terms of the model, it is suggested that visual structure had its effect on the perceptual grouping subcomponent.     

谱聚类算法在不同属性层级结构诊断评估中的应用

聚类分析已成功用于认知诊断评估(CDA)中,使用广泛的聚类分析方法为K-means算法,有研究已证明K-means在CDA中具有较好的聚类效果。而谱聚类算法通常比K-means分类效果更佳,本研究将谱聚类算法引进CDA,探讨了属性层级结构、属性个数、样本量和失误率对该方法的影响。研究发现：（1）谱聚类算法要比K-means提供更好的聚类结果,尤其在实验条件较苛刻时,谱聚类算法更加稳健;（2）线型结构聚类效果最好,收敛型和发散型相近,独立型结构表现较差;（3）属性个数和失误率增加后,聚类效果会下降;（4）样本量增加后,聚类效果有所提升,但K-means方法有时会有反向结果出现。     

Dimensions of interpersonal orientation: Cross-cultural studies regarding the structure of the ‘DOI Kit’

The DOI Kit is a four-version instrument (Junior Self and Other-Report, and Adult Self-and Other-Report) developed in Spain to measure ‘Dimensions of Interpersonal Orientation’ (Spanish: ‘Dimensiones de Orientación Interpersonal’ or DOI), which are defined as prevalent postures in interpersonal behaviour. The DOI Kit has six first-level scales that collapse into two broad oblique dimensions: Prosocial versus Antisocial Behaviour, and Sociability versus Unsociability.The aim of this article is to find out whether the first- and second-level structure of the Spanish DOI Kit can also be applied to data gathered in Chile and in Germany. A total of eight groups of data were studied. In order to find the best common structure for Spanish, Chilean, and German samples, Simultaneous Component Analysis (SCA) was applied. SCA shows an excellent replication of the original DOI Kit structure. This is also found when separate factor analyses (varimax and oblimin rotations) are performed.     

CHull as an alternative to AIC and BIC in the context of mixtures of factor analyzers

Mixture analysis is commonly used for clustering objects on the basis of multivariate data. When the data contain a large number of variables, regular mixture analysis may become problematic, because a large number of parameters need to be estimated for each cluster. To tackle this problem, the mixtures-of-factor-analyzers (MFA) model was proposed, which combines clustering with exploratory factor analysis. MFA model selection is rather intricate, as both the number of clusters and the number of underlying factors have to be determined. To this end, the Akaike (AIC) and Bayesian (BIC) information criteria are often used. AIC and BIC try to identify a model that optimally balances model fit and model complexity. In this article, the CHull (Ceulemans & Kiers, 2006) method, which also balances model fit and complexity, is presented as an interesting alternative model selection strategy for MFA. In an extensive simulation study, the performances of AIC, BIC, and CHull were compared. AIC performs poorly and systematically selects overly complex models, whereas BIC performs slightly better than CHull when considering the best model only. However, when taking model selection uncertainty into account by looking at the first three models retained, CHull outperforms BIC. This especially holds in more complex, and thus more realistic, situations (e.g., more clusters, factors, noise in the data, and overlap among clusters).     

Illness perception clusters and relationship quality are associated with diabetes distress in adults with Type 2 diabetes

This report aims to augment what is already known about emotional distress in Type 2 diabetes, by assessing the predictive value of illness perception clusters and relationship quality on four subcategories of Diabetes Distress.162 individuals with Type 2 diabetes responded to a postal questionnaire assessing demographics, depression, diabetes distress, illness perceptions and relationship quality. Long-term blood glucose was retrieved from participants’ General Practitioner. Three illness perception clusters emerged from the data, capturing three subgroups of participants sharing similar illness perception schemas. Regression analyses were performed across each diabetes distress subscale, with demographics, illness perception clusters, and relationship variables entered into three blocks. Covariates explained 51.1% of the variance in emotional burden, 41% of the variance in regimen-related distress, 20% of the variance in interpersonal distress, and 8.6% of the variance in physician-related distress. Cluster membership was strongly associated with emotional burden, regimen-related distress, and to a lesser degree interpersonal distress, but was not associated with physician-related distress. Relationship quality most strongly predicted regimen-related distress. Illness perception schemas and interpersonal issues influence emotional adjustment in diabetes. This study provides direction for the content of a novel approach to identifying and reducing diabetes distress in people with Type 2 diabetes.     

Simultaneous Decision on the Number of Latent Clusters and Classes for Multilevel Latent Class Models

The Multilevel Latent Class Model (MLCM) proposed by Vermunt (2003) has been shown to be an excellent framework for analyzing nested data with assumed discrete latent constructs. The nonparametric version of MLCM assumes 2 levels of discrete latent components to describe the dependency observed in data. Model selection is an important step in any statistical modeling. The task of model selection for MLCM amounts to the decision on the number of discrete latent components at both higher and lower levels and is more challenging than standard Latent Class Models. In this article, simulation studies were conducted to systematically examine the effects of sample sizes, clusters/classes distinctness, and the number of latent clusters and classes on the performance of various information criteria in recovering the true latent structure. Results of the simulation studies are summarized and presented. The final section presents the remarks and recommendations about the simultaneous decision regarding the number of latent classes and clusters when applying MLCMs to analyze empirical data.     

Flexible Work Arrangements Availability and their Relationship with Work‐to‐Family Conflict,Job Satisfaction,and Turnover Intentions: A Comparison of Three Country Clusters

The present study explored the availability of flexible work arrangements (FWA) and their relationship with manager outcomes of job satisfaction, turnover intentions, and work‐to‐family conflict (WFC) across country clusters. We used individualism and collectivism to explain differences in FWA availability across Latin American, Anglo, and Asian clusters. Managers from the Anglo cluster were more likely to report working in organisations that offer FWA compared to managers from other clusters. For Anglo managers, flextime was the only FWA that had significant favorable relationships with the outcome variables. For Latin Americans, part‐time work negatively related with turnover intentions and strain‐based WFC. For Asians, flextime was unrelated to time‐based WFC, and telecommuting was positively associated with strain‐based WFC. The clusters did not moderate the compressed work week and outcome relationships. Implications for practitioners adopting FWA practices across cultures are discussed.     

Neuropsychological syndromes in right hemisphere stroke patients

A specific set of syndromes, based on different intrahemispheric lesion locations, has not yet been described in patients with right hemisphere lesions. To explore whether statistically derived clusters could give clues to a syndrome structure, neuropsychological data from a sample of 106 patients with right hemisphere stroke were studied. CT data were available for 58 of the patients. Based on factor analysis of eight test and four ratings variables, six variables were chosen for a cluster analysis. A structure of 13 clusters was considered statistically valid. Combining clusters with parallel test profiles into main cluster classes, five right hemisphere syndromes proved clinically valid: an above average syndrome ( n =46), denial perseverance (frontal) syndrome ( n =14). a depressed mood syndrome ( n = 14), a focal RH syndrome ( n =18) and a global RH syndrome ( n =9). It is suggested that syndromes are related to intrahemispheric location of the lesion, such as the extent of anterior and posterior damage.     

An examination of procedures for determining the number of clusters in a data set

A Monte Carlo evaluation of 30 procedures for determining the number of clusters was conducted on artificial data sets which contained either 2, 3, 4, or 5 distinct nonoverlapping clusters. To provide a variety of clustering solutions, the data sets were analyzed by four hierarchical clustering methods. External criterion measures indicated excellent recovery of the true cluster structure by the methods at the correct hierarchy level. Thus, the clustering present in the data was quite strong. The simulation results for the stopping rules revealed a wide range in their ability to determine the correct number of clusters in the data. Several procedures worked fairly well, whereas others performed rather poorly. Thus, the latter group of rules would appear to have little validity, particularly for data sets containing distinct clusters. Applied researchers are urged to select one or more of the better criteria. However, users are cautioned that the performance of some of the criteria may be data dependent.The authors would like to express their appreciation to a number of individuals who provided assistance during the conduct of this research. Those who deserve recognition include Roger Blashfield, John Crawford, John Gower, James Lingoes, Wansoo Rhee, F. James Rohlf, Warren Sarle, and Tom Soon.     

What needs to be explained to account for age-related effects on multiple cognitive variables?

Structural equation methodology was used to investigate age-related influences across a number of cognitive variables in 204 adults ranging from 18 to 91 years of age with a hierarchical structure that contained 4 1st-order factors and 1 2nd-order common factor. Direct age relations were found to the common factor as well as to 1st-order speed and memory factors. Replicability of the findings was explored by investigating the same structure of age relations, using 2 different data sets, and a similar patten was found in each. These results suggest that at least 3 statistically distinct types of age-related influences are operating on a wide variety of cognitive variables and presumably require separate explanatory mechanisms.     

Profiling local optima in K-means clustering: developing a diagnostic technique

Using the cluster generation procedure proposed by D. Steinley and R. Henson (2005), the author investigated the performance of K-means clustering under the following scenarios: (a) different probabilities of cluster overlap; (b) different types of cluster overlap; (c) varying samples sizes, clusters, and dimensions; (d) different multivariate distributions of clusters; and (e) various multidimensional data structures. The results are evaluated in terms of the Hubert-Arabie adjusted Rand index, and several observations concerning the performance of K-means clustering are made. Finally, the article concludes with the proposal of a diagnostic technique indicating when the partitioning given by a K-means cluster analysis can be trusted. By combining the information from several observable characteristics of the data (number of clusters, number of variables, sample size, etc.) with the prevalence of unique local optima in several thousand implementations of the K-means algorithm, the author provides a method capable of guiding key data-analysis decisions.