Regularized Multiple-Set Canonical Correlation Analysis

Multiple-set canonical correlation analysis (Generalized CANO or GCANO for short) is an important technique because it subsumes a number of interesting multivariate data analysis techniques as special cases. More recently, it has also been recognized as an important technique for integrating information from multiple sources. In this paper, we present a simple regularization technique for GCANO and demonstrate its usefulness. Regularization is deemed important as a way of supplementing insufficient data by prior knowledge, and/or of incorporating certain desirable properties in the estimates of parameters in the model. Implications of regularized GCANO for multiple correspondence analysis are also discussed. Examples are given to illustrate the use of the proposed technique. The work reported in this paper is supported by Grants 10630 and 290439 from the Natural Sciences and Engineering Research Council of Canada to the first and the second authors, respectively. The authors would like to thank the two editors (old and new), the associate editor, and four anonymous reviewers for their insightful comments on earlier versions of this paper. Matlab programs that carried out the computations reported in the paper are available upon request.     

Regularized Generalized Structured Component Analysis

Generalized structured component analysis (GSCA) has been proposed as a component-based approach to structural equation modeling. In practice, GSCA may suffer from multi-collinearity, i.e., high correlations among exogenous variables. GSCA has yet no remedy for this problem. Thus, a regularized extension of GSCA is proposed that integrates a ridge type of regularization into GSCA in a unified framework, thereby enabling to handle multi-collinearity problems effectively. An alternating regularized least squares algorithm is developed for parameter estimation. A Monte Carlo simulation study is conducted to investigate the performance of the proposed method as compared to its non-regularized counterpart. An application is also presented to demonstrate the empirical usefulness of the proposed method.     

Functional Multiple-Set Canonical Correlation Analysis

We propose functional multiple-set canonical correlation analysis for exploring associations among multiple sets of functions. The proposed method includes functional canonical correlation analysis as a special case when only two sets of functions are considered. As in classical multiple-set canonical correlation analysis, computationally, the method solves a matrix eigen-analysis problem through the adoption of a basis expansion approach to approximating data and weight functions. We apply the proposed method to functional magnetic resonance imaging (fMRI) data to identify networks of neural activity that are commonly activated across subjects while carrying out a working memory task.     

Oblique Rotaton in Canonical Correlation Analysis Reformulated as Maximizing the Generalized Coefficient of Determination

To facilitate the interpretation of canonical correlation analysis (CCA) solutions, procedures have been proposed in which CCA solutions are orthogonally rotated to a simple structure. In this paper, we consider oblique rotation for CCA to provide solutions that are much easier to interpret, though only orthogonal rotation is allowed in the existing formulations of CCA. Our task is thus to reformulate CCA so that its solutions have the freedom of oblique rotation. Such a task can be achieved using Yanai’s (Jpn. J. Behaviormetrics 1:46–54, 1974; J. Jpn. Stat. Soc. 11:43–53, 1981) generalized coefficient of determination for the objective function to be maximized in CCA. The resulting solutions are proved to include the existing orthogonal ones as special cases and to be rotated obliquely without affecting the objective function value, where ten Berge’s (Psychometrika 48:519–523, 1983) theorems on suborthonormal matrices are used. A real data example demonstrates that the proposed oblique rotation can provide simple, easily interpreted CCA solutions.     

An Extension of Dominance Analysis to Canonical Correlation Analysis

Dominance analysis (Budescu, 1993 Budescu, D. V. 1993. Dominance analysis: A new approach to the problem of relative importance of predictors in multiple regression.. Psychological Bulletin, 114: 542–551. [Crossref], [Web of Science ®] , [Google Scholar]) offers a general framework for determination of relative importance of predictors in univariate and multivariate multiple regression models. This approach relies on pairwise comparisons of the contribution of predictors in all relevant subset models. In this article we extend dominance analysis to canonical correlation analysis to explore the relative importance of the variables in both sets. The proposed extension provides (a) a decomposition of the models' fit into components associated with the individual variables; (b) the ability to compare the relative importance of variables from the two sets; (c) the ability to perform multistage analyses, involving all canonical variates; and (d) a bootstrapping inference procedure. The approach is illustrated with an empirical data example involving parenting styles and youth outcomes and its unique features are highlighted and discussed.     

Revisiting Interpretation of Canonical Correlation Analysis: A Tutorial and Demonstration of Canonical Commonality Analysis

In the face of multicollinearity, researchers face challenges interpreting canonical correlation analysis (CCA) results. Although standardized function and structure coefficients provide insight into the canonical variates produced, they fall short when researchers want to fully report canonical effects. This article revisits the interpretation of CCA results, providing a tutorial and demonstrating canonical commonalty analysis. Commonality analysis fully explains the canonical effects produced by using the variables in a given canonical set to partition the variance of canonical variates produced from the other canonical set. Conducting canonical commonality analysis without the aid of software is laborious and may be untenable, depending on the number of noteworthy canonical functions and variables in either canonical set. Commonality analysis software is identified for the canonical correlation case and we demonstrate its use in facilitating model interpretation. Data from Holzinger and Swineford (1939) Holzinger, K. J. and Swineford, F. 1939. “A study in factor analysis: The stability of a bi-factor solution.”. In Supplementary Educational Monographs Vol. 48, 81–91. Chicago: Department of Education, University of Chicago.  [Google Scholar] are employed to test a hypothetical theory that problem-solving skills are predicted by fundamental math ability.     

Rotation in Correspondence Analysis from the Canonical Correlation Perspective

Psychometrika - Correspondence analysis (CA) is a statistical method for depicting the relationship between two categorical variables, and usually places an emphasis on graphical representations....     

Canonical Correlation and Chi-Square: Relationships and Interpretation

A 2 × 2 chi-square can be computed from a phi coefficient, which is the Pearson correlation between two binomial variables. Similarly, chi-square for larger contingency tables can be computed from canonical correlation coefficients. The authors address the following series of issues involving this relationship: (a) how to represent a contingency table in terms of a correlation matrix involving r - 1 row and c - 1 column dummy predictors; (b) how to compute chi-square from canonical correlations solved from this matrix; (c) how to compute loadings for the omitted row and column variables; and (d) the possible interpretive advantage of describing canonical relationships that comprise chi-square, together with some examples. The proposed procedures integrate chi-square analysis of contingency tables with general correlational theory and serve as an introduction to some recent methods of analysis more widely known by sociologists.     

An Empirical Investigation Of Step-Down Canonical Correlation With Cross-Validation

Three potential applications of stepwise procedures in canonical analysis and several alternative stepping decision rules are described. A stepdown procedure using smallest interest multiple correlation as the stepping criterion was applied to data on the Minnesota Importance Questionnaire and the Minnesota Vocational Interest Inventory for two random halves of a group of 500 males in a double cross-validation design. Results indicated that up to 75 percent of variables could be dropped from either set with little drop in the R[SUBc]. Cross-validation coefficients were usually higher after dropping several variables than for the full sets.     

K-fold Crossvalidation in Canonical Analysis

A computer-assisted, K-fold crossvalidation technique is discussed within the framework of canonical correlation analysis of randomly generated data sets. Results of the analysis suggest that this technique of multi-crossvalidation can be an effective method to reduce the contamination of canonical variates and canonical correlations by sample-specific variance components.     

Regularized Partial and/or Constrained Redundancy Analysis

Methods of incorporating a ridge type of regularization into partial redundancy analysis (PRA), constrained redundancy analysis (CRA), and partial and constrained redundancy analysis (PCRA) were discussed. The usefulness of ridge estimation in reducing mean square error (MSE) has been recognized in multiple regression analysis for some time, especially when predictor variables are nearly collinear, and the ordinary least squares estimator is poorly determined. The ridge estimation method was extended to PRA, CRA, and PCRA, where the reduced rank ridge estimates of regression coefficients were obtained by minimizing the ridge least squares criterion. It was shown that in all cases they could be obtained in closed form for a fixed value of ridge parameter. An optimal value of the ridge parameter is found by G-fold cross validation. Illustrative examples were given to demonstrate the usefulness of the method in practical data analysis situations. We thank Jim Ramsay for his insightful comments on an earlier draft of this paper. The work reported in this paper is supported by Grants 10630 from the Natural Sciences and Engineering Research Council of Canada to the first author.     

Studying Canonical Analysis: A Reply To Thorndike's Comments

Robert M. Thorndike (1976) commented on the results of a Monte Carlo study on the stability of canonical correlations, canonical weights, and canonical variate-variable correlations (Barcikowski and Stevens, 1975). In this paper each of his comments are examined by the authors of the Monte Carlo Study. In addition, a possible solution to the large number of subjects necessary for stable weights and variate-variable correlations using ridge regression procedures is suggested.     

Functional Generalized Structured Component Analysis

An extension of Generalized Structured Component Analysis (GSCA), called Functional GSCA, is proposed to analyze functional data that are considered to arise from an underlying smooth curve varying over time or other continua. GSCA has been geared for the analysis of multivariate data. Accordingly, it cannot deal with functional data that often involve different measurement occasions across participants and a large number of measurement occasions that exceed the number of participants. Functional GSCA addresses these issues by integrating GSCA with spline basis function expansions that represent infinite-dimensional curves onto a finite-dimensional space. For parameter estimation, functional GSCA minimizes a penalized least squares criterion by using an alternating penalized least squares estimation algorithm. The usefulness of functional GSCA is illustrated with gait data.     

Recalled Parental Behavior and Objective Personality Measures: A Canonical Analysis

Information about an individual's past relationships with his parents is an important component in psychological assessment procedures. To systematically connect these relationships and personality, two groups of college students completed an objective inventory designed to assess parental child rearing behaviors. Canonical correlational analysis was employed to relate these measures to the CPI and MMPI. A recalled parental behavior component described as Acceptance vs. Hostile Detachment/Involvement emerged as being highly related to aspects of both personality inventories, with greater recalled acceptance (and lesser recalled hostility) being associated with the more positive aspects of personality. Additional findings are enumerated.     

图的典范装饰与方程组的典范解

在集合论ZFC-＋AFA中,每个图有唯一装饰,每个方程组有唯一解。但是,在集合论ZFC-4-SAFA和ZFC-4-FAFA中,每个图并非只有一个装饰,每个方程组并非只有一个解。笔者通过定义互模拟坍塌概念,在可达点图的典范装饰概念的基础上导出方程组的典范解,提出并证明：在上述三种具体的非良基集合论中,每个可达点图都有唯一的典范装饰,每个方程组有唯一的典范解。