Canonical/redundancy factoring analysis

The interrelationships between two sets of measurements made on the same subjects can be studied by canonical correlation. Originally developed by Hotelling [1936], the canonical correlation is the maximum correlation betweenlinear functions (canonical factors) of the two sets of variables. An alternative statistic to investigate the interrelationships between two sets of variables is the redundancy measure, developed by Stewart and Love [1968]. Van Den Wollenberg [1977] has developed a method of extracting factors which maximize redundancy, as opposed to canonical correlation.A component method is presented which maximizes user specified convex combinations of canonical correlation and the two nonsymmetric redundancy measures presented by Stewart and Love. Monte Carlo work comparing canonical correlation analysis, redundancy analysis, and various canonical/redundancy factoring analyses on the Van Den Wollenberg data is presented. An empirical example is also provided.Wayne S. DeSarbo is a Member of Technical Staff at Bell Laboratories in the Mathematics and Statistics Research Group at Murray Hill, N.J. I wish to express my appreciation to J. Kettenring, J. Kruskal, C. Mallows, and R. Gnanadesikan for their valuable technical assistance and/or for comments on an earlier draft of this paper. I also wish to thank the editor and reviewers of this paper for their insightful remarks.     

Some symmetric,invariant measures of multivariate association

A distinction is drawn between redundancy measurement and the measurement of multivariate association for two sets of variables. Several measures of multivariate association between two sets of variables are examined. It is shown that all of these measures are generalizations of the (univariate) squared-multiple correlation; all are functions of the canonical correlations, and all are invariant under linear transformations of the original sets of variables. It is further shown that the measures can be considered to be symmetric and are strictly ordered for any two sets of observed variables. It is suggested that measures of multivariate relationship may be used to generalize the concept of test reliability to the case of vector random variables.     

On the optimality of the simultaneous redundancy transformations

The objective of this paper is to introduce and motivate additional properties and interpretations for the redundancy variables. It is shown that these variables can be derived by application of certain invariance arguments and without reference to the index of redundancy. In addition, an optimality property for the variables is presented which is important whenever one restricts attention in a study to a subset of the redundancy variables. This optimality property pertains to the subset rather than to the individual variables.This paper is based in part on the author's doctoral dissertation, Department of Statistics, Princeton, University. Research was conducted under the supervision of Lawrence S. Mayer.     

Linear relations amongk sets of variables

A family of solutions for linear relations amongk sets of variables is proposed. It is shown how these solutions apply fork=2, and how they can be generalized from there tok3.The family of solutions depends on three independent choices: (i) to what extent a solution may be influenced by differences in variances of components within each set; (ii) to what extent the sets may be differentially weighted with respect to their contribution to the solution—including orthogonality constraints; (iii) whether or not individual sets of variables may be replaced by an orthogonal and unit normalized basis.Solutions are compared with respect to their optimality properties. For each solution the appropriate stationary equations are given. For one example it is shown how the determinantal equation of the stationary equations can be interpreted.     

Global optimality of the successive Maxbet algorithm

The Maxbet method is an alternative to the method of generalized canonical correlation analysis and of Procrustes analysis. Contrary to these methods, it does not maximize the inner products (covariances) between linear composites, but also takes their sums of squares (variances) into account. It is well-known that the Maxbet algorithm, which has been proven to converge monotonically, may converge to local maxima. The present paper discusses an eigenvalue criterion which is sufficient, but not necessary for global optimality. However, in two special cases, the eigenvalue criterion is shown to be necessary and sufficient for global optimality. The first case is when there are only two data sets involved; the second case is when the inner products between all variables involved are positive, regardless of the number of data sets.The authors are obliged to Henk Kiers for critical comments on a previous draft.     

On a test of dimensionality in redundancy analysis

Lazraq and Cléroux (Psychometrika, 2002, 411–419) proposed a test for identifying the number of significant components in redundancy analysis. This test, however, is ill-conceived. A major problem is that it regards each redundancy component as if it were a single observed predictor variable, which cannot be justified except for the rare situations in which there is only one predictor variable. Consequently, the proposed test leads to drastically biased results, particularly when the number of predictor variables is large, and it cannot be recommended for use. This is shown both theoretically and by Monte Carlo studies.The work reported in this paper was supported by Grant A6394 to the first author from the Natural Sciences and Engineering Research Council of Canada.     

Interpreting canonical correlation analysis through biplots of structure correlations and weights

This paper extends the biplot technique to canonical correlation analysis and redundancy analysis. The plot of structure correlations is shown to the optimal for displaying the pairwise correlations between the variables of the one set and those of the second. The link between multivariate regression and canonical correlation analysis/redundancy analysis is exploited for producing an optimal biplot that displays a matrix of regression coefficients. This plot can be made from the canonical weights of the predictors and the structure correlations of the criterion variables. An example is used to show how the proposed biplots may be interpreted.     

Generalizations of the partial,part and bipartial canonical correlation analysis

Theg ₁- andg ₂-bipartial canonical correlation analyses are developed as generalizations of the partial, part, and bipartial canonical correlation analysis. Illustrative examples are provided.     

The comparison of interdependent correlations between optimal linear composites

A general procedure is provided for comparing correlation coefficients between optimal linear composites. The procedure allows computationally efficient significance tests on independent or dependent multiple correlations, partial correlations, and canonical correlations, with or without the assumption of multivariate normality. Evidence from some Monte Carlo studies on the effectiveness of the methods is also provided.This research was supported in part by an operating grant (#67-4640) to the first author from the National Sciences and Engineering Research Council of Canada. The authors would also like to acknowledge the helpful comments and encouragement of Alexander Shapiro, Stanley Nash, and Ingram Olkin.     

Influence in canonical correlation analysis

The perturbation theory of the generalized eigenproblem is used to derive influence functions of each squared canonical correlation coefficient and the corresponding canonical vector pair. Three sample versions of these functions are described and some properties are noted. As particular applications, the influence function of the squared multiple correlation coefficient and influence functions of eigenvalues and eigenvectors in correspondence analysis are obtained. Three numerical examples are briefly discussed.We thank the Editor and the anonymous reviewers for their helpful comments. This research was carried out with the financial support of the Italian Ministry of the University and the National Research Council.     

The second order approximation to sample influence curve in canonical correlation analysis

A second order approximation to the sample influence curve (SIC) in canonical correlation analysis has been derived in the literature. However, it does not seem satisfactory for some cases. In this paper, we present a more accurate second order approximation. As a particular case, the proposed method is exact for the SIC of the squared multiple correlation coefficient. An example is given. The authors are most grateful to the associate editor and three reviewers for valuable comments and suggestions which improved the presentation of the paper considerably. The first author was partly supported by a RGC earmarked research grant of Hong Kong.     

Canonical analysis of longitudinal and repeated measures data with stationary weights

When measuring the same variables on different occasions, two procedures for canonical analysis with stationary compositing weights are developed. The first, SUMCOV, maximizes the sum of the covariances of the canonical variates subject to norming constraints. The second, COLLIN, maximizes the largest root of the covariances of the canonical variates subject to norming constraints. A characterization theorem establishes a model building approach. Both methods are extended to allow for Cohort Sequential Designs. Finally a numerical illustration utilizing Nesselroade and Baltes data is presented.The authors wish to thank John Nesselroade for permitting us to use the data whose analysis we present.     

Properties of the maximum likelihood solution in factor analysis regression

Algebraic properties of the normal theory maximum likelihood solution in factor analysis regression are investigated. Two commonly employed measures of the within sample predictive accuracy of the factor analysis regression function are considered: the variance of the regression residuals and the squared correlation coefficient between the criterion variable and the regression function. It is shown that this within sample residual variance and within sample squared correlation may be obtained directly from the factor loading and unique variance estimates, without use of the original observations or the sample covariance matrix.     

Relationships among several methods of linearly constrained correspondence analysis

This paper shows essential equivalences among several methods of linearly constrained correspondence analysis. They include Fisher's method of additive scoring, Hayashi's second type of quantification method, ter Braak's canonical correspondence analysis, Nishisato's type of quantification method, ter Braak's canonical correspondence analysis, Nishisato's ANOVA of categorical data, correspondence analysis of manipulated contingency tables, Böckenholt and Böckenholt's least squares canonical analysis with linear constraints, and van der Heijden and Meijerink's zero average restrictions. These methods fall into one of two classes of methods corresponding to two alternative ways of imposing linear constraints, the reparametrization method and the null space method. A connection between the two is established through Khatri's lemma.The work reported in this paper has been supported by grant A6394 from the Natural Sciences and Engineering Research Council of Canada to the first author. We wish to thank Carolyn Anderson, Ulf Böckenholt, Henk Kiers, Shizuhiko Nishisato, Jim Ramsay, Tadashi Shibayama, Cajo ter Braak, and Peter van der Heijden for their helpful comments on earlier drafts of this paper.     

Model-based recursive partitioning of extended redundancy analysis with an application to nicotine dependence among US adults

Extended redundancy analysis (ERA) is used to reduce multiple sets of predictors to a smaller number of components and examine the effects of these components on a response variable. In various social and behavioural studies, auxiliary covariates (e.g., gender, ethnicity) can often lead to heterogeneous subgroups of observations, each of which involves distinctive relationships between predictor and response variables. ERA is currently unable to consider such covariate-dependent heterogeneity to examine whether the model parameters vary across subgroups differentiated by covariates. To address this issue, we combine ERA with model-based recursive partitioning in a single framework. This combined method, MOB-ERA, aims to partition observations into heterogeneous subgroups recursively based on a set of covariates while fitting a specified ERA model to data. Upon the completion of the partitioning procedure, one can easily examine the difference in the estimated ERA parameters across covariate-dependent subgroups. Moreover, it produces a tree diagram that aids in visualizing a hierarchy of partitioning covariates, as well as interpreting their interactions. In the analysis of public data concerning nicotine dependence among US adults, the method uncovered heterogeneous subgroups characterized by several sociodemographic covariates, each of which yielded different directional relationships between three predictor sets and nicotine dependence.     

Component analysis in cross-sectional and longitudinal data

An extension of component analysis to longitudinal or cross-sectional data is presented. In this method, components are derived under the restriction of invariant and/or stationary compositing weights. Optimal compositing weights are found numerically. The method can be generalized to allow differential weighting of the observed variables in deriving the component solution. Some choices of weightings are discussed. An illustration of the method using real data is presented.Preparation of this article was supported in part by PSC-CUNY Grant #665365 to Roger E. Millsap and by National Institute of Aging Grant NIA-AG03164-03 to William Meredith. The authors thank John Nesselroade for permitting the use of the data presented in the article.     

Generalized canonical correlation analysis of matrices with missing rows: a simulation study

A method is presented for generalized canonical correlation analysis of two or more matrices with missing rows. The method is a combination of Carroll’s (1968) method and the missing data approach of the OVERALS technique (Van der Burg, 1988). In a simulation study we assess the performance of the method and compare it to an existing procedure called GENCOM, proposed by Green and Carroll (1988). We find that the proposed method outperforms the GENCOM algorithm both with respect to model fit and recovery of the true structure. The research of Michel van de Velden was partly funded through EU Grant HPMF-CT-2000-00664. The authors would like to thank the associate editor and three anonymous referees for their constructive comments and suggestions that led to a considerable improvement of the paper.     

Direction of Effects in Multiple Linear Regression Models

Previous studies analyzed asymmetric properties of the Pearson correlation coefficient using higher than second order moments. These asymmetric properties can be used to determine the direction of dependence in a linear regression setting (i.e., establish which of two variables is more likely to be on the outcome side) within the framework of cross-sectional observational data. Extant approaches are restricted to the bivariate regression case. The present contribution extends the direction of dependence methodology to a multiple linear regression setting by analyzing distributional properties of residuals of competing multiple regression models. It is shown that, under certain conditions, the third central moments of estimated regression residuals can be used to decide upon direction of effects. In addition, three different approaches for statistical inference are discussed: a combined D’Agostino normality test, a skewness difference test, and a bootstrap difference test. Type I error and power of the procedures are assessed using Monte Carlo simulations, and an empirical example is provided for illustrative purposes. In the discussion, issues concerning the quality of psychological data, possible extensions of the proposed methods to the fourth central moment of regression residuals, and potential applications are addressed.     

A test of the hypothesis that Cronbach's alpha reliability coefficient is the same for two tests administered to the same sample

In measurement studies the researcher may wish to test the hypothesis that Cronbach's alpha reliability coefficient is the same for two measurement procedures. A statistical test exists for independent samples of subjects. In this paper three procedures are developed for the situation in which the coefficients are determined from the same sample. All three procedures are computationally simple and give tight control of Type I error when the sample size is 50 or greater.The author is indebted to Jerry S. Gilmer for development of the computer programs used in this study.