Note on the squared multiple correlation as a lower bound to communality

It is demonstrated that the squared multiple correlation of a variable with the remaining variables in a set of variables is a function of the communalities and the squared canonical correlations between the observed variables and common factors. This equation is shown to imply a strict inequality between the squared multiple correlation and communality.     

The rowwise correlation between two proximity matrices and the partial rowwise correlation

This paper discusses rowwise matrix correlation, based on the weighted sum of correlations between all pairs of corresponding rows of two proximity matrices, which may both be square (symmetric or asymmetric) or rectangular. Using the correlation coefficients usually associated with Pearson, Spearman, and Kendall, three different rowwise test statistics and their normalized coefficients are discussed, and subsequently compared with their nonrowwise alternatives like Mantel'sZ. It is shown that the rowwise matrix correlation coefficient between two matricesX andY is the partial correlation between the entries ofX andY controlled for the nominal variable that has the row objects as categories. Given this fact, partial rowwise correlations (as well as multiple regression extensions in the case of Pearson's approach) can be easily developed.The author wishes to thank the Editor, two referees, Jan van Hooff, and Ruud Derix for their useful comments, and E. J. Dietz for a copy of the algorithm of the Mantel permutation test.     

Circumplex models for correlation matrices

Structural models that yield circumplex inequality patterns for the elements of correlation matrices are reviewed. Particular attention is given to a stochastic process defined on the circle proposed by T. W. Anderson. It is shown that the Anderson circumplex contains the Markov Process model for a simplex as a limiting case when a parameter tends to infinity.Anderson's model is intended for correlation matrices with positive elements. A replacement for Anderson's correlation function that permits negative correlations is suggested. It is shown that the resulting model may be reparametrzed as a factor analysis model with nonlinear constraints on the factor loadings. An unrestricted factor analysis, followed by an appropriate rotation, is employed to obtain parameter estimates. These estimates may be used as initial approximations in an iterative procedure to obtain minimum discrepancy estimates.Practical applications are reported.Presented as the 1992 Psychometric Society Presidential Address. I am greatly indebted to Stephen Du Toit for help in the development of the computer program employed here. Part of this research was carried out at the University of South Africa and at the Institute for Statistical Research of the South African Human Sciences Research Council.     

A note on structural models for the circumplex

Jöreskog (1974) developed a latent variable model for the covariance structure of the circumplex which, under certain conditions, includes a model for a patterned correlation matrix (Browne, 1977). This model is of limited usefulness, however, in that it employs a known matrix that is rank deficient for many problems. Furthermore, the model is inappropriate for the circumplex which contains negative covariances. This paper presents alternative models for the perfect circumplex and quasi-circumplex that avoids these difficulties, and that includes the important model for a patterned correlation circumplex matrix. Two numerical examples are provided.This research was supported in part by a grant from the Graduate School of the University of Minnesota. I wish to thank M. W. Browne for suggesting the final model presented in this paper. James Steiger and the Editor also made several valuable suggestions.     

Constrained canonical correlation

This paper explores some of the problems associated with traditional canonical correlation. A response surface methodology is developed to examine the stability of the derived linear functions, where one wishes to investigate how much the coefficients can change and still be in an -neighborhood of the globally optimum canonical correlation value. In addition, a discrete (or constrained) canonical correlation method is formulated where the derived coefficients of these linear functions are constrained to be in some small set, e.g., {1, 0, –1}, to aid in the interpretation of the results. An example concerning the psychographic responses of Wharton MBA students of the University of Pennsylvania regarding driving preferences and life-style considerations is provided.Wayne S. DeSarbo, Robert Jausman, Shen Lin, and Wesley Thompson are all Members of Technical Staff at Bell Laboratories. We wish to express our gratitude to the editor and reviewers of this paper for their insightful remarks.     

Minimum rank and minimum trace of covariance matrices

This paper considers some mathematical aspects of minimum trace factor analysis (MTFA). The uniqueness of an optimal point of MTFA is proved and necessary and sufficient conditions for a point x to be optimal are established. Finally, some results about the connection between MTFA and the classical minimum rank factor analysis will be presented.     

Least-squares approximation of an improper correlation matrix by a proper one

An algorithm is presented for the best least-squares fitting correlation matrix approximating a given missing value or improper correlation matrix. The proposed algorithm is based upon a solution for Mosier's oblique Procrustes rotation problem offered by ten Berge and Nevels. A necessary and sufficient condition is given for a solution to yield the unique global minimum of the least-squares function. Empirical verification of the condition indicates that the occurrence of non-optimal solutions with the proposed algorithm is very unlikely. A possible drawback of the optimal solution is that it is a singular matrix of necessity. In cases where singularity is undesirable, one may impose the additional nonsingularity constraint that the smallest eigenvalue of the solution be , where is an arbitrary small positive constant. Finally, it may be desirable to weight the squared errors of estimation differentially. A generalized solution is derived which satisfies the additional nonsingularity constraint and also allows for weighting. The generalized solution can readily be obtained from the standard unweighted singular solution by transforming the observed improper correlation matrix in a suitable way.     

An inequality between the weighted average and the rowwise correlation coefficient for proximity matrices

De Vries (1993) discusses Pearson's product-moment correlation, Spearman's rank correlation, and Kendall's rank-correlation coefficient for assessing the association between the rows of two proximity matrices. For each of these he introduces a weighted average variant and a rowwise variant. In this note it is shown that for all three types, the absolute value of the first variant is greater than or equal to the absolute value of the second.The author is obliged to Frits E. Zegers for useful comments on an earlier version of this paper.     

A note on moran's measure of multiple rank correlation

A measure of multiple rank correlation,T _y.12 ², is proposed for the situation with no tied observations in the variables. The measure is a weighted average of two squared Kendall taus. It is shown thatT _y.12 ² is equivalent to a statistic previously proposed by Moran and thus a new interpretation is given to Moran's statistic.The author wishes to thank Nancy Anderson, Willard Larkin, and Kent Norman for their helpful comments.     

Influence in canonical correlation analysis

The perturbation theory of the generalized eigenproblem is used to derive influence functions of each squared canonical correlation coefficient and the corresponding canonical vector pair. Three sample versions of these functions are described and some properties are noted. As particular applications, the influence function of the squared multiple correlation coefficient and influence functions of eigenvalues and eigenvectors in correspondence analysis are obtained. Three numerical examples are briefly discussed.We thank the Editor and the anonymous reviewers for their helpful comments. This research was carried out with the financial support of the Italian Ministry of the University and the National Research Council.     

General and supplementary factors of personality in genetic and environmental correlation matrices

Genetic and environmental correlation matrices among 7 personality scales were estimated, using samples of adult and young adult twins from Australia (2081 and 1178 pairs, respectively). A general factor of personality and two supplemental factors were obtained in each. The supplemental factors were tentatively identified as Social conformity and Other-dependence. Factors from the genetic and environmental correlations replicated well across samples, and they were similar in both the genetic and environmental covariation. It was concluded that the structure of personality is inherent in the evolved phenotype, and is not the immediate consequence of either genetic or environmental organizing factors.     

Redundancy analysis an alternative for canonical correlation analysis

A component method is presented maximizing Stewart and Love's redundancy index. Relationships with multiple correlation and principal component analysis are pointed out and a rotational procedure for obtaining bi-orthogonal variates is given. An elaborate example comparing canonical correlation analysis and redundancy analysis on artificial data is presented.A Fortran IV program for the method of redundancy analysis described in this paper can be obtained from the author upon request.     

Interpreting canonical correlation analysis through biplots of structure correlations and weights

This paper extends the biplot technique to canonical correlation analysis and redundancy analysis. The plot of structure correlations is shown to the optimal for displaying the pairwise correlations between the variables of the one set and those of the second. The link between multivariate regression and canonical correlation analysis/redundancy analysis is exploited for producing an optimal biplot that displays a matrix of regression coefficients. This plot can be made from the canonical weights of the predictors and the structure correlations of the criterion variables. An example is used to show how the proposed biplots may be interpreted.     

Fungible Correlation Matrices: A Method for Generating Nonsingular,Singular, and Improper Correlation Matrices for Monte Carlo Research

For a fixed set of standardized regression coefficients and a fixed coefficient of determination (R-squared), an infinite number of predictor correlation matrices will satisfy the implied quadratic form. I call such matrices fungible correlation matrices. In this article, I describe an algorithm for generating positive definite (PD), positive semidefinite (PSD), or indefinite (ID) fungible correlation matrices that have a random or fixed smallest eigenvalue. The underlying equations of this algorithm are reviewed from both algebraic and geometric perspectives. Two simulation studies illustrate that fungible correlation matrices can be profitably used in Monte Carlo research. The first study uses PD fungible correlation matrices to compare penalized regression algorithms. The second study uses ID fungible correlation matrices to compare matrix-smoothing algorithms. R code for generating fungible correlation matrices is presented in the supplemental materials.     

The second order approximation to sample influence curve in canonical correlation analysis

A second order approximation to the sample influence curve (SIC) in canonical correlation analysis has been derived in the literature. However, it does not seem satisfactory for some cases. In this paper, we present a more accurate second order approximation. As a particular case, the proposed method is exact for the SIC of the squared multiple correlation coefficient. An example is given. The authors are most grateful to the associate editor and three reviewers for valuable comments and suggestions which improved the presentation of the paper considerably. The first author was partly supported by a RGC earmarked research grant of Hong Kong.     

The comparison of interdependent correlations between optimal linear composites

A general procedure is provided for comparing correlation coefficients between optimal linear composites. The procedure allows computationally efficient significance tests on independent or dependent multiple correlations, partial correlations, and canonical correlations, with or without the assumption of multivariate normality. Evidence from some Monte Carlo studies on the effectiveness of the methods is also provided.This research was supported in part by an operating grant (#67-4640) to the first author from the National Sciences and Engineering Research Council of Canada. The authors would also like to acknowledge the helpful comments and encouragement of Alexander Shapiro, Stanley Nash, and Ingram Olkin.     

Approximate uniqueness estimates for singular correlation matrices

The residual variance (one minus the squared multiple correlation) is often used as an approximation to the uniqueness in factor analysis. An upper bound approximation to the residual variance is given for the case when the correlation matrix is singular. The approximation is computationally simpler than the exact solution, especially since it can be applied routinely without prior knowledge as to the singularity or nonsingularity of the correlation matrix.     

Generalized canonical correlation analysis of matrices with missing rows: a simulation study

A method is presented for generalized canonical correlation analysis of two or more matrices with missing rows. The method is a combination of Carroll’s (1968) method and the missing data approach of the OVERALS technique (Van der Burg, 1988). In a simulation study we assess the performance of the method and compare it to an existing procedure called GENCOM, proposed by Green and Carroll (1988). We find that the proposed method outperforms the GENCOM algorithm both with respect to model fit and recovery of the true structure. The research of Michel van de Velden was partly funded through EU Grant HPMF-CT-2000-00664. The authors would like to thank the associate editor and three anonymous referees for their constructive comments and suggestions that led to a considerable improvement of the paper.     

Sensitivity analysis in factor analysis: Difference between using covariance and correlation matrices

Influence curves of some parameters under various methods of factor analysis have been given in the literature. These influence curves depend on the influence curves for either the covariance or the correlation matrix used in the analysis. The differences between the influence curves based on the covariance and the correlation matrices are derived in this paper. Simple formulas for the differences of the influence curves, based on the two matrices, for the unique variance matrix, factor loadings and some other parameter are obtained under scale-invariant estimation methods, though the influence curves themselves are in complex forms.The authors are most grateful to the referees, the Associate Editor, the Editor and Raymond Lam for helpful suggestions for improving the clarity of the paper.     

Ezekiel’s classic estimator of the population squared multiple correlation coefficient: Monte Carlo-based extensions and refinements

Abstract

Ezekiel’s adjusted R² is widely used in linear regression analysis. The present study examined the statistical properties of Ezekiel’s measure through a series of Monte Carlo simulations. Specifically, we examined the bias and root mean squared error (RMSE) of Ezekiel’s adjusted R² relative to (a) the sample R² statistic, and (b) the sample R² minus the expected value of R². Simulation design factors consisted of sample sizes (N?=?50, 100, 200, 400), number of predictors (2, 3, 4, 5, 6), and population squared multiple correlations (ρ² = 0, .10, .25, .40, .60). Factorially crossing these design factors resulted in 100 simulation conditions. All populations were normal/Gaussian, and for each condition, we drew 10,000 Monte Carlo samples. Regarding systematic variation (bias), results indicated that with few exceptions, Ezekiel’s adjusted R² demonstrated the lowest bias. Regarding unsystematic variation (RMSE), the performance of Ezekiel’s measure was comparable to the other statistics, suggesting that the bias-variance tradeoff is minimal for Ezekiel’s adjusted R². Additional findings indicated that sample size-to-predictor ratios of 66.67 and greater were associated with low bias and that ratios of this magnitude were accompanied by large sample sizes (N?=?200 and 400), thus suggesting that researchers using Ezekiel’s adjusted R² should aim for sample sizes of 200 or greater in order to minimize bias when estimating the population squared multiple correlation coefficient. Overall, these findings indicate that Ezekiel’s adjusted R² has desirable properties and, in addition, these findings bring needed clarity to the statistical literature on Ezekiel’s classic estimator.