Matrix formulas for part and partial correlation

General formulas for part and partial correlation of any order are derived in terms of multiple correlation coefficients, standard partial regression weights, and validities. The relationship between part correlation and the independent contribution of a predictor is discussed.     

Revisiting Interpretation of Canonical Correlation Analysis: A Tutorial and Demonstration of Canonical Commonality Analysis

In the face of multicollinearity, researchers face challenges interpreting canonical correlation analysis (CCA) results. Although standardized function and structure coefficients provide insight into the canonical variates produced, they fall short when researchers want to fully report canonical effects. This article revisits the interpretation of CCA results, providing a tutorial and demonstrating canonical commonalty analysis. Commonality analysis fully explains the canonical effects produced by using the variables in a given canonical set to partition the variance of canonical variates produced from the other canonical set. Conducting canonical commonality analysis without the aid of software is laborious and may be untenable, depending on the number of noteworthy canonical functions and variables in either canonical set. Commonality analysis software is identified for the canonical correlation case and we demonstrate its use in facilitating model interpretation. Data from Holzinger and Swineford (1939) Holzinger, K. J. and Swineford, F. 1939. “A study in factor analysis: The stability of a bi-factor solution.”. In Supplementary Educational Monographs Vol. 48, 81–91. Chicago: Department of Education, University of Chicago.  [Google Scholar] are employed to test a hypothetical theory that problem-solving skills are predicted by fundamental math ability.     

Some simple procedures for handling missing data in multivariate analysis

For analyses with missing data, some popular procedures delete cases with missing values, perform analysis with missing value correlation or covariance matrices, or estimate missing values by sample means. There are objections to each of these procedures. Several procedures are outlined here for replacing missing values by regression values obtained in various ways, and for adjusting coefficients (such as factor score coefficients) when data are missing. None of the procedures are complex or expensive.This research was supported by NIH Special Research Resources Grant RR-3. The author expresses his gratitude to Robert I. Jennrich and the referees for their suggestions.     

Property Designators,Predicates, and Rigidity

The article discusses an idea of how to extend the notion of rigidity to predicates, namely the idea that predicates stand in a certain systematic semantic relation to properties, such that this relation may hold rigidly or nonrigidly. The relation (which I call signification) can be characterised by recourse to canonical property designators which are derived from predicates (or general terms) by means of nominalization: a predicate signifies that property which the derived property designator designates. Whether signification divides into rigid and non-rigid cases will then depend uponwhether canonical property designators divide into rigid and non-rigid ones. But, I shall argue, they do not, and so the only notion of rigidity gained this way is trivial. To show this, I first focus on the kind of canonical property designators which could be thought to be nonrigid, canonical designators such as having the colour of ripe tomatoes which themselves contain non-rigid property designators. An argument to the effect that such complex canonical designators are non-rigid is rebutted, five arguments to the effect that they are rigid are formulated, and finally an explanation of their rigidity based on the general nature of canonical property designators is presented.     

Canonical Correlation and Chi-Square: Relationships and Interpretation

A 2 × 2 chi-square can be computed from a phi coefficient, which is the Pearson correlation between two binomial variables. Similarly, chi-square for larger contingency tables can be computed from canonical correlation coefficients. The authors address the following series of issues involving this relationship: (a) how to represent a contingency table in terms of a correlation matrix involving r - 1 row and c - 1 column dummy predictors; (b) how to compute chi-square from canonical correlations solved from this matrix; (c) how to compute loadings for the omitted row and column variables; and (d) the possible interpretive advantage of describing canonical relationships that comprise chi-square, together with some examples. The proposed procedures integrate chi-square analysis of contingency tables with general correlational theory and serve as an introduction to some recent methods of analysis more widely known by sociologists.     

Least-squares approximation of an improper correlation matrix by a proper one

An algorithm is presented for the best least-squares fitting correlation matrix approximating a given missing value or improper correlation matrix. The proposed algorithm is based upon a solution for Mosier's oblique Procrustes rotation problem offered by ten Berge and Nevels. A necessary and sufficient condition is given for a solution to yield the unique global minimum of the least-squares function. Empirical verification of the condition indicates that the occurrence of non-optimal solutions with the proposed algorithm is very unlikely. A possible drawback of the optimal solution is that it is a singular matrix of necessity. In cases where singularity is undesirable, one may impose the additional nonsingularity constraint that the smallest eigenvalue of the solution be , where is an arbitrary small positive constant. Finally, it may be desirable to weight the squared errors of estimation differentially. A generalized solution is derived which satisfies the additional nonsingularity constraint and also allows for weighting. The generalized solution can readily be obtained from the standard unweighted singular solution by transforming the observed improper correlation matrix in a suitable way.     

Maximization of sums of quotients of quadratic forms and some generalizations

Monotonically convergent algorithms are described for maximizing six (constrained) functions of vectors x, or matricesX with columns x₁, ..., x _r . These functions are h₁(x)= _k (xA _kx)(xC _kx)^–1, H₁(X)= _k tr (XA _k X)(XC _k X)^–1, h₁(X)= _{k l} (x _l A _kx _l ) (x _l C _kx _l )^–1 withX constrained to be columnwise orthonormal, h₂(x)= _k (xA _kx)²(xC _kx)^–1 subject to xx=1, H₂(X)= _k tr(XA _kX)(XA_kX)(XC_kX)^–1 subject toXX=I, and h₂(X)= _{k l} (x _l A _kx _l )² (x _l C _kX _l )^–1 subject toXX=I. In these functions the matricesC _k are assumed to be positive definite. The matricesA _k can be arbitrary square matrices. The general formulation of the functions and the algorithms allows for application of the algorithms in various problems that arise in multivariate analysis. Several applications of the general algorithms are given. Specifically, algorithms are given for reciprocal principal components analysis, binormamin rotation, generalized discriminant analysis, variants of generalized principal components analysis, simple structure rotation for one of the latter variants, and set component analysis. For most of these methods the algorithms appear to be new, for the others the existing algorithms turn out to be special cases of the newly derived general algorithms.This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the author. The author is obliged to Jos ten Berge for stimulating this research and for helpful comments on an earlier version of this paper.     

Bounded contraction and Gentzen-style formulation of Łukasiewicz logics

In this paper, we consider multiplicative-additive fragments of affine propositional classical linear logic extended with n-contraction. To be specific, n-contraction (n 2) is a version of the contraction rule where (n+ 1) occurrences of a formula may be contracted to n occurrences. We show that expansions of the linear models for (n + 1)- valued ukasiewicz logic are models for the multiplicative-additive classical linear logic, its affine version and their extensions with n-contraction. We prove the finite axiomatizability for the classes of finite models, as well as for the class of infinite linear models based on the set of rational numbers in the interval [0, 1]. The axiomatizations obtained in a Gentzen-style formulation are equivalent to finite and infinite-valued ukasiewicz logics.Presented by Jan Zygmunt     

Canonical analysis of longitudinal and repeated measures data with stationary weights

When measuring the same variables on different occasions, two procedures for canonical analysis with stationary compositing weights are developed. The first, SUMCOV, maximizes the sum of the covariances of the canonical variates subject to norming constraints. The second, COLLIN, maximizes the largest root of the covariances of the canonical variates subject to norming constraints. A characterization theorem establishes a model building approach. Both methods are extended to allow for Cohort Sequential Designs. Finally a numerical illustration utilizing Nesselroade and Baltes data is presented.The authors wish to thank John Nesselroade for permitting us to use the data whose analysis we present.     

Clustering with relational constraint

The paper deals with clustering problems where grouping is constrained by a symmetric and reflexive relation. For solving clustering problems with relational constraints two methods are adapted: the standard hierarchical clustering procedure based on the Lance and Williams formula, and local optimization procedure, CLUDIA. To illustrate these procedures, clusterings of the European countries are given based on the developmental indicators where the relation is determined by the geographical neighbourhoods of countries.Extended version of the paper presented at the European meeting of the Psychometric Society, Groningen, June, 19–21, 1980.This work was supported in part by the Boris Kidri Fund, Yugoslavia.     

New lower and upper bounds for communality in factor analysis

We derive several relationships between communalities and the eigenvalues for ap ×p correlation matrix under the usual factor analysis model. For suitable choices ofj, _j (), where _j () is thej-th largest eigenvalue of , provides either a lower or an upper bound to the communalities for some of the variables. We show that for at least one variable, 1 - _p () improves on the use of squared mulitiple correlation coefficient as a lower bound.This research was done while the second author was at Tokyo Institute of Technology.     

Interpreting canonical correlation analysis through biplots of structure correlations and weights

This paper extends the biplot technique to canonical correlation analysis and redundancy analysis. The plot of structure correlations is shown to the optimal for displaying the pairwise correlations between the variables of the one set and those of the second. The link between multivariate regression and canonical correlation analysis/redundancy analysis is exploited for producing an optimal biplot that displays a matrix of regression coefficients. This plot can be made from the canonical weights of the predictors and the structure correlations of the criterion variables. An example is used to show how the proposed biplots may be interpreted.     

A thurstonian pairwise choice model with univariate and multivariate spline transformations

A probabilistic choice model is developed for paired comparisons data about psychophysical stimuli. The model is based on Thurstone's Law of Comparative Judgment Case V and assumes that each stimulus is measured on a small number of physical variables. The utility of a stimulus is related to its values on the physical variables either by means of an additive univariate spline model or by means of multivariate spline model. In the additive univariate spline model, a separate univariate spline transformation is estimated for each physical dimension and the utility of a stimulus is assumed to be an additive combination of these transformed values. In the multivariate spline model, the utility of a stimulus is assumed to be a general multivariate spline function in the physical variables. The use of B splines for estimating the transformation functions is discussed and it is shown how B splines can be generalized to the multivariate case by using as basis functions tensor products of the univariate basis functions. A maximum likelihood estimation procedure for the Thurstone Case V model with spline transformation is described and applied for illustrative purposes to various artificial and real data sets. Finally, the model is extended using a latent class approach to the case where there are unreplicated paired comparisons data from a relatively large number of subjects drawn from a heterogeneous population. An EM algorithm for estimating the parameters in this extended model is outlined and illustrated on some real data.The first author is supported as Bevoegdverklaard Navorser of the Belgian Nationaal Fonds voor Wetenschappelijk Onderzoek. The authors are indebted to Ulf Böckenholt and Yoshio Takane for useful comments on an earlier draft of this paper.     

To be and not to be: Dialectical tense logic

The paper concerns time, change and contradiction, and is in three parts. The first is an analysis of the problem of the instant of change. It is argued that some changes are such that at the instant of change the system is in both the prior and the posterior state. In particular there are some changes from p being true to p being true where a contradiction is realized. The second part of the paper specifies a formal logic which accommodates this possibility. It is a tense logic based on an underlying paraconsistent prepositional logic, the logic of paradox. (See the author's article of the same name Journal of Philosophical Logic 8 (1979).) Soundness and completeness are established, the latter by the canonical model construction, and extensions of the basic system briefly considered. The final part of the paper discusses Leibniz's principle of continuity: Whatever holds up to the limit holds at the limit. It argues that in the context of physical changes this is a very plausible principle. When it is built into the logic of the previous part, it allows a rigorous proof that change entails contradictions. Finally the relation of this to remarks on dialectics by Hegel and Engels is briefly discussed.     

Sets and Classes as Many

In this paper the view is developed that classes should not be understood as individuals, but, rather, as classes as many of individuals. To correlate classes with individuals labelling and colabelling functions are introduced and sets identified with a certain subdomain of the classes on which the labelling and colabelling functions are mutually inverse. A minimal axiomatization of the resulting system is formulated and some of its extensions are related to various systems of set theory, including nonwellfounded set theories.     

A note on Roy's largest root

The name Roy's largest root and similar names are used in practice to label two different but functionally related statistics—one proportional to anF, and the other, a squared canonical correlation. This note presents the logic that leads to the two formulations, states which statistic some popular statistical packages use, and shows the possible source of this inconsistency in the original work of Roy (1953) and Heck (1960).     

Partial Monotonic Protothetics

This paper has four parts. In the first part, I present Leniewski's protothetics and the complete system provided for that logic by Henkin. The second part presents a generalized notion of partial functions in propositional type theory. In the third part, these partial functions are used to define partial interpretations for protothetics. Finally, I present in the fourth part a complete system for partial protothetics. Completeness is proved by Henkin's method [4] using saturated sets instead of maximally saturated sets. This technique provides a canonical representation of a partial semantic space and it is suggested that this space can be interpreted as an epistemic state of a non-omniscient agent.     

Predictive probability and analogy by similarity in inductive logic

The-continuum of inductive methods was derived from an assumption, called-condition, which says that the probability of finding an individual having propertyx _j depends only on the number of observed individuals having propertyx _j and on the total number of observed individuals. So, according to that assumption, all individuals with properties which are different fromx _j have equal weight with respect to that probability and, in particular, it does not matter whether any individual was observed having some propertysimilar tox _j (the most complete proof of this result is presented in Carnap, 1980).The problem thus remained open to find some general condition, weaker than the-condition, which would allow for thederivation of probability functions which might be sensitive to similarity. Carnap himself suggested a weakening of the-condition which might allow for similarity sensitive probability functions (Carnap, 1980, p. 45) but he did not find the family of probability functions derivable from that principle. The aim of this paper is to present the family of probability functions derivable from Carnap's suggestion and to show how it is derived.In Section 1 the general problem of analogy by similarity in inductive logic is presented, Section 2 outlines the notation and the conceptual background involved in the proof, Section 3 gives the proof, Section 4 discusses Carnap's principle and the result, Section 5 is a brief review of the solutions which have previously been proposed.     

A mixture model for distributions of correlation coefficients

An old problem in personnel psychology is to characterize distributions of test validity correlation coefficients. The proposed model views histograms of correlation coefficients as observations from a mixture distribution which, for a fixed sample sizen, is a conditional mixture distributionh(r|n) = _{j j} h(r; _j ,n), whereR is the correlation coefficient, _j are population correlation coefficients and _j are the mixing weights. The associated marginal distribution ofR is regarded as the parent distribution underlying histograms of empirical correlation coefficients. Maximum likelihood estimates of the parameters _j and _j can be obtained with an EM algorithm solution and tests for the number of componentst are achieved after the (one-component) density ofR is replaced with a tractable modeling densityh(r; _j ,n). Two illustrative examples are provided.     

Canonical/redundancy factoring analysis

The interrelationships between two sets of measurements made on the same subjects can be studied by canonical correlation. Originally developed by Hotelling [1936], the canonical correlation is the maximum correlation betweenlinear functions (canonical factors) of the two sets of variables. An alternative statistic to investigate the interrelationships between two sets of variables is the redundancy measure, developed by Stewart and Love [1968]. Van Den Wollenberg [1977] has developed a method of extracting factors which maximize redundancy, as opposed to canonical correlation.A component method is presented which maximizes user specified convex combinations of canonical correlation and the two nonsymmetric redundancy measures presented by Stewart and Love. Monte Carlo work comparing canonical correlation analysis, redundancy analysis, and various canonical/redundancy factoring analyses on the Van Den Wollenberg data is presented. An empirical example is also provided.Wayne S. DeSarbo is a Member of Technical Staff at Bell Laboratories in the Mathematics and Statistics Research Group at Murray Hill, N.J. I wish to express my appreciation to J. Kettenring, J. Kruskal, C. Mallows, and R. Gnanadesikan for their valuable technical assistance and/or for comments on an earlier draft of this paper. I also wish to thank the editor and reviewers of this paper for their insightful remarks.