Machine short-cuts in the computation of chisquare and the contingency coefficient

Rapid computational routines are presented for calculatingx ² from frequency data in the following cases: (1) test of goodness of fit between an observed and a theoretical distribution; (2) test of independence of distributions displayed in anr ×c table; (3) test of independence of distributions displayed in anr × 2 table. A rapid method of computing the contingency coefficient also follows from the procedure used in the second of these cases.     

An algebraic solution for the communalities

Factorial analysis begins with ann ×n correlation matrixR, whose principal diagonal entries are unknown. If the common test space of the battery is under investigation, the communality of each test is entered in the appropriate diagonal cell. This value is the portion of the test's variance shared with others in the battery. The communalities must be so estimated thatR will maintain the rank determined by its side entries, after the former have been inserted. Previous methods of estimating the communalities have involved a certain arbitrariness, since they depended on selecting test subgroups or parts of the data inR. A theory is presented showing that this difficulty can be avoided in principle. In its present form, the theory is not offered as a practical computing procedure. The basis of the new method lies in the Cayley-Hamilton theorem: Any square matrix satisfies its own characteristic equation.     

A simplification of a result by zellini on the maximal rank of symmetric three-way arrays

Zellini (1979, Theorem 3.1) has shown how to decompose an arbitrary symmetric matrix of ordern ×n as a linear combination of 1/2n(n+1) fixed rank one matrices, thus constructing an explicit tensor basis for the set of symmetricn ×n matrices. Zellini's decomposition is based on properties of persymmetric matrices. In the present paper, a simplified tensor basis is given, by showing that a symmetric matrix can also be decomposed in terms of 1/2n(n+1) fixed binary matrices of rank one. The decomposition implies that ann ×n ×p array consisting ofp symmetricn ×n slabs has maximal rank 1/2n(n+1). Likewise, an unconstrained INDSCAL (symmetric CANDECOMP/PARAFAC) decomposition of such an array will yield a perfect fit in 1/2n(n+1) dimensions. When the fitting only pertains to the off-diagonal elements of the symmetric matrices, as is the case in a version of PARAFAC where communalities are involved, the maximal number of dimensions can be further reduced to 1/2n(n–1). However, when the saliences in INDSCAL are constrained to be nonnegative, the tensor basis result does not apply. In fact, it is shown that in this case the number of dimensions needed can be as large asp, the number of matrices analyzed.     

Matrix reorganization and dynamic programming: Applications to paired comparisons and unidimensional seriation

A recursive dynamic programming strategy is discussed for optimally reorganizing the rows and simultaneously the columns of ann ×n proximity matrix when the objective function measuring the adequacy of a reorganization has a fairly simple additive structure. A number of possible objective functions are mentioned along with several numerical examples using Thurstone's paired comparison data on the relative seriousness of crime. Finally, the optimization tasks we propose to attack with dynamic programming are placed in a broader theoretical context of what is typically referred to as the quadratic assignment problem and its extension to cubic assignment.Partial support for this research was provided by NIJ Grant 80-IJ-CX-0061.     

Determining a simple structure when loadings for certain tests are known

A rigorous and an approximate solution are found for the problem: Given a primary trait matrix forn tests andr ₁ traits, and a matrix for the samen tests andr ₂ reference axes, to discover the transformation which will transform the second matrix into the first, or primary trait matrix. Formulas for determining the limits of the effect of using the approximate solution are presented. The method is applied to a set of twenty hypothetical tests, defined by their loadings on four orthogonal primary traits. After factoring the inter-correlations of these variables by Thurstone's centroid method, approximating the diagonals, the original hypothetical matrix is reproduced with a root mean square discrepancy of .014 by assuming as known the primary trait loadings of only the first eight tests. The method is applied to the results of factoring two batteries of 14 tests, having 8 tests in common, to give the factor loadings of the two batteries on the same reference axes. The method provides a means of comparing directly and quantitatively the results of two different factor studies, provided they have tests in common, and of testing the stability of simple structure under changes in the battery. The relations of the method here developed to certain problems in multiple correlation are shown.     

Matrix correlation

A correlational measure for ann byp matrixX and ann byq matrixY assesses their relation without specifying either as a fixed target. This paper discusses a number of useful measures of correlation, with emphasis on measures which are invariant with respect to rotations or changes in singular values of either matrix. The maximization of matrix correlation with respect to transformationsXL andYM is discussed where one or both transformations are constrained to be orthogonal. Special attention is focussed on transformations which causeXL andYM to ben bys, wheres may be any number between 1 and min (p, q). An efficient algorithm is described for maximizing the correlation betweenXL andYM where analytic solutions do not exist. A factor analytic example is presented illustrating the advantages of various coefficients and of varying the number of columns of the transformed matrices.This research was supported by grant APA 0320 from the Natural Sciences and Engineering Research Council of Canada. The authors wish to acknowledge valuable discussion of this problem with Jan de Leeuw, University of Leiden.     

General theory and methods for matric factoring

Methods are developed for factoring an arbitrary rectangular matrixS of rankr into the formFP, whereF hasr columns andP hasr rows. For the statistical problem of factor analysis,S may be the score matrix of a population of individuals on a battery of tests. ThenF is a matrix of factor loadings,P is a matrix of factor scores, andr is the number of factor variates. (As in current procedures, there remains a subsequent problem of rotation of axes and interpretation of factors, which is not discussed here.) Methods are also developed for factoring an arbitrary Gramian matrixG of rankr into the formFF, whereF hasr columns andF denotesF transposed. For the statistical problem of factor analysis,G may be the matrix of intercorrelations,R, of a battery of tests, with unity, communalities, or other parameters in the principal diagonal.R is proportional toSS, and it is shown thatS can be factored by factoringR. This may usually be the most economical procedure in practice; it should not be overlooked, however, thatS can be factored directly. The general methods build up anF (andP) in as many stages as desired; as many factors as may be deemed computationally practical can be extracted at a time. Perhaps it will usually be found convenient to extract not more than three factors at a time. Current procedures, like the centroid and principal axes, are special cases of a general method presented here for extracting one factor at a time.     

Generating fixed-length sequences satisfying any givennth-order transition probability matrix

An experimental design involving sequences ofm distinct events can be conceptualized as annthorder transition probability matrix specifying the probabilities with which each of themdistinct events is to follow certainn-grams. This paper describes a general method for constructing sequences of shortest possible length that satisfy any such matrix and presents a computer program that randomly generates such sequences.     

Exact probabilities for contingency tables using binomial coefficients

The use of binomial coefficients in place of factorials to shorten the calculation of exact probabilities for 2 × 2 and 2 ×r contingency tables is discussed. A useful set of inequalities for estimating the cumulative probabilities in the tail of the distribution from the probability of a single table is given. A table of binomial coefficients with four significant places andn through 60 is provided.     

EM algorithms for ML factor analysis

The details of EM algorithms for maximum likelihood factor analysis are presented for both the exploratory and confirmatory models. The algorithm is essentially the same for both cases and involves only simple least squares regression operations; the largest matrix inversion required is for aq ×q symmetric matrix whereq is the matrix of factors. The example that is used demonstrates that the likelihood for the factor analysis model may have multiple modes that are not simply rotations of each other; such behavior should concern users of maximum likelihood factor analysis and certainly should cast doubt on the general utility of second derivatives of the log likelihood as measures of precision of estimation.     

Some properties of the communality in multiple factor theory

Several theorems concerning properties of the communaltiy of a test in the Thurstone multiple factor theory are established. The following theorems are applicable to a battery ofn tests which are describable in terms ofr common factors, with orthogonal reference vectors.1. The communality of a testj is equal to the square of the multiple correlation of testj with ther reference vectors.2. The communality of a testj is equal to the square of the multiple correlation of testj with ther reference vectors and then—1 remaining tests. Corollary: The square of the multiple correlation of a testj with then—1 remaining tests is equal to or less than the communality of testj. It cannot exceed the communality.3. The square of the multiple correlation of a testj with then—1 remaining tests equals the communality of testj if the group of tests containsr statistically independent ests teach with a communality of unity.4. With correlation coefficients corrected for attenuation, when the number of tests increases indefinitely while the rank of the correlational matrix remains unchanged, the communality of a testj equals the square of the multiple correlation of testj with then—1 remaining tests.5. With raw correlation coefficients, it is shown in a special case that the square of the multiple correlation of a testj with then—1 remaining tests approaches the communality of testj as a limit when the number of tests increases indefinitely while the rank of correlational matrix remains the same. This has not yet been proved for the general case.The author wishes to express his appreciation of the encouragement and assistance given him by Dr. L. L. Thurstone.     

Comparison of ranks of cross-product and covariance solutions in component analysis

SupposeD is a data matrix forN persons andn variables, and is the matrix obtained fromD by expressing the variables in deviation-score form. It is shown that ifD has rankr, will always have rank (r−1) ifr=N<n, otherwise it will generally have rankr. If has ranks,D will always have ranks ifs=n, but ifs<n it will generally have rank (s+1). Thus two cases can arise, Case A in whichD has rank one greater than , and Case B in whichD has rank equal to . Implications of this distinction for analysis of cross products versus analysis of covariances are briefly indicated.     

Multiple rectilinear prediction and the resolution into components

It is assumed that a battery ofn tests has been resolved into components in a common factor space ofr dimensions and a unique factor space of at mostn dimensions, wherer is much less thann. Simplified formulas for ordinary multiple and partial correlation of tests are derived directly in terms of the components. The best (in the sense of least squares) linear regression equations for predicting factor scores from test scores are derived also in terms of the components. Spearman's single factor prediction formulas emerge as special cases. The last part of the paper shows how the communality is an upper bound for multiple correlation. A necessary and sufficient condition is established for the square of the multiple correlation coefficient of testj on the remainingn—1 tests to approach the communality of testj as a limit asn increases indefinitely whiler remains constant. Limits are established for partial correlation and regression coefficients and for the prediction of factor scores.I am indebted to Professor Dunham Jackson for helpful criticism of most of this paper.     

A semi-orthogonal dependent factor solution

The paper presents a general framework for the dependent factor method in which judgmental as well as analytic criteria may be employed.The procedure involves a semi-orthogonal transformation of an oblique solution comprising a number of reference factors and a number of experimental factors (composites). It determines analytically the residual factors in the experimental field keeping the reference field constant. The method is shown to be a generalization of the multiple factor procedure [Thurstone, 1947] in so far as it depends on the use of generalized inverse in derivingpartial structure from total pattern. It is also shown to provide an example of the previously empty category (Case III) of the Harris-Kaiser generalization [1964]. A convenient computational procedure is provided. It is based on an extension of Aitken's [1937] method of pivotal condensation of a triple-product matrix to the evaluation of a matrix of the formH — VA ^–1 U' (for a nonsingularA).     

Retrieving the correlation matrix from a truncated PCA solution: The inverse principal component problem

Whenr Principal Components are available fork variables, the correlation matrix is approximated in the least squares sense by the loading matrix times its transpose. The approximation is generally not perfect unlessr =k. In the present paper it is shown that, whenr is at or above the Ledermann bound,r principal components are enough to perfectly reconstruct the correlation matrix, albeit in a way more involved than taking the loading matrix times its transpose. In certain cases just below the Ledermann bound, recovery of the correlation matrix is still possible when the set of all eigenvalues of the correlation matrix is available as additional information.     

Multiple rectilinear prediction and the resolution into components: II

Given a battery ofn tests that has already been resolved intor orthogonal common factors andn unique factors, procedures are outlined for computing the following types of linear multiple regressions directly from the factor loadings: (i) the regression of any one test on then?1 remaining tests; (ii) all then different regressions of ordern?1 for then tests, computed simultaneously; (iii) the regression of any common factor on then tests; (iv) the regressions of all the common factors on then tests computed simultaneously; (v) the regression of any unique factor on then tests; (vi) the regressions of all the unique factors on then tests, computed simultaneously. Multiple and partial correlations are then determined by ordinary formulas from the regression coefficients. A worksheet with explicit instructions is provided, with a completely worked out example. Computing these regressions directly from the factor loadings is a labor-saving device, the efficiency of which increases as the number of tests increases. The amount of labor depends essentially on the number of common factors. This is in contrast to computations based on the original test intercorrelations, where the amount of labor increases more than proportionately as the number of tests increases. The procedures evaluate formulas developed in a previous paper (2). They are based essentially on a shortened way of computing the inverse of the test intercorrelation matrix by use of the factor loadings.     

Enumeration of paths in digraphs

The solution of the problem of enumeration of then-paths in a digraph has so far been attempted through an indirect approach of enumerating the redundant chains. The approach has yielded an algorithm for determination of the general formula for the matrix of redundantn-chains and also a partial recurrence formula for the same. This paper presents a direct approach to the problem. It gives a recurrence relation expressing the matrix ofn-paths of a digraph in terms of the matrices of (n – 1)-paths of its first-order subgraphs. The result is exploited to give an algorithm for computing the matrix ofn-paths. The algorithm is illustrated with a 6 × 6 matrix.The writer wishes to express his indebtedness to Dr. A. K. Gayen of Indian Institute of Technology, Kharagpur, for constant guidance and encouragement. Thanks are also due to Dr. S. N. N. Pandit of the Indian Institute of Technology, Kharagpur, and Shri M. T. Subrahmanya of Indian Statistical Institute, Calcutta, for helpful suggestions.     

The contribution of an orthogonal multiple factor solution to multiple correlation

A method is indicated by which multiple factor analysis may be used in determining a number,r, and then in selectingr predicting variables out ofn variables so that each of the remainingn-r variables may be predicted almost as well from ther variables as it could be predicted from all then—1 variables.     

Kruskal's polynomial for 2×2×2 arrays and a generalization to 2×n×n arrays

A remarkable difference between the concept of rank for matrices and that for three-way arrays has to do with the occurrence of non-maximal rank. The set ofn×n matrices that have a rank less thann has zero volume. Kruskal pointed out that a 2×2×2 array has rank three or less, and that the subsets of those 2×2×2 arrays for which the rank is two or three both have positive volume. These subsets can be distinguished by the roots of a certain polynomial. The present paper generalizes Kruskal's results to 2×n×n arrays. Incidentally, it is shown that twon ×n matrices can be diagonalized simultaneously with positive probability.The author is obliged to Joe Kruskal and Henk Kiers for commenting on an earlier draft, and to Tom Wansbeek for raising stimulating questions.     

Investigating the Performance of Alternate Regression Weights by Studying All Possible Criteria in Regression Models with a Fixed Set of Predictors

We describe methods for assessing all possible criteria (i.e., dependent variables) and subsets of criteria for regression models with a fixed set of predictors, x (where x is an n×1 vector of independent variables). Our methods build upon the geometry of regression coefficients (hereafter called regression weights) in n-dimensional space. For a full-rank predictor correlation matrix, R _xx, of order n, and for regression models with constant R ² (coefficient of determination), the OLS weight vectors for all possible criteria terminate on the surface of an n-dimensional ellipsoid. The population performance of alternate regression weights—such as equal weights, correlation weights, or rounded weights—can be modeled as a function of the Cartesian coordinates of the ellipsoid. These geometrical notions can be easily extended to assess the sampling performance of alternate regression weights in models with either fixed or random predictors and for models with any value of R ². To illustrate these ideas, we describe algorithms and R (R Development Core Team, 2009) code for: (1) generating points that are uniformly distributed on the surface of an n-dimensional ellipsoid, (2) populating the set of regression (weight) vectors that define an elliptical arc in ℝ ⁿ , and (3) populating the set of regression vectors that have constant cosine with a target vector in ℝ ⁿ . Each algorithm is illustrated with real data. The examples demonstrate the usefulness of studying all possible criteria when evaluating alternate regression weights in regression models with a fixed set of predictors.