Minimum rank and minimum trace of covariance matrices

This paper considers some mathematical aspects of minimum trace factor analysis (MTFA). The uniqueness of an optimal point of MTFA is proved and necessary and sufficient conditions for a point x to be optimal are established. Finally, some results about the connection between MTFA and the classical minimum rank factor analysis will be presented.     

Constructing a covariance matrix that yields a specified minimizer and a specified minimum discrepancy function value

A method is presented for constructing a covariance matrix Σ*₀ that is the sum of a matrix Σ(γ₀) that satisfies a specified model and a perturbation matrix,E, such that Σ*₀=Σ(γ0) +E. The perturbation matrix is chosen in such a manner that a class of discrepancy functionsF(Σ*₀, Σ(γ₀)), which includes normal theory maximum likelihood as a special case, has the prespecified parameter value γ₀ as minimizer and a prespecified minimum δ A matrix constructed in this way seems particularly valuable for Monte Carlo experiments as the covariance matrix for a population in which the model does not hold exactly. This may be a more realistic conceptualization in many instances. An example is presented in which this procedure is employed to generate a covariance matrix among nonnormal, ordered categorical variables which is then used to study the performance of a factor analysis estimator. We are grateful to Alexander Shapiro for suggesting the proof of the solution in section 2.     

A note on the uniqueness of minimum rank solutions in factor analysis

A revised theorem is presented concerning uniqueness of minimum rank solutions in common factor analysis.     

Iterative least squares estimates of communality: Initial estimate need not affect stabilized value

A common criticism of iterative least squares estimates of communality is that method of initial estimation may influence stabilized values. As little systematic research on this topic has been performed, the criticism appears to be based on cumulated experience with empirical data sets. In the present paper, two studies are reported in which four types of initial estimate (unities, squared multiple correlations, highestr, and zeroes) and four levels of convergence criterion were employed using four widely available computer packages (BMDP, SAS, SPSS, and SOUPAC). The results suggest that initial estimates have no effect on stabilized communality estimates when a stringent criterion for convergence is used, whereas initial estimates appear to affect stabilized values employing rather gross convergence criteria. There were no differences among the four computer packages for matrices without Heywood cases.     

A note on multiple and canonical correlation for a singular covariance matrix

A weaker generalized inverse (Rao's g-inverse; Graybill's c-inverse) can be used in place of the Moore-Penrose generalized inverse to obtain multiple and canonical correlations from singular covariance matrices.The author expresses his gratitude to a referee for his suggestions.     

Rank-reducibility of a symmetric matrix and sampling theory of minimum trace factor analysis

One of the intriguing questions of factor analysis is the extent to which one can reduce the rank of a symmetric matrix by only changing its diagonal entries. We show in this paper that the set of matrices, which can be reduced to rankr, has positive (Lebesgue) measure if and only ifr is greater or equal to the Ledermann bound. In other words the Ledermann bound is shown to bealmost surely the greatest lower bound to a reduced rank of the sample covariance matrix. Afterwards an asymptotic sampling theory of so-called minimum trace factor analysis (MTFA) is proposed. The theory is based on continuous and differential properties of functions involved in the MTFA. Convex analysis techniques are utilized to obtain conditions for differentiability of these functions.     

Computational aspects of the greatest lower bound to the reliability and constrained minimum trace factor analysis

In the last decade several algorithms for computing the greatest lower bound to reliability or the constrained minimum-trace communality solution in factor analysis have been developed. In this paper convergence properties of these methods are examined. Instead of using Lagrange multipliers a new theorem is applied that gives a sufficient condition for a symmetric matrix to be Gramian. Whereas computational pitfalls for two methods suggested by Woodhouse and Jackson can be constructed it is shown that a slightly modified version of one method suggested by Bentler and Woodward can safely be applied to any set of data. A uniqueness proof for the solution desired is offered.The authors are obliged to Charles Lewis and Dirk Knol for helpful comments, and to Frank Brokken and Henk Camstra for developing computer programs.     

Statistical inference of minimum rank factor analysis

For any given number of factors, Minimum Rank Factor Analysis yields optimal communalities for an observed covariance matrix in the sense that the unexplained common variance with that number of factors is minimized, subject to the constraint that both the diagonal matrix of unique variances and the observed covariance matrix minus that diagonal matrix are positive semidefinite. As a result, it becomes possible to distinguish the explained common variance from the total common variance. The percentage of explained common variance is similar in meaning to the percentage of explained observed variance in Principal Component Analysis, but typically the former is much closer to 100 than the latter. So far, no statistical theory of MRFA has been developed. The present paper is a first start. It yields closed-form expressions for the asymptotic bias of the explained common variance, or, more precisely, of the unexplained common variance, under the assumption of multivariate normality. Also, the asymptotic variance of this bias is derived, and also the asymptotic covariance matrix of the unique variances that define a MRFA solution. The presented asymptotic statistical inference is based on a recently developed perturbation theory of semidefinite programming. A numerical example is also offered to demonstrate the accuracy of the expressions.This work was supported, in part, by grant DMS-0073770 from the National Science Foundation.     

An alternative to the methodology for analysis of covariance

A general statistical model for simultaneous analysis of data from several groups is described. The model is primarily designed to be used for the analysis of covariance. The model can handle any number of covariates and criterion variables, and any number of treatment groups. Treatment effects may be assessed when the treatment groups are not randomized. In addition, the model allows for measurement errors in the criterion variables as well as in the covariates. A wide variety of hypotheses concerning the parameters of the model can be tested by means of a large sample likelihood ratio test. In particular, the usual assumptions of ANCOVA may be tested.Research reported in this paper has been partly supported by the Swedish Council for Social Science Research under project Statistical methods for analysis of longitudinal data, project director Karl G. Jöreskog, and partly by the Bank of Sweden Tercentenary Foundation under project Structural Equation Models in the Social Sciences, project director Karl G. Jöreskog.     

An estimate of the covariance between variables which are not jointly observed

Situations sometimes arise in which variables collected in a study are not jointly observed. This typically occurs because of study design. An example is an equating study where distinct groups of subjects are administered different sections of a test. In the normal maximum likelihood function to estimate the covariance matrix among all variables, elements corresponding to those that are not jointly observed are unidentified. If a factor analysis model holds for the variables, however, then all sections of the matrix can be accurately estimated, using the fact that the covariances are a function of the factor loadings. Standard errors of the estimated covariances can be obtained by the delta method. In addition to estimating the covariance matrix in this design, the method can be applied to other problems such as regression factor analysis. Two examples are presented to illustrate the method. This research was partially supported by NIMH grant MH5-4576     

A simplification of a result by zellini on the maximal rank of symmetric three-way arrays

Zellini (1979, Theorem 3.1) has shown how to decompose an arbitrary symmetric matrix of ordern ×n as a linear combination of 1/2n(n+1) fixed rank one matrices, thus constructing an explicit tensor basis for the set of symmetricn ×n matrices. Zellini's decomposition is based on properties of persymmetric matrices. In the present paper, a simplified tensor basis is given, by showing that a symmetric matrix can also be decomposed in terms of 1/2n(n+1) fixed binary matrices of rank one. The decomposition implies that ann ×n ×p array consisting ofp symmetricn ×n slabs has maximal rank 1/2n(n+1). Likewise, an unconstrained INDSCAL (symmetric CANDECOMP/PARAFAC) decomposition of such an array will yield a perfect fit in 1/2n(n+1) dimensions. When the fitting only pertains to the off-diagonal elements of the symmetric matrices, as is the case in a version of PARAFAC where communalities are involved, the maximal number of dimensions can be further reduced to 1/2n(n–1). However, when the saliences in INDSCAL are constrained to be nonnegative, the tensor basis result does not apply. In fact, it is shown that in this case the number of dimensions needed can be as large asp, the number of matrices analyzed.     

A note on the factor analysis of partial covariance matrices

A relationship is given between the joint common factor structure of two sets of variables, and the factor structure of the partial covariance matrix of one of the sets with the other partialled out.     

The asymptotic bias of minimum trace factor analysis, with applications to the greatest lower bound to reliability

In theory, the greatest lower bound (g.l.b.) to reliability is the best possible lower bound to the reliability based on single test administration. Yet the practical use of the g.l.b. has been severely hindered by sampling bias problems. It is well known that the g.l.b. based on small samples (even a sample of one thousand subjects is not generally enough) may severely overestimate the population value, and statistical treatment of the bias has been badly missing. The only results obtained so far are concerned with the asymptotic variance of the g.l.b. and of its numerator (the maximum possible error variance of a test), based on first order derivatives and the asumption of multivariate normality. The present paper extends these results by offering explicit expressions for the second order derivatives. This yields a closed form expression for the asymptotic bias of both the g.l.b. and its numerator, under the assumptions that the rank of the reduced covariance matrix is at or above the Ledermann bound, and that the nonnegativity constraints on the diagonal elements of the matrix of unique variances are inactive. It is also shown that, when the reduced rank is at its highest possible value (i.e., the number of variables minus one), the numerator of the g.l.b. is asymptotically unbiased, and the asymptotic bias of the g.l.b. is negative. The latter results are contrary to common belief, but apply only to cases where the number of variables is small. The asymptotic results are illustrated by numerical examples.This research was supported by grant DMI-9713878 from the National Science Foundation.     

Multi‐set factor analysis by means of Parafac2

We consider multi‐set data consisting of observations, k = 1,…, K (e.g., subject scores), on J variables in K different samples. We introduce a factor model for the J × J covariance matrices , k = 1,…, K, where the common part is modelled by Parafac2 and the unique variances , k = 1,…, K, are diagonal. The Parafac2 model implies a common loadings matrix that is rescaled for each k, and a common factor correlation matrix. We estimate the unique variances by minimum rank factor analysis on for each k. The factors can be chosen orthogonal or oblique. We present a novel algorithm to estimate the Parafac2 part and demonstrate its performance in a simulation study. Also, we fit our model to a data set in the literature. Our model is easy to estimate and interpret. The unique variances, the factor correlation matrix and the communalities are guaranteed to be proper, and a percentage of explained common variance can be computed for each k. Also, the Parafac2 part is rotationally unique under mild conditions.     

Asymptotic distributions of the estimators of communalities in factor analysis

Asymptotic distributions of the estimators of communalities are derived for the maximum likelihood method in factor analysis. It is shown that the common practice of equating the asymptotic standard error of the communality estimate to the unique variance estimate is correct for standardized communality but not correct for unstandardized communality. In a Monte Carlo simulation the accuracy of the normal approximation to the distributions of the estimators are assessed when the sample size is 150 or 300. This study was carried out in part under the ISM Cooperative Research Program (90-ISM-CRP-9).     

Structural analysis of covariance and correlation matrices

A general approach to the analysis of covariance structures is considered, in which the variances and covariances or correlations of the observed variables are directly expressed in terms of the parameters of interest. The statistical problems of identification, estimation and testing of such covariance or correlation structures are discussed.Several different types of covariance structures are considered as special cases of the general model. These include models for sets of congeneric tests, models for confirmatory and exploratory factor analysis, models for estimation of variance and covariance components, regression models with measurement errors, path analysis models, simplex and circumplex models. Many of the different types of covariance structures are illustrated by means of real data.1978 Psychometric Society Presidential Address.This research has been supported by the Bank of Sweden Tercentenary Foundation under the project entitledStructural Equation Models in the Social Sciences, Karl G. Jöreskog, project director.     

Population models and simulation methods: The case of the Spearman rank correlation

The purpose of this paper is to highlight the importance of a population model in guiding the design and interpretation of simulation studies used to investigate the Spearman rank correlation. The Spearman rank correlation has been known for over a hundred years to applied researchers and methodologists alike and is one of the most widely used non‐parametric statistics. Still, certain misconceptions can be found, either explicitly or implicitly, in the published literature because a population definition for this statistic is rarely discussed within the social and behavioural sciences. By relying on copula distribution theory, a population model is presented for the Spearman rank correlation, and its properties are explored both theoretically and in a simulation study. Through the use of the Iman–Conover algorithm (which allows the user to specify the rank correlation as a population parameter), simulation studies from previously published articles are explored, and it is found that many of the conclusions purported in them regarding the nature of the Spearman correlation would change if the data‐generation mechanism better matched the simulation design. More specifically, issues such as small sample bias and lack of power of the t‐test and r‐to‐z Fisher transformation disappear when the rank correlation is calculated from data sampled where the rank correlation is the population parameter. A proof for the consistency of the sample estimate of the rank correlation is shown as well as the flexibility of the copula model to encompass results previously published in the mathematical literature.     

A note on the minres method

This paper discusses the advantages and problems related to factor analysis by minimizing residuals (minres). It is shown that this method fails if the starting point of iterations is not well chosen. A suitable starting point is suggested.     

An incomplete data approach to the analysis of covariance structures

In this paper, linear structural equation models with latent variables are considered. It is shown how many common models arise from incomplete observation of a relatively simple system. Subclasses of models with conditional independence interpretations are also discussed. Using an incomplete data point of view, the relationships between the incomplete and complete data likelihoods, assuming normality, are highlighted. For computing maximum likelihood estimates, the EM algorithm and alternatives are surveyed. For the alternative algorithms, simplified expressions for computing function values and derivatives are given. Likelihood ratio tests based on complete and incomplete data are related, and an example on using their relationship to improve the fit of a model is given.This research forms part of the author's doctoral thesis and was supported by a Commonwealth Postgraduate Research Award. The author also wishes to acknowledge the support of CSIRO during the preparation of this paper and the referees' comments which led to substantial improvements.     

Relationship of social rank in mice to growth,endurance, and fertility

The influence of the social rank of male mice, determined by the outcome of fights, was assessed on their growth, treadmill performance, open-field behavior, and morphometric traits. Fertility of mated females was also investigated. Special attention was paid to the relationship between the male's social rank and body weight. Winners of fights were heavier than losers; their latencies were shorter, and they showed more locomotor activity in an open field test. Winners also had higher absolute testicular and epididymal weights. These males had a positive influence on the reproductive fitness of the females with which they were mated.