Properties of the maximum likelihood solution in factor analysis regression

Algebraic properties of the normal theory maximum likelihood solution in factor analysis regression are investigated. Two commonly employed measures of the within sample predictive accuracy of the factor analysis regression function are considered: the variance of the regression residuals and the squared correlation coefficient between the criterion variable and the regression function. It is shown that this within sample residual variance and within sample squared correlation may be obtained directly from the factor loading and unique variance estimates, without use of the original observations or the sample covariance matrix.     

A note on some modifications of latent roots and vectors

When an arbitrary positive scalar matrix is added to a correlation matrix the latent roots of the sum are equal to the corresponding roots of the correlation matrix plus an amount equal to the scalar number of the scalar matrix. The latent vectors of the sum are identical with those of the correlation matrix. An approximation to these relationships is suggested for the case in which the sum is of a correlation matrix and of a positive semidefinite diagonal matrix. The approximation is used to allow the solution of a characteristic problem for a correlation matrix with unities in the main diagonal to provide a family of solutions for the same correlation matrix.This research has been supported by a grant from the National Institute of Mental Health, MH 7864-01.     

Validating a selection test,a predictive probability approach

The problem of inferring the validity of a selection test (x) as a predictor of some criterion (y) when completexy data are not available is investigated. The basic approach is to construct the predictive probability distribution of the unobservedy scores and then derive interval estimates of the least squares regression weights, the difference in averagey scores for selected and unselected cases, and the residual variance in predictingy fromx. Further, an approximation to the predictive distribution of the squared correlation betweenx andy in a future group is derived.     

Retrieving the correlation matrix from a truncated PCA solution: The inverse principal component problem

Whenr Principal Components are available fork variables, the correlation matrix is approximated in the least squares sense by the loading matrix times its transpose. The approximation is generally not perfect unlessr =k. In the present paper it is shown that, whenr is at or above the Ledermann bound,r principal components are enough to perfectly reconstruct the correlation matrix, albeit in a way more involved than taking the loading matrix times its transpose. In certain cases just below the Ledermann bound, recovery of the correlation matrix is still possible when the set of all eigenvalues of the correlation matrix is available as additional information.     

Explained Variance and Intraclass Correlation in a Two-Level AR(1) Model

The total variance of a first-order autoregressive AR(1) time series is well known in time series literature. However, despite the increased use and interest in two-level AR(1) models, an equation for the total variance of these models does not exist. This paper presents an approximation of this total variance. It will be used to compute the unexplained and explained variance at each level of the model, the proportion of explained variance, and the intraclass correlation (ICC). The use of these variances and the ICC will be illustrated using an example concerning structured diary data about the positive affect of 96 married women.     

Bias and error in multiple-choice tests

A formula for estimating real scores on a multiple-choice test from a knowledge of raw scores is derived. This formula does not involve the assumption of a binomial distribution of real scores as does the Calandra formula. Other important formulas derived show: the variance of real scores in terms of the variance of raw scores and the correlation between real scores and raw scores. If the variance of real scores (or of raw scores also) is binomial, the regression of real scores on raw scores is linear; but, otherwise the regression is curvilinear. Yet the linear estimating formula is a close approximation to the curvilinear relationship. Factors affecting the regression of real scores on raw scores and the correlation coefficient are: (1) the number of choices per question; (2) the number of questions answered; (3) the ratio of the average group raw score to the variance of raw scores.     

Relationships between two systems of factor analysis

Considering only population values, it is shown that the complete set of factors of a correlation matrix with units in the diagonal cells may be transformed into the factors derived by factoring these correlations with communalities in the diagonal cells. When the correlations are regarded as observed values, the common factors derived as a transformation of the complete set of factors of the correlation matrix with units in the diagonal cells satisfy Lawley's requirement for a maximum likelihood solution and are a first approximation to Rao's canonical factors.     

Multilevel models for multiple-baseline data: modeling across-participant variation in autocorrelation and residual variance

Multilevel models (MLM) have been used as a method for analyzing multiple-baseline single-case data. However, some concerns can be raised because the models that have been used assume that the Level-1 error covariance matrix is the same for all participants. The purpose of this study was to extend the application of MLM of single-case data in order to accommodate across-participant variation in the Level-1 residual variance and autocorrelation. This more general model was then used in the analysis of single-case data sets to illustrate the method, to estimate the degree to which the autocorrelation and residual variances differed across participants, and to examine whether inferences about treatment effects were sensitive to whether or not the Level-1 error covariance matrix was allowed to vary across participants. The results from the analyses of five published studies showed that when the Level-1 error covariance matrix was allowed to vary across participants, some relatively large differences in autocorrelation estimates and error variance estimates emerged. The changes in modeling the variance structure did not change the conclusions about which fixed effects were statistically significant in most of the studies, but there was one exception. The fit indices did not consistently support selecting either the more complex covariance structure, which allowed the covariance parameters to vary across participants, or the simpler covariance structure. Given the uncertainty in model specification that may arise when modeling single-case data, researchers should consider conducting sensitivity analyses to examine the degree to which their conclusions are sensitive to modeling choices.     

Matrix reduction and approximation to principal axes

A modification of Hotelling's iteration method of factor analysis is presented which is much more rapid and almost as accurate. At any stage of the approximation for a factor vector its major product moment reduces the rank of the residual matrix by precisely one. Each approximation to an eigenvalue is larger than the preceding one. By observing the decline in these increments one can often stop the iterations at early stages without serious loss. If subsequent rotational procedures are used, the method gives practically the same results as the more exact methods and in a small fraction of the time.This study was supported in part by Office of Naval Research Contract Nonr-477(08) and Public Health Research Grant M-743 (C6).     

Meta-analysis of the relationship between congruence and well-being measures

This paper summarizes the results of a meta-analysis on findings of 41 different studies, yielding 77 correlation coefficients between measures of personality-environment congruence and well-being. Results show congruence-achievement and congruence-stability correlations of .06 and .15, respectively, with negligible residual variance. The mean congruence-satisfaction correlation was .21, and after two further breakdowns—by environmental reference and congruence measuring method—mean congruence-satisfaction correlations exceeding .35 were found, with almost all total nonrandom variance among findings of different congruence studies explained.     

Residual Normality Assumption and the Estimation of Multiple Membership Random Effects Models

While conventional hierarchical linear modeling is applicable to purely hierarchical data, a multiple membership random effects model (MMrem) is appropriate for nonpurely nested data wherein some lower-level units manifest mobility across higher-level units. Although a few recent studies have investigated the influence of cluster-level residual nonnormality on hierarchical linear modeling estimation for purely hierarchical data, no research has examined the statistical performance of an MMrem given residual non-normality. The purpose of the present study was to extend prior research on the influence of residual non-normality from purely nested data structures to multiple membership data structures. Employing a Monte Carlo simulation study, this research inquiry examined two-level MMrem parameter estimate biases and inferential errors. Simulation factors included the level-two residual distribution, sample sizes, intracluster correlation coefficient, and mobility rate. Results showed that estimates of fixed effect parameters and the level-one variance component were robust to level-two residual non-normality. The level-two variance component, however, was sensitive to level-two residual non-normality and sample size. Coverage rates of the 95% credible intervals deviated from the nominal value assumed when level-two residuals were non-normal. These findings can be useful in the application of an MMrem to account for the contextual effects of multiple higher-level units.     

The evaluation of multiple and partial correlation coefficients from the factorial matrix

This paper shows how to compute multiple correlation coefficients, partial correlation coefficients, and regression coefficients from the factorial matrix. Special emphasis is given to computation technique and to approximation formulas. The method is extremely flexible in application since it may be applied to any subset of the original set of observed variables. It is also extremely useful when many of these coefficients are desired.     

Testing for independence between pairs of autocorrelated binomial data sequences

A problem arises in analyzing the existence of interdependence between the behavioral sequences of two individuals: tests involving a statistic such as chi-square assume independent observations within each behavioral sequence, a condition which may not exist in actual practice. Using Monte Carlo simulations of binomial data sequences, we found that the use of a chi-square test frequently results in unacceptable Type I error rates when the data sequences are autocorrelated. We compared these results to those from two other methods designed specifically for testing for intersequence independence in the presence of intrasequence autocorrelation. The first method directly tests the intersequence correlation using an approximation of the variance of the intersequence correlation estimated from the sample autocorrelations. The second method uses tables of critical values of the intersequence correlation computed by Nakamuraet al. (J. Am. Stat. Assoc., 1976,71, 214–222). Although these methods were originally designed for normally distributed data, we found that both methods produced much better results than the uncorrected chi-square test when applied to binomial autocorrelated sequences. The superior method appears to be the variance approximation method, which resulted in Type I error rates that were generally less than or equal to 5% when the level of significance was set at .05.     

The determination of successive principal components without computation of tables of residual correlation coefficients

A procedure is presented for determining the successive principal components of a correlation matrix where it is not necessary to compute the successive tables of residual correlations. The original correlation matrix is bordered with a new row and column for each principal component that is determined.This paper is one of a series of reports on the development of multiple factor analysis in the study of primary human abilities which have been supported by research grants from the Carnegie Corporation of New York and the facilities of the Psychometric Laboratory, which have been provided by the Social Science Research Committee of The University of Chicago.     

Fitting the factor analysis model in ℓ1 norm

The well‐known problem of fitting the exploratory factor analysis model is reconsidered where the usual least squares goodness‐of‐fit function is replaced by a more resistant discrepancy measure, based on a smooth approximation of the ?₁ norm. Fitting the factor analysis model to the sample correlation matrix is a complex matrix optimization problem which requires the structure preservation of the unknown parameters (e.g. positive definiteness). The projected gradient approach is a natural way of solving such data matching problems as especially designed to follow the geometry of the model parameters. Two reparameterizations of the factor analysis model are considered. The approach leads to globally convergent procedures for simultaneous estimation of the factor analysis matrix parameters. Numerical examples illustrate the algorithms and factor analysis solutions.     

Statistical inference of minimum rank factor analysis

For any given number of factors, Minimum Rank Factor Analysis yields optimal communalities for an observed covariance matrix in the sense that the unexplained common variance with that number of factors is minimized, subject to the constraint that both the diagonal matrix of unique variances and the observed covariance matrix minus that diagonal matrix are positive semidefinite. As a result, it becomes possible to distinguish the explained common variance from the total common variance. The percentage of explained common variance is similar in meaning to the percentage of explained observed variance in Principal Component Analysis, but typically the former is much closer to 100 than the latter. So far, no statistical theory of MRFA has been developed. The present paper is a first start. It yields closed-form expressions for the asymptotic bias of the explained common variance, or, more precisely, of the unexplained common variance, under the assumption of multivariate normality. Also, the asymptotic variance of this bias is derived, and also the asymptotic covariance matrix of the unique variances that define a MRFA solution. The presented asymptotic statistical inference is based on a recently developed perturbation theory of semidefinite programming. A numerical example is also offered to demonstrate the accuracy of the expressions.This work was supported, in part, by grant DMS-0073770 from the National Science Foundation.     

STATISTICAL AND PSYCHOMETRIC INFERENCE IN PRINCIPAL COMPONENTS ANALYSIS

Statistical inference applied to principal components analysis deals with estimating the parameters of the correlation matrix, R, found in the population, from the characteristics of the sample matrix, R*. On the other hand, psychometric inference refers to estimating the internal consistency of the components themselves, so that the decisions about retaining a component for further analysis can be based upon psychometric criteria. A slightly modified approach to statistical inference, which focuses upon the variance of the Components in the population, has been suggested. This viewpoint can be extended to estimating the true score variance and the reliabilities of the components in the population of subjects. Psychometric tests of significance can then be made statistical in nature.     

Sensitivity analysis in factor analysis: Difference between using covariance and correlation matrices

Influence curves of some parameters under various methods of factor analysis have been given in the literature. These influence curves depend on the influence curves for either the covariance or the correlation matrix used in the analysis. The differences between the influence curves based on the covariance and the correlation matrices are derived in this paper. Simple formulas for the differences of the influence curves, based on the two matrices, for the unique variance matrix, factor loadings and some other parameter are obtained under scale-invariant estimation methods, though the influence curves themselves are in complex forms.The authors are most grateful to the referees, the Associate Editor, the Editor and Raymond Lam for helpful suggestions for improving the clarity of the paper.     

Estimation of the correlation coefficient in the case of a bivariate normal population when one of the variables is dichotomized

It is shown that the problem of estimation of the correlation coefficient of a bivariate normal population when one of the variables is dichotomized may be attacked with “probit analysis” methods. This represents an extension of the work of Gillman and Goode (3), as it was possible to find by this approach an approximation to the large-sample variance of the resulting estimateG ofρ. An empirical investigation was undertaken with the object of obtaining some information about the distribution ofG for large sample size. Methods for determining the “pass-fail” cut-off are considered.     

A criterion for significant common factor variance

Up to the present only empirical methods have been available for determining the number of factors to be extracted from a matrix of correlations. The problem has been confused by the implicit attitude that a matrix of intercorrelations between psychological variables has a rank which is determinable. A table of residuals always contains error variance and common factor variance. The extraction of successive factors increases the proportion of error variance remaining to common factor variance remaining, and a point is reached where the extraction of more dimensions would contain so much error variance that the common factor variance would be overshadowed. The critical value for this point is determined by probability theory and does not take into account the size of the residuals. Interpretation of the criterion is discussed.