Estimation of covariance structure models with parameters subject to functional restraints

This paper demonstrates the feasibility of using the penalty function method to estimate parameters that are subject to a set of functional constraints in covariance structure analysis. Both types of inequality and equality constraints are studied. The approaches of maximum likelihood and generalized least squares estimation are considered. A modified Scoring algorithm and a modified Gauss-Newton algorithm are implemented to produce the appropriate constrained estimates. The methodology is illustrated by its applications to Heywood cases in confirmatory factor analysis, quasi-Weiner simplex model, and multitrait-multimethod matrix analysis.The author is indebted to several anonymous reviewers for creative suggestions for improvement of this paper. Computer funding is provided by the Computer Services Centre, The Chinese University of Hong Kong.     

A note on the sampling properties of the Vincentizing (quantile averaging) procedure

To assess the effect of a manipulation on a response time distribution, psychologists often use Vincentizing or quantile averaging to construct group or “average” distributions. We provide a theorem characterizing the large sample properties of the averaged quantiles when the individual RT distributions all belong to the same location-scale family. We then apply the theorem to estimating parameters for the quantile-averaged distributions. From the theorem, it is shown that parameters of the group distribution can be estimated by generalized least squares. This method provides accurate estimates of standard errors of parameters and can therefore be used in formal inference. The method is benchmarked in a small simulation study against both a maximum likelihood method and an ordinary least-squares method. Generalized least squares essentially is the only method based on the averaged quantiles that is both unbiased and provides accurate estimates of parameter standard errors. It is also proved that for location-scale families, performing generalized least squares on quantile averages is formally equivalent to averaging parameter estimates from generalized least squares performed on individuals. A limitation on the method is that individual RT distributions must be members of the same location-scale family.     

Estimating parameters of the diffusion model: approaches to dealing with contaminant reaction times and parameter variability

Three methods for fitting the diffusion model (Ratcliff, 1978) to experimental data are examined. Sets of simulated data were generated with known parameter values, and from fits of the model, we found that the maximum likelihood method was better than the chi-square and weighted least squares methods by criteria of bias in the parameters relative to the parameter values used to generate the data and standard deviations in the parameter estimates. The standard deviations in the parameter values can be used as measures of the variability in parameter estimates from fits to experimental data. We introduced contaminant reaction times and variability into the other components of processing besides the decision process and found that the maximum likelihood and chi-square methods failed, sometimes dramatically. But the weighted least squares method was robust to these two factors. We then present results from modifications of the maximum likelihood and chi-square methods, in which these factors are explicitly modeled, and show that the parameter values of the diffusion model are recovered well. We argue that explicit modeling is an important method for addressing contaminants and variability in nondecision processes and that it can be applied in any theoretical approach to modeling reaction time.     

Covariance structure analysis in several populations

This paper is concerned with the study of covariance structural models in several populations. Estimation theory of the parameters that are subject to general functional restraints is developed based on the generalized least squares approach. Asymptotic properties of the constrained estimator are studied; and asymptotic chi-square tests are presented to evaluate appropriate model comparisons. The method of multipliers and the standard reparametrization technique are discussed in obtaining the estimates. The methodology is demonstrated by a set of real data.Computer facilities were provided by the Computer Services Center, The Chinese University of Hong Kong. The authors are indebted to several anonymous reviewers for suggestions for improvement of this paper.     

Allowing for correlations between correlations in random-effects meta-analysis of correlation matrices

Practical meta-analysis of correlation matrices generally ignores covariances (and hence correlations) between correlation estimates. The authors consider various methods for allowing for covariances, including generalized least squares, maximum marginal likelihood, and Bayesian approaches, illustrated using a 6-dimensional response in a series of psychological studies concerning prediction of exercise behavior change. Quantities of interest include the overall population mean correlation matrix, the contrast between the mean correlations, the predicted correlation matrix in a new study, and the conflict between the existing studies and a new correlation matrix. The authors conclude that accounting for correlations between correlations is unnecessary when interested in individual correlations but potentially important if concerned with a composite measure involving 2 or more correlations. A simulation study indicates the asymptotic normal assumption appears reasonable. Because of potential instability in the generalized least squares methods, they recommend a model-based approach, either the maximum marginal likelihood approach or a full Bayesian analysis.     

A class of factor analysis estimation procedures with common asymptotic sampling properties

A general class of estimation procedures for the factor model is considered. The procedures are shown to yield estimates possessing the same asymptotic sampling properties as those from estimation by maximum likelihood or generalized least squares, both of which are special members of the class. General expressions for the derivatives needed for Newton-Raphson determination of the estimates are derived. Numerical examples are given, and the effect of the choice of estimation procedure is discussed.The author wishes to thank Dr. W. N. Venables for his encouragement and helpful suggestions throughout the preparation of this paper, and a reviewer whose comments on an earlier version led to the basic approach used in appendix B to the asymptotic theory.     

Structural equation models with continuous and polytomous variables

A two-stage procedure is developed for analyzing structural equation models with continuous and polytomous variables. At the first stage, the maximum likelihood estimates of the thresholds, polychoric covariances and variances, and polyserial covariances are simultaneously obtained with the help of an appropriate transformation that significantly simplifies the computation. An asymptotic covariance matrix of the estiates is also computed. At the second stage, the parameters in the structural covariance model are obtained via the generalized least squares approach. Basic statistical properties of the estimates are derived and some illustrative examples and a small simulation study are reported.This research was supported in part by a research grant DA01070 from the U. S. Public Health Service. We are indebted to several referees and the editor for very valuable comments and suggestions for improvement of this paper. The computing assistance of King-Hong Leung and Man-Lai Tang is also gratefully acknowledged.     

A study of algorithms for covariance structure analysis with specific comparisons using factor analysis

Several algorithms for covariance structure analysis are considered in addition to the Fletcher-Powell algorithm. These include the Gauss-Newton, Newton-Raphson, Fisher Scoring, and Fletcher-Reeves algorithms. Two methods of estimation are considered, maximum likelihood and weighted least squares. It is shown that the Gauss-Newton algorithm which in standard form produces weighted least squares estimates can, in iteratively reweighted form, produce maximum likelihood estimates as well. Previously unavailable standard error estimates to be used in conjunction with the Fletcher-Reeves algorithm are derived. Finally all the algorithms are applied to a number of maximum likelihood and weighted least squares factor analysis problems to compare the estimates and the standard errors produced. The algorithms appear to give satisfactory estimates but there are serious discrepancies in the standard errors. Because it is robust to poor starting values, converges rapidly and conveniently produces consistent standard errors for both maximum likelihood and weighted least squares problems, the Gauss-Newton algorithm represents an attractive alternative for at least some covariance structure analyses.Work by the first author has been supported in part by Grant No. Da01070 from the U. S. Public Health Service. Work by the second author has been supported in part by Grant No. MCS 77-02121 from the National Science Foundation.     

Tests of homogeneity of means and covariance matrices for multivariate incomplete data

Existing test statistics for assessing whether incomplete data represent a missing completely at random sample from a single population are based on a normal likelihood rationale and effectively test for homogeneity of means and covariances across missing data patterns. The likelihood approach cannot be implemented adequately if a pattern of missing data contains very few subjects. A generalized least squares rationale is used to develop parallel tests that are expected to be more stable in small samples. Three factors were varied for a simulation: number of variables, percent missing completely at random, and sample size. One thousand data sets were simulated for each condition. The generalized least squares test of homogeneity of means performed close to an ideal Type I error rate for most of the conditions. The generalized least squares test of homogeneity of covariance matrices and a combined test performed quite well also.Preliminary results on this research were presented at the 1999 Western Psychological Association convention, Irvine, CA, and in the UCLA Statistics Preprint No. 265 (http://www.stat.ucla.edu). The assistance of Ke-Hai Yuan and several anonymous reviewers is gratefully acknowledged.     

A gauss-newton algorithm for exploratory factor analysis

It is shown that the scoring algorithm for maximum likelihood estimation in exploratory factor analysis can be developed in a way that is many times more efficient than a direct development based on information matrices and score vectors. The algorithm offers a simple alternative to current algorithms and when used in one-step mode provides the simplest and fastest method presently available for moving from consistent to efficient estimates. Perhaps of greater importance is its potential for extension to the confirmatory model. The algorithm is developed as a Gauss-Newton algorithm to facilitate its application to generalized least squares and to maximum likelihood estimation.This research was supported by NSF Grant MCS-8301587.     

A bayesian approach to confirmatory factor analysis

Confirmatory factor analysis is considered from a Bayesian viewpoint, in which prior information on parameter is incorporated in the analysis. An iterative algorithm is developed to obtain the Bayes estimates. A numerical example based on longitudinal data is presented. A simulation study is designed to compare the Bayesian approach with the maximum likelihood method.Computer facilities were provided by the Computer Services Center, The Chinese University of Hong Kong.     

Combining standardized mean differences using the method of maximum likelihood

A maximum likelihood procedure for combining standardized mean differences based on a noncentratt-distribution is proposed. With a proper data augmentation technique, an EM-algorithm is developed. Information and likelihood ratio statistics are discussed in detail for reliable inference. Simulation results favor the proposed procedure over both the existing normal theory maximum likelihood procedure and the commonly used generalized least squares procedure.     

Replies to comments on "Which method delivers greater signal-to-noise ratio: Structural equation modelling or regression analysis with weighted composites?" by Yuan and Fang (2023)

Observational data typically contain measurement errors. Covariance-based structural equation modelling (CB-SEM) is capable of modelling measurement errors and yields consistent parameter estimates. In contrast, methods of regression analysis using weighted composites as well as a partial least squares approach to SEM facilitate the prediction and diagnosis of individuals/participants. But regression analysis with weighted composites has been known to yield attenuated regression coefficients when predictors contain errors. Contrary to the common belief that CB-SEM is the preferred method for the analysis of observational data, this article shows that regression analysis via weighted composites yields parameter estimates with much smaller standard errors, and thus corresponds to greater values of the signal-to-noise ratio (SNR). In particular, the SNR for the regression coefficient via the least squares (LS) method with equally weighted composites is mathematically greater than that by CB-SEM if the items for each factor are parallel, even when the SEM model is correctly specified and estimated by an efficient method. Analytical, numerical and empirical results also show that LS regression using weighted composites performs as well as or better than the normal maximum likelihood method for CB-SEM under many conditions even when the population distribution is multivariate normal. Results also show that the LS regression coefficients become more efficient when considering the sampling errors in the weights of composites than those that are conditional on weights.     

Dynamical system approach to factor analysis parameter estimation

A new unified approach to solving and studying the factor analysis parameter estimation problem is proposed. The maximum likelihood and least squares formulations of factor analysis are considered. The approach leads to globally convergent procedures for simultaneous estimation of the factor analysis parameters. The method presented necessarily leads to proper factor analysis estimations.     

Comparison of alternative estimation methods in confirmatory factor analyses of the General Health Questionnaire

This paper examines the implications of violating assumptions concerning the continuity and distributional properties of data in establishing measurement models in social science research. The General Health Questionnaire-12 uses an ordinal response scale. Responses to the GHQ-12 from 201 Hong Kong immigrants on arrival in Australia showed that the data were not normally distributed. A series of confirmatory factor analyses using either a Pearson product-moment or a polychoric correlation input matrix and employing either maximum likelihood, weighted least squares or diagonally weighted least squares estimation methods were conducted on the data. The parameter estimates and goodness-of-fit statistics provided support for using polychoric correlations and diagonally weighted least squares estimation when analyzing ordinal, nonnormal data.     

Improved Regression Calibration

The likelihood for generalized linear models with covariate measurement error cannot in general be expressed in closed form, which makes maximum likelihood estimation taxing. A popular alternative is regression calibration which is computationally efficient at the cost of inconsistent estimation. We propose an improved regression calibration approach, a general pseudo maximum likelihood estimation method based on a conveniently decomposed form of the likelihood. It is both consistent and computationally efficient, and produces point estimates and estimated standard errors which are practically identical to those obtained by maximum likelihood. Simulations suggest that improved regression calibration, which is easy to implement in standard software, works well in a range of situations.     

How to fit a response time distribution

Among the most valuable tools in behavioral science is statistically fitting mathematical models of cognition to data—response time distributions, in particular. However, techniques for fitting distributions vary widely, and little is known about the efficacy of different techniques. In this article, we assess several fitting techniques by simulating six widely cited models of response time and using the fitting procedures to recover model parameters. The techniques include the maximization of likelihood and least squares fits of the theoretical distributions to different empirical estimates of the simulated distributions. A running example is used to illustrate the different estimation and fitting procedures. The simulation studies reveal that empirical density estimates are biased even for very large sample sizes. Some fitting techniques yield more accurate and less variable parameter estimates than do others. Methods that involve least squares fits to density estimates generally yield very poor parameter estimates.     

A comparison of three methods of fitting the normal ogive

The Mueller-Urban method of fitting the normal ogive is derived, and the inadequacies of its inherent assumptions are discussed. This and the unweighted least squares method are compared to the maximum likelihood solution which is shown to be very close to the ideal least squares solution. As an empirical demonstration of the superiority of the maximum likelihood solution, random ogives are fitted by all three methods and they are compared on the basis of the expected values and the standard errors of the estimates. It is concluded that the maximum likelihood solution is uniformly superior to the others in all respects.This research was done under Contract Nonr-248(55) between the Office of Naval Research and The Johns Hopkins University. This is Report No. 18 under that contract. Reproduction in whole or in part is permitted for any purpose of the United States Government. This paper is part of a dissertation submitted to The Johns Hopkins University. Part of this work was done while the author was a National Institutes of Health Research Fellow.Now at The Biometric Laboratory, The George Washington University. The author is indebted to Dr. Wendell R. Garner for his valuable advice and encouragement, and to Jerome Cornfield for several helpful discussions.     

The Infinitesimal Jackknife with Exploratory Factor Analysis

The infinitesimal jackknife, a nonparametric method for estimating standard errors, has been used to obtain standard error estimates in covariance structure analysis. In this article, we adapt it for obtaining standard errors for rotated factor loadings and factor correlations in exploratory factor analysis with sample correlation matrices. Both maximum likelihood estimation and ordinary least squares estimation are considered.