On Bayesian estimation in unrestricted factor analysis

It is shown that the common and unique variance estimates produced by Martin & McDonald's Bayesian estimation procedure for the unrestricted common factor model have a predictable sum which is always greater than the maximum likelihood estimate of the total variance. This fact is used to justify a suggested simple alternative method of specifying the Bayesian parameters required by the procedure.     

Bayesian inference for graphical factor analysis models

We generalize factor analysis models by allowing the concentration matrix of the residuals to have nonzero off-diagonal elements. The resulting model is named graphical factor analysis model. Allowing a structure of associations gives information about the correlation left unexplained by the unobserved variables, which can be used both in the confirmatory and exploratory context. We first present a sufficient condition for global identifiability of this class of models with a generic number of factors, thereby extending the results in Stanghellini (1997) and Vicard (2000). We then consider the issue of model comparison and show that fast local computations are possible for this purpose, if the conditional independence graphs on the residuals are restricted to be decomposable and a Bayesian approach is adopted. To achieve this aim, we propose a new reversible jump MCMC method to approximate the posterior probabilities of the considered models. We then study the evolution of political democracy in 75 developing countries based on eight measures of democracy in two different years. We acknowledge support from M.U.R.S.T. of Italy and from the European Science Foundation H.S.S.S. Network. We are grateful to the referees and the Editor for many useful suggestions and comments which led to a substantial improvement of the paper. We also thank Nanny Wermuth for stimulating discussions and Kenneth A. Bollen for kindly providing us with the data-set.     

On robust estimation of effect size under semiparametric models

Estimation of effect size is of interest in many applied fields such as Psychology, Sociology and Education. However there are few nonparametric estimators of effect size proposed in the existing literature, and little is known about the distributional characteristics of these estimators. In this article, two estimators based on the sample quantiles are proposed and studied. The first one is the estimator suggested by Hedges and Olkin (see page 93 of Hedges & Olkin, 1985) for the situation where a treatment effect is evaluated against a control group (Case A). A modified version of the robust estimator by Hedges and Olkin is also proposed for the situation where two parallel treatments are compared (Case B). Large sample distributions of both estimators are derived. Their asymptotic relative efficiencies with respect to the normal maximum likelihood estimators under several common distributions are evaluated. The robust properties of the proposed estimators are discussed with respect to the sample-wise breakdown points proposed by Akritas (1991). Simulation studies are provided in which the performing characteristics of the proposed estimator are compared to that of the nonparametric estimators by Kraemer and Andrews (1982). Interval estimation of the effect sizes is also discussed. In an example, interval estimates for the data set in Kraemer and Andrews (1982) are calculated for both cases A and B.     

Numerical methods for polynomial models in nonlinear factor analysis

The basic concepts of nonlinear factor analysis are introduced and some extensions of the general theory are developed. An elementary account of the class of multiple-factor polynomial models is presented, using more elementary algebraic methods than have been employed in earlier accounts of this theory. Working formulas are developed for the multiple-factor polynomial model without product terms.Some empirical results are presented.This work was carried out while the author was a Visiting Research Fellow at the Educational Testing Service, Princeton, N. J. It made use of computer facilities at Princeton University that are supported in part by National Science Foundation Grant NSF-Gp579.     

Bayesian estimation and testing of structural equation models

The Gibbs sampler can be used to obtain samples of arbitrary size from the posterior distribution over the parameters of a structural equation model (SEM) given covariance data and a prior distribution over the parameters. Point estimates, standard deviations and interval estimates for the parameters can be computed from these samples. If the prior distribution over the parameters is uninformative, the posterior is proportional to the likelihood, and asymptotically the inferences based on the Gibbs sample are the same as those based on the maximum likelihood solution, for example, output from LISREL or EQS. In small samples, however, the likelihood surface is not Gaussian and in some cases contains local maxima. Nevertheless, the Gibbs sample comes from the correct posterior distribution over the parameters regardless of the sample size and the shape of the likelihood surface. With an informative prior distribution over the parameters, the posterior can be used to make inferences about the parameters underidentified models, as we illustrate on a simple errors-in-variables model.We thank David Spiegelhalter for suggesting applying the Gibbs sampler to structural equation models to the first author at a 1994 workshop in Wiesbaden. We thank Ulf Böckenholt, Chris Meek, Marijtje van Duijn, Clark Glymour, Ivo Molenaar, Steve Klepper, Thomas Richardson, Teddy Seidenfeld, and Tom Snijders for helpful discussions, mathematical advice, and critiques of earlier drafts of this paper.     

Bayesian estimation in unrestricted factor analysis: A treatment for heywood cases

A Bayesian procedure is given for estimation in unrestricted common factor analysis. A choice of the form of the prior distribution is justified. It is shown empirically that the procedure achieves its objective of avoiding inadmissible estimates of unique variances, and is reasonably insensitive to certain variations in the shape of the prior distribution.     

A Bayesian approach to nonlinear latent variable models using the Gibbs sampler and the metropolis-hastings algorithm

Nonlinear latent variable models are specified that include quadratic forms and interactions of latent regressor variables as special cases. To estimate the parameters, the models are put in a Bayesian framework with conjugate priors for the parameters. The posterior distributions of the parameters and the latent variables are estimated using Markov chain Monte Carlo methods such as the Gibbs sampler and the Metropolis-Hastings algorithm. The proposed estimation methods are illustrated by two simulation studies and by the estimation of a non-linear model for the dependence of performance on task complexity and goal specificity using empirical data.     

Bayesian and maximum likelihood estimation of hierarchical response time models

Hierarchical (or multilevel) statistical models have become increasingly popular in psychology in the last few years. In this article, we consider the application of multilevel modeling to the ex-Gaussian, a popular model of response times. We compare single-level and hierarchical methods for estimation of the parameters of ex-Gaussian distributions. In addition, for each approach, we compare maximum likelihood estimation with Bayesian estimation. A set of simulations and analyses of parameter recovery show that although all methods perform adequately well, hierarchical methods are better able to recover the parameters of the ex-Gaussian, by reducing variability in the recovered parameters. At each level, little overall difference was observed between the maximum likelihood and Bayesian methods.     

Bayesian analysis of two‐level nonlinear structural equation models with continuous and polytomous data

Two‐level structural equation models with mixed continuous and polytomous data and nonlinear structural equations at both the between‐groups and within‐groups levels are important but difficult to deal with. A Bayesian approach is developed for analysing this kind of model. A Markov chain Monte Carlo procedure based on the Gibbs sampler and the Metropolis‐Hasting algorithm is proposed for producing joint Bayesian estimates of the thresholds, structural parameters and latent variables at both levels. Standard errors and highest posterior density intervals are also computed. A procedure for computing Bayes factor, based on the key idea of path sampling, is established for model comparison.     

Alternative factor models and factorial invariance of the GHQ-12: a large sample analysis using confirmatory factor analysis

This study tested alternative factor models of the General Health Questionnaire-12 (GHQ-12), based on previous research findings, with a large sample using confirmatory factor analysis. An alternative models framework was used to test 6 factor analytic models. A 3-factor model was the best explanation of the sample data. The 3 factors were labeled Anxiety-Depression, Social Dysfunction, and Loss of Confidence. The model was found to be factorially invariant between men and women. The utility of the 3 subscales, as opposed to the total GHQ-12 score, is questioned as they appear to provide little information beyond that of a general factor.     

Bayesian analysis of order-statistics models for ranking data

In this paper, a class of probability models for ranking data, the order-statistics models, is investigated. We extend the usual normal order-statistics model into one where the underlying random variables follow a multivariate normal distribution. Bayesian approach and the Gibbs sampling technique are used for parameter estimation. In addition, methods to assess the adequacy of model fit are introduced. Robustness of the model is studied by considering a multivariate-t distribution. The proposed method is applied to analyze the presidential election data of the American Psychological Association (APA).The author is grateful to K. Lam, K.F. Lam, the Editor, an associate editor, and three reviewers for their valuable comments and suggestions. This research was substantially supported by the CRCG grant 335/017/0015 of the University of Hong Kong and a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. HKU 7169/98H). Upon completion of this paper, I became aware that similar work had been done independently by K.G. Yao and U. Böckenholt (1999).     

Three case studies in the Bayesian analysis of cognitive models

Bayesian statistical inference offers a principled and comprehensive approach for relating psychological models to data. This article presents Bayesian analyses of three influential psychological models: multidimensional scaling models of stimulus representation, the generalized context model of category learning, and a signal detection theory model of decision making. In each case, the model is recast as a probabilistic graphical model and is evaluated in relation to a previously considered data set. In each case, it is shown that Bayesian inference is able to provide answers to important theoretical and empirical questions easily and coherently. The generality of the Bayesian approach and its potential for the understanding of models and data in psychology are discussed.     

Bayesian estimation of a multilevel IRT model using gibbs sampling

In this article, a two-level regression model is imposed on the ability parameters in an item response theory (IRT) model. The advantage of using latent rather than observed scores as dependent variables of a multilevel model is that it offers the possibility of separating the influence of item difficulty and ability level and modeling response variation and measurement error. Another advantage is that, contrary to observed scores, latent scores are test-independent, which offers the possibility of using results from different tests in one analysis where the parameters of the IRT model and the multilevel model can be concurrently estimated. The two-parameter normal ogive model is used for the IRT measurement model. It will be shown that the parameters of the two-parameter normal ogive model and the multilevel model can be estimated in a Bayesian framework using Gibbs sampling. Examples using simulated and real data are given.     

Bayesian latent variable models for the analysis of experimental psychology data

In this paper, we address the use of Bayesian factor analysis and structural equation models to draw inferences from experimental psychology data. While such application is non-standard, the models are generally useful for the unified analysis of multivariate data that stem from, e.g., subjects’ responses to multiple experimental stimuli. We first review the models and the parameter identification issues inherent in the models. We then provide details on model estimation via JAGS and on Bayes factor estimation. Finally, we use the models to re-analyze experimental data on risky choice, comparing the approach to simpler, alternative methods.     

Robustness of the maximum-likelihood estimation procedure in factor analysis

In order to determine the robustness of the maximum-likelihood estimation procedure, random variates from six distributions were generated independently on a high speed computer and then used to represent the common and specific factors in a factor analysis model in which the coefficients of these factors had been specified. Using Lawley's approximatex ² statistic in evaluating the estimates obtained, the estimation procedure is found to be insensitive to changes in the distributions considered.This study was conducted while the authors were members of the Statistical Laboratory at Iowa State University. All computations were done at Iowa State on an IBM 7074.     

Local influence analysis of nonlinear structural equation models

By regarding the latent random vectors as hypothetical missing data and based on the conditional expectation of the complete-data log-likelihood function in the EM algorithm, we investigate assessment of local influence of various perturbation schemes in a nonlinear structural equation model. The basic building blocks of local influence analysis are computed via observations of the latent variables generated by the Metropolis-Hastings algorithm, while the diagnostic measures are obtained via the conformal normal curvature. Seven perturbation schemes, including some perturbation schemes on latent vectors, are investigated. The proposed procedure is illustrated by a simulation study and a real example. Acknowledgment: This research is fully supported by a grant (CUHK 4243/02H) from the Research Grant Council of the Hong Kong Special Administration Region. The authors are indebted to ICPSR and the relevant funding agency for allowing use of their data, and to the Editor and reviewers for their valuable comments for improving the paper.     

Bayesian model comparison of nonlinear structural equation models with missing continuous and ordinal categorical data

Missing data are very common in behavioural and psychological research. In this paper, we develop a Bayesian approach in the context of a general nonlinear structural equation model with missing continuous and ordinal categorical data. In the development, the missing data are treated as latent quantities, and provision for the incompleteness of the data is made by a hybrid algorithm that combines the Gibbs sampler and the Metropolis‐Hastings algorithm. We show by means of a simulation study that the Bayesian estimates are accurate. A Bayesian model comparison procedure based on the Bayes factor and path sampling is proposed. The required observations from the posterior distribution for computing the Bayes factor are simulated by the hybrid algorithm in Bayesian estimation. Our simulation results indicate that the correct model is selected more frequently when the incomplete records are used in the analysis than when they are ignored. The methodology is further illustrated with a real data set from a study concerned with an AIDS preventative intervention for Filipina sex workers.     

Estimation of nonlinear mixed-effects continuous-time models using the continuous-discrete extended Kalman filter

Many intensive longitudinal measurements are collected at irregularly spaced time intervals, and involve complex, possibly nonlinear and heterogeneous patterns of change. Effective modelling of such change processes requires continuous-time differential equation models that may be nonlinear and include mixed effects in the parameters. One approach of fitting such models is to define random effect variables as additional latent variables in a stochastic differential equation (SDE) model of choice, and use estimation algorithms designed for fitting SDE models, such as the continuous-discrete extended Kalman filter (CDEKF) approach implemented in the dynr R package, to estimate the random effect variables as latent variables. However, this approach's efficacy and identification constraints in handling mixed-effects SDE models have not been investigated. In the current study, we analytically inspect the identification constraints of using the CDEKF approach to fit nonlinear mixed-effects SDE models; extend a published model of emotions to a nonlinear mixed-effects SDE model as an example, and fit it to a set of irregularly spaced ecological momentary assessment data; and evaluate the feasibility of the proposed approach to fit the model through a Monte Carlo simulation study. Results show that the proposed approach produces reasonable parameter and standard error estimates when some identification constraint is met. We address the effects of sample size, process noise variance, and data spacing conditions on estimation results.     

A probabilistic analysis of cross-examination using Bayesian networks

The legal scholar Henry Wigmore asserted that cross-examination is ‘the greatest legal engine ever invented for the discovery of truth.’ Was Wigmore right? Instead of addressing this question upfront, this paper offers a conceptual ground clearing. It is difficult to say whether Wigmore was right or wrong without becoming clear about what we mean by cross-examination; how it operates at trial; what it is intended to accomplish. Despite the growing importance of legal epistemology, there is virtually no philosophical work that discusses cross-examination, its scope and function at trial. This paper makes a first attempt at clearing the ground by articulating an analysis of cross-examination using probability theory and Bayesian networks. This analysis relies on the distinction between undercutting and rebutting evidence. A preliminary assessment of the truth-seeking function of cross-examination is offered at the end of the paper.