A Robust Bayesian Approach for Structural Equation Models with Missing Data

In this paper, normal/independent distributions, including but not limited to the multivariate t distribution, the multivariate contaminated distribution, and the multivariate slash distribution, are used to develop a robust Bayesian approach for analyzing structural equation models with complete or missing data. In the context of a nonlinear structural equation model with fixed covariates, robust Bayesian methods are developed for estimation and model comparison. Results from simulation studies are reported to reveal the characteristics of estimation. The methods are illustrated by using a real data set obtained from diabetes patients.     

Semiparametric Thurstonian Models for Recurrent Choices: A Bayesian Analysis

We develop semiparametric Bayesian Thurstonian models for analyzing repeated choice decisions involving multinomial, multivariate binary or multivariate ordinal data. Our modeling framework has multiple components that together yield considerable flexibility in modeling preference utilities, cross-sectional heterogeneity and parameter-driven dynamics. Each component of our model is specified semiparametrically using Dirichlet process (DP) priors. The utility (latent variable) component of our model allows the alternative-specific utility errors to semiparametrically deviate from a normal distribution. This generates a robust alternative to popular Thurstonian specifications that are based on underlying normally distributed latent variables. Our second component focuses on flexibly modeling cross-sectional heterogeneity. The semiparametric specification allows the heterogeneity distribution to mimic either a finite mixture distribution or a continuous distribution such as the normal, whichever is supported by the data. Thus, special features such as multimodality can be readily incorporated without the need to overtly search for the best heterogeneity specification across a series of models. Finally, we allow for parameter-driven dynamics using a semiparametric state-space approach. This specification adds to the literature on robust Kalman filters. The resulting framework is very general and integrates divergent strands of the literatures on flexible choice models, Bayesian nonparametrics and robust time series specifications. Given this generality, we show how several existing Thurstonian models can be obtained as special forms of our model. We describe Markov chain Monte Carlo methods for the inference of model parameters, report results from two simulation studies and apply the model to consumer choice data from a frequently purchased product category. The results from our simulations and application highlight the benefits of using our semiparametric approach.     

Hierarchical Multinomial Processing Tree Models: A Latent-Trait Approach

Multinomial processing tree models are widely used in many areas of psychology. A hierarchical extension of the model class is proposed, using a multivariate normal distribution of person-level parameters with the mean and covariance matrix to be estimated from the data. The hierarchical model allows one to take variability between persons into account and to assess parameter correlations. The model is estimated using Bayesian methods with weakly informative hyperprior distribution and a Gibbs sampler based on two steps of data augmentation. Estimation, model checks, and hypotheses tests are discussed. The new method is illustrated using a real data set, and its performance is evaluated in a simulation study.     

An elicitation method for multiple linear regression models

This paper describes a method of quantifying subjective opinion about a normal linear regression model. Opinion about the regression coefficients and experimental error is elicited and modeled by a multivariate probability distribution (a Bayesian conjugate prior distribution). The distribution model is richly parameterized and various assessment tasks are used to estimate its parameters. These tasks include the revision of opinion in the light of hypothetical data, the assessment of credible intervals, and a task commonly performed in cue-weighting experiments. A new assessment task is also introduced. In addition, implementation of the method in an interactive computer program is described and the method is illustrated with a practical example.     

Bayesian inference for finite mixtures of generalized linear models with random effects

We present an hierarchical Bayes approach to modeling parameter heterogeneity in generalized linear models. The model assumes that there are relevant subpopulations and that within each subpopulation the individual-level regression coefficients have a multivariate normal distribution. However, class membership is not known a priori, so the heterogeneity in the regression coefficients becomes a finite mixture of normal distributions. This approach combines the flexibility of semiparametric, latent class models that assume common parameters for each sub-population and the parsimony of random effects models that assume normal distributions for the regression parameters. The number of subpopulations is selected to maximize the posterior probability of the model being true. Simulations are presented which document the performance of the methodology for synthetic data with known heterogeneity and number of sub-populations. An application is presented concerning preferences for various aspects of personal computers.     

Approximations to the distribution of a test statistic in covariance structure analysis: A comprehensive study

In structural equation modelling (SEM), a robust adjustment to the test statistic or to its reference distribution is needed when its null distribution deviates from a χ² distribution, which usually arises when data do not follow a multivariate normal distribution. Unfortunately, existing studies on this issue typically focus on only a few methods and neglect the majority of alternative methods in statistics. Existing simulation studies typically consider only non-normal distributions of data that either satisfy asymptotic robustness or lead to an asymptotic scaled χ² distribution. In this work we conduct a comprehensive study that involves both typical methods in SEM and less well-known methods from the statistics literature. We also propose the use of several novel non-normal data distributions that are qualitatively different from the non-normal distributions widely used in existing studies. We found that several under-studied methods give the best performance under specific conditions, but the Satorra–Bentler method remains the most viable method for most situations.     

Loglinear representations of multivariate Bernoulli Rasch models

In this paper, the extended Rasch model for dichotomously scored items is derived from the general multivariate Bernoulli distribution. The necessary and sufficient conditions for the multivariate Bernoulli distribution to be equal to the extended Rasch model provide a new loglinear representation of the extended Rasch model. Conditions are also given under which the extended Rasch model is equal to the random effects Rasch model, and it is shown under what conditions the extended Rasch model is equal to a random effects Rasch model in which the underlying variable has a normal distribution. In addition, alternative models for the construction of likelihood ratio tests are proposed. One of these alternative models is Haberman's extended interaction model. Furthermore, it is shown how both the SPSS and SAS programs can be used to estimate and test loglinear representations of extended Rasch models.     

Probabilistic multidimensional scaling: Complete and incomplete data

Simple procedures are described for obtaining maximum likelihood estimates of the location and uncertainty parameters of the Hefner model. This model is a probabilistic, multidimensional scaling model, which assigns a multivariate normal distribution to each stimulus point. It is shown that for such a model, standard nonmetric and metric algorithms are not appropriate. A procedure is also described for constructing incomplete data sets, by taking into consideration the degree of familiarity the subject has for each stimulus. Maximum likelihood estimates are developed both for complete and incomplete data sets. This research was supported by National Science Grant No. SOC76-20517. The first author would especially like to express his gratitude to the Netherlands Institute for Advanced Study for its very substantial help with this research.     

Multiple imputation for item scores when test data are factorially complex

Multiple imputation under a two‐way model with error is a simple and effective method that has been used to handle missing item scores in unidimensional test and questionnaire data. Extensions of this method to multidimensional data are proposed. A simulation study is used to investigate whether these extensions produce biased estimates of important statistics in multidimensional data, and to compare them with lower benchmark listwise deletion, two‐way with error and multivariate normal imputation. The new methods produce smaller bias in several psychometrically interesting statistics than the existing methods of two‐way with error and multivariate normal imputation. One of these new methods clearly is preferable for handling missing item scores in multidimensional test data.     

Testing and modelling non‐normality within the one‐factor model

Maximum likelihood estimation in the one‐factor model is based on the assumption of multivariate normality for the observed data. This general distributional assumption implies three specific assumptions for the parameters in the one‐factor model: the common factor has a normal distribution; the residuals are homoscedastic; and the factor loadings do not vary across the common factor scale. When any of these assumptions is violated, non‐normality arises in the observed data. In this paper, a model is presented based on marginal maximum likelihood to enable explicit tests of these assumptions. In addition, the model is suitable to incorporate the detected violations, to enable statistical modelling of these effects. Two simulation studies are reported in which the viability of the model is investigated. Finally, the model is applied to IQ data to demonstrate its practical utility as a means to investigate ability differentiation.     

Parallel Factor Analysis of gait waveform data: A multimode extension of Principal Component Analysis

Gait data are typically collected in multivariate form, so some multivariate analysis is often used to understand interrelationships between observed data. Principal Component Analysis (PCA), a data reduction technique for correlated multivariate data, has been widely applied by gait analysts to investigate patterns of association in gait waveform data (e.g., interrelationships between joint angle waveforms from different subjects and/or joints). Despite its widespread use in gait analysis, PCA is for two-mode data, whereas gait data are often collected in higher-mode form. In this paper, we present the benefits of analyzing gait data via Parallel Factor Analysis (Parafac), which is a component analysis model designed for three- or higher-mode data. Using three-mode joint angle waveform data (subjects×time×joints), we demonstrate Parafac's ability to (a) determine interpretable components revealing the primary interrelationships between lower-limb joints in healthy gait and (b) identify interpretable components revealing the fundamental differences between normal and perturbed subjects' gait patterns across multiple joints. Our results offer evidence of the complex interconnections that exist between lower-limb joints and limb segments in both normal and abnormal gaits, confirming the need for the simultaneous analysis of multi-joint gait waveform data (especially when studying perturbed gait patterns).     

Bayesian analysis of order-statistics models for ranking data

In this paper, a class of probability models for ranking data, the order-statistics models, is investigated. We extend the usual normal order-statistics model into one where the underlying random variables follow a multivariate normal distribution. Bayesian approach and the Gibbs sampling technique are used for parameter estimation. In addition, methods to assess the adequacy of model fit are introduced. Robustness of the model is studied by considering a multivariate-t distribution. The proposed method is applied to analyze the presidential election data of the American Psychological Association (APA).The author is grateful to K. Lam, K.F. Lam, the Editor, an associate editor, and three reviewers for their valuable comments and suggestions. This research was substantially supported by the CRCG grant 335/017/0015 of the University of Hong Kong and a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. HKU 7169/98H). Upon completion of this paper, I became aware that similar work had been done independently by K.G. Yao and U. Böckenholt (1999).     

Generating multivariate data from nonnormal distributions: Mihal and Barrett revisited

An algorithm described by Graybill (1969) factors a population correlation matrix, R, into an upper and lower triangular matrix, T and T′, such that R=T′T. The matrix T is used to generate multivariate data sets from a multinormal distribution. When this algorithm is used to generate data for nonnormal distributions, however, the sample correlations are systematically biased downward. We describe an iterative technique that removes this bias by adjusting the initial correlation matrix. R, factored by the Graybill algorithm. The method is illustrated by simulating a multivariate study by Mihal and Barrett (1976). Large-N simulations indicate that the iterative technique works: multivariate data sets generated with this approach successfully model both the univariate distributions of the individual variables and their multivariate structure (as assessed by intercorrelation and regression analyses).     

Effects of skewness and kurtosis on normal-theory based maximum likelihood test statistic in multilevel structural equation modeling

A simulation study investigated the effects of skewness and kurtosis on level-specific maximum likelihood (ML) test statistics based on normal theory in multilevel structural equation models. The levels of skewness and kurtosis at each level were manipulated in multilevel data, and the effects of skewness and kurtosis on level-specific ML test statistics were examined. When the assumption of multivariate normality was violated, the level-specific ML test statistics were inflated, resulting in Type I error rates that were higher than the nominal level for the correctly specified model. Q-Q plots of the test statistics against a theoretical chi-square distribution showed that skewness led to a thicker upper tail and kurtosis led to a longer upper tail of the observed distribution of the level-specific ML test statistic for the correctly specified model.     

Fitting and Testing Conditional Multinormal Partial Credit Models

A multinormal partial credit model for factor analysis of polytomously scored items with ordered response categories is derived using an extension of the Dutch Identity (Holland in Psychometrika 55:5?C18, 1990). In the model, latent variables are assumed to have a multivariate normal distribution conditional on unweighted sums of item scores, which are sufficient statistics. Attention is paid to maximum likelihood estimation of item parameters, multivariate moments of latent variables, and person parameters. It is shown that the maximum likelihood estimates can be found without the use of numerical integration techniques. More general models are discussed which can be used for testing the model, and it is shown how models with different numbers of latent variables can be tested against each other. In addition, multi-group extensions are proposed, which can be used for testing both measurement invariance and latent population differences. Models and procedures discussed are demonstrated in an empirical data example.     

Discriminant Analysis Using Mixed Continuous,Dichotomous, and Ordered Categorical Variables

This article proposes an intuitive approach for predictive discriminant analysis with mixed continuous, dichotomous, and ordered categorical variables that are defined via an underlying multivariate normal distribution with a threshold specification. The classification rule is based on the comparison of the observed data logarithm probability density functions. To reduce the computational burden, the analysis is conducted in the context of a confirmatory factor analysis model with independent error measurements. Identification of the dichotomous and ordered categorical variables is discussed. Results are obtained by implementations of a Monte Carlo expectation maximization (MCEM)algorithm and a path sampling procedure. Probabilities of misclassification are estimated via the idea of the “jackknife” method. A real example is given to illustrate the proposed method.     

Contextualized Personality Questionnaires: A Case for Copulas in Structural Equation Models for Categorical Data

For structural equation models (SEMs) with categorical data, correlated measurement residuals are not easily implemented. The problem lies mainly in the absence of a categorical analogue to the multivariate normal distribution and the absence of closed form formulas in SEMs for categorical data. We present a novel technique to handle measurement residuals that keeps the attractive SEM mainframe intact yet adds flexibility in dependence modeling without excessive computational burden. The technique is based upon the concept of copula functions and is introduced with a data set of ordinal responses originating from a contextualized personality study on aggression. Focus is on models arising in a multitrait-multimethod context, where the flexibility in dependence structures allows for method effects that can vary across the latent trait dimension. The empirical application illustrates that ignoring design-implied correlated measurement residuals can potentially influence study results and conclusions in both a quantitative as well as a qualitative way. Model parameter estimates can be biased, but more important, model inferences can be heavily distorted.     

Analysis of Interactions and Nonlinear Effects with Missing Data: A Factored Regression Modeling Approach Using Maximum Likelihood Estimation

Abstract

When estimating multiple regression models with incomplete predictor variables, it is necessary to specify a joint distribution for the predictor variables. A convenient assumption is that this distribution is a multivariate normal distribution, which is also the default in many statistical software packages. This distribution will in general be misspecified if predictors with missing data have nonlinear effects (e.g., x²) or are included in interaction terms (e.g., x·z). In the present article, we introduce a factored regression modeling approach for estimating regression models with missing data that is based on maximum likelihood estimation. In this approach, the model likelihood is factorized into a part that is due to the model of interest and a part that is due to the model for the incomplete predictors. In three simulation studies, we showed that the factored regression modeling approach produced valid estimates of interaction and nonlinear effects in regression models with missing values on categorical or continuous predictor variables under a broad range of conditions. We developed the R package mdmb, which facilitates a user-friendly application of the factored regression modeling approach, and present a real-data example that illustrates the flexibility of the software.     

Structural Equation Modeling with Small Samples: Test Statistics

Structural equation modeling is a well-known technique for studying relationships among multivariate data. In practice, high dimensional nonnormal data with small to medium sample sizes are very common, and large sample theory, on which almost all modeling statistics are based, cannot be invoked for model evaluation with test statistics. The most natural method for nonnormal data, the asymptotically distribution free procedure, is not defined when the sample size is less than the number of nonduplicated elements in the sample covariance. Since normal theory maximum likelihood estimation remains defined for intermediate to small sample size, it may be invoked but with the probable consequence of distorted performance in model evaluation. This article studies the small sample behavior of several test statistics that are based on maximum likelihood estimator, but are designed to perform better with nonnormal data. We aim to identify statistics that work reasonably well for a range of small sample sizes and distribution conditions. Monte Carlo results indicate that Yuan and Bentler's recently proposed F-statistic performs satisfactorily.