Finite mixtures in confirmatory factor-analysis models

In this paper, various types of finite mixtures of confirmatory factor-analysis models are proposed for handling data heterogeneity. Under the proposed mixture approach, observations are assumed to be drawn from mixtures of distinct confirmatory factor-analysis models. But each observation does not need to be identified to a particular model prior to model fitting. Several classes of mixture models are proposed. These models differ by their unique representations of data heterogeneity. Three different sampling schemes for these mixture models are distinguished. A mixed type of the these three sampling schemes is considered throughout this article. The proposed mixture approach reduces to regular multiple-group confirmatory factor-analysis under a restrictive sampling scheme, in which the structural equation model for each observation is assumed to be known. By assuming a mixture of multivariate normals for the data, maximum likelihood estimation using the EM (Expectation-Maximization) algorithm and the AS (Approximate-Scoring) method are developed, respectively. Some mixture models were fitted to a real data set for illustrating the application of the theory. Although the EM algorithm and the AS method gave similar sets of parameter estimates, the AS method was found computationally more efficient than the EM algorithm. Some comments on applying the mixture approach to structural equation modeling are made.Note: This paper is one of the Psychometric Society's 1995 Dissertation Award papers.—EditorThis article is based on the dissertation of the author. The author would like to thank Peter Bentler, who was the dissertation chair, for guidance and encouragement of this work. Eric Holman, Robert Jennrich, Bengt Muthén, and Thomas Wickens, who served as the committee members for the dissertation, had been very supportive and helpful. Michael Browne is appreciated for discussing some important points about the use of the approximate information in the dissertation. Thanks also go to an anonymous associate editor, whose comments were very useful for the revision of an earlier version of this article.     

Distinguishing Between Latent Classes and Continuous Factors with Categorical Outcomes: Class Invariance of Parameters of Factor Mixture Models

Factor mixture models are latent variable models with categorical and continuous latent variables that can be used as a model-based approach to clustering. A previous article covered the results of a simulation study showing that in the absence of model violations, it is usually possible to choose the correct model when fitting a series of models with different numbers of classes and factors within class. The response format in the first study was limited to normally distributed outcomes. This article has 2 main goals, first, to replicate parts of the first study with 5-point Likert scale and binary outcomes, and second, to address the issue of testing class invariance of thresholds and loadings. Testing for class invariance of parameters is important in the context of measurement invariance and when using mixture models to approximate nonnormal distributions. Results show that it is possible to discriminate between latent class models and factor models even if responses are categorical. Comparing models with and without class-specific parameters can lead to incorrectly accepting parameter invariance if the compared models differ substantially with respect to the number of estimated parameters. The simulation study is complemented with an illustration of a factor mixture analysis of 10 binary depression items obtained from a female subsample of the Virginia Twin Registry.     

Gennclus: New models for general nonhierarchical clustering analysis

A general class of nonhierarchical clustering models and associated algorithms for fitting them are presented. These (metric) clustering models generalize the Shepard-Arabie Additive Clusters model in allowing for: (1). either overlapping or nonoverlapping clusters; (2). either symmetric (one-way clustering) or nonsymmetric (two-way clustering) proximities (input data); and, (3). either symmetric or diagonal weights. The GENNCLUS algorithms utilize alternating least-squares methods combining ordinary and constrained least-squares, nonlinear constrained mathematical programming, and combinatorial optimization techniques in estimating model parameters. In addition to developing the mathematical bases of these models, a comprehensive set of Monte Carlo simulations of the different models is reported. Two applications concerning brand-switching data and celebrity-brand proximities are discussed. Finally, extensions to three-way models, nonmetric analyses, and other model specifications are provided.Wayne S. DeSarbo is a Member of Technical Staff in the Mathematics and Statistics Research Center at Bell Laboratories in Murray Hill, N.J. I wish to thank R. Gnanadesikan, J. D. Carroll, and P. Arabie for their comments on a previous draft of this paper. I also wish to acknowledge the computer assistance provided by Linda Clark. Finally, I wish to thank the reviewers and editor for their very complete reviews and comments.     

Estimation of nonlinear mixed-effects continuous-time models using the continuous-discrete extended Kalman filter

Many intensive longitudinal measurements are collected at irregularly spaced time intervals, and involve complex, possibly nonlinear and heterogeneous patterns of change. Effective modelling of such change processes requires continuous-time differential equation models that may be nonlinear and include mixed effects in the parameters. One approach of fitting such models is to define random effect variables as additional latent variables in a stochastic differential equation (SDE) model of choice, and use estimation algorithms designed for fitting SDE models, such as the continuous-discrete extended Kalman filter (CDEKF) approach implemented in the dynr R package, to estimate the random effect variables as latent variables. However, this approach's efficacy and identification constraints in handling mixed-effects SDE models have not been investigated. In the current study, we analytically inspect the identification constraints of using the CDEKF approach to fit nonlinear mixed-effects SDE models; extend a published model of emotions to a nonlinear mixed-effects SDE model as an example, and fit it to a set of irregularly spaced ecological momentary assessment data; and evaluate the feasibility of the proposed approach to fit the model through a Monte Carlo simulation study. Results show that the proposed approach produces reasonable parameter and standard error estimates when some identification constraint is met. We address the effects of sample size, process noise variance, and data spacing conditions on estimation results.     

Regime-Switching Bivariate Dual Change Score Model

Mixture structural equation model with regime switching (MSEM-RS) provides one possible way of representing over-time heterogeneities in dynamic processes by allowing a system to manifest qualitatively or quantitatively distinct change processes conditional on the latent “regime” the system is in at a particular time point. Unlike standard mixture structural equation models such as growth mixture models, MSEM-RS allows individuals to transition between latent classes over time. This class of models, often referred to as regime-switching models in the time series and econometric applications, can be specified as regime-switching mixture structural equation models when the number of repeated measures involved is not large. We illustrate the empirical utility of such models using one special case—a regime-switching bivariate dual change score model in which two growth processes are allowed to manifest regime-dependent coupling relations with one another. The proposed model is illustrated using a set of longitudinal reading and arithmetic performance data from the Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 study (ECLS-K; U.S. Department of Education, National Center for Education Statistics, 2010).     

Accuracy, Scope, and Flexibility of Models

Traditionally, models are compared on the basis of their accuracy, their scope, and their simplicity. Simplicity is often represented by parameter counts; the fewer the parameters, the simpler the model. Arguments are presented here suggesting that simplicity has little place in discussions of modeling; instead, the concept of flexibility should be substituted. When comparing two models one should be wary of the possibility of their differential flexibility. Several methods for assessing relative flexibility are possible, as represented in this special issue of the Journal of Mathematical Psychology. Here, the method of cross-validation is applied in the comparison of two models, a linear integration model (LIM) and the fuzzy-logical model of perception (FLMP), in the fitting of 44 data sets concerning the perception of layout seen among three panels with the presence or absence of four sources of information for depth. Prior to cross-validation the two models performed about equally well; after cross-validation LIM was statistically superior to FLMP, but the overall pattern of fits remained nearly the same for both models. Copyright 2000 Academic Press.     

Estimating the probability and fidelity of memory

Research on memory may benefit from paradigms that permit graded characterization of memory performance, but a simple variance-based approach to the analysis of such graded data confounds two potential sources of error: the probability of memory and the fidelity of memory. Such data are more properly modeled by a mixture distribution, thereby permitting explicit estimation of both the probability and fidelity of memory. An expectation-maximization algorithm is presented for fitting such data to a mixture model, and Monte Carlo validation of this tool reveals circumstances under which it may be expected to be most effective. Limitations of the tool are outlined with respect to potential confounds in experiment design and interpretation of results. Finally, approaches to ameliorating such confounds are discussed. An R procedure for fitting response error data to mixture models may be downloaded from http://brm.psychonomic-journals.org/content/supplemental.     

Generalized latent variable models with non‐linear effects

Until recently, item response models such as the factor analysis model for metric responses, the two‐parameter logistic model for binary responses and the multinomial model for nominal responses considered only the main effects of latent variables without allowing for interaction or polynomial latent variable effects. However, non‐linear relationships among the latent variables might be necessary in real applications. Methods for fitting models with non‐linear latent terms have been developed mainly under the structural equation modelling approach. In this paper, we consider a latent variable model framework for mixed responses (metric and categorical) that allows inclusion of both non‐linear latent and covariate effects. The model parameters are estimated using full maximum likelihood based on a hybrid integration–maximization algorithm. Finally, a method for obtaining factor scores based on multiple imputation is proposed here for the non‐linear model.     

Bayesian Model Comparison for the Order Restricted RC Association Model

Association models constitute an attractive alternative to the usual log-linear models for modeling the dependence between classification variables. They impose special structure on the underlying association by assigning scores on the levels of each classification variable, which can be fixed or parametric. Under the general row-column (RC) association model, both row and column scores are unknown parameters without any restriction concerning their ordinality. However, when the classification variables are ordinal, order restrictions on the scores arise naturally. Under such restrictions, we adopt an alternative parameterization and draw inferences about the equality of adjacent scores using the Bayesian approach. To achieve that, we have constructed a reversible jump Markov chain Monte Carlo algorithm for moving across models of different dimension and estimate accurately the posterior model probabilities which can be used either for model comparison or for model averaging. The proposed methodology is evaluated through a simulation study and illustrated using actual datasets.     

Taxometric analysis as a general strategy for distinguishing categorical from dimensional latent structure

Statistical analyses investigating latent structure can be divided into those that estimate structural model parameters and those that detect the structural model type. The most basic distinction among structure types is between categorical (discrete) and dimensional (continuous) models. It is a common, and potentially misleading, practice to apply some method for estimating a latent structural model such as factor analysis without first verifying that the latent structure type assumed by that method applies to the data. The taxometric method was developed specifically to distinguish between dimensional and 2-class models. This study evaluated the taxometric method as a means of identifying categorical structures in general. We assessed the ability of the taxometric method to distinguish between dimensional (1-class) and categorical (2-5 classes) latent structures and to estimate the number of classes in categorical datasets. Based on 50,000 Monte Carlo datasets (10,000 per structure type), and using the comparison curve fit index averaged across 3 taxometric procedures (Mean Above Minus Below A Cut, Maximum Covariance, and Latent Mode Factor Analysis) as the criterion for latent structure, the taxometric method was found superior to finite mixture modeling for distinguishing between dimensional and categorical models. A multistep iterative process of applying taxometric procedures to the data often failed to identify the number of classes in the categorical datasets accurately, however. It is concluded that the taxometric method may be an effective approach to distinguishing between dimensional and categorical structure but that other latent modeling procedures may be more effective for specifying the model.     

Bayesian analysis of two‐level nonlinear structural equation models with continuous and polytomous data

Two‐level structural equation models with mixed continuous and polytomous data and nonlinear structural equations at both the between‐groups and within‐groups levels are important but difficult to deal with. A Bayesian approach is developed for analysing this kind of model. A Markov chain Monte Carlo procedure based on the Gibbs sampler and the Metropolis‐Hasting algorithm is proposed for producing joint Bayesian estimates of the thresholds, structural parameters and latent variables at both levels. Standard errors and highest posterior density intervals are also computed. A procedure for computing Bayes factor, based on the key idea of path sampling, is established for model comparison.     

Bayesian inference for finite mixtures of generalized linear models with random effects

We present an hierarchical Bayes approach to modeling parameter heterogeneity in generalized linear models. The model assumes that there are relevant subpopulations and that within each subpopulation the individual-level regression coefficients have a multivariate normal distribution. However, class membership is not known a priori, so the heterogeneity in the regression coefficients becomes a finite mixture of normal distributions. This approach combines the flexibility of semiparametric, latent class models that assume common parameters for each sub-population and the parsimony of random effects models that assume normal distributions for the regression parameters. The number of subpopulations is selected to maximize the posterior probability of the model being true. Simulations are presented which document the performance of the methodology for synthetic data with known heterogeneity and number of sub-populations. An application is presented concerning preferences for various aspects of personal computers.     

Fitting direct covariance structures by the MSTRUCT modeling language of the CALIS procedure

This paper demonstrates the usefulness and flexibility of the general structural equation modelling (SEM) approach to fitting direct covariance patterns or structures (as opposed to fitting implied covariance structures from functional relationships among variables). In particular, the MSTRUCT modelling language (or syntax) of the CALIS procedure (SAS/STAT version 9.22 or later: SAS Institute, 2010) is used to illustrate the SEM approach. The MSTRUCT modelling language supports a direct covariance pattern specification of each covariance element. It also supports the input of additional independent and dependent parameters. Model tests, fit statistics, estimates, and their standard errors are then produced under the general SEM framework. By using numerical and computational examples, the following tests of basic covariance patterns are illustrated: sphericity, compound symmetry, and multiple‐group covariance patterns. Specification and testing of two complex correlation structures, the circumplex pattern and the composite direct product models with or without composite errors and scales, are also illustrated by the MSTRUCT syntax. It is concluded that the SEM approach offers a general and flexible modelling of direct covariance and correlation patterns. In conjunction with the use of SAS macros, the MSTRUCT syntax provides an easy‐to‐use interface for specifying and fitting complex covariance and correlation structures, even when the number of variables or parameters becomes large.     

A general latent assignment approach for modeling psychological contaminants

Data from psychological experiments are rife with ‘contaminants’, which can generally be defined as data generated by psychological processes different from those intended as the object of study. Contaminant data can interfere with the testing of substantive psychological models and their parameters, so it is important to have methods for their identification and removal. After noting that current practices in cognitive modeling for dealing with contaminants are not completely satisfactory, we argue for a general latent mixture approach to the problem. We demonstrate the tractability and effectiveness of the approach concretely, through a series of four applications. These applications involve a simple choice problem, a diffusion model of a response time and accuracy in decision-making, a hierarchical signal detection model of recognition memory, and a reinforcement learning model of decision-making on bandit problems. We conclude that developing models of contaminant processes requires the same sort of creative effort that is needed to model substantive psychological processes, but that it is a necessary endeavour that can be coherently and usefully pursued within the latent mixture modeling approach.     

Bayesian generic priors for causal learning

The article presents a Bayesian model of causal learning that incorporates generic priors--systematic assumptions about abstract properties of a system of cause-effect relations. The proposed generic priors for causal learning favor sparse and strong (SS) causes--causes that are few in number and high in their individual powers to produce or prevent effects. The SS power model couples these generic priors with a causal generating function based on the assumption that unobservable causal influences on an effect operate independently (P. W. Cheng, 1997). The authors tested this and other Bayesian models, as well as leading nonnormative models, by fitting multiple data sets in which several parameters were varied parametrically across multiple types of judgments. The SS power model accounted for data concerning judgments of both causal strength and causal structure (whether a causal link exists). The model explains why human judgments of causal structure (relative to a Bayesian model lacking these generic priors) are influenced more by causal power and the base rate of the effect and less by sample size. Broader implications of the Bayesian framework for human learning are discussed.     

Dynamic analysis of multivariate panel data with non-linear transformations

Many models for multivariate data analysis can be seen as special cases of the linear dynamic or state space model. Contrary to the classical approach to linear dynamic systems analysis, in which high-dimensional exact solutions are sought, the model presented here is developed from a social science framework where low-dimensional approximate solutions are preferred. Borrowing concepts from the theory on mixture distributions, the linear dynamic model can be viewed as a multi-layered regression model, in which the output variables are imprecise manifestations of an unobserved continuous process. An additional layer of mixing makes it possible to incorporate non-normal as well as ordinal variables.Using the EM-algorithm, we find estimates of the unknown model parameters, simultaneously providing stability estimates. The model is very general and cannot be well estimated by other estimation methods. We illustrate the applicability of the obtained procedure through an example with generated data.