A taxonomy of latent structure assumptions for probability matrix decomposition models

A taxonomy of latent structure assumptions (LSAs) for probability matrix decomposition (PMD) models is proposed which includes the original PMD model (Maris, De Boeck, & Van Mechelen, 1996) as well as a three-way extension of the multiple classification latent class model (Maris, 1999). It is shown that PMD models involving different LSAs are actually restricted latent class models with latent variables that depend on some external variables. For parameter estimation a combined approach is proposed that uses both a mode-finding algorithm (EM) and a sampling-based approach (Gibbs sampling). A simulation study is conducted to investigate the extent to which information criteria, specific model checks, and checks for global goodness of fit may help to specify the basic assumptions of the different PMD models. Finally, an application is described with models involving different latent structure assumptions for data on hostile behavior in frustrating situations.Note: The research reported in this paper was partially supported by the Fund for Scientific Research-Flanders (Belgium) (project G.0207.97 awarded to Paul De Boeck and Iven Van Mechelen), and the Research Fund of K.U. Leuven (F/96/6 fellowship to Andrew Gelman, OT/96/10 project awarded to Iven Van Mechelen and GOA/2000/02 awarded to Paul De Boeck and Iven Van Mechelen). We thank Marcel Croon and Kristof Vansteelandt for commenting on an earlier draft of this paper.     

Psychometric latent response models

In this paper, some psychometric models will be presented that belong to the larger class oflatent response models (LRMs). First, LRMs are introduced by means of an application in the field ofcomponential item response theory (Embretson, 1980, 1984). Second, a general definition of LRMs (not specific for the psychometric subclass) is given. Third, some more psychometric LRMs, and examples of how they can be applied, are presented. Fourth, a method for obtaining maximum likelihood (ML) and some maximum a posteriori (MAP) estimates of the parameters of LRMs is presented. This method is then applied to theconjunctive Rasch model. Fifth and last, an application of the conjunctive Rasch model is presented. This model was applied to responses to typical verbal ability items (open synonym items).This paper presents theoretical and empirical results of a research project supported by the Research Council [Onderzoeksraad] of the University of Leuven (grant number 89-9) to Paul De Boeck and Luc Delbeke.     

EM algorithms for ML factor analysis

The details of EM algorithms for maximum likelihood factor analysis are presented for both the exploratory and confirmatory models. The algorithm is essentially the same for both cases and involves only simple least squares regression operations; the largest matrix inversion required is for aq ×q symmetric matrix whereq is the matrix of factors. The example that is used demonstrates that the likelihood for the factor analysis model may have multiple modes that are not simply rotations of each other; such behavior should concern users of maximum likelihood factor analysis and certainly should cast doubt on the general utility of second derivatives of the log likelihood as measures of precision of estimation.     

Modelling asynchrony in automatic speech recognition using loosely coupled hidden Markov models

Hidden Markov models (HMMs) have been successful for modelling the dynamics of carefully dictated speech, but their performance degrades severely when used to model conversational speech. Since speech is produced by a system of loosely coupled articulators, stochastic models explicitly representing this parallelism may have advantages for automatic speech recognition (ASR), particularly when trying to model the phonological effects inherent in casual spontaneous speech. This paper presents a preliminary feasibility study of one such model class: loosely coupled HMMs. Exact model estimation and decoding is potentially expensive, so possible approximate algorithms are also discussed. Comparison of one particular loosely coupled model on an isolated word task suggests loosely coupled HMMs merit further investigation. An approximate algorithm giving performance which is almost always statistically indistinguishable from the exact algorithm is also identified, making more extensive research computationally feasible.     

Estimating multiple classification latent class models

This paper presents a new class of models for persons-by-items data. The essential new feature of this class is the representation of the persons: every person is represented by its membership tomultiple latent classes, each of which belongs to onelatent classification. The models can be considered as a formalization of the hypothesis that the responses come about in a process that involves the application of a number ofmental operations. Two algorithms for maximum likelihood (ML) and maximum a posteriori (MAP) estimation are described. They both make use of the tractability of the complete data likelihood to maximize the observed data likelihood. Properties of the MAP estimators (i.e., uniqueness and goodness-of-recovery) and the existence of asymptotic standard errors were examined in a simulation study. Then, one of these models is applied to the responses to a set of fraction addition problems. Finally, the models are compared to some related models in the literature.Thanks are to Paul De Boeck for creating the intellectually stimulating atmosphere in which this class of models came about, Iven van Mechelen for theone-sided idea, Kikumi Tatsuoka for the use of her data, and Theodoor Bouw for running part of the simulation study.     

Circle graphs and monadic second-order logic

This article is part of a project consisting in expressing, whenever possible, graph properties and graph transformations in monadic second-order logic or in its extensions using modulo p cardinality set predicates or auxiliary linear orders. A circle graph is the intersection graph of a set of chords of a circle. Such a set is called a chord diagram. It can also be described by a word with two occurrences of each letter, called a double occurrence word. If a circle graph is prime for the split (or join) decomposition defined by Cunnigham, it has a unique representation by a chord diagram, and this diagram can be defined by monadic second-order formulas with the even cardinality set predicate. By using the (canonical) split decomposition of a circle graph, we define in monadic second-order logic with auxiliary linear orders all its chord diagrams. This construction uses the fact that the canonical split decomposition of a graph can be constructed in monadic second-order logic with help of an arbitrary linear order. We prove that the order of first occurrences of the letters in a double occurrence word w that represents a connected circle graph determines this word in a unique way. The word w can be defined by a monadic second-order formula from the word of first occurrences of letters. We also prove that a set of circle graphs has bounded clique-width if and only if all the associated chord diagrams have bounded tree-width.     

An improved stochastic EM algorithm for large-scale full-information item factor analysis

In this paper, we explore the use of the stochastic EM algorithm (Celeux & Diebolt (1985) Computational Statistics Quarterly, 2, 73) for large-scale full-information item factor analysis. Innovations have been made on its implementation, including an adaptive-rejection-based Gibbs sampler for the stochastic E step, a proximal gradient descent algorithm for the optimization in the M step, and diagnostic procedures for determining the burn-in size and the stopping of the algorithm. These developments are based on the theoretical results of Nielsen (2000, Bernoulli, 6, 457), as well as advanced sampling and optimization techniques. The proposed algorithm is computationally efficient and virtually tuning-free, making it scalable to large-scale data with many latent traits (e.g. more than five latent traits) and easy to use for practitioners. Standard errors of parameter estimation are also obtained based on the missing-information identity (Louis, 1982, Journal of the Royal Statistical Society, Series B, 44, 226). The performance of the algorithm is evaluated through simulation studies and an application to the analysis of the IPIP-NEO personality inventory. Extensions of the proposed algorithm to other latent variable models are discussed.     

A maximum likelihood approach to test validation with missing and censored dependent variables

A maximum likelihood approach is described for estimating the validity of a test (x) as a predictor of a criterion variable (y) when there are both missing and censoredy scores present in the data set. The missing data are due to selection on a latent variable (y _s ) which may be conditionally related toy givenx. Thus, the missing data may not be missing random. The censoring process in due to the presence of a floor or ceiling effect. The maximum likelihood estimates are constructed using the EM algorithm. The entire analysis is demonstrated in terms of hypothetical data sets.     

The CHIC Model: A Global Model for Coupled Binary Data

Often problems result in the collection of coupled data, which consist of different N-way N-mode data blocks that have one or more modes in common. To reveal the structure underlying such data, an integrated modeling strategy, with a single set of parameters for the common mode(s), that is estimated based on the information in all data blocks, may be most appropriate. Such a strategy implies a global model, consisting of different N-way N-mode submodels, and a global loss function that is a (weighted) sum of the partial loss functions associated with the different submodels. In this paper, such a global model for an integrated analysis of a three-way three-mode binary data array and a two-way two-mode binary data matrix that have one mode in common is presented. A simulated annealing algorithm to estimate the model parameters is described and evaluated in a simulation study. An application of the model to real psychological data is discussed. T. Wilderjans is a Research Assistant of the Fund for Scientific Research—Flanders (Belgium). The research reported in this paper was partially supported by the Research Council of K.U. Leuven (GOA/2005/04). We are grateful to Kristof Vansteelandt for providing us with an interesting data set. We also thank three anonymous reviewers for their useful comments.     

A multistage ranking model

In this paper, we propose a (n–1)² parameter, multistage ranking model, which represents a generalization of Luce's model. We propose then×n item-rank relative frequency matrix (p-matrix) as a device for summarizing a set of rankings. As an alternative to the traditional maximum likelihood estimation, for the proposed model we suggest a method which estimates the parameters from thep-matrix. An illustrative numerical example is given. The proposed model and its differences from Luce's model are briefly discussed. We also show some specialp-matrix patterns possessed by the Thurstonian models and distance-based models.     

The Local Minima Problem in Hierarchical Classes Analysis: An Evaluation of a Simulated Annealing Algorithm and Various Multistart Procedures

Hierarchical classes models are quasi-order retaining Boolean decomposition models for N-way N-mode binary data. To fit these models to data, rationally started alternating least squares (or, equivalently, alternating least absolute deviations) algorithms have been proposed. Extensive simulation studies showed that these algorithms succeed quite well in recovering the underlying truth but frequently end in a local minimum. In this paper we evaluate whether or not this local minimum problem can be mitigated by means of two common strategies for avoiding local minima in combinatorial data analysis: simulated annealing (SA) and use of a multistart procedure. In particular, we propose a generic SA algorithm for hierarchical classes analysis and three different types of random starts. The effectiveness of the SA algorithm and the random starts is evaluated by reanalyzing data sets of previous simulation studies. The reported results support the use of the proposed SA algorithm in combination with a random multistart procedure, regardless of the properties of the data set under study. Eva Ceulemans is a post-doctoral fellow of the Fund for Scientific Research Flanders (Belgium). Iwin Leenen is a post-doctoral researcher of the Spanish Ministerio de Educación y Ciencia (programa Ramón y Cajal). The research reported in this paper was partially supported by the Research Council of K.U. Leuven (GOA/05/04).     

Semi-sparse PCA

It is well known that the classical exploratory factor analysis (EFA) of data with more observations than variables has several types of indeterminacy. We study the factor indeterminacy and show some new aspects of this problem by considering EFA as a specific data matrix decomposition. We adopt a new approach to the EFA estimation and achieve a new characterization of the factor indeterminacy problem. A new alternative model is proposed, which gives determinate factors and can be seen as a semi-sparse principal component analysis (PCA). An alternating algorithm is developed, where in each step a Procrustes problem is solved. It is demonstrated that the new model/algorithm can act as a specific sparse PCA and as a low-rank-plus-sparse matrix decomposition. Numerical examples with several large data sets illustrate the versatility of the new model, and the performance and behaviour of its algorithmic implementation.

     

A simplification of a result by zellini on the maximal rank of symmetric three-way arrays

Zellini (1979, Theorem 3.1) has shown how to decompose an arbitrary symmetric matrix of ordern ×n as a linear combination of 1/2n(n+1) fixed rank one matrices, thus constructing an explicit tensor basis for the set of symmetricn ×n matrices. Zellini's decomposition is based on properties of persymmetric matrices. In the present paper, a simplified tensor basis is given, by showing that a symmetric matrix can also be decomposed in terms of 1/2n(n+1) fixed binary matrices of rank one. The decomposition implies that ann ×n ×p array consisting ofp symmetricn ×n slabs has maximal rank 1/2n(n+1). Likewise, an unconstrained INDSCAL (symmetric CANDECOMP/PARAFAC) decomposition of such an array will yield a perfect fit in 1/2n(n+1) dimensions. When the fitting only pertains to the off-diagonal elements of the symmetric matrices, as is the case in a version of PARAFAC where communalities are involved, the maximal number of dimensions can be further reduced to 1/2n(n–1). However, when the saliences in INDSCAL are constrained to be nonnegative, the tensor basis result does not apply. In fact, it is shown that in this case the number of dimensions needed can be as large asp, the number of matrices analyzed.     

Large-scale latent semantic analysis

Latent semantic analysis (LSA) is a statistical technique for representing word meaning that has been widely used for making semantic similarity judgments between words, sentences, and documents. In order to perform an LSA analysis, an LSA space is created in a two-stage procedure, involving the construction of a word frequency matrix and the dimensionality reduction of that matrix through singular value decomposition (SVD). This article presents LANSE, an SVD algorithm specifically designed for LSA, which allows extremely large matrices to be processed using off-the-shelf computer hardware.     

Integer programs for one- and two-mode blockmodeling based on prespecified image matrices for structural and regular equivalence

Establishing blockmodels for one- and two-mode binary network matrices has typically been accomplished using multiple restarts of heuristic algorithms that minimize functions of inconsistency with an ideal block structure. Although these algorithms likely yield exceptional performance, they are not assured to provide blockmodels that optimize the functional indices. In this paper, we present integer programming models that, for a prespecified image matrix, can produce guaranteed optimal solutions for matrices of nontrivial size. Accordingly, analysts performing a confirmatory analysis of a prespecified blockmodel structure can apply our models directly to obtain an optimal solution. In exploratory cases where a blockmodel structure is not prespecified, we recommend a two-stage procedure, where a heuristic method is first used to identify an image matrix and the integer program is subsequently formulated and solved to identify the optimal solution for that image matrix. Although best suited for ideal block structures associated with structural equivalence, the integer programming models have the flexibility to accommodate functional indices pertaining to regular equivalence. Computational results are reported for a variety of one- and two-mode matrices from the blockmodeling literature.     

Information matrix estimation procedures for cognitive diagnostic models

Two new methods to estimate the asymptotic covariance matrix for marginal maximum likelihood estimation of cognitive diagnosis models (CDMs), the inverse of the observed information matrix and the sandwich-type estimator, are introduced. Unlike several previous covariance matrix estimators, the new methods take into account both the item and structural parameters. The relationships between the observed information matrix, the empirical cross-product information matrix, the sandwich-type covariance matrix and the two approaches proposed by de la Torre (2009, J. Educ. Behav. Stat., 34, 115) are discussed. Simulation results show that, for a correctly specified CDM and Q-matrix or with a slightly misspecified probability model, the observed information matrix and the sandwich-type covariance matrix exhibit good performance with respect to providing consistent standard errors of item parameter estimates. However, with substantial model misspecification only the sandwich-type covariance matrix exhibits robust performance.     

Hierarchical classes: Model and data analysis

A discrete, categorical model and a corresponding data-analysis method are presented for two-way two-mode (objects × attributes) data arrays with 0, 1 entries. The model contains the following two basic components: a set-theoretical formulation of the relations among objects and attributes; a Boolean decomposition of the matrix. The set-theoretical formulation defines a subset of the possible decompositions as consistent with it. A general method for graphically representing the set-theoretical decomposition is described. The data-analysis algorithm, dubbed HICLAS, aims at recovering the underlying structure in a data matrix by minimizing the discrepancies between the data and the recovered structure. HICLAS is evaluated with a simulation study and two empirical applications.This research was supported in part by a grant from the Belgian NSF (NFWO) to Paul De Boeck and in part by NSF Grant BNS-83-01027 to Seymour Rosenberg. We thank Iven Van Mechelen for clarifying several aspects of the Boolean algebraic formulation of the model and Phipps Arabie for his comments on an earlier draft.     

Consistent estimation in the rasch model based on nonparametric margins

Consider the class of two parameter marginal logistic (Rasch) models, for a test ofm True-False items, where the latent ability is assumed to be bounded. Using results of Karlin and Studen, we show that this class of nonparametric marginal logistic (NML) models is equivalent to the class of marginal logistic models where the latent ability assumes at most (m + 2)/2 values. This equivalence has two implications. First, estimation for the NML model is accomplished by estimating the parameters of a discrete marginal logistic model. Second, consistency for the maximum likelihood estimates of the NML model can be shown (whenm is odd) using the results of Kiefer and Wolfowitz. An example is presented which demonstrates the estimation strategy and contrasts the NML model with a normal marginal logistic model.This research was supported by NIMH traning grant, 2 T32 MH 15758-06 and by ONR contract N00014-84-K-0588. The author would like to thank Diane Lambert, John Rolph, and Stephen Fienberg for their assistance. Also, the comments of the referees helped to substantially improve the final version of this paper.     

A Finite Mixture of Nonlinear Random Coefficient Models for Continuous Repeated Measures Data

Nonlinear random coefficient models (NRCMs) for continuous longitudinal data are often used for examining individual behaviors that display nonlinear patterns of development (or growth) over time in measured variables. As an extension of this model, this study considers the finite mixture of NRCMs that combine features of NRCMs with the idea of finite mixture (or latent class) models. The efficacy of this model is that it allows the integration of intrinsically nonlinear functions where the data come from a mixture of two or more unobserved subpopulations, thus allowing the simultaneous investigation of intra-individual (within-person) variability, inter-individual (between-person) variability, and subpopulation heterogeneity. Effectiveness of this model to work under real data analytic conditions was examined by executing a Monte Carlo simulation study. The simulation study was carried out using an R routine specifically developed for the purpose of this study. The R routine used maximum likelihood with the expectation–maximization algorithm. The design of the study mimicked the output obtained from running a two-class mixture model on task completion data.     

Relating latent class membership to external variables: An overview

In this article we provide an overview of existing approaches for relating latent class membership to external variables of interest. We extend on the work of Nylund-Gibson et al. (Structural Equation Modeling: A Multidisciplinary Journal, 2019, 26, 967), who summarize models with distal outcomes by providing an overview of most recommended modeling options for models with covariates and larger models with multiple latent variables as well. We exemplify the modeling approaches using data from the General Social Survey for a model with a distal outcome where underlying model assumptions are violated, and a model with multiple latent variables. We discuss software availability and provide example syntax for the real data examples in Latent GOLD.