MCMC estimation and some model-fit analysis of multidimensional IRT models

A Bayesian procedure to estimate the three-parameter normal ogive model and a generalization of the procedure to a model with multidimensional ability parameters are presented. The procedure is a generalization of a procedure by Albert (1992) for estimating the two-parameter normal ogive model. The procedure supports analyzing data from multiple populations and incomplete designs. It is shown that restrictions can be imposed on the factor matrix for testing specific hypotheses about the ability structure. The technique is illustrated using simulated and real data. The authors would like to thank Norman Verhelst for his valuable comments and ACT, CITO group and SweSAT for the use of their data.     

Testing the efficiency of Markov chain Monte Carlo with People using facial affect categories

Exploring how people represent natural categories is a key step toward developing a better understanding of how people learn, form memories, and make decisions. Much research on categorization has focused on artificial categories that are created in the laboratory, since studying natural categories defined on high-dimensional stimuli such as images is methodologically challenging. Recent work has produced methods for identifying these representations from observed behavior, such as reverse correlation (RC). We compare RC against an alternative method for inferring the structure of natural categories called Markov chain Monte Carlo with People (MCMCP). Based on an algorithm used in computer science and statistics, MCMCP provides a way to sample from the set of stimuli associated with a natural category. We apply MCMCP and RC to the problem of recovering natural categories that correspond to two kinds of facial affect (happy and sad) from realistic images of faces. Our results show that MCMCP requires fewer trials to obtain a higher quality estimate of people's mental representations of these two categories.     

Bayesian inference for finite mixtures of generalized linear models with random effects

We present an hierarchical Bayes approach to modeling parameter heterogeneity in generalized linear models. The model assumes that there are relevant subpopulations and that within each subpopulation the individual-level regression coefficients have a multivariate normal distribution. However, class membership is not known a priori, so the heterogeneity in the regression coefficients becomes a finite mixture of normal distributions. This approach combines the flexibility of semiparametric, latent class models that assume common parameters for each sub-population and the parsimony of random effects models that assume normal distributions for the regression parameters. The number of subpopulations is selected to maximize the posterior probability of the model being true. Simulations are presented which document the performance of the methodology for synthetic data with known heterogeneity and number of sub-populations. An application is presented concerning preferences for various aspects of personal computers.     

Bayesian Hierarchical Classes Analysis

Hierarchical classes models are models for N-way N-mode data that represent the association among the N modes and simultaneously yield, for each mode, a hierarchical classification of its elements. In this paper we present a stochastic extension of the hierarchical classes model for two-way two-mode binary data. In line with the original model, the new probabilistic extension still represents both the association among the two modes and the hierarchical classifications. A fully Bayesian method for fitting the new model is presented and evaluated in a simulation study. Furthermore, we propose tools for model selection and model checking based on Bayes factors and posterior predictive checks. We illustrate the advantages of the new approach with applications in the domain of the psychology of choice and psychiatric diagnosis. Iwin Leenen is now at the Instituto Mexicano de Investigación de Familia y Población (IMIFAP), Mexico. The research reported in this paper was partially supported by the Spanish Ministerio de Educación y Ciencia (programa Ramón y Cajal) and by the Research Council of K.U.Leuven (PDM/99/037, GOA/2000/02, and GOA/2005/04). The authors are grateful to Johannes Berkhof for fruitful discussions.     

Bayesian adaptive estimation: The next dimension

We propose a new psychometric model for two-dimensional stimuli, such as color differences, based on parameterizing the threshold of a one-dimensional psychometric function as an ellipse. The Ψ Bayesian adaptive estimation method applied to this model yields trials that vary in multiple stimulus dimensions simultaneously. Simulations indicate that this new procedure can be much more efficient than the more conventional procedure of estimating the psychometric function on one-dimensional lines independently, requiring only one-fourth or less the number of trials for equivalent performance in typical situations. In a real psychophysical experiment with a yes-no task, as few as 22 trials per estimated threshold ellipse were enough to consistently demonstrate certain color appearance phenomena. We discuss the practical implications of the multidimensional adaptation. In order to make the application of the model practical, we present two significantly faster algorithms for running the Ψ method: a discretized algorithm utilizing the Fast Fourier Transform for better scaling with the sampling rates and a Monte Carlo particle filter algorithm that should be able to scale into even more dimensions.     

Bayesian estimation of a multilevel IRT model using gibbs sampling

In this article, a two-level regression model is imposed on the ability parameters in an item response theory (IRT) model. The advantage of using latent rather than observed scores as dependent variables of a multilevel model is that it offers the possibility of separating the influence of item difficulty and ability level and modeling response variation and measurement error. Another advantage is that, contrary to observed scores, latent scores are test-independent, which offers the possibility of using results from different tests in one analysis where the parameters of the IRT model and the multilevel model can be concurrently estimated. The two-parameter normal ogive model is used for the IRT measurement model. It will be shown that the parameters of the two-parameter normal ogive model and the multilevel model can be estimated in a Bayesian framework using Gibbs sampling. Examples using simulated and real data are given.     

Estimation and visualization of confusability matrices from adaptive measurement data

We present a simple but effective method based on Luce’s choice axiom [Luce, R.D. (1959). Individual choice behavior: A theoretical analysis. New York: John Wiley & Sons] for consistent estimation of the pairwise confusabilities of items in a multiple-choice recognition task with arbitrarily chosen choice-sets. The method combines the exact (non-asymptotic) Bayesian way of assessing uncertainty with the unbiasedness emphasized in the classical frequentist approach.We apply the method to data collected using an adaptive computer game designed for prevention of reading disability. A player’s estimated confusability of phonemes (or more accurately, phoneme-grapheme connections) and larger units of language is visualized in an easily understood way with color cues and explicit indication of the accuracy of the estimates. Visualization of learning-related changes in the player’s performance is considered.The empirical validity of the choice axiom is evaluated using the game data itself. The axiom appears to hold reasonably well although a small systematic violation is observable for the smallest choice-set sizes.