Hierarchical Multinomial Processing Tree Models: A Latent-Class Approach

Multinomial processing tree models are widely used in many areas of psychology. Their application relies on the assumption of parameter homogeneity, that is, on the assumption that participants do not differ in their parameter values. Tests for parameter homogeneity are proposed that can be routinely used as part of multinomial model analyses to defend the assumption. If parameter homogeneity is found to be violated, a new family of models, termed latent-class multinomial processing tree models, can be applied that accommodates parameter heterogeneity and correlated parameters, yet preserves most of the advantages of the traditional multinomial method. Estimation, goodness-of-fit tests, and tests of other hypotheses of interest are considered for the new family of models. The author thanks Bill Batchelder, Edgar Erdfelder, Thorsten Meiser, and Christoph Stahl for helpful comments on a previous version of this paper. The author is also grateful to Edgar Erdfelder for making available the data set analyzed in this paper.     

Pooling data versus averaging model fits for some prototypical multinomial processing tree models

For a study with multinomial data where there are n_g individuals and with each person having n_r test trials, the question arises as to how to fit the parameters of a multinomial processing tree (MPT) model. Should each parameter be estimated for each individual and then averaged to obtain a group estimate, or should the frequencies in the multinomial categories be pooled so that the model is fit once for the entire group? This basic question is explored with a series of Monte Carlo simulations for some prototypical MPT models. There is a general finding of a pooling advantage for the case where there is a single experimental condition. Also when there are different experimental conditions, there is reduced bias for detecting condition differences for a method based on the pooled data. Although the focus of the paper is on multinomial models, a general theorem is advanced that establishes a basic condition that determines whether there is or is not a difference between the averaging of individual estimates and the estimate based on the pooled data.     

Using Excel to estimate parameters from observed data: An example from source memory data

We present an application, using Excel, that can solve best-fitting parameters for multinomial models. Multinomial modeling has become increasingly popular and can be used in a variety of domains, such as memory, perception, and other domains in which processes are assumed to be dissociable. We offer an application that can be used for a variety of psychological models and can be used on both PC and Macintosh platforms. We illustrate the use of our program by analyzing data from a source memory experiment.     

Parameter identification in multinomial processing tree models

Multinomial processing tree models form a popular class of statistical models for categorical data that have applications in various areas of psychological research. As in all statistical models, establishing which parameters are identified is necessary for model inference and selection on the basis of the likelihood function, and for the interpretation of the results. The required calculations to establish global identification can become intractable in complex models. We show how to establish local identification in multinomial processing tree models, based on formal methods independently proposed by Catchpole and Morgan (1997) and by Bekker, Merckens, and Wansbeek (1994). This approach is illustrated with multinomial processing tree models for the source-monitoring paradigm in memory research.     

A multinomial model to assess central characteristics of mental operators

Knowledge about potential operators, about the preconditions of their applicability, and about their effects is essential to interact effectively with the physical world. Four classes of representational units of this knowledge can be distinguished: I) rules, II) structures, III) instances, and IV) episodes. Two important characteristics of these units are the abstractness of content and the directionality of access. A multinomial model is presented that enables the measurement of these characteristics. Three experiments were conducted to validate the parameters of the model. The multinomial model could be fitted very well to the empirical data of each experiment. Moreover, the parameter estimates showed the expected effects. The model allows the investigation of the influence of important variables (for example, knowledge domain, type of instruction, or amount of practice) on characteristics of mental operators without a strong commitment to any specific process theory. Debates regarding the contribution of different kinds of knowledge can be converted into statistical tests of the corresponding model parameters.     

Assessing individual differences in categorical data

In cognitive modeling, data are often categorical observations taken over participants and items. Usually subsets of these observations are pooled and analyzed by a cognitive model assuming the category counts come from a multinomial distribution with the same model parameters underlying all observations. It is well known that if there are individual differences in participants and/or items, a model analysis of the pooled data may be quite misleading, and in such cases it may be appropriate to augment the cognitive model with parametric random effects assumptions. On the other hand, if random effects are incorporated into a cognitive model that is not needed, the resulting model may be more flexible than the multinomial model that assumes no heterogeneity, and this may lead to overfitting. This article presents Monte Carlo statistical tests for directly detecting individual participant and/or item heterogeneity that depend only on the data structure itself. These tests are based on the fact that heterogeneity in participants and/or items results in overdispersion of certain category count statistics. It is argued that the methods developed in the article should be applied to any set of participant 3 item categorical data prior to cognitive model-based analyses.     

Assessing the evidence for response time mixture distributions

I describe a technique for comparing two simple accounts of a distribution of response times: A mixture model and a generalized-shift model. In the mixture model, a target distribution is assumed to be a mixture of response times from two other (reference) distributions. In the generalized-shift model, the target distribution is assumed to be a quantile average of the reference distributions. In order to distinguish these two possibilities, quantiles for the target distribution are estimated from the quantiles of the reference distributions assuming either a shift or a mixture, and the predicted quantiles are used to calculate the multinomial likelihood of the obtained data. Monte Carlo simulations reported here demonstrate that the index is relatively unbiased, is effective with moderate sample sizes and modest spreads between the reference distributions, is relatively unaffected by changes in the number of bins or by data trimming, can be used with data aggregated across subjects, and is relatively insensitive to a range of subject variations in distribution shape and in mixture or shift proportion. As an illustration, the index is applied to the interpretation of three effects from distinct paradigms: residual switch costs in the task-switching paradigm, the psychological refractory period effect, and sequential effects in the Simon task. I conclude that the multinomial likelihood index provides a useful and easily applied tool for the interpretation of effects on response time distributions.     

Carnap's inductive probabilities as a contribution to decision theory

Common probability theories only allow the deduction of probabilities by using previously known or presupposed probabilities. They do not, however, allow the derivation of probabilities from observed data alone. The question thus arises as to how probabilities in the empirical sciences, especially in medicine, may be arrived at. Carnap hoped to be able to answer this question byhis theory of inductive probabilities. In the first four sections of the present paper the above mentioned problem is discussed in general. After a short presentation of Carnap's theory it is then shown that this theory cannot claim validity for arbitrary random processes. It is suggested that the theory be only applied to binomial and multinomial experiments. By application of de Finetti's theorem Carnap's inductive probabilities are interpreted as consecutive probabilities of the Bayesian kind. Through the introduction of a new axiom the decision parameter λ can be determined even if no a priori knowledge is given. Finally, it is demonstrated that the fundamental problem of Wald's decision theory, i.e., the determination of a plausible criterion where no a priori knowledge is available, can be solved for the cases of binomial and multinomial experiments.     

The derivation of some tests for the rasch model from the multinomial distribution

The present paper is concerned with testing the fit of the Rasch model. It is shown that this can be achieved by constructing functions of the data, on which model tests can be based that have power against specific model violations. It is shown that the asymptotic distribution of these tests can be derived by using the theoretical framework of testing model fit in general multinomial and product-multinomial models. The model tests are presented in two versions: one that can be used in the context of marginal maximum likelihood estimation and one that can be applied in the context of conditional maximum likelihood estimation.I am indebted to Norman Verhelst and Niels Veldhuijzen for their helpful comments. Requests for reprints should be sent to Cees A. W. Glas, Cito, PO Box 1034, 6801 MG Arnhem, THE NETHERLANDS.     

Accounting for belief bias in a mental model framework: comment on Klauer, Musch, and Naumer (2000)

K. C. Klauer, J. Musch, and B. Naumer (2000; see record 2000-02818-008) presented a general multinomial model of belief bias effects in syllogistic reasoning. They claimed to map a particular mental model account of belief bias (J. V. Oakhill, P. N. Johnson-Laird, & A. Garnham, 1989; see record 1989-38845-001)) onto this model and to show empirically that it is incorrect. The authors argue that this mental model account does not map onto the multinomial model and that it can account for the data presented by Klauer et al. (Experiments 1-4). The authors further argue that additional data Klauer et al. presented in support of a new model of their own (Experiments 5-8) are explained by this mental model account. The mental model account is, therefore, refuted neither by Klauer et al.'s theoretical analysis nor by any of the results they presented. Furthermore, the account can accommodate more recent findings on belief bias in a more satisfactory way than can alternative models that have been proposed.     

Implicit memory for auditorily presented threatening stimuli: a process-dissociation approach

The investigation of unconscious cognition involves especially problems with the methodology of measuring implicit and explicit proportions of different task performances. In this study the process dissociation procedure of Jacoby and its modification within the multinomial modelling framework for an indirect word-nonword-discrimination task is applied to a sample of 45 healthy students. The paradigm includes acoustically presented stimuli. During a learning phase, subjects listened to a series of neutral and threatening words. Performance was tested by letting subjects decide whether a presented stimulus (masked with white noise at signal-noise ratio of -17 dB or unmasked) had been a word or a nonword. Within this paradigm, implicit cognition occurs when (a) a word is more probably correctly recognized as "word" after presentation during the learning phase (typical priming effect) or when (b) a nonword derived from a word is more probably falsely recognized as "word" after its corresponding word had been presented during the learning phase (effect of implicit cognition given perceptual fluency). Frequencies for hits and false alarms were analyzed within the multinomial model which allows estimating parameters for the correct discrimination of words (c), the response bias (b), the classical priming effect (u1), and the parameter for the priming effect of "old" nonwords (u2). Under masked stimuli the multinomial model showed implicit cognition, an effect not equally found for neutral and threatening words. Threatening words exhibited a significantly higher portion of implicit cognition than neutral ones. Given the statistical complexity of multinomial models, the application of this method was explained in detail.     

A General Family of Limited Information Goodness-of-Fit Statistics for Multinomial Data

Maydeu-Olivares and Joe (J. Am. Stat. Assoc. 100:1009–1020, 2005; Psychometrika 71:713–732, 2006) introduced classes of chi-square tests for (sparse) multidimensional multinomial data based on low-order marginal proportions. Our extension provides general conditions under which quadratic forms in linear functions of cell residuals are asymptotically chi-square. The new statistics need not be based on margins, and can be used for one-dimensional multinomials. We also provide theory that explains why limited information statistics have good power, regardless of sparseness. We show how quadratic-form statistics can be constructed that are more powerful than X ² and yet, have approximate chi-square null distribution in finite samples with large models. Examples with models for truncated count data and binary item response data are used to illustrate the theory.     

An integrative review of process dissociation and related models in social cognition

This chapter reviews the use of formal dual process models in social psychology, with a focus on the process dissociation model and related multinomial models. The utility of the models is illustrated using studies of social and affective influences on memory, judgement and decision making, and social attitudes and stereotypes. We then compare and contrast the process dissociation model with other approaches, including implicit and explicit tests, signal detection theory, and multinomial models. Finally we show how several recently proposed multinomial models can be integrated into a single family of models, of which process dissociation is a specific instance. We describe how these process models can be used as both theoretical and measurement tools to answer questions about the role of automatic and controlled processes in social behaviour.     

On belief bias in syllogistic reasoning

A multinomial model is used to disentangle the respective contributions of reasoning processes and response bias in conclusion-acceptance data that exhibit belief bias. A model-based meta-analysis of 22 studies reveals that such data are structurally too sparse to allow discrimination of different accounts of belief bias. Four experiments are conducted to obtain richer data, allowing deeper tests through the use of the multinomial model. None of the current accounts of belief bias is consistent with the complex pattern of results. A new theory of belief bias is proposed that assumes that most reasoners construct only one mental model representing the premises as well as the conclusion or, in the case of an unbelievable conclusion, its logical negation. New predictions derived from the theory are confirmed in 4 additional studies.     

Measuring Response Styles Across the Big Five: A Multiscale Extension of an Approach Using Multinomial Processing Trees

This study shows how to address the problem of trait-unrelated response styles (RS) in rating scales using multidimensional item response theory. The aim is to test and correct data for RS in order to provide fair assessments of personality. Expanding on an approach presented by Böckenholt (2012), observed rating data are decomposed into multiple response processes based on a multinomial processing tree. The data come from a questionnaire consisting of 50 items of the International Personality Item Pool measuring the Big Five dimensions administered to 2,026 U.S. students with a 5-point rating scale. It is shown that this approach can be used to test if RS exist in the data and that RS can be differentiated from trait-related responses. Although the extreme RS appear to be unidimensional after exclusion of only 1 item, a unidimensional measure for the midpoint RS is obtained only after exclusion of 10 items. Both RS measurements show high cross-scale correlations and item response theory-based (marginal) reliabilities. Cultural differences could be found in giving extreme responses. Moreover, it is shown how to score rating data to correct for RS after being proved to exist in the data.     

A context-free language for binary multinomial processing tree models

This paper provides a new formalization for the class of binary multinomial processing tree (BMPT) models, and theorems for the class are developed using the formalism. MPT models are a popular class of information processing models for categorical data in specific cognitive paradigms. They have a recursive structure that is productively described with the tools of formal language and computation theory. We provide an axiomatization that characterizes BMPT models as strings in a context-free language, and then we add model-theoretic axioms and definitions to interpret the strings as parameterized probabilistic models for categorical data. The language for BMPT models is related to the Full Binary Tree language, a well-studied context-free language. Once BMPT models are viewed from the perspective of the Full Binary Tree language, a number of theoretical and computational results can be developed. In particular, we have a number of results concerning the enumerations of BMPT models as well as the identifiability of subclasses of these models.     

HMMTree: A computer program for latent-class hierarchical multinomial processing tree models

Latent-class hierarchical multinomial models are an important extension of the widely used family of multinomial processing tree models, in that they allow for testing the parameter homogeneity assumption and provide a framework for modeling parameter heterogeneity. In this article, the computer program HMMTree is introduced as a means of implementing latent-class hierarchical multinomial processing tree models. HMMTree computes parameter estimates, confidence intervals, and goodness-of-fit statistics for such models, as well as the Fisher information, expected category means and variances, and posterior probabilities for class membership. A brief guide to using the program is provided.     

Polytomous Latent Scales for the Investigation of the Ordering of Items

We propose three latent scales within the framework of nonparametric item response theory for polytomously scored items. Latent scales are models that imply an invariant item ordering, meaning that the order of the items is the same for each measurement value on the latent scale. This ordering property may be important in, for example, intelligence testing and person-fit analysis. We derive observable properties of the three latent scales that can each be used to investigate in real data whether the particular model adequately describes the data. We also propose a methodology for analyzing test data in an effort to find support for a latent scale, and we use two real-data examples to illustrate the practical use of this methodology.     

IQ heritability estimation: analyzing genetically-informative data with structural equation models

When analyzing genetic data, Structural Equations Modeling (SEM) provides a straightforward methodology to decompose phenotypic variance using a model-based approach. Furthermore, several models can be easily implemented, tested, and compared using SEM, allowing the researcher to obtain valuable information about the sources of variability. This methodology is briefly described and applied to re-analyze a Spanish set of IQ data using the biometric ACE model. In summary, we report heritability estimates that are consistent with those of previous studies and support substantial genetic contribution to phenotypic IQ; around 40% of the variance can be attributable to it. With regard to the environmental contribution, shared environment accounts for 50% of the variance, and non-shared environment accounts for the remaining 10%. These results are discussed in the text.     

A revised methodology for research on metamemory: Pre-judgment Recall and Monitoring (PRAM)

A revised methodology is described for research on metacognitive monitoring, especially judgments of learning (JOLs), to investigate psychological processing that previously has been only hypothetical and unobservable. During data collection a new stage of recall occurs just prior to the JOL, so that during data analysis the items can be partitioned into subcategories to measure the degree of JOL accuracy in ways that are more analytic than was previously possible. A weighted-average combinatorial rule allows the component measures of JOL accuracy to be combined into the usual overall measure of metacognitive accuracy. An example using the revised methodology offers a new explanation for the delayed-JOL effect, in which delayed JOLs are more accurate than immediate JOLs for predicting recall.