TreeBUGS: An R package for hierarchical multinomial-processing-tree modeling

Multinomial processing tree (MPT) models are a class of measurement models that account for categorical data by assuming a finite number of underlying cognitive processes. Traditionally, data are aggregated across participants and analyzed under the assumption of independently and identically distributed observations. Hierarchical Bayesian extensions of MPT models explicitly account for participant heterogeneity by assuming that the individual parameters follow a continuous hierarchical distribution. We provide an accessible introduction to hierarchical MPT modeling and present the user-friendly and comprehensive R package TreeBUGS, which implements the two most important hierarchical MPT approaches for participant heterogeneity—the beta-MPT approach (Smith & Batchelder, Journal of Mathematical Psychology 54:167-183, 2010) and the latent-trait MPT approach (Klauer, Psychometrika 75:70-98, 2010). TreeBUGS reads standard MPT model files and obtains Markov-chain Monte Carlo samples that approximate the posterior distribution. The functionality and output are tailored to the specific needs of MPT modelers and provide tests for the homogeneity of items and participants, individual and group parameter estimates, fit statistics, and within- and between-subjects comparisons, as well as goodness-of-fit and summary plots. We also propose and implement novel statistical extensions to include continuous and discrete predictors (as either fixed or random effects) in the latent-trait MPT model.     

Hierarchical Multinomial Processing Tree Models: A Latent-Class Approach

Multinomial processing tree models are widely used in many areas of psychology. Their application relies on the assumption of parameter homogeneity, that is, on the assumption that participants do not differ in their parameter values. Tests for parameter homogeneity are proposed that can be routinely used as part of multinomial model analyses to defend the assumption. If parameter homogeneity is found to be violated, a new family of models, termed latent-class multinomial processing tree models, can be applied that accommodates parameter heterogeneity and correlated parameters, yet preserves most of the advantages of the traditional multinomial method. Estimation, goodness-of-fit tests, and tests of other hypotheses of interest are considered for the new family of models. The author thanks Bill Batchelder, Edgar Erdfelder, Thorsten Meiser, and Christoph Stahl for helpful comments on a previous version of this paper. The author is also grateful to Edgar Erdfelder for making available the data set analyzed in this paper.     

A hierarchical model for estimating response time distributions

We present a statistical model for inference with response time (RT) distributions. The model has the following features. First, it provides a means of estimating the shape, scale, and location (shift) of RT distributions. Second, it is hierarchical and models between-subjects and within-subjects variability simultaneously. Third, inference with the model is Bayesian and provides a principled and efficient means of pooling information across disparate data from different individuals. Because the model efficiently pools information across individuals, it is particularly well suited for those common cases in which the researcher collects a limited number of observations from several participants. Monte Carlo simulations reveal that the hierarchical Bayesian model provides more accurate estimates than several popular competitors do. We illustrate the model by providing an analysis of the symbolic distance effect in which participants can more quickly ascertain the relationship between nonadjacent digits than that between adjacent digits.     

多项式加工树模型在再认启发式中的应用

摘 要 再认启发式利用再认线索进行决策。以往研究采用一致率、击中率、虚报率和区分指数来表示再认启发式使用,然而这些方法都存在局限。多项式加工树模型能够分离不同的认知加工过程,为了解决再认使用与知识使用的混淆,研究者提出一种多项式加工树模型 r-model 测量再认启发式的使用。本文将重 点介绍 r-model,具体包括 r-model 的内容、数据分析以及考虑个体差异的分层 r-model。最后,从 r-model 的模型修正和边界条件两个方面提出未来研究方向。 关键词 再认启发式;流畅启发式;多项式加工树;贝叶斯分层模型     

因果模型在类比推理中的作用

通过操纵因果模型的特征维度及推理方向, 探讨因果模型在类比推理中的作用。实验一探讨了当结果特征未知时进行类比推理的情况, 发现在一果多因时, 被试采用因果模型进行类比推理, 而在一因多果时, 被试同时采用因果模型和计算模型进行类比推理。实验二探讨当原因特征未知时进行类比推理的情况, 发现在一果多因和一因多果时, 被试均采用因果模型进行类比推理。结果表明：(1)当结果特征未知时, 人们会建构因果模型进行类比推理。且当因果模型和计算模型处于冲突情境时, 人们会采用因果模型进行类比推理; 但当因果模型和计算模型处于非冲突情境时, 人们会同时采用因果模型和计算模型。(2)当原因特征未知时, 即按照因果模型推理的难度增加时, 人们仍会建构因果模型进行类比推理。     

On normal theory based inference for multilevel models with distributional violations

Data in social and behavioral sciences are often hierarchically organized though seldom normal, yet normal theory based inference procedures are routinely used for analyzing multilevel models. Based on this observation, simple adjustments to normal theory based results are proposed to minimize the consequences of violating normality assumptions. For characterizing the distribution of parameter estimates, sandwich-type covariance matrices are derived. Standard errors based on these covariance matrices remain consistent under distributional violations. Implications of various covariance estimators are also discussed. For evaluating the quality of a multilevel model, a rescaled statistic is given for both the hierarchical linear model and the hierarchical structural equation model. The rescaled statistic, improving the likelihood ratio statistic by estimating one extra parameter, approaches the same mean as its reference distribution. A simulation study with a 2-level factor model implies that the rescaled statistic is preferable.This research was supported by grants DA01070 and DA00017 from the National Institute on Drug Abuse and a University of North Texas faculty research grant. We would like to thank the Associate Editor and two reviewers for suggestions that helped to improve the paper.     

Theoretical and empirical review of multinomial process tree modeling

We review a current and popular class of cognitive models calledmultinomial processing tree (MPT) models. MPT models are simple, substantively motivated statistical models that can be applied to categorical data. They are useful as data-analysis tools for measuring underlying or latent cognitive capacities and as simple models for representing and testing competing psychological theories. We formally describe the cognitive structure and parametric properties of the class of MPT models and provide an inferential statistical analysis for the entire class. Following this, we provide a comprehensive review of over 80 applications of MPT models to a variety of substantive areas in cognitive psychology, including various types of human memory, visual and auditory perception, and logical reasoning. We then address a number of theoretical issues relevant to the creation and evaluation of MPT models, including model development, model validity, discrete-state assumptions, statistical issues, and the relation between MPT models and other mathematical models. In the conclusion, we consider the current role of MPT models in psychological research and possible future directions.     

多项式加工树模型次序约束的量化分析及其应用

多项式加工树（MPT）模型是一种认知测量模型,能够对潜在认知过程进行测量和检验。已有研究探讨了二链MPT模型次序约束的重新参数化问题,本研究探讨了MPT模型次序约束的量化分析方法并从二链推广到多链,同时归纳出MPT模型参数向量内和参数向量间两参数次序约束量化分析的结论。数据分析结果表明该方法不仅在MPT模型框架下验证了潜在参数次序关系,而且给出了约束的量化指标,为潜在认知测量提供更有意义的解释。     

On the use of heterogeneous thresholds ordinal regression models to account for individual differences in response style

This paper proposes a general approach to accounting for individual differences in the extreme response style in statistical models for ordered response categories. This approach uses a hierarchical ordinal regression modeling framework with heterogeneous thresholds structures to account for individual differences in the response style. Markov chain Monte Carlo algorithms for Bayesian inference for models with heterogeneous thresholds structures are discussed in detail. A simulation and two examples based on ordinal probit models are given to illustrate the proposed methodology. The simulation and examples also demonstrate that failing to account for individual differences in the extreme response style can have adverse consequences for statistical inferences.The author is grateful to Ulf Böckenholt, an associate editor, and three anonymous reviewers for helpful comments, and Kristine Kuhn and Kshiti Joshi for providing the data.     

Multilevel Model Prediction

Multilevel models are proven tools in social research for modeling complex, hierarchical systems. In multilevel modeling, statistical inference is based largely on quantification of random variables. This paper distinguishes among three types of random variables in multilevel modeling—model disturbances, random coefficients, and future response outcomes—and provides a unified procedure for predicting them. These predictors are best linear unbiased and are commonly known via the acronym BLUP; they are optimal in the sense of minimizing mean square error and are Bayesian under a diffuse prior. For parameter estimation purposes, a multilevel model can be written as a linear mixed-effects model. In this way, parameters of the many equations can be estimated simultaneously and hence efficiently. For prediction purposes, we show that it is more convenient to retain the multiple equation feature of multilevel models. In this way, the efficient BLUPs are easy to compute and retain their intuitively appealing recursive form. We also derive explicit equations for standard errors of these different types of predictors. Prediction in multilevel modeling is important in a wide range of applications. To demonstrate the applicability of our results, this paper discusses prediction in the context of a study of school effectiveness. This research was supported by a grant from the Graduate School at the University of Wisconsin at Madision and the National Science Foundation, Grant number SES-0436274. We are grateful to Norman Webb at Wisconsin Center for Education Research for making available the data used in the reported application.     

Action understanding as inverse planning

Humans are adept at inferring the mental states underlying other agents’ actions, such as goals, beliefs, desires, emotions and other thoughts. We propose a computational framework based on Bayesian inverse planning for modeling human action understanding. The framework represents an intuitive theory of intentional agents’ behavior based on the principle of rationality: the expectation that agents will plan approximately rationally to achieve their goals, given their beliefs about the world. The mental states that caused an agent’s behavior are inferred by inverting this model of rational planning using Bayesian inference, integrating the likelihood of the observed actions with the prior over mental states. This approach formalizes in precise probabilistic terms the essence of previous qualitative approaches to action understanding based on an “intentional stance” [Dennett, D. C. (1987). The intentional stance. Cambridge, MA: MIT Press] or a “teleological stance” [Gergely, G., Nádasdy, Z., Csibra, G., & Biró, S. (1995). Taking the intentional stance at 12 months of age. Cognition, 56, 165-193]. In three psychophysical experiments using animated stimuli of agents moving in simple mazes, we assess how well different inverse planning models based on different goal priors can predict human goal inferences. The results provide quantitative evidence for an approximately rational inference mechanism in human goal inference within our simplified stimulus paradigm, and for the flexible nature of goal representations that human observers can adopt. We discuss the implications of our experimental results for human action understanding in real-world contexts, and suggest how our framework might be extended to capture other kinds of mental state inferences, such as inferences about beliefs, or inferring whether an entity is an intentional agent.     

Understanding memory impairment with memory models and hierarchical Bayesian analysis

The study of human episodic memory is a topic that interests cognitive and mathematical psychologists as well as clinicians interested in the diagnosis and assessment of Alzheimer’s disease and related disorders (ADRD). In this paper, we use simple cognitive models for the recognition and recall tasks typically applied in clinical assessments of ADRD to study memory performance in ADRD patients. Our models make use of hierarchical Bayesian methods as a way to model individual differences in patient performance and to facilitate the modeling of performance changes that occur during multiple recall tasks. We show how the models are able to account for different aspects of patient performance, and also discuss some of the predictive capabilities of the model. We conclude with a discussion on the scope to improve on our results by discussing the link between memory theory in psychology and clinical practice.     

A novel method for assessing rival models of recognition memory

A general comparison is made between the multinomial processing tree (MPT) approach and a strength-based approach for modeling recognition memory measurement. Strength models include the signal-detection model and the dual-process model. Existing MPT models for recognition memory and a new generic MPT model, called the Multistate (MS) model, are contrasted with the strength models. Although the ROC curves for the MS model and strength model are similar, there is a critical difference between existing strength models and MPT models that goes beyond the assessment of the ROC. This difference concerns the question of stochastic mixtures for foil test trials. The hazard function and the reverse hazard function are powerful methods for detecting the presence of a probabilistic mixture. Several new theorems establish a novel method for obtaining information about the hazard function and reverse hazard function for the latent continuous distributions that are assumed in the strength approach to recognition memory. Evidence is provided that foil test trials involve a stochastic mixture. This finding occurred for both short-term memory procedures, such as the Brown–Peterson task, and long-term list-learning procedures, such as the paired-associate task. The effect of mixtures on foil trials is problematic for existing strength models but can be readily handled by MPT models such as the MS model. Other phenomena, such as the mirror effect and the effect of target-foil similarity, are also predicted accurately by the MPT modeling framework.     

Evaluating models of recognition memory using first- and second-choice responses

Swets, Tanner Jr., and Birdsall (1961) proposed a 4-alternative forced-choice task with two choices (4AFC-2R) for distinguishing between the Equal-Variance Signal Detection model and the One-High Threshold model. This task was recently implemented in the field of recognition memory (Parks & Yonelinas, 2009), a field in which several candidate models have been proposed. One advantage of the 4AFC-2R task is that it permits parameter estimation and goodness of fit testing, something which so far was only possible through the use of Receiver Operating Characteristic (ROC) functions for the more complex candidate models. The present article provides a thorough characterization and comparison of the main recognition memory models in the context of this task. Results are illustrated by a reanalysis of Parks and Yonelinas’ original data, revealing a preference for hybrid approaches to recognition memory, more specifically for the dual-process model (Yonelinas, 1997), whereas pure signal detection models performed poorly. The present analysis provides an assessment of the merits and limitations of this task, highlighting future research applications.     

Re-conceptualizing the past: Historical data in vocational interest research

Noteworthy progress has been made in the development of statistical models for evaluating the structure of vocational interests over the past three decades. It is proposed that historically significant interest datasets, when combined with modern structural methods of data analysis, provide an opportunity to re-examine the underlying assumptions of J.L. Holland’s [Holland, J. L. (1959). A theory of vocational choice. Journal of Counseling Psychology, 6, 35–45; Holland, J. L. (1997). Making vocational choices (3rd ed.). Odessa, FL: Psychological Assessment Resources] RIASEC model. To illustrate this potential, data obtained from J. P. Guilford’s study of interest structure were re-analyzed using modern circumplex and hierarchical clustering techniques to evaluate Holland’s and I. Gati’s [Gati, I. (1979). A hierarchical model for the structure of interests. Journal of Vocational Behavior, 15, 90–106; Gati, I. (1991). The structure of vocational interests. Psychological Bulletin, 109, 309–324] interest structures. Obtained results indicate that a circumplex model can be used to effectively represent the structure underlying Guilford’s interest measures. However, hierarchical clustering results suggest that Holland’s RIASEC types may not be the most effective categories for grouping specific interest measures into broader interest areas. The current findings provide support for the continued investigation of alternatives to Holland’s interest categories using modern measures of basic interests.     

Improving precision of ability estimation: Getting more from response times

By considering information about response time (RT) in addition to response accuracy (RA), joint models for RA and RT such as the hierarchical model (van der Linden, 2007) can improve the precision with which ability is estimated over models that only consider RA. The hierarchical model, however, assumes that only the person's speed is informative of ability. This assumption of conditional independence between RT and ability given speed may be violated in practice, and ignores collateral information about ability that may be present in the residual RTs. We propose a posterior predictive check for evaluating the assumption of conditional independence between RT and ability given speed. Furthermore, we propose an extension of the hierarchical model that contains cross-loadings between ability and RT, which enables one to take additional collateral information about ability into account beyond what is possible in the standard hierarchical model. A Bayesian estimation procedure is proposed for the model. Using simulation studies, the performance of the model is evaluated in terms of parameter recovery, and the possible gain in precision over the standard hierarchical model and an RA-only model is considered. The model is applied to data from a high-stakes educational test.     

Bayesian hierarchical modeling of the temporal dynamics of subjective well-being: A 10 year longitudinal analysis

This study demonstrates, for the first time, how Bayesian hierarchical modeling can be applied to yield novel insights into the long-term temporal dynamics of subjective well-being (SWB). Several models were proposed and examined using Bayesian methods. The models were assessed using a sample of Australian adults (n = 1081) who provided annual SWB scores on between 5 and 10 occasions. The best fitting models involved a probit transformation, allowed error variance to vary across participants, and did not include a lag parameter. Including a random linear and quadratic effect resulted in only a small improvement over the intercept only model. Examination of individual-level fits suggested that most participants were stable with a small subset exhibiting patterns of systematic change.     

Biased belief in the Bayesian brain: A deeper look at the evidence

A recent critique of hierarchical Bayesian models of delusion argues that, contrary to a key assumption of these models, belief formation in the healthy (i.e., neurotypical) mind is manifestly non-Bayesian. Here we provide a deeper examination of the empirical evidence underlying this critique. We argue that this evidence does not convincingly refute the assumption that belief formation in the neurotypical mind approximates Bayesian inference. Our argument rests on two key points. First, evidence that purports to reveal the most damning violation of Bayesian updating in human belief formation is counterweighted by substantial evidence that indicates such violations are the rare exception—not a common occurrence. Second, the remaining evidence does not demonstrate convincing violations of Bayesian inference in human belief updating; primarily because this evidence derives from study designs that produce results that are not obviously inconsistent with Bayesian principles.