We favor formal models of heuristics rather than lists of loose dichotomies: a reply to Evans and Over

In their comment on Marewski et al. (good judgments do not require complex cognition, 2009) Evans and Over (heuristic thinking and human intelligence: a commentary on Marewski, Gaissmaier and Gigerenzer, 2009) conjectured that heuristics can often lead to biases and are not error free. This is a most surprising critique. The computational models of heuristics we have tested allow for quantitative predictions of how many errors a given heuristic will make, and we and others have measured the amount of error by analysis, computer simulation, and experiment. This is clear progress over simply giving heuristics labels, such as availability, that do not allow for quantitative comparisons of errors. Evans and Over argue that the reason people rely on heuristics is the accuracy-effort trade-off. However, the comparison between heuristics and more effortful strategies, such as multiple regression, has shown that there are many situations in which a heuristic is more accurate with less effort. Finally, we do not see how the fast and frugal heuristics program could benefit from a dual-process framework unless the dual-process framework is made more precise. Instead, the dual-process framework could benefit if its two “black boxes” (Type 1 and Type 2 processes) were substituted by computational models of both heuristics and other processes.     

An Introduction to Network Psychometrics: Relating Ising Network Models to Item Response Theory Models

In recent years, network models have been proposed as an alternative representation of psychometric constructs such as depression. In such models, the covariance between observables (e.g., symptoms like depressed mood, feelings of worthlessness, and guilt) is explained in terms of a pattern of causal interactions between these observables, which contrasts with classical interpretations in which the observables are conceptualized as the effects of a reflective latent variable. However, few investigations have been directed at the question how these different models relate to each other. To shed light on this issue, the current paper explores the relation between one of the most important network models—the Ising model from physics—and one of the most important latent variable models—the Item Response Theory (IRT) model from psychometrics. The Ising model describes the interaction between states of particles that are connected in a network, whereas the IRT model describes the probability distribution associated with item responses in a psychometric test as a function of a latent variable. Despite the divergent backgrounds of the models, we show a broad equivalence between them and also illustrate several opportunities that arise from this connection.     

Turing redux: Enculturation and computation

Many of our cognitive capacities are shaped by enculturation. Enculturation is the acquisition of cognitive practices such as symbol-based mathematical practices, reading, and writing during ontogeny. Enculturation is associated with significant changes to the organization and connectivity of the brain and to the functional profiles of embodied actions and motor programs. Furthermore, it relies on scaffolded cultural learning in the cognitive niche. The purpose of this paper is to explore the components of symbol-based mathematical practices. Phylogenetically, these practices are the result of concerted organism-niche interactions that have led from approximate number estimations to the emergence of discrete, symbol-based mathematical operations. Ontogenetically, symbol-based mathematical practices are associated with plastic changes to neural circuitry, action schemata, and motor programs. It will be suggested that these practices rely on previously acquired capacities such as subitizing and counting. With these considerations in place, I will argue that computations, understood in the sense of Turing (1936), are a specific kind of symbol-based mathematical practices that can be realized by human organisms, machines, or by hybrid organism-machine systems. In sum, this paper suggests a new way to think about mathematical cognition and computation.     

Using Response Times and Response Accuracy to Measure Fluency Within Cognitive Diagnosis Models

Psychometrika - The recent “Every Student Succeed Act" encourages schools to use an innovative assessment to provide feedback about students’ mastery level of grade-level content...     

Speed-Accuracy Response Models: Scoring Rules based on Response Time and Accuracy

Starting from an explicit scoring rule for time limit tasks incorporating both response time and accuracy, and a definite trade-off between speed and accuracy, a response model is derived. Since the scoring rule is interpreted as a sufficient statistic, the model belongs to the exponential family. The various marginal and conditional distributions for response accuracy and response time are derived, and it is shown how the model parameters can be estimated. The model for response accuracy is found to be the two-parameter logistic model. It is found that the time limit determines the item discrimination, and this effect is illustrated with the Amsterdam Chess Test II.     

Truth and meaning redux

In this paper, we defend Davidson's program in truth-theoretical semantics against recent criticisms by Scott Soames. We argue that Soames has misunderstood Davidson's project, that in consequence his criticisms miss the mark, that appeal to meanings as entities in the alternative approach that Soames favors does no work, and that the approach is no advance over truth-theoretic semantics.     

Log-Multiplicative Association Models as Item Response Models

Log-multiplicative association (LMA) models, which are special cases of log-linear models, have interpretations in terms of latent continuous variables. Two theoretical derivations of LMA models based on item response theory (IRT) arguments are presented. First, we show that Anderson and colleagues (Anderson &; Vermunt, 2000; Anderson &; Böckenholt, 2000; Anderson, 2002), who derived LMA models from statistical graphical models, made the equivalent assumptions as Holland (1990) when deriving models for the manifest probabilities of response patterns based on an IRT approach. We also present a second derivation of LMA models where item response functions are specified as functions of rest-scores. These various connections provide insights into the behavior of LMA models as item response models and point out philosophical issues with the use of LMA models as item response models. We show that even for short tests, LMA and standard IRT models yield very similar to nearly identical results when data arise from standard IRT models. Log-multiplicative association models can be used as item response models and do not require numerical integration for estimation.     

Hidden Markov Item Response Theory Models for Responses and Response Times

Current approaches to model responses and response times to psychometric tests solely focus on between-subject differences in speed and ability. Within subjects, speed and ability are assumed to be constants. Violations of this assumption are generally absorbed in the residual of the model. As a result, within-subject departures from the between-subject speed and ability level remain undetected. These departures may be of interest to the researcher as they reflect differences in the response processes adopted on the items of a test. In this article, we propose a dynamic approach for responses and response times based on hidden Markov modeling to account for within-subject differences in responses and response times. A simulation study is conducted to demonstrate acceptable parameter recovery and acceptable performance of various fit indices in distinguishing between different models. In addition, both a confirmatory and an exploratory application are presented to demonstrate the practical value of the modeling approach.     

Stochastic Dynamic Models of Response Time and Accuracy: A Foundational Primer

A large class of statistical decision models for performance in simple information processing tasks can be described by linear, first-order, stochastic differential equations (SDEs), whose solutions are diffusion processes. In such models, the first passage time for the diffusion process through a response criterion determines the time at which an observer makes a decision about the identity of a stimulus. Because the assumptions of many cognitive models lead to SDEs that are time inhomogeneous, classical methods for solving such first passage time problems are usually inapplicable. In contrast, recent integral equation methods often yield solutions to both the one-sided and the two-sided first passage time problems, even in the presence of time inhomogeneity. These methods, which are of particular relevance to the cognitive modeler, are described in detail, together with illustrative applications. Copyright 2000 Academic Press.