The dynamics of multimodal integration: The averaging diffusion model

We combine extant theories of evidence accumulation and multi-modal integration to develop an integrated framework for modeling multimodal integration as a process that unfolds in real time. Many studies have formulated sensory processing as a dynamic process where noisy samples of evidence are accumulated until a decision is made. However, these studies are often limited to a single sensory modality. Studies of multimodal stimulus integration have focused on how best to combine different sources of information to elicit a judgment. These studies are often limited to a single time point, typically after the integration process has occurred. We address these limitations by combining the two approaches. Experimentally, we present data that allow us to study the time course of evidence accumulation within each of the visual and auditory domains as well as in a bimodal condition. Theoretically, we develop a new Averaging Diffusion Model in which the decision variable is the mean rather than the sum of evidence samples and use it as a base for comparing three alternative models of multimodal integration, allowing us to assess the optimality of this integration. The outcome reveals rich individual differences in multimodal integration: while some subjects’ data are consistent with adaptive optimal integration, reweighting sources of evidence as their relative reliability changes during evidence integration, others exhibit patterns inconsistent with optimality.     

The intrinsic constraint model for stereo-motion integration

How the visual system integrates the information provided by several depth cues is central for vision research. Here, we present a model for how the human visual system combines disparity and velocity information. The model provides a depth interpretation to a subspace defined by the covariation of the two signals. We show that human performance is consistent with the predictions of the model, and compare them with those of another theoretical approach, the modified weak-fusion model. We discuss the validity of each approach as a model for human perception of 3-D shape from multiple cues to depth.     

Language integration in bilingual sentence production

To what extent are processes used in sentence production integrated between the different languages of a bilingual and to what extent are they kept separate? We consider three models that differ in their assumptions about the degree of integration: De Bot's [De Bot, K. (1992). A bilingual production model: Levelt's Speaking model adapted. Applied Linguistics, 13, 1-24] bilingual blueprint of the speaker, Ullman's [Ullman, M. T. (2001). The neural basis of lexicon and grammar in first and second language: The declarative/procedural model. Bilingualism: Language and Cognition, 4, 105-122] declarative/procedural model of bilingualism, and Hartsuiker et al.'s [Hartsuiker, R. J., Pickering, M. J., & Veltkamp, E. (2004). Is syntax separate or shared between languages? Cross-linguistic syntactic priming in Spanish/English bilinguals. Psychological Science, 15, 409-414] integrated model. A review of the evidence from bilingual sentence production studies shows that Hartsuiker et al.'s predictions are supported, but argues against the other two models. We discuss some repercussions for bilingual language use.     

A diffusion model for discrimination of temporal numerosity

This paper presents a model for sequential discrimination between the temporal frequencies of two discrete events. The model results from adjoining a theory for continuous estimation of temporal numerosity to a mathematical representation of the response criterion. A simple set of assumptions is used to describe the time evolution of numerosity estimates by a stochastic diffusion process. The predictions of the model are extensively compared to a body of results presented in a previous report, with an emphasis on individual performances. It is shown that a number of response strategies, observed in actual experiments, can be predicted by generalizing the classical notion of a single-valued response threshold to a stochastic, time-varying process.     

A reinforcement learning diffusion decision model for value-based decisions

Psychonomic Bulletin & Review - Psychological models of value-based decision-making describe how subjective values are formed and mapped to single choices. Recently, additional efforts have...     

The integration problem for naive realism

This paper makes explicit the basic problem perfect hallucinations pose for perceptual naive realists, more fundamental than the well-trodden Screening-off Problem. The deeper problem offers the basis for an overarching classification of the available naive- realist-friendly approaches to perfect hallucinations. In the course of laying out the challenges to the different types of response, the paper makes a case for the superiority of a particular approach to perfect hallucinations, on which they would be understood as a special kind of perceptual anomaly—arising from a secondary mode of perceptual processing.     

Fast-dm: A free program for efficient diffusion model analysis

In the present article, a flexible and fast computer program, called fast-dm, for diffusion model data analysis is introduced. Fast-dm is free software that can be downloaded from the authors' websites. The program allows estimating all parameters of Ratcliff's (1978) diffusion model from the empirical response time distributions of any binary classification task. Fast-dm is easy to use: it reads input data from simple text files, while program settings are specified by commands in a control file. With fast-dm, complex models can be fitted, where some parameters may vary between experimental conditions, while other parameters are constrained to be equal across conditions. Detailed directions for use of fast-dm are presented, as well as results from three short simulation studies exemplifying the utility of fast-dm.     

Neural model for the Stevens’ power law

A neural model, compatible with known “all or none” neurological principles, duplicates or closely approximates Stevens’ power law with any exponent—greater or less than unity. The model has several electronic analogues, and one is discussed. The model predicts experimental neurological and psychological findings.     

A diffusion model of perceptual memory

A model is presented of the perceptual process through which an observer compares two consecutively observed stimuli. Emphasis is placed on the marmer in which a memory of the first stimulus is maintained until the comparison stimulus is observed. It is argued that the role of this perceptual memory process provides the primary distinction between detection and recognition tasks. Two experiments are reported: an experiment in which the observer is asked to judge the similarity in position of two points of light presented serially in a dark room; and an experiment in which the observer judges the similarity in loudness of two serially presented tones. The visual experiment is discussed in relation to the analysis of autokinesis and involuntary eye movements, while the auditory experiment is shown to have special relevance to the issue of time-order errors.     

A crossed random effects diffusion model for speeded semantic categorization decisions

Choice reaction times (RTs) are often used as a proxy measure of typicality in semantic categorization studies. However, other item properties have been linked to choice RTs as well. We apply a tailored process model of choice RT to a speeded semantic categorization task in order to deconfound different sources of variability in RT. Our model is based on a diffusion model of choice RT, extended to include crossed random effects (of items and participants). This model retains the interesting process interpretation of the diffusion model’s parameters, but it can be applied to choice RTs even in the case where there are few or no repeated measurements of each participant-item combination. Different aspects of the response process are then linked to different types of item properties. A typicality measure turns out to predict the rate of information uptake, while a lexicographic measure predicts the stimulus encoding time. Accessibility measures cannot reliably predict any component of the decision process.     

A fast numerical algorithm for the estimation of diffusion model parameters

In this paper, we describe a new algorithmic approach for parameter estimation in Ratcliff's [(1978). A theory of memory retrieval. Psychological Review, 85 (2), 59-108] diffusion model. This problem, especially if inter-trial variabilities of parameters are included in the model, is computationally very expensive; the parameter estimation procedure often takes a long time even with today's high-speed computers. The algorithm described here makes the calculation of the cumulative distribution functions for predicted process durations computationally much less expensive. This improvement is achieved by solving the Kolmogorov backward equation numerically instead of employing the previously used closed form solution. Additionally, the algorithm can determine the optimum fit for one of the model parameters (the starting point z) directly, thereby reducing the dimension of the parameter search space by one. The resulting method is shown to be notably faster than the standard (closed-form solution) method for parameter estimation.     

Fluency breaks in older speakers: Implications for a model of stuttering throughout the life cycle

In order to test a model describing the occurence of fluency breaks throughout the life cycle, spontaneous speech samples of 40 nonstutterers and 4 stutterers above the age of 50 were perceptually analyzed. Fluency breaks were classified into categories of formulative or motoric breaks. The results supported research, noting an increase in formulative fluency breaks (especially fillers and interjections) for older speakers. There was, however, a significant decrease in the occurence of formulative fluency breaks during the final years of late adulthood. No change was found in the occurence of motoric fluency breaks in the nonstuttering speakers. Stuttering speakers had significantly less formulative breaks and significantly more motoric breaks than nonstuttering speakers.     

A production system model for human problem solving

Summary A new production system model was developed for a class of transformation problems, based upon the results of an experiment which evaluated the psychological validity of a noticing order in problem solving. This model specifies how general problem solving heuristics interact with domain knowledge and how possible actions are assembled from externally presented information. The production system may conceptually be divided into three groups: move generation, move evaluation, and move execution productions. The simulation model was evaluated by a second experiment. With a common set of parameter values, good fits were obtained between the predicted and observed data. The significance of individual productions was assessed by running simulations with individual productions deleted. While previous simulations have described subjects' behavior on a single problem, the present model successfully predicted human problem solving in three structurally different problems. The proposed production system thus gives a detailed account of the procedural knowledge that subjects use in transformation problems.     

Decomposing task-switching costs with the diffusion model

In four experiments, task-switching processes were investigated with variants of the alternating runs paradigm and the explicit cueing paradigm. The classical diffusion model for binary decisions (Ratcliff, 1978) was used to dissociate different components of task-switching costs. Findings can be reconciled with the view that task-switching processes take place in successive phases as postulated by multiple-components models of task switching (e.g., Mayr & Kliegl, 2003; Ruthruff, Remington, & Johnston, 2001). At an earlier phase, task-set reconfiguration (Rogers & Monsell, 1995) or cue-encoding (Schneider & Logan, 2005) takes place, at a later phase, the response is selected in accord with constraints set in the first phase. Inertia effects (Allport, Styles, & Hsieh, 1994; Allport & Wylie, 2000) were shown to affect this later stage. Additionally, findings support the notion that response caution contributes to both global as well as to local switching costs when task switches are predictable.     

Hedonic integration: Test of a linear model

A simple adding model was tested for the integration of the hedonic component of taste stimuli. Subjects tasted mixtures of quinine sulfate and apple juice and rated the pleasantness of the composite taste, The rating data showed significant nonadditivity, which could reflect failure of the model itself or merely a nonlinear response output function. Since the rating data could be transformed to additivity, justification for such transformation was sought through a second task based on the logic of two-stage integration. In this task, subjects rated the difference between taste mixtures. The subtracting model underlying this judgment provided functional scale values for the taste mixtures on validated interval scales. These functional scale values were consistent with the predictions of the simple adding model, and thus justified the transformation of the single rating data to additivity. The application of functional measurement to obtain the psychophysical functions of the two taste components is also illustrated.