How to use the diffusion model: Parameter recovery of three methods: EZ, fast-dm, and DMAT

Parameter recovery of three different implementations of the Ratcliff diffusion model was investigated: the EZ model (Wagenmakers, van der Maas, & Grasman, 2007), fast-dm (Voss & Voss, 2007), and DMAT (Vandekerckhove & Tuerlinckx, 2007). Their capacity to recover both the mean structure and individual differences in parameter values was explored. The three methods were applied to simulated data generated by the diffusion model, by the leaky, competing accumulator (LCA) model (Usher & McClelland, 2001) and by the linear ballistic accumulator (LBA) model (Brown & Heathcote, 2008). Results show that EZ and DMAT are better capable than fast-dm in recovering experimental effects on parameters. EZ was best in recovering individual differences in parameter values. When data were generated by the LCA model, the diffusion model estimates obtained with all three methods correlated well with corresponding LCA model parameters. No such one-on-one correspondence could be established between parameters of the LBA model and the diffusion model.     

Timing and reaction time.

Because reaction time (RT) tasks are generally repetitive and temporally regular, participants may use timing strategies that affect response speed and accuracy. This hypothesis was tested in 3 serial choice RT experiments in which participants were presented with stimuli that sometimes arrived earlier or later than normal. RTs increased and errors decreased when stimuli came earlier than normal, and RTs decreased and errors increased when stimuli came later than normal. The results were consistent with an elaboration of R. Ratcliff's diffusion model (R. Ratcliff, 1978; R. Ratcliff & J. N. Rouder, 1998; R. Ratcliff, T. Van Zandt, & G. McKoon, 1999), supplemented by a hypothesis developed by D. Laming (1979a, 1979b), according to which participants initiate stimulus sampling before the onset of the stimulus at a time governed by an internal timekeeper. The success of this model suggests that timing is used in the service of decision making.     

Temporal expectation and information processing: a model-based analysis

People are able to use temporal cues to anticipate the timing of an event, enabling them to process that event more efficiently. We conducted two experiments, using the fixed-foreperiod paradigm (Experiment 1) and the temporal-cueing paradigm (Experiment 2), to assess which components of information processing are speeded when subjects use such temporal cues to predict the onset of a target stimulus. We analyzed the observed temporal expectation effects on task performance using sequential-sampling models of decision making: the Ratcliff diffusion model and the shifted-Wald model. The results from the two experiments were consistent: temporal expectation affected the duration of nondecision processes (target encoding and/or response preparation) but had little effect on the two main components of the decision process: response-threshold setting and the rate of evidence accumulation. Our findings provide novel evidence about the psychological processes underlying temporal-expectation effects on reaction time.     

A random-walk model of accuracy and reaction time applied to three experiments on pigeon visual discrimination

A random-walk model of visual discrimination is described and applied to reaction time (RT) distributions from three discrete-trial experiments with pigeons. Experiment 1 was a two-choice hue discrimination task with multiple hues. Choice percentages changed with hue discriminability; RTs were shortest for the least and most discriminable stimuli. Experiments 2 and 3 used go/no-go hue discriminations. Blocks of sessions differed in reward probability associated with a variable red stimulus in Experiment 2 and with a constant green stimulus in Experiment 3. Changes in hue had a large effect on response percentage and a small effect on RT; changes in reward shifted RT distributions on the time axis. The "random-walk, pigeon" model applied to these data is closely related to Ratcliff's diffusion model (Ratcliff, 1978; Ratcliff & Rouder, 1998). Simulations showed that stimulus discriminability affected the speed with which evidence accumulated toward a response threshold, in line with comparable effects in human subjects. Reward probability affected bias, modeled as the amount of evidence needed to reach one threshold rather than the other. The effects of reward probability are novel, and their isolation from stimulus effects within the decision process can guide development of a broader model of discrimination.     

Estimating parameters of the diffusion model: approaches to dealing with contaminant reaction times and parameter variability

Three methods for fitting the diffusion model (Ratcliff, 1978) to experimental data are examined. Sets of simulated data were generated with known parameter values, and from fits of the model, we found that the maximum likelihood method was better than the chi-square and weighted least squares methods by criteria of bias in the parameters relative to the parameter values used to generate the data and standard deviations in the parameter estimates. The standard deviations in the parameter values can be used as measures of the variability in parameter estimates from fits to experimental data. We introduced contaminant reaction times and variability into the other components of processing besides the decision process and found that the maximum likelihood and chi-square methods failed, sometimes dramatically. But the weighted least squares method was robust to these two factors. We then present results from modifications of the maximum likelihood and chi-square methods, in which these factors are explicitly modeled, and show that the parameter values of the diffusion model are recovered well. We argue that explicit modeling is an important method for addressing contaminants and variability in nondecision processes and that it can be applied in any theoretical approach to modeling reaction time.     

A diffusion model explanation of the worst performance rule for reaction time and IQ

The worst performance rule for cognitive tasks [Coyle, T.R. (2003). IQ, the worst performance rule, and Spearman's law: A reanalysis and extension. Intelligence, 31, 567-587] in which reaction time is measured is the result that IQ scores correlate better with longer (i.e., 0.7 and 0.9 quantile) reaction times than shorter (i.e., 0.1 and 0.3 quantile) reaction times. We show that this pattern of correlations can be predicted by the diffusion model [Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59-108], in two ways: either assuming that the rate of accumulation of information toward a decision is higher for higher IQ subjects or assuming that the criterial amounts of information they require before a decision are lower. Importantly, the model explains both reaction times and accuracy, so the two possibilities can be distinguished.     

Making strides in modeling individual differences: reply to Leite, Ratcliff, and White (2007)

Leite, Ratcliff, and White (2007) claimed that the diffusion model (Ratcliff, 1978) could simulate the molar patterns in response times (RTs) from the multiple tasks observed by Chen, Hale, and Myerson (2007). We present our own simulations to clarify the underlying mechanisms and show that, as is predicted by the difference engine model (Myerson, Hale, Zheng, Jenkins, & Widaman, 2003), correlations across tasks are the key to the molar patterns in individual RTs. Although the diffusion model and other sequential-sampling models may be able to accommodate patterns of RTs across tasks like those studied by Chen et al., the difference engine is the only current model that actually predicts them.     

On the relation between the mean and the variance of a diffusion model response time distribution

Almost every empirical psychological study finds that the variance of a response time (RT) distribution increases with the mean. Here we present a theoretical analysis of the nature of the relationship between RT mean and RT variance, based on the assumption that a diffusion model (e.g., Ratcliff (1978) Psychological Review, 85, 59-108; Ratcliff (2002). Psychonomic Bulletin & Review, 9, 278-291), adequately captures the shape of empirical RT distributions. We first derive closed-form analytic solutions for the mean and variance of a diffusion model RT distribution. Next, we study how systematic differences in two important diffusion model parameters simultaneously affect the mean and the variance of the diffusion model RT distribution. Within the range of plausible values for the drift rate parameter, the relation between RT mean and RT standard deviation is approximately linear. Manipulation of the boundary separation parameter also leads to an approximately linear relation between RT mean and RT standard deviation, but only for low values of the drift rate parameter.     

Interpreting the parameters of the diffusion model: an empirical validation

The diffusion model (Ratcliff, 1978) allows for the statistical separation of different components of a speeded binary decision process (decision threshold, bias, information uptake, and motor response). These components are represented by different parameters of the model. Two experiments were conducted to test the interpretational validity of the parameters. Using a color discrimination task, we investigated whether experimental manipulations of specific aspects of the decision process had specific effects on the corresponding parameters in a diffusion model data analysis (see Ratcliff, 2002; Ratcliff & Rouder, 1998; Ratcliff, Thapar, & McKoon, 2001, 2003). In support of the model, we found that (1) decision thresholds were higher when we induced accuracy motivation, (2) drift rates (i.e., information uptake) were lower when stimuli were harder to discriminate, (3) the motor components were increased when a more difficult form of response was required, and (4) the process was biased toward rewarded responses.     

A diffusion model account of masking in two-choice letter identification

The diffusion model developed by R. Ratcliff (1978, 1981, 1985, 1988) for 2-choice decisions was applied to data from 2 letter identification experiments. In the experiments, stimulus letters were displayed and then masked, and the stimulus onset asynchrony between letter and mask was manipulated to vary accuracy from near chance to near ceiling. A standard reaction time procedure was used in one experiment and a deadline procedure in the other. Two hypotheses about the effect of masking on the information provided to the decision process were contrasted: (a) The output of perception to the decision process varies with time, so that the information used by the decision process rises and falls, reflecting the stimulus onset and mask onset. (b) The output of perception to the decision is constant over time, reflecting information integrated over the time between the stimulus and mask onsets. The data were well fit by the diffusion model only with the assumption of constant information over time.     

An integrated perspective on the relation between response speed and intelligence

Research in the field of mental chronometry and individual differences has revealed several robust regularities (Jensen, 2006). These include right-skewed response time (RT) distributions, the worst performance rule, correlations with general intelligence (g) that are more pronounced for RT standard deviations (RTSD) than they are for RT means (RTm), an almost perfect linear relation between individual differences in RTSD and RTm, linear Brinley plots, and stronger correlations between g and inspection time (IT) than between g and RTm. Here we show how all these regularities are manifestations of a single underlying relationship, when viewed through the lens of Ratcliff’s diffusion model ( [Ratcliff, 1978] and [Ratcliff et al., 2008] ). The single underlying relationship is between individual differences in general intelligence and individual differences in “drift rate”, which is just the speed of information processing in Ratcliff’s model. We also test and confirm a strong prediction of the diffusion model, namely that the worst performance rule generalizes to phenomena outside of the field of intelligence. Our approach provides an integrative perspective on intelligence findings.     

Individual differences on speeded cognitive tasks: Comment on Chen,Hale, and Myerson (2007)

Chen, Hale, and Myerson (2007) recently reported a test of the difference engine model (Myerson, Hale, Zheng, Jenkins, & Widaman, 2003). This test evaluated whether the standard deviation (SD) is proportional to the amount of processing—that is, mean reaction time (RT)—in a speeded cognitive task. We show that this evaluation is not a test of the model because its finding is a consequence of relationships in the data. We argue any model structure that produces increasing values of RT as a function of difficulty, with different slopes for different individuals, necessarily produces a correlation between SD and mean RT. We illustrate this with a different model structure—that is, the diffusion model proposed by Ratcliff (1978)—showing that it produces a fan out between fast- and slow-group means and produces the correlation between SD and mean RT that matches the empirical result.     

A diffusion model decomposition of the practice effect

When people repeatedly perform the same cognitive task, their mean response times (RTs) invariably decrease. The mathematical function that best describes this decrease has been the subject of intense debate. Here, we seek a deeper understanding of the practice effect by simultaneously taking into account the changes in accuracy and in RT distributions with practice, both for correct and error responses. To this end, we used the Ratcliff diffusion model, a successful model of two-choice RTs that decomposes the effect of practice into its constituent psychological processes. Analyses of data from a 10,000-trial lexical decision task demonstrate that practice not only affects the speed of information processing, but also response caution, response bias, and peripheral processing time. We conclude that the practice effect consists of multiple subcomponents, and that it may be hazardous to abstract the interactive combination of these subcomponents in terms of a single output measure such as mean RT for correct responses. Supplemental materials may be downloaded from http://pbr .psychonomic-journals.org/content/supplemental.     

Using diffusion models to understand clinical disorders

Sequential sampling models provide an alternative to traditional analyses of reaction times and accuracy in two-choice tasks. These models are reviewed with particular focus on the diffusion model (Ratcliff, 1978) and how its application can aid research on clinical disorders. The advantages of a diffusion model analysis over traditional comparisons are shown through simulations and a simple lexical decision experiment. Application of the diffusion model to a clinically relevant topic is demonstrated through an analysis of data from nonclinical participants with high- and low-trait anxiety in a recognition memory task. The model showed that after committing an error, participants with high-trait anxiety responded more cautiously by increasing their boundary separation, whereas participants with low-trait anxiety did not. The article concludes with suggestions for ways to improve and broaden the application of these models to studies of clinical disorders.     

The EZ diffusion method: Too EZ?

The diffusion model (Ratcliff, 1978) for fast two-choice decisions has been successful in a number of domains. Wagenmakers, van der Maas, and Grasman (2007) proposed a new method for fitting the model to data (“EZ”) that is simpler than the standard chisquare method (Ratcliff & Tuerlinckx, 2002). For an experimental condition, EZ can estimate parameter values for the main components of processing using only correct response times (RTs), their variance, and accuracy, not error RTs or the shapes of RT distributions. Wagenmakers et al. suggested that EZ produces accurate parameter estimates in cases in which the chi-square method would fail-specifically, experimental conditions with small numbers of observations or with accuracy near ceiling. In this article, I counter these claims and discuss EZ’s limitations. Unlike the chi-square method, EZ is extremely sensitive to outlier RTs and is usually less efficient in recovering parameter values, and it can lead to errors in interpretation when the data do not meet its assumptions, when the number of observations in an experimental condition is small, or when accuracy in an experimental condition is high. The conclusion is that EZ can be useful in the exploration of parameter spaces, but it should not be used for meaningful estimates of parameter values or for assessing whether or not a model fits data.     

Fitting the ratcliff diffusion model to experimental data

Many experiments in psychology yield both reaction time and accuracy data. However, no off-the-shelf methods yet exist for the statistical analysis of such data. One particularly successful model has been the diffusion process, but using it is difficult in practice because of numerical, statistical, and software problems. We present a general method for performing diffusion model analyses on experimental data. By implementing design matrices, a wide range of across-condition restrictions can be imposed on model parameters, in a flexible way. It becomes possible to fit models with parameters regressed onto predictors. Moreover, data analytical tools are discussed that can be used to handle various types of outliers and contaminants. We briefly present an easy-touse software tool that helps perform diffusion model analyses.     

Diffusion versus linear ballistic accumulation: different models but the same conclusions about psychological processes?

Quantitative models for response time and accuracy are increasingly used as tools to draw conclusions about psychological processes. Here we investigate the extent to which these substantive conclusions depend on whether researchers use the Ratcliff diffusion model or the Linear Ballistic Accumulator model. Simulations show that the models agree on the effects of changes in the rate of information accumulation and changes in non-decision time, but that they disagree on the effects of changes in response caution. In fits to empirical data, however, the models tend to agree closely on the effects of an experimental manipulation of response caution. We discuss the implications of these conflicting results, concluding that real manipulations of caution map closely, but not perfectly to response caution in either model. Importantly, we conclude that inferences about psychological processes made from real data are unlikely to depend on the model that is used.     

A ballistic model of choice response time

Almost all models of response time (RT) use a stochastic accumulation process. To account for the benchmark RT phenomena, researchers have found it necessary to include between-trial variability in the starting point and/or the rate of accumulation, both in linear (R. Ratcliff & J. N. Rouder, 1998) and nonlinear (M. Usher & J. L. McClelland, 2001) models. The authors show that a ballistic (deterministic within-trial) model using a simplified version of M. Usher and J. L. McClelland's (2001) nonlinear accumulation process with between-trial variability in accumulation rate and starting point is capable of accounting for the benchmark behavioral phenomena. The authors successfully fit their model to R. Ratcliff and J. N. Rouder's (1998) data, which exhibit many of the benchmark phenomena.     

Instructional and probability manipulations of bias in multiletter matching

Ratcliff (1985) performed fits of his diffusion model to the results of multiletter-matching experiments conducted by Ratcliff and Hacker (1981) and Proctor, Rao, and Hurst (1984), in which bias to respond "same" or "different" was manipulated by instructions and probabilities, respectively. The fits showed that both bias manipulations affected settings of a goodness-of-match criterion, whereas instructions also affected sensitivity. Evaluations of the experimental procedures and of Ratcliff's model-fitting procedures were performed in the present study. Three experiments showed that instructions and probabilities had similar effects, regardless of whether the different pairs were blocked or randomized according to the number of mismatching positions. The most salient feature of the results--that "same" reaction times were traded off more than were "different" reaction times, with no corresponding asymmetry in the error rates--was evident in all situations. The evaluation of Ratcliff's model-fitting procedures indicated that the apparent influence of instructions on sensitivity likely is an artifact of unequal variance for the sets of same and different pairs. Moreover, the effects of bias can be explained in terms of settings of response criteria, rather than of the goodness-of-match criterion, as in Ratcliff's fits.