Drawing conclusions from choice response time models: A tutorial using the linear ballistic accumulator

Cognitive models of choice and response times can lead to deeper insights into the processes underlying decisions than standard analyses of accuracy and response time data. The application of these models, however, has historically been reserved for the authors of the models, and their associates. Recently, choice response time models have become more accessible through the release of user-friendly software for estimating their parameters. The aim of this tutorial is to provide guidance about the process of using these parameter estimates and associated model fits to make conclusions about experimental data. We use an application of one response time model, the linear ballistic accumulator, as an example to demonstrate the steps required to select an appropriate parametric characterization of a data set. We also discuss how to evaluate the quality of the agreement between model and data, including guidelines for presenting model predictions for group-level data.     

Bayes factors for the linear ballistic accumulator model of decision-making

Evidence accumulation models of decision-making have led to advances in several different areas of psychology. These models provide a way to integrate response time and accuracy data, and to describe performance in terms of latent cognitive processes. Testing important psychological hypotheses using cognitive models requires a method to make inferences about different versions of the models which assume different parameters to cause observed effects. The task of model-based inference using noisy data is difficult, and has proven especially problematic with current model selection methods based on parameter estimation. We provide a method for computing Bayes factors through Monte-Carlo integration for the linear ballistic accumulator (LBA; Brown and Heathcote, 2008), a widely used evidence accumulation model. Bayes factors are used frequently for inference with simpler statistical models, and they do not require parameter estimation. In order to overcome the computational burden of estimating Bayes factors via brute force integration, we exploit general purpose graphical processing units; we provide free code for this. This approach allows estimation of Bayes factors via Monte-Carlo integration within a practical time frame. We demonstrate the method using both simulated and real data. We investigate the stability of the Monte-Carlo approximation, and the LBA’s inferential properties, in simulation studies.     

The simplest complete model of choice response time: linear ballistic accumulation

We propose a linear ballistic accumulator (LBA) model of decision making and reaction time. The LBA is simpler than other models of choice response time, with independent accumulators that race towards a common response threshold. Activity in the accumulators increases in a linear and deterministic manner. The simplicity of the model allows complete analytic solutions for choices between any number of alternatives. These solutions (and freely-available computer code) make the model easy to apply to both binary and multiple choice situations. Using data from five previously published experiments, we demonstrate that the LBA model successfully accommodates empirical phenomena from binary and multiple choice tasks that have proven difficult for other theoretical accounts. Our results are encouraging in a field beset by the tradeoff between complexity and completeness.     

A diffusion model account of response time and accuracy in a brightness discrimination task: fitting real data and failing to fit fake but plausible data

A brightness discrimination experiment was performed to examine how subjects decide whether a patch of pixels is “bright” or “dark,” and stimulus duration, brightness, and speed versus accuracy instructions were manipulated. The diffusion model (Ratcliff, 1978) was fit to the data, and it accounted for all the dependent variables: mean correct and error response times, the shapes of response time distributions for correct and error responses, and accuracy values. Speed-accuracy manipulations affected only boundary separation (response criteria settings) in the model. Drift rate (the rate of accumulation of evidence) in the diffusion model, which represents stimulus quality, increased as a function of stimulus duration and stimulus brightness but asymptoted as stimulus duration increased from 100 to 150 msec. To address the argument that the diffusion model can fit any pattern of data, simulated patterns of plausible data are presented that the model cannot fit.     

An EZ-diffusion model for response time and accuracy

The EZ-diffusion model for two-choice response time tasks takes mean response time, the variance of response time, and response accuracy as inputs. The model transforms these data via three simple equations to produce unique values for the quality of information, response conservativeness, and nondecision time. This transformation of observed data in terms of unobserved variables addresses the speed—accuracy trade-off and allows an unambiguous quantification of performance differences in two-choice response time tasks. The EZ-diffusion model can be applied to data-sparse situations to facilitate individual subject analysis. We studied the performance of the EZ-diffusion model in terms of parameter recovery and robustness against misspecification by using Monte Carlo simulations. The EZ model was also applied to a real-world data set.     

A hierarchical approach for fitting curves to response time measurements

Understanding how response time (RT) changes with manipulations has been critical in distinguishing among theories in cognition. It is well known that aggregating data distorts functional relationships (e.g., Estes, 1956). Less well appreciated is a second pitfall: Minimizing squared errors (i.e., OLS regression) also distorts estimated functional forms with RT data. We discuss three properties of RT that should be modeled for accurate analysis and, on the basis of these three properties, provide a hierarchical Weibull regression model for regressing RT onto covariates. Hierarchical regression model analysis of lexical decision task data reveals that RT decreases as a power function of word frequency with the scale of RT decreasing 11% for every doubling of word frequency. A detailed discussion of the model and analysis techniques are presented as archived materials and may be downloaded from www.psychonomic.org/archive.     

Modeling response signal and response time data

The diffusion model (Ratcliff, 1978) and the leaky competing accumulator model (LCA, Usher & McClelland, 2001) were tested against two-choice data collected from the same subjects with the standard response time procedure and the response signal procedure. In the response signal procedure, a stimulus is presented and then, at one of a number of experimenter-determined times, a signal to respond is presented. The models were fit to the data from the two procedures simultaneously under the assumption that responses in the response signal procedure were based on a mixture of decision processes that had already terminated at response boundaries before the signal and decision processes that had not yet terminated. In the latter case, decisions were based on partial information in one variant of each model or on guessing in a second variant. Both variants of the diffusion model fit the data well and both fit better than either variant of the LCA model, although the differences in numerical goodness-of-fit measures were not large enough to allow decisive selection between the models.     

Getting more from success: standard raising as esteem maintenance

In 4 studies, the authors tested the hypothesis that success prompts standard raising and that doing so helps maintain performance-related esteem by augmenting perceptions of competence. Study 1 showed that students raised examination standards following success in meeting them. In Study 2, professors who were recently promoted to tenure raised what they thought were the publication standards at their university, compared with the standards of their nonpromoted counterparts. Supporting the authors' esteem-maintenance account, participants in Study 3 reported that they would feel better meeting a high standard by a small margin than meeting a low standard by a large margin, and competence attributions mediated this relationship. Participants in Study 4 raised standards when their esteem was threatened but not when it was affirmed, also supporting an esteem-maintenance account. Alternative interpretations of standard raising, including goals for future improvement and initial standard lowering, were not supported in these studies.     

An ARC-REM model for accuracy and response time in recognition and recall

This article presents a model for accuracy and response time (RT) in recognition and cued recall, fitted to free-response and signal-to-respond data from Experiment 1 of P. A. Nobel and R. M. Shiffrin (2001). The model posits that recognition operates through parallel activation in a single retrieval step and cued recall operates as a sequential search. Because the data for recognition showed that variations in list length and study time per list had a large effect on accuracy but a small or negligible effect on (a) free-response RT distributions and (b) retrieval dynamics in signal-to-respond, the timing of the recognition decision is based on an assessment of retrieval completion (ARC), rather than on a sufficiency of evidence in favor of 1 of the response options. By assuming within-trial forgetting, the model predicts both the dissociation of accuracy and RT and the finding that errors are slower than correct responses. For cued recall, this model was incorporated as the 1st step in a search consisting of cycles of sampling and recovery.     

Check Vocal: A program to facilitate checking the accuracy and response time of vocal responses from DMDX

CheckVocal is a Windows application that facilitates checking the accuracy and response time of recorded vocal responses in naming and other experimental tasks using the DMDX display and response collection software. CheckVocal handles all keeping-track and presents each recorded response audiovisually (as waveform, spectrogram, and sound played out) along with the correspondingprinted correct response andregistered responsetime. The user simply decides whether the response was correct, wrong, or missing, with a single mouse click advancing to the next response. Response ti me correction can be done manually or automatically (retriggering by apower threshold). Data safety and integrity is ensured by cross-checking and status saving, so t hat interruptedsessions can be resumed later. CheckVocal is freely available to the DMDX community via a dedicated Web page.     

The point of no return in choice reaction time: controlled and ballistic stages of response preparation

A countermanding procedure and race model are used to assess separately the effects of experimental factors before and after the "point of no return" in response preparation. The results reveal details about processes that so closely precede the initiation of movement that they cannot be inhibited. These processes appear to be affected by the repetition of stimulus-response pairs, but not by the physical or semantic properties of the stimuli. A model of response preparation is supported in which response inhibition depends upon the outcome of a race between independent excitatory and inhibitory processes, and reaction time is the sum of the durations of at least two stages, separated by the point of no return.     

Strength of stereotype and response time for linear displays and rotary controls.

101 Hong Kong Chinese subjects were tested on six configurations of linear displays and rotary controls. A significant linear relationship was found between the average response times and the strength of direction-of-turn stereotype as measured by the proportion of response made by the majority of subjects.     

Separating perception time from response time: the slope transition paradigm

This paper describes the slope transition paradigm (STP), a variant of rapid serial visual presentation (RSVP) that separates early (perceptual) processing time from total response time. The paradigm is based on a very simple idea: provide varying amounts of time for perceptual processing and find the moment when the subject begins to waste time waiting for more data. That moment is a measure of how much time was actually needed. The method was used in two experiments. Results are discussed in relation to set-size effects, perceptual capacity limits, attentional dwell times, and some related neurophysiological findings. The method appears to tap aspects of information processing that differ from those tapped in studies of the psychological refractory period, the attentional blink, and repetition blindness.     

The time course of perceptual choice: the leaky, competing accumulator model

The time course of perceptual choice is discussed in a model of gradual, leaky, stochastic, and competitive information accumulation in nonlinear decision units. Special cases of the model match a classical diffusion process, but leakage and competition work together to address several challenges to existing diffusion, random walk, and accumulator models. The model accounts for data from choice tasks using both time-controlled (e.g., response signal) and standard reaction time paradigms and its adequacy compares favorably with other approaches. A new paradigm that controls the time of arrival of information supporting different choice alternatives provides further support. The model captures choice behavior regardless of the number of alternatives, accounting for the log-linear relation between reaction time and number of alternatives (Hick's law) and explains a complex pattern of visual and contextual priming in visual word identification.     

Effects of alternating set for speed or accuracy on response time,accuracy and confidence in a unidimensional discrimination task

Three approaches to a theoretical analysis of confidence judgments are considered: one linking confidence to the number of sensory observations, one based on a distinction between ‘state’ and ‘process’ factors, and a ‘balance of evidence’ hypothesis developed from an accumulator model of discrimination. An experiment is described in which observers were asked to decide which of two parallel lines was the longer, and to rate their confidence in each judgment. Each observer's set for speed or accuracy was manipulated over successive blocks of trials, and effects on response time, accuracy, and confidence were examined. Under an accuracy set, observers produced a higher percentage of correct responses, had longer response times, and made more confident judgments than under a set for speed. Within both speed and accuracy blocks, however, confidence ratings were inversely related to response time. The data on response accuracy, time, and confidence indicate certain deficiencies in either of the first two approaches, but were well accounted for by the third.     

Knowing where is different from knowing what: Distinct response time profiles and accuracy effects for target location,orientation, and color probability

When a location is cued, targets appearing at that location are detected more quickly. When a target feature is cued, targets bearing that feature are detected more quickly. These attentional cueing effects are only superficially similar. More detailed analyses find distinct temporal and accuracy profiles for the two different types of cues. This pattern parallels work with probability manipulations, where both feature and spatial probability are known to affect detection accuracy and reaction times. However, little has been done by way of comparing these effects. Are probability manipulations on space and features distinct? In a series of five experiments, we systematically varied spatial probability and feature probability along two dimensions (orientation or color). In addition, we decomposed response times into initiation and movement components. Targets appearing at the probable location were reported more quickly and more accurately regardless of whether the report was based on orientation or color. On the other hand, when either color probability or orientation probability was manipulated, response time and accuracy improvements were specific for that probable feature dimension. Decomposition of the response time benefits demonstrated that spatial probability only affected initiation times, whereas manipulations of feature probability affected both initiation and movement times. As detection was made more difficult, the two effects further diverged, with spatial probability disproportionally affecting initiation times and feature probability disproportionately affecting accuracy. In conclusion, all manipulations of probability, whether spatial or featural, affect detection. However, only feature probability affects perceptual precision, and precision effects are specific to the probable attribute.     

Managing data for integrity: Policies and procedures for ensuring the accuracy and quality of the data in the laboratory

Management of the research data is an extremely important responsibility of the Principal Investigator (PI) and other members of the research team. Without accurate data, no worthwhile conclusions can be drawn from the research study. Integrity in data management is critical to the success of the research group and to public trust in the research outcomes. One of the primary responsibilities of the PI is to provide proper training to the junior members of the lab. This effort can be buttressed by institutional data policies that are implemented at the group level. Extensive and frequent guidance in good research practices by the PI and other senior research staff is critical to the proper training of new scientists.     

Fitting Wald and ex-Wald distributions to response time data: An example using functions for the S-PLUS package

Schwarz (2001, 2002) proposed the ex-Wald distribution, obtained from the convolution of Wald and exponential random variables, as a model of simple and go/no-go response time. This article provides functions for the S-PLUS package that produce maximum likelihood estimates of the parameters for the ex-Wald, as well as for the shifted Wald and ex-Gaussian, distributions. In a Monte Carlo study, the efficiency and bias of parameter estimates were examined. Results indicated that samples of at least 400 are necessary to obtain adequate estimates of the ex-Wald and that, for some parameter ranges, much larger samples may be required. For shifted Wald estimation, smaller samples of around 100 were adequate, at least when fits identified by the software as having ill-conditioned maximums were excluded. The use of all functions is illustrated using data from Schwarz (2001). The S-PLUS functions and Schwarz’s data may be downloaded from the Psychonomic Society’s Web archive, www. psychonomic.org/archive/.     

Recall accuracy and illusory memories: when more is less

In two experiments it was revealed that manipulations that increased recall of studied list items also increased false recall of theme-related, critical nonpresented words. In Experiment 1 subjects listened to a series of short word lists, each containing items associatively related to a theme, while engaging in either semantic or nonsemantic processing. On an immediate free recall test semantic processors demonstrated greater correct recall as well as more illusory memories of critical nonpresented items than nonsemantic processors. In Experiment 2, the short study lists were combined to form longer lists that were presented either blocked by theme or in a random presentation order. Retention interval was also varied as participants were tested either immediately, one week after, or three weeks after the study phase. Presenting the target items in a blocked, as opposed to random, format increased recall accuracy, but this was at the expense of a higher intrusion rate for theme-consistent items. Interestingly, the level of false memories was not affected by retention interval even though typical decrements in the recall of study items were observed over time. The results of these experiments highlight the persistence of the false memory effect, as well as pointing to several factors, primarily semantic processing, that may lead to the creation of false memories. Interpretations are offered within the theoretical frameworks of source monitoring and fuzzy trace theory.