Drawing conclusions from choice response time models: A tutorial using the linear ballistic accumulator

Cognitive models of choice and response times can lead to deeper insights into the processes underlying decisions than standard analyses of accuracy and response time data. The application of these models, however, has historically been reserved for the authors of the models, and their associates. Recently, choice response time models have become more accessible through the release of user-friendly software for estimating their parameters. The aim of this tutorial is to provide guidance about the process of using these parameter estimates and associated model fits to make conclusions about experimental data. We use an application of one response time model, the linear ballistic accumulator, as an example to demonstrate the steps required to select an appropriate parametric characterization of a data set. We also discuss how to evaluate the quality of the agreement between model and data, including guidelines for presenting model predictions for group-level data.     

A ballistic model of choice response time

Almost all models of response time (RT) use a stochastic accumulation process. To account for the benchmark RT phenomena, researchers have found it necessary to include between-trial variability in the starting point and/or the rate of accumulation, both in linear (R. Ratcliff & J. N. Rouder, 1998) and nonlinear (M. Usher & J. L. McClelland, 2001) models. The authors show that a ballistic (deterministic within-trial) model using a simplified version of M. Usher and J. L. McClelland's (2001) nonlinear accumulation process with between-trial variability in accumulation rate and starting point is capable of accounting for the benchmark behavioral phenomena. The authors successfully fit their model to R. Ratcliff and J. N. Rouder's (1998) data, which exhibit many of the benchmark phenomena.     

Modeling response signal and response time data

The diffusion model (Ratcliff, 1978) and the leaky competing accumulator model (LCA, Usher & McClelland, 2001) were tested against two-choice data collected from the same subjects with the standard response time procedure and the response signal procedure. In the response signal procedure, a stimulus is presented and then, at one of a number of experimenter-determined times, a signal to respond is presented. The models were fit to the data from the two procedures simultaneously under the assumption that responses in the response signal procedure were based on a mixture of decision processes that had already terminated at response boundaries before the signal and decision processes that had not yet terminated. In the latter case, decisions were based on partial information in one variant of each model or on guessing in a second variant. Both variants of the diffusion model fit the data well and both fit better than either variant of the LCA model, although the differences in numerical goodness-of-fit measures were not large enough to allow decisive selection between the models.     

Vocational choice: A decision making perspective

We propose a model of vocational choice that can be used for analyzing and guiding the decision processes underlying career and job choices. Our model is based on research in behavioral decision making (BDM), in particular the choice goals framework developed by Bettman, Luce, and Payne (1998). The basic model involves two major processes. First, the selection of a decision strategy according to four choice goals: maximizing decision accuracy, minimizing cognitive effort, minimizing negative emotion, and maximizing justifiability of the decision. Second, the construction of situation-specific preferences, which can reflect irrelevant task and context factors such as the evaluation mode. This basic model is extended to account for social influences and the long decision time typical of most career and job decisions. We review research on vocational choice in light of this model, discuss normative implications for counseling, and outline a research agenda for studying vocational choice from a behavioral decision making perspective.     

The response dynamics of preferential choice

The ubiquity of psychological process models requires an increased degree of sophistication in the methods and metrics that we use to evaluate them. We contribute to this venture by capitalizing on recent work in cognitive science analyzing response dynamics, which shows that the bearing information processing dynamics have on intended action is also revealed in the motor system. This decidedly “embodied” view suggests that researchers are missing out on potential dependent variables with which to evaluate their models—those associated with the motor response that produces a choice. The current work develops a method for collecting and analyzing such data in the domain of decision making. We first validate this method using widely normed stimuli from the International Affective Picture System (Experiment 1), and demonstrate that curvature in response trajectories provides a metric of the competition between choice options. We next extend the method to risky decision making (Experiment 2) and develop predictions for three popular classes of process model. The data provided by response dynamics demonstrate that choices contrary to the maxim of risk seeking in losses and risk aversion in gains may be the product of at least one “online” preference reversal, and can thus begin to discriminate amongst the candidate models. Finally, we incorporate attentional data collected via eye-tracking (Experiment 3) to develop a formal computational model of joint information sampling and preference accumulation. In sum, we validate response dynamics for use in preferential choice tasks and demonstrate the unique conclusions afforded by response dynamics over and above traditional methods.     

Multi-attribute,multi-alternative models of choice: Choice,reaction time,and process tracing

The first aim of this research is to compare computational models of multi-alternative, multi-attribute choice when attribute values are explicit. The choice predictions of utility (standard random utility & weighted valuation), heuristic (elimination-by-aspects, lexicographic, & maximum attribute value), and dynamic (multi-alternative decision field theory, MDFT, & a version of the multi-attribute linear ballistic accumulator, MLBA) models are contrasted on both preferential and risky choice data. Using both maximum likelihood and cross-validation fit measures on choice data, the utility and dynamic models are preferred over the heuristic models for risky choice, with a slight overall advantage for the MLBA for preferential choice. The response time predictions of these models (except the MDFT) are then tested. Although the MLBA accurately predicts response time distributions, it only weakly accounts for stimulus-level differences. The other models completely fail to account for stimulus-level differences. Process tracing measures, i.e., eye and mouse tracking, were also collected. None of the qualitative predictions of the models are completely supported by that data. These results suggest that the models may not appropriately represent the interaction of attention and preference formation. To overcome this potential shortcoming, the second aim of this research is to test preference-formation assumptions, independently of attention, by developing the models of attentional sampling (MAS) model family which incorporates the empirical gaze patterns into a sequential sampling framework. An MAS variant that includes attribute values, but only updates the currently viewed alternative and does not contrast values across alternatives, performs well in both experiments. Overall, the results support the dynamic models, but point to the need to incorporate a framework that more accurately reflects the relationship between attention and the preference-formation process.     

Response-level probability effects on reaction time: Now you see them,now you don't

Many reaction time (RT) experiments have tested for response-level probability effects. Their results have been mixed, which is surprising because psychophysiological studies provide clear evidence of motor-level changes associated with an anticipated response. A survey of the designs used in the RT studies reveals many potential problems that could conceal the effects of response probability. We report five new RT experiments testing for response-level probability effects with the most promising of the previous designs—that of Blackman (1972 Blackman, A. R. 1972. Influence of stimulus and response probability on decision and movement latency in a discrete choice reaction task. Journal of Experimental Psychology, 92: 128–133. doi:10.1037/h0032169[Crossref], [PubMed] , [Google Scholar])—and with new designs. Some of these experiments yield evidence of response-level probability effects, but others do not. It appears that response-level probability effects are present primarily in simple tasks with a strong emphasis on response preparation, possibly because participants only expend effort on response preparation in these tasks.     

Strategic flexibility in response preparation: Effects of cue validity on reaction time and pupil dilation

This study examined the ability of participants to strategically adapt their level of response preparation to the predictive value of preparatory cues. Participants performed the finger-precuing task under three levels of cue validity: 100, 75 and 50% valid. Response preparation was indexed by means of reaction time (RT) and pupil dilation, the latter providing a psychophysiological index of invested effort. Results showed a systematic increase in RT benefits (generated by valid cues) and RT costs (generated by invalid cues) with increments in the predictive value of cues. Converging with these behavioural effects, pupil dilation also increased systematically with greater cue validity during the cue-stimulus interval, suggesting more effortful response preparation with increases in cue validity. Together, these findings confirm the hypothesis that response preparation is flexible and that it can be strategically allocated in proportion to the relative frequency of valid/invalid preparatory cues.     

A comparison of bounded diffusion models for choice in time controlled tasks

The Wiener diffusion model (WDM) for 2-alternative tasks assumes that sensory information is integrated over time. Recent neurophysiological studies have found neural correlates of this integration process in certain neuronal populations. This paper analyses the properties of the WDM with two different boundary conditions in decision making tasks in which the time of response is indicated by a cue. A dual reflecting boundary mechanism is proposed and its performance is compared with a well-established absorbing boundary in the cases of the WDM, the WDM with extensions, and the WDM with prior probability. The two types of boundary influence the dynamics of the model and introduce differential weighting of evidence. Comparisons with Ornstein-Uhlenbeck models are also done, and it is shown that the WDM with both types of boundary achieves similar performance and produces similar fits to existing behavioural data. Further studies are proposed to distinguish which boundary mechanism is more consistent with experimental data.     

On the mean and variance of response times under the diffusion model with an application to parameter estimation

We give closed form expressions for the mean and variance of RTs for Ratcliff’s diffusion model [Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59-108] under the simplifying assumption that there is no variability across trials in the parameters. These expressions are more general than those currently available. As an application, we demonstrate their use in a method-of-moments estimation procedure that addresses some of the weaknesses of the EZ method [Wagenmakers, E.-J., van der Maas, H. L. J., & Grasman, R. P. P. P. (2007). An EZ-diffusion model for response time and accuracy. Psychonomic Bulletin & Review, 14, 3-22], and illustrate this with lexical decision data. We discuss further possible applications.     

The structure of choice

An unsolved fundamental problem in decision science concerns the extent to which the nature of the perceived relationships among items in a set of alternatives influences how they are chosen. More specifically, given a choice set with n items, how does human choice behavior differ as a function of the perceived relationships between the items of the set? In what follows, we study this problem empirically and theoretically from the standpoint of the dimensional structure of the choice set. In particular, we use generalized invariance structure theory (GIST; Vigo, 2013, 2014) to propose an inverse relationship between the degree of concept learning difficulty of a choice set (as determined by its degree of invariance or internal coherence) and choice response times on its members. To our knowledge, this is the first model that precisely unifies these two fundamental constructs. On average, the model, without free parameters, accounts for nearly 90% of the variance in the data from our two response-time experiments.     

The dynamics of deferred decision

Decision makers are often unable to choose between the options that they are offered. In these settings they typically defer their decision, that is, delay the decision to a later point in time or avoid the decision altogether. In this paper, we outline eight behavioral findings regarding the causes and consequences of choice deferral that cognitive theories of decision making should be able to capture. We show that these findings can be accounted for by a deferral-based time limit applied to existing sequential sampling models of preferential choice. Our approach to modeling deferral as a time limit in a sequential sampling model also makes a number of novel predictions regarding the interactions between choice probabilities, deferral probabilities, and decision times, and we confirm these predictions in an experiment. Choice deferral is a key feature of everyday decision making, and our paper illustrates how established theoretical approaches can be used to understand the cognitive underpinnings of this important behavioral phenomenon.     

Reaction time and intelligence: Comparing associations based on two response modes

People who score highly on intelligence tests also tend to have faster and less variable reaction times. Effect size estimates for the reaction time–intelligence association are larger in samples that are more representative of the population. However, such samples have often been tested on a reaction time device that requires reading a number and processing its association with a specific response location (Cox, Huppert, & Whichelow, 1993). Here, we use this device and another reaction time device (Dykiert et al., 2010) that is similar, except that the responses require less processing; subjects simply press a button that is adjacent to the stimulus light. We focus on the possibility that lights as stimuli require less higher-order cognitive engagement than numbers, and then test whether parameters from these two tasks are highly correlated and similarly associated with age and higher cognitive abilities. Both tasks measured simple and choice reaction times and their intra-individual variation across trials. The parameters of the two tasks were very highly correlated and parameters from both tasks were similarly associated with age, social factors, and differences in higher cognitive abilities. The respective choice reaction time parameters from either task accounted for much of the age- and higher cognitive ability-associations of the other task's parameters. These findings are important in establishing that the effect sizes of higher cognitive ability associations with processing speed measures may be found when the processing demands are minimal.     

The point of no return in choice reaction time: controlled and ballistic stages of response preparation

A countermanding procedure and race model are used to assess separately the effects of experimental factors before and after the "point of no return" in response preparation. The results reveal details about processes that so closely precede the initiation of movement that they cannot be inhibited. These processes appear to be affected by the repetition of stimulus-response pairs, but not by the physical or semantic properties of the stimuli. A model of response preparation is supported in which response inhibition depends upon the outcome of a race between independent excitatory and inhibitory processes, and reaction time is the sum of the durations of at least two stages, separated by the point of no return.     

Response-repetition effects depend on motor set: Evidence for anatomical coding in response selection

Successful motor performance requires a process of response selection that chooses the correct response out of a set of possible ones. Most theories of response selection assume that this selection process operates on spatial codes, which define the location of stimuli and responses in environmental coordinates, with little or no role for the anatomical codes of the effectors involved. In this study, we tested this assumption by investigating response-repetition effects in a response-cuing paradigm using two motor sets (fingers on one hand vs. fingers on two hands). Reaction time results demonstrated a robust response-repetition benefit that was greater for the one-hand set than for the two-hands set. Furthermore, with the one-hand set the repetition benefit was independent of cue type and cue-stimulus interval on the previous trial, whereas with the two-hands set it was strongly modulated by these two factors. These differential response-repetition effects for one- and two-hands motor sets demonstrate the important role of the neuro-anatomical hand distinction in response selection, thereby supporting multiple coding notions.     

Suppressing the truth as a mechanism of deception: Delta plots reveal the role of response inhibition in lying

Lying takes more time than telling the truth. Because lying involves withholding the truth, this “lie effect” has been related to response inhibition. We investigated the response inhibition hypothesis of lying using the delta-plot method: A leveling-off of the standard increase of the lie effect with slower reaction times would be indicative of successful response inhibition. Participants performed a reaction-time task that required them to alternate between lying and truth telling in response to autobiographical questions. In two experiments, we found that the delta plot of the lie effect leveled off with longer response latencies, but only in a group of participants who had better inhibitory skills as indexed by relatively small lie effects. This finding supports the role of response inhibition in lying. We elaborate on repercussions for cognitive models of deception and the data analysis of reaction-time based lie tests.     

Simple matrix methods for analyzing diffusion models of choice probability, choice response time, and simple response time

Diffusion processes (e.g., Wiener process, Ornstein-Uhlenbeck process) are powerful approaches to model human information processes in a variety of psychological tasks. Lack of mathematical tractability, however, has prevented broad applications of these models to empirical data. This tutorial explains step by step, using a matrix approach, how to construct these models, how to implement them on a computer, and how to calculate the predictions made by these models. In particular, we present models for binaries choices for unidimensional and multiattribute choice alternatives; for simple reaction time tasks; and for three alternatives choice problems.     

From Poisson shot noise to the integrated Ornstein-Uhlenbeck process: Neurally principled models of information accumulation in decision-making and response time

In the diffusion model of decision-making, evidence is accumulated by a Wiener diffusion process. A neurally motivated account of diffusive evidence accumulation is given, in which diffusive accumulation arises from an interaction between neural integration processes operating on short and long time scales. The short time scale process is modeled as a Poisson shot noise process with exponential decay. Stimulus information is coded by excitatory-inhibitory shot noise pairs. The long time scale process is modeled as algebraic integration, possibly implemented as a first-order autoregressive process realized by recurrent connections within a population of neurons. At high intensities, an excitatory-inhibitory shot noise pair converges weakly to an Ornstein-Uhlenbeck (OU) velocity process. The integrated OU process, or OU displacement process, obtained by integrating the velocity process over time, is indistinguishable at long times from the Wiener process. Diffusive information accumulation may therefore be characterized as an integrated OU process whose properties mimic those of the Wiener process.     

The precise time course of lexical activation: MEG measurements of the effects of frequency, probability, and density in lexical decision

Visually presented letter strings consistently yield three MEG response components: the M170, associated with letter-string processing (Tarkiainen, Helenius, Hansen, Cornelissen, & Salmelin, 1999); the M250, affected by phonotactic probability, (Pylkk?nen, Stringfellow, & Marantz, 2002); and the M350, responsive to lexical frequency (Embick, Hackl, Schaeffer, Kelepir, & Marantz, 2001). Pylkk?nen et al. found evidence that the M350 reflects lexical activation prior to competition among phonologically similar words. We investigate the effects of lexical and sublexical frequency and neighborhood density on the M250 and M350 through orthogonal manipulation of phonotactic probability, density, and frequency. The results confirm that probability but not density affects the latency of the M250 and M350; however, an interaction between probability and density on M350 latencies suggests an earlier influence of neighborhoods than previously reported.     

Decision by sampling: The role of the decision environment in risky choice

Decision by sampling (DbS) is a theory about how our environment shapes the decisions that we make. Here, I review the application of DbS to risky decision making. According to classical theories of risky decision making, people make stable transformations between outcomes and probabilities and their subjective counterparts using fixed psychoeconomic functions. DbS offers a quite different account. In DbS, the subjective value of an outcome or probability is derived from a series of binary, ordinal comparisons with a sample of other outcomes or probabilities from the decision environment. In this way, the distribution of attribute values in the environment determines the subjective valuations of outcomes and probabilities. I show how DbS interacts with the real-world distributions of gains, losses, and probabilities to produce the classical psychoeconomic functions. I extend DbS to account for preferences in benchmark data sets. Finally, in a challenge to the classical notion of stable subjective valuations, I review evidence that manipulating the distribution of attribute values in the environment changes our subjective valuations just as DbS predicts.