Assessing the evidence for response time mixture distributions

I describe a technique for comparing two simple accounts of a distribution of response times: A mixture model and a generalized-shift model. In the mixture model, a target distribution is assumed to be a mixture of response times from two other (reference) distributions. In the generalized-shift model, the target distribution is assumed to be a quantile average of the reference distributions. In order to distinguish these two possibilities, quantiles for the target distribution are estimated from the quantiles of the reference distributions assuming either a shift or a mixture, and the predicted quantiles are used to calculate the multinomial likelihood of the obtained data. Monte Carlo simulations reported here demonstrate that the index is relatively unbiased, is effective with moderate sample sizes and modest spreads between the reference distributions, is relatively unaffected by changes in the number of bins or by data trimming, can be used with data aggregated across subjects, and is relatively insensitive to a range of subject variations in distribution shape and in mixture or shift proportion. As an illustration, the index is applied to the interpretation of three effects from distinct paradigms: residual switch costs in the task-switching paradigm, the psychological refractory period effect, and sequential effects in the Simon task. I conclude that the multinomial likelihood index provides a useful and easily applied tool for the interpretation of effects on response time distributions.     

Quantile maximum likelihood estimation of response time distributions

Queen’s University, Kingston, Ontario, Canada We introduce and evaluate via a Monte Carlo study a robust new estimation technique that fits distribution functions to grouped response time (RT) data, where the grouping is determined by sample quantiles. The new estimator, quantile maximum likelihood (QML), is more efficient and less biased than the best alternative estimation technique when fitting the commonly used ex-Gaussian distribution. Limitations of the Monte Carlo results are discussed and guidance provided for the practical application of the new technique. Because QML estimation can be computationally costly, we make fast open source code for fitting available that can be easily modified     

A hierarchical model for estimating response time distributions

We present a statistical model for inference with response time (RT) distributions. The model has the following features. First, it provides a means of estimating the shape, scale, and location (shift) of RT distributions. Second, it is hierarchical and models between-subjects and within-subjects variability simultaneously. Third, inference with the model is Bayesian and provides a principled and efficient means of pooling information across disparate data from different individuals. Because the model efficiently pools information across individuals, it is particularly well suited for those common cases in which the researcher collects a limited number of observations from several participants. Monte Carlo simulations reveal that the hierarchical Bayesian model provides more accurate estimates than several popular competitors do. We illustrate the model by providing an analysis of the symbolic distance effect in which participants can more quickly ascertain the relationship between nonadjacent digits than that between adjacent digits.     

An evaluation of the reliability of probability judgments across response modes and over time

Despite the importance of probability assessment methods in behavioral decision theory and decision analysis, little attention has been directed at evaluating their reliability and validity. In fact, no comprehensive study of reliability has been undertaken. Since reliability is a necessary condition for validity, this oversight is significant. The present study was motivated by that oversight. We investigated the reliability of probability measures derived from three response modes: numerical probabilities, pie diagrams, and odds. Unlike previous studies, the experiment was designed to distinguish systematic deviations in probability judgments, such as those due to experience or practice, from random deviations. It was found that subjects assessed probabilities reliably for all three assessment methods regardless of the reliability measures employed. However, a small but statistically significant decrease over time in the magnitudes of assessed probabilities was observed. This effect was linked to a decrease in subjects overconfidence during the course of the experiment.     

Location cuing and response time distributions in visual attention

The allocation of visual attention was investigated in two experiments. In Experiment 1 (n = 24), a peripheral cue was presented, and in Experiment 2 (n = 24), a central cue was used. In both experiments, cue validity was 90%, and the task was four-choice target identification. Response time distributions were collected for valid trials over five cue-target stimulus onset asynchronies (SOAs), and ex-Gaussian parameters were extracted. In both experiments, only the mean of the Gaussian component decreased as a function of cue-target SOA, which implied a strict time axis translation of the distributions. The results were consistent with sequential sampling models featuring a variable delay in the onset of information uptake.     

A hierarchical bayesian statistical framework for response time distributions

This paper provides a statistical framework for estimating higher-order characteristics of the response time distribution, such as the scale (variability) and shape. Consideration of these higher order characteristics often provides for more rigorous theory development in cognitive and perceptual psychology (e.g., Luce, 1986). RT distribution for a single participant depends on certain participant characteristics, which in turn can be thought of as arising from a distribution of latent variables. The present work focuses on the three-parameter Weibull distribution, with parameters for shape, scale, and shift (initial value). Bayesian estimation in a hierarchical framework is conceptually straightforward. Parameter estimates, both for participant quantities and population parameters, are obtained through Markov Chain Monte Carlo methods. The methods are illustrated with an application to response time data in an absolute identification task. The behavior of the Bayes estimates are compared to maximum likelihood (ML) estimates through Monte Carlo simulations. For small sample size, there is an occasional tendency for the ML estimates to be unreasonably extreme. In contrast, by borrowing strength across participants, Bayes estimation shrinks extreme estimates. The results are that the Bayes estimators are more accurate than the corresponding ML estimators.We are grateful to Michael Stadler who allowed us use of his data. This research is supported by (a) National Science Foundation Grant SES-0095919 to J. Rouder, D. Sun, and P. Speckman, (b) University of Missouri Research Board Grant 00-77 to J. Rouder, (c) National Science Foundation grant DMS-9972598 to Sun and Speckman, and (d) a grant from the Missouri Department of Conservation to D. Sun.     

Fitting Wald and ex-Wald distributions to response time data: An example using functions for the S-PLUS package

Schwarz (2001, 2002) proposed the ex-Wald distribution, obtained from the convolution of Wald and exponential random variables, as a model of simple and go/no-go response time. This article provides functions for the S-PLUS package that produce maximum likelihood estimates of the parameters for the ex-Wald, as well as for the shifted Wald and ex-Gaussian, distributions. In a Monte Carlo study, the efficiency and bias of parameter estimates were examined. Results indicated that samples of at least 400 are necessary to obtain adequate estimates of the ex-Wald and that, for some parameter ranges, much larger samples may be required. For shifted Wald estimation, smaller samples of around 100 were adequate, at least when fits identified by the software as having ill-conditioned maximums were excluded. The use of all functions is illustrated using data from Schwarz (2001). The S-PLUS functions and Schwarz’s data may be downloaded from the Psychonomic Society’s Web archive, www. psychonomic.org/archive/.     

Levels of selective attention revealed through analyses of response time distributions

The present research examines the nature of the interference effects in a number of selective attention tasks. All of these tasks result in interference in performance by presenting information that is irrelevant to task performance but competes for selection. The interference from this competing information slows the response time (RT) of participants relative to a condition where the competition is minimized. The authors use a convolution of an exponential and a Gaussian (ex-Gaussian) distribution to examine the influence of interference on the characteristics of RT distributions. Consistent with previous research, the authors show that interference in the Stroop task is reflected by both the Gaussian and exponential portions of the ex-Gaussian. In contrast, in 4 experiments they show that several other interference tasks evidence interference that is reflected only in the Gaussian portion of the ex-Gaussian distribution. The authors suggest that these differences reflect the operation of different selection mechanisms, and they examine how sequential sampling models accommodate these effects.     

Decomposing the problem-size effect: a comparison of response time distributions across cultures

Is the locus of the problem-size effect in mental arithmetic different across cultures? In a novel approach to this question, the ex-Gaussian distributional model was applied to response times for large (e.g., 8 x 9) and small (e.g., 2 x 3) problems obtained from Chinese and Canadian graduate students in a multiplication production task (LeFevre & Liu, 1997). The problem-size effect for the Chinese group occurred in mu (the mean of the normal component), whereas the problem-size effect for the Canadian group occurred in both mu and tau (the mean of the exponential component). The results support the position that the problem-size effect for the Chinese group is purely a memory-retrieval effect, whereas for the Canadian group, it is an effect of both retrieval and the use of nonretrieval solution procedures.     

Dynamics of the attentional control of word retrieval: analyses of response time distributions

Since W. Wundt (1904) and H. J. Watt (1906), researchers have found no agreement on how goals direct word retrieval. A prevailing associative account (E. K. Miller & J. D. Cohen, 2001) holds that goals bias association strength, which determines retrieval latency and whether irrelevant words interfere. A symbolic account (A. Roelofs, 2003) holds that goals enable retrieval rules and predicts no strict dependence of interference on latency. Here, 3 chronometric experiments in which the role of relative retrieval latency was investigated through distributional analyses, following Watt, are reported. Participants verbally categorized picture-word pairs that were semantically related or unrelated, or they categorized single pictures or words. The pairs yielded semantic latency effects in both word and picture categorizing, although single words were categorized slower than single pictures. Semantic effects occurred in word categorizing even when postexposure of the pictures compensated for the difference in categorizing latency. Vincentile and ex-Gaussian analyses revealed that the semantic effects occurred throughout the latency distributions, excluding goal neglect as the cause of the effects. The results were interpreted as most consistent with the symbolic account, which was corroborated by computer simulations.     

Using response time distributions to examine top-down influences on attentional capture

Three experiments examined contingent attentional capture, which is the finding that cuing effects are larger when cues are perceptually similar to a target than when they are dissimilar to the target. This study also analyzed response times (RTs) in terms of the underlying distributions for valid cues and invalid cues. Specifically, an ex-Gaussian analysis and a vincentile analysis examined the influence of top-down attentional control settings on the shift and skew of RT distributions and how the shift and the skew contributed to the cuing effects in the mean RTs. The results showed that cue/target similarity influenced the size of cuing effects. The RT distribution analyses showed that the cuing effects reflected only a shifting effect, not a skewing effect, in the RT distribution between valid cues and invalid cues. That is, top-down attentional control moderated the cuing effects in the mean RTs through distribution shifting, not distribution skewing. The results support the contingent orienting hypothesis (Folk, Remington, & Johnston, Journal of Experimental Psychology: Human Perception and Performance, 18, 1030–1044, 1992) over the attentional disengagement account (Theeuwes, Atchley, & Kramer, 2000) as an explanation for when top-down attentional settings influence the selection of salient stimuli.     

Aging and intraindividual variability in performance: analyses of response time distributions

It has been suggested that older adults are more variable in their performance because they are more prone to lapses of either attention or intention. In the present experiment, 9 young and 9 older adults each performed nearly 2000 trials of a same-different judgment task. As expected, older adults were slower and more variable than young adults. When the age-related difference in speed was taken into account, however, the older adults were, if anything, less variable than the young adults. When younger and older adults' RT distributions were analyzed using quantile-quantile plots and by fitting ex-Gaussian and Weibull functions, there was no consistent evidence that older adults' distributions were more skewed than young adults', as would be predicted by age-related increases in lapses of attention or intention. Importantly, there was a positive, linear relation between RT and intraindividual variability, and the same relation was observed both within subjects (practice increased speed and reduced variability) as well as between subjects (regardless of age, slower individuals were more variable). Thus, the present results suggest that there may be a general law governing the relation between average RT and variability, and that the greater performance variability of older adults primarily reflects their greater average RTs.     

Additive and interactive effects on response time distributions in visual word recognition

Across 3 different word recognition tasks, distributional analyses were used to examine the joint effects of stimulus quality and word frequency on underlying response time distributions. Consistent with the extant literature, stimulus quality and word frequency produced additive effects in lexical decision, not only in the means but also in the shape of the response time distributions, supporting an early normalization process that is separate from processes influenced by word frequency. In contrast, speeded pronunciation and semantic classification produced interactive influences of word frequency and stimulus quality, which is a fundamental prediction from interactive activation models of lexical processing. These findings suggest that stimulus normalization is specific to lexical decision and is driven by the task's emphasis on familiarity-based information.     

Predicting group-level outcome variables: An empirical comparison of analysis strategies

This study provides a review of two methods for analyzing multilevel data with group-level outcome variables and compares them in a simulation study. The analytical methods included an unadjusted ordinary least squares (OLS) analysis of group means and a two-step adjustment of the group means suggested by Croon and van Veldhoven (2007). The Type I error control, power, bias, standard errors, and RMSE in parameter estimates were compared across design conditions that included manipulations of number of predictor variables, level of correlation between predictors, level of intraclass correlation, predictor reliability, effect size, and sample size. The results suggested that an OLS analysis of the group means, with White’s heteroscedasticity adjustment, provided more power for tests of group-level predictors, but less power for tests of individual-level predictors. Furthermore, this simple analysis avoided the extreme bias in parameter estimates and inadmissible solutions that were encountered with other strategies. These results were interpreted in terms of recommended analytical methods for applied researchers.     

群体水平IRT模型及其应用——兼与IRT的比较

传统上,群体评估都是以个体的评估结果的平均值为基础进行的。而群体水平IRT理论则可以避开对个体的评估,直接实现对群体的评估,它具有许多传统方法难以企及的优点。本文将群体水平IRT模型应用于2007年某省高考英语阅读理解的410所学校的能力评估,评估结果发现：410所学校的英语阅读理解能力几乎都在[-1,1]区间内,没有能力极高或极低的学校。对这些学校而言,测验中所有项目的难度较易,区分度适中。所有的评估结果与IRT模型的评估结果在 的水平上相关显著,表明GIRT模型在实践中是可以选择的一种群体评估方法。     

Reaction time and movement time as measures of stimulus evaluation and response processes

Reaction time has been divided into the time to initiate a response (RT) and the time to execute the motor response (MT) in many chronometric studies of intelligence. Our purpose was to determine which cognitive processes are reflected by RT and MT. To accomplish this, the latency of the P300 wave of the event-related potential was recorded concurrently with RT and MT measures in three experiments. P300 latency reflects the duration of stimulus evaluation relatively independently of response processes. In the first experiment, a Sternberg memory task was employed to manipulate stimulus classification requirements. In the second, the subjects' emphasis on either speed or accuracy of responding was examined during a task that also manipulated stimulus evaluation time. In the third experiment, a Stroop-like task was used to examine response processes. RT and P300 latency varied with manipulations of stimulus evaluation time, whereas MT varied with difficulty of motor execution. MT was also affected by stimulus classification processes. Both RT and MT were sensitive to processes involved in response bias and preparation. The possibility that the correlations of RT and MT with measures of intelligence are due to effects on a common stage of information processing cannot be rejected in the light of these results.     

Attention, spatial integration, and the tail of response time distributions in Stroop task performance

A few studies have examined selective attention in Stroop task performance through ex-Gaussian analyses of response time (RT) distributions. It has remained unclear whether the tail of the RT distribution in vocal responding reflects spatial integration of relevant and irrelevant attributes, as suggested by Spieler, Balota, and Faust (2000). Here, two colour-word Stroop experiments with vocal responding are reported in which the spatial relation between colour and word was manipulated. Participants named colours (e.g., green; say "green") while trying to ignore distractors that were incongruent or congruent words (e.g., red or green), or neutral series of Xs. The vocal RT was measured. Colour words in colour, white words superimposed onto colour rectangles (Experiment 1), and colour rectangles combined with auditory words (Experiment 2) yielded Stroop effects in both the leading edge and the tail of the RT distributions. These results indicate that spatial integration is not necessary for effects in the tail to occur in vocal responding. It is argued that the findings are compatible with an association of the tail effects with task conflict.