A Bayesian approach for estimating the probability of trigger failures in the stop-signal paradigm

Response inhibition is frequently investigated using the stop-signal paradigm, where participants perform a two-choice response time task that is occasionally interrupted by a stop signal instructing them to withhold their response. Stop-signal performance is formalized as a race between a go and a stop process. If the go process wins, the response is executed; if the stop process wins, the response is inhibited. Successful inhibition requires fast stop responses and a high probability of triggering the stop process. Existing methods allow for the estimation of the latency of the stop response, but are unable to identify deficiencies in triggering the stop process. We introduce a Bayesian model that addresses this limitation and enables researchers to simultaneously estimate the probability of trigger failures and the entire distribution of stopping latencies. We demonstrate that trigger failures are clearly present in two previous studies, and that ignoring them distorts estimates of stopping latencies. The parameter estimation routine is implemented in the BEESTS software (Matzke et al., Front. Quantitative Psych. Measurement, 4, 918; 2013a) and is available at http://dora.erbe-matzke.com/software.html.     

Bayesian inference for psychology. Part II: Example applications with JASP

Bayesian hypothesis testing presents an attractive alternative to p value hypothesis testing. Part I of this series outlined several advantages of Bayesian hypothesis testing, including the ability to quantify evidence and the ability to monitor and update this evidence as data come in, without the need to know the intention with which the data were collected. Despite these and other practical advantages, Bayesian hypothesis tests are still reported relatively rarely. An important impediment to the widespread adoption of Bayesian tests is arguably the lack of user-friendly software for the run-of-the-mill statistical problems that confront psychologists for the analysis of almost every experiment: the t-test, ANOVA, correlation, regression, and contingency tables. In Part II of this series we introduce JASP (http://www.jasp-stats.org), an open-source, cross-platform, user-friendly graphical software package that allows users to carry out Bayesian hypothesis tests for standard statistical problems. JASP is based in part on the Bayesian analyses implemented in Morey and Rouder’s BayesFactor package for R. Armed with JASP, the practical advantages of Bayesian hypothesis testing are only a mouse click away.     

Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications

Bayesian parameter estimation and Bayesian hypothesis testing present attractive alternatives to classical inference using confidence intervals and p values. In part I of this series we outline ten prominent advantages of the Bayesian approach. Many of these advantages translate to concrete opportunities for pragmatic researchers. For instance, Bayesian hypothesis testing allows researchers to quantify evidence and monitor its progression as data come in, without needing to know the intention with which the data were collected. We end by countering several objections to Bayesian hypothesis testing. Part II of this series discusses JASP, a free and open source software program that makes it easy to conduct Bayesian estimation and testing for a range of popular statistical scenarios (Wagenmakers et al. this issue).     

On the importance of avoiding shortcuts in applying cognitive models to hierarchical data

Psychological experiments often yield data that are hierarchically structured. A number of popular shortcut strategies in cognitive modeling do not properly accommodate this structure and can result in biased conclusions. To gauge the severity of these biases, we conducted a simulation study for a two-group experiment. We first considered a modeling strategy that ignores the hierarchical data structure. In line with theoretical results, our simulations showed that Bayesian and frequentist methods that rely on this strategy are biased towards the null hypothesis. Secondly, we considered a modeling strategy that takes a two-step approach by first obtaining participant-level estimates from a hierarchical cognitive model and subsequently using these estimates in a follow-up statistical test. Methods that rely on this strategy are biased towards the alternative hypothesis. Only hierarchical models of the multilevel data lead to correct conclusions. Our results are particularly relevant for the use of hierarchical Bayesian parameter estimates in cognitive modeling.     

Failures of cognitive control or attention? The case of stop-signal deficits in schizophrenia

We used Bayesian cognitive modelling to identify the underlying causes of apparent inhibitory deficits in the stop-signal paradigm. The analysis was applied to stop-signal data reported by Badcock et al. (Psychological Medicine 32: 87-297, 2002) and Hughes et al. (Biological Psychology 89: 220-231, 2012), where schizophrenia patients and control participants made rapid choice responses, but on some trials were signalled to stop their ongoing response. Previous research has assumed an inhibitory deficit in schizophrenia, because estimates of the mean time taken to react to the stop signal are longer in patients than controls. We showed that these longer estimates are partly due to failing to react to the stop signal (“trigger failures”) and partly due to a slower initiation of inhibition, implicating a failure of attention rather than a deficit in the inhibitory process itself. Correlations between the probability of trigger failures and event-related potentials reported by Hughes et al. are interpreted as supporting the attentional account of inhibitory deficits. Our results, and those of Matzke et al. (2016), who report that controls also display a substantial although lower trigger-failure rate, indicate that attentional factors need to be taken into account when interpreting results from the stop-signal paradigm.     

A Simple Method for Comparing Complex Models: Bayesian Model Comparison for Hierarchical Multinomial Processing Tree Models Using Warp-III Bridge Sampling

Psychometrika - Multinomial processing trees (MPTs) are a popular class of cognitive models for categorical data. Typically, researchers compare several MPTs, each equipped with many parameters,...     

Bayesian inference for psychology,part III: Parameter estimation in nonstandard models

We demonstrate the use of three popular Bayesian software packages that enable researchers to estimate parameters in a broad class of models that are commonly used in psychological research. We focus on WinBUGS, JAGS, and Stan, and show how they can be interfaced from R and MATLAB. We illustrate the use of the packages through two fully worked examples; the examples involve a simple univariate linear regression and fitting a multinomial processing tree model to data from a classic false-memory experiment. We conclude with a comparison of the strengths and weaknesses of the packages. Our example code, data, and this text are available via https://osf.io/ucmaz/.     

A test of the diffusion model explanation for the worst performance rule using preregistration and blinding

People with higher IQ scores also tend to perform better on elementary cognitive-perceptual tasks, such as deciding quickly whether an arrow points to the left or the right Jensen (2006). The worst performance rule (WPR) finesses this relation by stating that the association between IQ and elementary-task performance is most pronounced when this performance is summarized by people’s slowest responses. Previous research has shown that the WPR can be accounted for in the Ratcliff diffusion model by assuming that the same ability parameter—drift rate—mediates performance in both elementary tasks and higher-level cognitive tasks. Here we aim to test four qualitative predictions concerning the WPR and its diffusion model explanation in terms of drift rate. In the first stage, the diffusion model was fit to data from 916 participants completing a perceptual two-choice task; crucially, the fitting happened after randomly shuffling the key variable, i.e., each participant’s score on a working memory capacity test. In the second stage, after all modeling decisions were made, the key variable was unshuffled and the adequacy of the predictions was evaluated by means of confirmatory Bayesian hypothesis tests. By temporarily withholding the mapping of the key predictor, we retain flexibility for proper modeling of the data (e.g., outlier exclusion) while preventing biases from unduly influencing the results. Our results provide evidence against the WPR and suggest that it may be less robust and less ubiquitous than is commonly believed.