Signal detection theory with finite mixture distributions: theoretical developments with applications to recognition memory

An extension of signal detection theory (SDT) that incorporates mixtures of the underlying distributions is presented. The mixtures can be motivated by the idea that a presentation of a signal shifts the location of an underlying distribution only if the observer is attending to the signal; otherwise, the distribution is not shifted or is only partially shifted. Thus, trials with a signal presentation consist of a mixture of 2 (or more) latent classes of trials. Mixture SDT provides a general theoretical framework that offers a new perspective on a number of findings. For example, mixture SDT offers an alternative to the unequal variance signal detection model; it can also account for nonlinear normal receiver operating characteristic curves, as found in recent research.     

Process dissociation and mixture signal detection theory

The process dissociation procedure was developed in an attempt to separate different processes involved in memory tasks. The procedure naturally lends itself to a formulation within a class of mixture signal detection models. The dual process model is shown to be a special case. The mixture signal detection model is applied to data from a widely analyzed study. The results suggest that a process other than recollection may be involved in the process dissociation procedure.     

Assessment of shock discrimination in rats with signal detection theory

Nociception in six rats was evaluated with signal detection theory (TSD) techniques using a modified “yes-no” paradigm. A .1-sec, .075-mA shock was the discriminative stimulus for a choice to one side of a T-maze, and a .1-sec, .25-mA shock (capable of reliably eliciting a “flinch”) was the discriminative stimulus for the opposite choice. Response bias was manipulated through differential reinforcement for the two choices, and signal sensitivity was manipulated by varying the intensity of the higher intensity shock signal. Individual detection ROCs appeared linear with slopes of one when plotted on double-probability scales. Indices of signal sensitivity and response bias were independent, as predicted by TSD. Morphine sulfate (4 mg/kg) significantly reduced d’ but did not affect β, suggesting that, if appropriate parameters are used, TSD analyses may have value in assessing nociception in rats.     

The mirror effect and mixture signal detection theory

The mirror effect for word frequency refers to the finding that low-frequency words have higher hit rates and lower false alarm rates than high-frequency words. This result is typically interpreted in terms of conventional signal detection theory (SDT), in which case it indicates that the order of the underlying old item distributions mirrors the order of the new item distributions. However, when viewed in terms of a mixture version of SDT, the order of hits and false alarms does not necessarily imply the same order in the underlying distributions because of possible effects of mixing. A reversal in underlying distributions did not appear for fits of mixture SDT models to data from 4 experiments.     

An introduction to Bayesian hierarchical models with an application in the theory of signal detection

Although many nonlinear models of cognition have been proposed in the past 50 years, there has been little consideration of corresponding statistical techniques for their analysis. In analyses with nonlinear models, unmodeled variability from the selection of items or participants may lead to asymptotically biased estimation. This asymptotic bias, in turn, renders inference problematic. We show, for example, that a signal detection analysis of recognition memory data leads to asymptotic underestimation of sensitivity. To eliminate asymptotic bias, we advocate hierarchical models in which participant variability, item variability, and measurement error are modeled simultaneously. By accounting for multiple sources of variability, hierarchical models yield consistent and accurate estimates of participant and item effects in recognition memory. This article is written in tutorial format; we provide an introduction to Bayesian statistics, hierarchical modeling, and Markov chain Monte Carlo computational techniques.     

Decision processes in discrimination: fundamental misrepresentations of signal detection theory.

A new approach to studying decision making in discrimination tasks is described that does not depend on the technical assumptions of signal detection theory (e.g., normality of the encoding distributions). In 3 different experiments, results of these new distribution-free tests converge on a single, surprising conclusion: response biases had substantial effects on the encoding distributions but no effect on the decision rule, which was uniformly unbiased in equal and unequal base rate conditions and in symmetric and asymmetric payoff conditions. This seemingly paradoxical result is fundamentally inconsistent with the entire family of signal detection theory models, raising some important questions about the significance of many published results in the human performance literature.     

An experiment on shape discrimination and signal detection in octopus

Ten octopuses were trained to perform a successive discrimination between the two shapes shown in Figure I (a). After 7 days of training, when performance was significantly above chance, transfer tests were given with other shapes that were either rotations or parts of the original training shapes. At least six theories have been put forward to explain shape discrimination in the octopus, but none of these are capable of explaining the present results. The transfer tests suggest that the discrimination was performed in terms of component parts of the shapes (vertical bars projecting upwards or downwards), and their relationship to the shape as a whole (terminal or central).

During successive discrimination training the general level of attack varies between animals, and fluctuates from day to day. As a result there are often more attacks on both the positive and negative shapes on some occasions than others, making it difficult to compare the levels of discrimination achieved. It is suggested that the concepts of signal detection theory can help overcome this difficulty. Attacks on the positive shape (“hits”) plotted against attacks on the negative shape (“false positives”) constitute an ROC curve from which a value of d', independant of the general level of attack, can be obtained.     

Right hemisphere sensitivity to novel metaphoric relations: application of the signal detection theory

The present study used the signal detection theory to test the hypothesis that the right hemisphere (RH) is more sensitive than the left hemisphere (LH) to the distant semantic relations in novel metaphoric expressions. In two divided visual field experiments, sensitivity (d') and criterion (beta) were calculated for responses to different types of word pairs. In the first experiment, subjects were presented with unfamiliar two-word novel metaphoric expressions ("signal") and unrelated word-pairs ("noise"). In the second experiment, literal expressions ("signal") and unrelated word pairs ("noise") were presented. In line with the Coarse Semantic Coding Theory [Beeman, M. (1998). Coarse semantic coding and discourse comprehension. In: M. Beeman & C. Chiarello (Eds.). Right hemisphere language comprehension: Perspectives from cognitive neuroscience. Mahwah, NJ: Erlbaum, pp. 255-284.] as well as with the Graded Salience Hypothesis [Giora, R. (2003). On our mind: Salience, context and figurative language. New York: Oxford University Press.], the findings suggest that the RH is more sensitive than the LH to unfamiliar metaphoric relations. Furthermore, this RH advantage in processing distant semantic relations did not extend to familiar semantic relations.     

An application of the optic sphere theory in discrimination of slant with minimal information

In Johansson and Börjesson (1989), a new theory of visual space perception—the optic sphere theory—was presented in which the hemispheric shape of the retina is utilized for determination of slant of plane surfaces in wide-angle perception. The process of the optic sphere mechanism can be described as the projection of a translating distal line on the optic sphere, and an extrapolation of this projection to a great circle. The determination of the 3-D slant of the distal line is made by identification of points of no change on the great circle during its rotation. The main objective of the present study was to investigate this process as applied to central stimulation of the retina with reduced and minimal information of slant or horizontal orientation. Each stimulus pattern consisted of either two continuous lines or two pairs of dots in motion presented on a computer screen. The pairwise lines and the pairs of dots defined simulated 3-D slants (or horizontal orientations) of different magnitude within each pair, and the subjects' task was to discriminate between these simulated slants. It was shown that the simulations evoke percepts of 3-D slants, and of horizontal orientations, and that it is possible to discriminate between them even from minimal information (pairs of dots). Further, the empirical findings of Börjesson (1994) indicated that longer extrapolations of the projected arc to a point of no change yield less accurate discriminations of slant. We failed to replicate this in Experiment 4, in which case stimulus variables that covaried with extrapolation length were eliminated or minimized. It is suggested that this raises some doubt about discrimination accuracy as dependent on extrapolation length per se. The overall conclusion, however, is that the optic sphere theory represents a possible explanation of, or analogy to, the process utilized by the visual system for determination of the simulated 3-D slants and horizontal orientations in the present study.     

An introduction to mixture item response theory models

Mixture item response theory (IRT) allows one to address situations that involve a mixture of latent subpopulations that are qualitatively different but within which a measurement model based on a continuous latent variable holds. In this modeling framework, one can characterize students by both their location on a continuous latent variable as well as by their latent class membership. For example, in a study of risky youth behavior this approach would make it possible to estimate an individual's propensity to engage in risky youth behavior (i.e., on a continuous scale) and to use these estimates to identify youth who might be at the greatest risk given their class membership. Mixture IRT can be used with binary response data (e.g., true/false, agree/disagree, endorsement/not endorsement, correct/incorrect, presence/absence of a behavior), Likert response scales, partial correct scoring, nominal scales, or rating scales. In the following, we present mixture IRT modeling and two examples of its use. Data needed to reproduce analyses in this article are available as supplemental online materials at http://dx.doi.org/10.1016/j.jsp.2016.01.002.     

Calculation of signal detection theory measures

Signal detection theory (SDT) may be applied to any area of psychology in which two different types of stimuli must be discriminated. We describe several of these areas and the advantages that can be realized through the application of SDT. Three of the most popular tasks used to study discriminability are then discussed, together with the measures that SDT prescribes for quantifying performance in these tasks. Mathematical formulae for the measures are presented, as are methods for calculating the measures with lookup tables, computer software specifically developed for SDT applications, and general purpose computer software (including spreadsheets and statistical analysis software).     

Decision noise: An explanation for observed violations of signal detection theory

In signal detection theory (SDT), responses are governed by perceptual noise and a flexible decision criterion. Recent criticisms of SDT (see, e.g., Balakrishnan, 1999) have identified violations of its assumptions, and researchers have suggested that SDT fundamentally misrepresents perceptual and decision processes. We hypothesize that, instead, these violations of SDT stem from decision noise: the inability to use deterministic response criteria. In order to investigate this hypothesis, we present a simple extension of SDT—the decision noise model—with which we demonstrate that shifts in a decision criterion can be masked by decision noise. In addition, we propose a new statistic that can help identify whether the violations of SDT stem from perceptual or from decision processes. The results of a stimulus classification experiment—together with model fits to past experiments—show that decision noise substantially affects performance. These findings suggest that decision noise is important across a wide range of tasks and needs to be better understood in order to accurately measure perceptual processes.     

Threshold nonlinearities and signal detection theory

The review summarizes Kulikowski's contributions to the mechanisms involved in visual detection and discrimination. His findings provide strong evidence for the existence of 'barriers' which block weak activity in the visual system. It is proposed that the properties of these barriers may underlie near-optimal detection in conditions of signal uncertainty.     

Functions for traditional and multilevel approaches to signal detection theory

In the present article, functions written in the freeware R are presented that calculate several measures from traditional signal detection theory for each individual in a sample, along with summary statistics for the sample. Bias-corrected and accelerated bootstrap confidence intervals are also produced. Arguments are made for using an alternative approach—multilevel generalized linear models—and a function is presented for it. These functions are part of the R package sdtalt, which is available on the Comprehensive R Archive Network. Recent data from memory recognition studies are used to illustrate these functions.     

A contextual effect in feature detection with application of signal detection methodology

These experiments address the question of whether the perception of a form is exclusively determined by a prior analysis of its elements. Employing a signal detection task, Experiment 1 replicated the finding of Weisstein and Harris (1974) that embedding a line segment feature in a unitary figural context facilitates its detection. The results of Experiment 2 suggest that the most probable agent of this context effect on feature detection is the three-dimensionality of the unitary context. With bi-hemiretinal stimulus presentation. Experiment 3 showed a significant context effect in the right, but not in the left visual field. Some current paradigms are applied to these results.     

BayesSDT: Software for Bayesian inference with signal detection theory

This article describes and demonstrates the BayesSDT MATLAB-based software package for performing Bayesian analysis with equal-variance Gaussian signal detection theory (SDT). The software uses WinBUGS to draw samples from the posterior distribution of six SDT parameters: discriminability, hit rate, false alarm rate, criterion, and two alternative measures of bias. The software either provides a simple MATLAB graphical user interface or allows a more general MATLAB function call to produce graphs of the posterior distribution for each parameter of interest for each data set, as well as to return the full set of posteriorsamples.     

Alternatives to a table of criterion values in signal detection theory

Gardner, Dalsing, Reyes, and Brake (1984) supplied a table of criterion values (β) related to hit and false-alarm rates in signal detection theory. Other methods of calculatingβ are suggested as more accurate alternatives to using that table. A short computer program is provided to calculate β and the sensitivity indexd’.     

An investigation of the sampling distributions of item discrimination indices

The sampling properties of four item discrimination indices (biserialr, Cook's indexB, theU–L 27 per cent index, and DeltaP) were investigated in order to ascertain their sampling properties when small samples drawn from actual test data rather than constructed data were employed. The empirical results indicated that the mean index values approximated the population values and that values of the standard deviations computed from large sample formulas were good approximations to the standard deviations of the observed distributions based on samples of size 120 or less. Goodness of fit tests comparing the observed distributions with the corresponding distribution of the product-moment correlation coefficient based upon a bivariate normal population indicated that this correlational model was inappropriate for the data. The lack of adequate mathematical models for the sampling distributions of item discrimination indices suggests that one should avoid indices whose only real reason for existence was the simplification of computational procedures.This research reported herein was performed pursuant to a contract (OE-2-10-071) with the United States Office of Education, Department of Health, Education and Welfare.