Searching for the sequential line-up advantage: A distinctiveness explanation

Signal Detection models as well as the Two-High-Threshold model (2HTM) have been used successfully as measurement models in recognition tasks to disentangle memory performance and response biases. A popular method in recognition memory is to elicit confidence judgements about the presumed old/new status of an item, allowing for the easy construction of ROCs. Since the 2HTM assumes fewer latent memory states than response options are available in confidence ratings, the 2HTM has to be extended by a mapping function which models individual rating scale usage. Unpublished data from 2 experiments in Bröder and Schütz (2009) validate the core memory parameters of the model, and 3 new experiments show that the response mapping parameters are selectively affected by manipulations intended to affect rating scale use, and this is independent of overall old/new bias. Comparisons with SDT show that both models behave similarly, a case that highlights the notion that both modelling approaches can be valuable (and complementary) elements in a researcher's toolbox.     

On the importance of models in interpreting remember-know experiments: Comments on Gardiner et al.'s (2002) meta-analysis

From a meta-analysis of recognition experiments using the remember-know-guess paradigm, Gardiner, Ramponi, and Richardson-Klavehn (2002) reported two findings that they viewed as evidence against the one-dimensional model for that paradigm: (1) Memory strength increased when know responses were added to remember responses, decreasing when guess responses were also included. (2) The accuracy of guess responses was correlated with the location of the old-new criterion in the one-dimensional model for the paradigm, implying that guesses were influenced by decision processes. We question both findings. The first result is contradicted by a signal-detection (SDT) analysis, which shows that both know and guess responses reduced estimated memory strength. The discrepancy results from the properties of A′, the measure of accuracy used by Gardiner et al., which we argue is flawed. The second result follows directly from the one-dimensional model, in which accuracy and response criteria are fixed. The authors' reasons for rejecting the one-dimensional model are thus not persuasive, but it can nonetheless be rejected because ROC curves implied by the data are inconsistent with ROCs derived from ratings experiments. A two-dimensional SDT model (Rotello, Macmillan, & Reeder, 2004) accounts for both sets of data. The analysis illustrates the importance of models in interpreting remember-know data.     

On the importance of models in interpreting remember-know experiments: comments on Gardiner et al.'s (2002) meta-analysis

From a meta-analysis of recognition experiments using the remember-know-guess paradigm, Gardiner, Ramponi, and Richardson-Klavehn (2002) reported two findings that they viewed as evidence against the one-dimensional model for that paradigm: (1) Memory strength increased when know responses were added to remember responses, decreasing when guess responses were also included. (2) The accuracy of guess responses was correlated with the location of the old-new criterion in the one-dimensional model for the paradigm, implying that guesses were influenced by decision processes. We question both findings. The first result is contradicted by a signal-detection (SDT) analysis, which shows that both know and guess responses reduced estimated memory strength. The discrepancy results from the properties of A', the measure of accuracy used by Gardiner et al., which we argue is flawed. The second result follows directly from the one-dimensional model, in which accuracy and response criteria are fixed. The authors' reasons for rejecting the one-dimensional model are thus not persuasive, but it can nonetheless be rejected because ROC curves implied by the data are inconsistent with ROCs derived from ratings experiments. A two-dimensional SDT model (Rotello, Macmillan, & Reeder, 2004) accounts for both sets of data. The analysis illustrates the importance of models in interpreting remember--know data.     

The influence of averaging and noisy decision strategies on the recognition memory ROC

Many single- and dual-process models of recognition memory predict that the ratings and remember-know receiver operating characteristics (ROCs) are the same, but Rotello, Macmillan, and Reeder (2004) reported that the slopes of the remember-know and ratingsz-transformed ROCs (zROCs) are different. The authors show that averaging introduces nonlinearities to the form of thezROC and that ratings and remember-knowzROCs are indistinguishable when constructed in a conventional manner. The authors show, further, that some nonoptimal decision strategies have a distinctive, nonlinear effect on the form of the single-process continuous-statezROC. The conclusion is that many factors having nothing to do with the nature of recognition memory can affect the shape ofzROCs, and that therefore, the shape of thezROC does not, alone, characterize different memory models.     

The belief bias effect is aptly named: a reply to Klauer and Kellen (2011)

In "Assessing the Belief Bias Effect With ROCs: It's a Response Bias Effect," Dube, Rotello, and Heit (2010) examined the form of receiver operating characteristic (ROC) curves for reasoning and the effects of belief bias on measurement indices that differ in whether they imply a curved or linear ROC function. We concluded that the ROC data are in fact curved and that analyses using statistics that assume a linear ROC are likely to produce Type I errors. Importantly, we showed that the interaction between logic and belief that has inspired much of the theoretical work on belief bias is in fact an error stemming from inappropriate reliance on a contrast (hit rate-false alarm rate) that implies linear ROCs. Dube et al. advanced a new model of belief bias, which, in light of their data, is currently the only plausible account of the effect. Klauer and Kellen (2011) disputed these conclusions, largely on the basis of speculation about the data collection method used by Dube et al. to construct the ROCs. New data and model-based analyses are presented that refute the speculations made by Klauer and Kellen. We also show that new modeling results presented by Klauer and Kellen actually support the conclusions advanced by Dube et al. Together, these data show that the methods used by Dube et al. are valid and that the belief bias effect is simply a response bias effect.     

Traditional difference-score analyses of reasoning are flawed

Studies of the belief bias effect in syllogistic reasoning have relied on three traditional difference score measures: the logic index, belief index, and interaction index. Dube, Rotello, and Heit (2010, 2011) argued that the interaction index incorrectly assumes a linear receiver operating characteristic (ROC). Here, all three measures are addressed. Simulations indicated that traditional analyses of reasoning experiments are likely to lead to incorrect conclusions. Two new experiments examined the role of instructional manipulations on the belief bias effect. The form of the ROCs violated assumptions of traditional measures. In comparison, signal detection theory (SDT) model-based analyses were a better match for the form of the ROCs, and implied that belief bias and instructional manipulations are predominantly response bias effects. Finally, reanalyses of previous studies of conditional reasoning also showed non-linear ROCs, violating assumptions of traditional analyses. Overall, reasoning research using traditional measures is at risk of drawing incorrect conclusions.     

Recollection is a continuous process: Evidence from plurality memory receiver operating characteristics

Is recollection a continuous/graded process or a threshold/all-or-none process? Receiver operating characteristic (ROC) analysis can answer this question as the continuous model and the threshold model predict curved and linear recollection ROCs, respectively. As memory for plurality, an item's previous singular or plural form, is assumed to rely on recollection, the nature of recollection can be investigated by evaluating plurality memory ROCs. The present study consisted of four experiments. During encoding, words (singular or plural) or objects (single/singular or duplicate/plural) were presented. During retrieval, old items with the same plurality or different plurality were presented. For each item, participants made a confidence rating ranging from “very sure old”, which was correct for same plurality items, to “very sure new”, which was correct for different plurality items. Each plurality memory ROC was the proportion of same versus different plurality items classified as “old” (i.e., hits versus false alarms). Chi-squared analysis revealed that all of the plurality memory ROCs were adequately fit by the continuous unequal variance model, whereas none of the ROCs were adequately fit by the two-high threshold model. These plurality memory ROC results indicate recollection is a continuous process, which complements previous source memory and associative memory ROC findings.     

Support for a continuous (single-process) model of recognition memory and source memory

Does memory retrieval occur in a continuous or an all-or-none manner? The shape of the receiver operating characteristic (ROC) has been used to answer this question, with curvilinear and linear memory ROCs indicating continuous and all-or-none retrieval processes, respectively. Signal detection models (e.g., the unequal variance model) correspond to a continuous retrieval process, whereas threshold models (including the multinomial model and the recollection component of the dual-process model) correspond to an all-or-none process. In studies of source memory, Slotnick et al. (2000) and others have observed curvilinear ROCs (supporting the unequal variance model), whereas Yonelinas (1999) observed linear ROCs (supporting the dual-process model). We resolve these seemingly inconsistent results, showing that source memory ROCs are naturally curvilinear but can appear linear when nondiagnostic source information is included in the analysis. Furthermore, the unequal variance model accounted for both recognition memory and source memory ROCs, supporting a continuous process of memory retrieval.     

Positive and negative remember judgments and ROCs in the plurals paradigm: Evidence for alternative decision strategies

Using old-new ratings and remember-know judgments we explored the plurals paradigm, in which studied words must be distinguished from plurality-changed lures. The paradigm allowed us to investigate negative remembering—that is, the remembering of a plural-altered study item; capacity for this judgment was found to be poorer than or equivalent to the conventional positive remembering. A response-bias manipulation affected positive but not negative remembering. The ratings were used to construct ROC curves and test the prediction of the most common dual-process theory of recognition memory (Yonelinas, 2001) that the amount of recollection can be independently estimated from ROC curves and from remember judgments. By fitting the individual data with pure signal detection theory (SDT) models and dual-process models that combined SDT and high-threshold components (HTSDT), we identified two types of subjects. For those who were better described by HTSDT, the predicted convergence of remember-know and ROC measures was observed. For those who were better described by SDT, the ROC intercept could not predict the remember rate. The data are consistent with the idea that all subjects rely on the same representation but base their decisions on different partitions of a decision space.     

On the form of ROCs constructed from confidence ratings

A classical question for memory researchers is whether memories vary in an all-or-nothing, discrete manner (e.g., stored vs. not stored, recalled vs. not recalled), or whether they vary along a continuous dimension (e.g., strength, similarity, or familiarity). For yes-no classification tasks, continuous- and discrete-state models predict nonlinear and linear receiver operating characteristics (ROCs), respectively (D. M. Green & J. A. Swets, 1966; N. A. Macmillan & C. D. Creelman, 1991). Recently, several authors have assumed that these predictions are generalizable to confidence ratings tasks (J. Qin, C. L. Raye, M. K. Johnson, & K. J. Mitchell, 2001; S. D. Slotnick, S. A. Klein, C. S. Dodson, & A. P. Shimamura, 2000, and A. P. Yonelinas, 1999). This assumption is shown to be unwarranted by showing that discrete-state ratings models predict both linear and nonlinear ROCs. The critical factor determining the form of the discrete-state ROC is the response strategy adopted by the classifier.     

Disambiguating conscious and unconscious influences: do exclusion paradigms demonstrate unconscious perception?

Does exclusion failure--responding with previously presented words despite instructions to avoid doing so--demonstrate unconscious influences? This article examines exclusion-based evidence for unconscious perception. I propose an alternative signal detection theory (SDT) framework that can account for exclusion failure and ostensibly convergent qualitative differences without positing additional unconscious perceptual mechanisms. In the proposed SDT model, exclusion failure is a criterion artifact, similar to classic SDT-based critiques of subjective threshold approaches. However, it is suggested that exclusion approaches do demonstrate that response strategies are applied only to above-criterion stimuli and thereby illustrate important qualitative differences between two conscious processes: phenomenal awareness itself and higher-order (i.e., metacognitive) decision processes.     

Characterizing belief bias in syllogistic reasoning: A hierarchical Bayesian meta-analysis of ROC data

The belief-bias effect is one of the most-studied biases in reasoning. A recent study of the phenomenon using the signal detection theory (SDT) model called into question all theoretical accounts of belief bias by demonstrating that belief-based differences in the ability to discriminate between valid and invalid syllogisms may be an artifact stemming from the use of inappropriate linear measurement models such as analysis of variance (Dube et al., Psychological Review, 117(3), 831–863, 2010). The discrepancy between Dube et al.’s, Psychological Review, 117(3), 831–863 (2010) results and the previous three decades of work, together with former’s methodological criticisms suggests the need to revisit earlier results, this time collecting confidence-rating responses. Using a hierarchical Bayesian meta-analysis, we reanalyzed a corpus of 22 confidence-rating studies (N =?993). The results indicated that extensive replications using confidence-rating data are unnecessary as the observed receiver operating characteristic functions are not systematically asymmetric. These results were subsequently corroborated by a novel experimental design based on SDT’s generalized area theorem. Although the meta-analysis confirms that believability does not influence discriminability unconditionally, it also confirmed previous results that factors such as individual differences mediate the effect. The main point is that data from previous and future studies can be safely analyzed using appropriate hierarchical methods that do not require confidence ratings. More generally, our results set a new standard for analyzing data and evaluating theories in reasoning. Important methodological and theoretical considerations for future work on belief bias and related domains are discussed.     

Discrete threshold versus continuous strength models of perceptual recognition.

Two experiments were designed to test discrete-threshold models of letter and word recognition against models that assume that decision criteria are applied to measures of continuous strength. Although our goal is to adjudicate this matter with respect to broad classes of models, some of the specific predictions for discrete-threshold are generated from Grainger and Jacobs' (1994) Dual-Readout Model (DROM) and some of the predictions for continuous strength are generated from a revised version of the Activation-Verification Model (Paap, Newsome, McDonald, & Schvaneveldt, 1982). Experiment 1 uses a two-alternative forced-choice task that is followed by an assessment of confidence and then a whole report if a word is recognized. Factors are manipulated to assess the presence or magnitude of a neighbourhood-frequency effect, a lexical-bias effect, a word-superiority effect, and a pseudoword advantage. Several discrepancies between DROM's predictions and the obtained data are noted. Both types of models were also used to predict the distribution of responses across the levels of confidence for each individual participant. The predictions based on continuous strength were superior. Experiment 2 used a same-different task and confidence ratings to enable the generation of receiver operating characteristics (ROCs). The shapes of the ROCs are more consistent with the continuous strength assumption than with a discrete threshold.     

A signal-detection analysis of fast-and-frugal trees

Models of decision making are distinguished by those that aim for an optimal solution in a world that is precisely specified by a set of assumptions (a so-called "small world") and those that aim for a simple but satisfactory solution in an uncertain world where the assumptions of optimization models may not be met (a so-called "large world"). Few connections have been drawn between these 2 families of models. In this study, the authors show how psychological concepts originating in the classic signal-detection theory (SDT), a small-world approach to decision making, can be used to understand the workings of a class of simple models known as fast-and-frugal trees (FFTs). Results indicate that (a) the setting of the subjective decision criterion in SDT corresponds directly to the choice of exit structure in an FFT; (b) the sensitivity of an FFT (measured in d') is reflected by the order of cues searched and the properties of cues in an FFT, including the mean and variance of cues' individual d's, the intercue correlation, and the number of cues; and (c) compared with the ideal and the optimal sequential sampling models in SDT and a majority model with an information search component, FFTs are extremely frugal (i.e., do not search for much cue information), highly robust, and well adapted to the payoff structure of a task. These findings demonstrate the potential of theory integration in understanding the common underlying psychological structures of apparently disparate theories of cognition.     

Variations in recollection: the effects of complexity on source recognition

Whether recollection is a threshold or signal detection process is highly controversial, and the controversy has centered in part on the shape of receiver operating characteristics (ROCs) and z-transformed ROCs (zROCs). U-shaped zROCs observed in tests thought to rely heavily on recollection, such as source memory tests, have provided evidence in favor of the threshold assumption, but zROCs are not always as U-shaped as threshold theory predicts. Source zROCs have been shown to become more linear when the contribution of familiarity to source discriminations is increased, and this may account for the existing results. However, another way in which source zROCs may become more linear is if the recollection threshold begins to break down and recollection becomes more graded and Gaussian. We tested the "graded recollection" account in the current study. We found that increasing stimulus complexity (i.e., changing from single words to sentences) or increasing source complexity (i.e., changing the sources from audio to videos of speakers) resulted in flatter source zROCs. In addition, conditions expected to reduce recollection (i.e., divided attention and amnesia) had comparable effects on source memory in simple and complex conditions, suggesting that differences between simple and complex conditions were due to differences in the nature of recollection, rather than differences in the utility of familiarity. The results suggest that under conditions of high complexity, recollection can appear more graded, and it can produce curved ROCs. The results have implications for measurement models and for current theories of recognition memory.     

Observers detecting a signal in two multiple observation tasks

Signal detection theory (SDT) is tested against a high threshold and a low threshold theory for two kinds of unusual tasks. In the first, the detection of the signal uses the central level of information processing; in the second, the lack of information originates in the stimulus itself rather than in the sensory organs' limitations. The observation is repeated a fixed number of times for the two tasks, and the observed integration of successive information is compared with predictions derived from the SDT and from threshold theories. All results strongly support the SDT which can thus be considered as a general model for risky binary decision-making insofar as these experiments are representative. This would be an argument to replace the classical analysis of the performance in terms of time and number of errors by a new analysis in terms of capability and strategy. The data show how the subjects are more selective for detecting the signal when the task is more difficult, and at the beginning and the end of a series of presentations. The organization of the decision-criteria of the rating-procedure varies with the difficulty of the condition, and with the number of observations in order to allow the subject to discriminate between likelihood ratios which become more precise.     

Signal-Detection, Threshold, and Dual-Process Models of Recognition Memory: ROCs and Conscious Recollection

Threshold- and signal-detection-based models have dominated theorizing about recognition memory. Building upon these theoretical frameworks, we have argued for a dual-process model in which conscious recollection (a threshold process) and familiarity (a signal-detection process) contribute to memory performance. In the current paper we assessed several memory models by examining the effects of levels of processing and the number of presentations on recognition memory receiver operating characteristics (ROCs). In general, when the ROCs were plotted in probability space they exhibited an inverted U shape; however, when they were plotted inzspace they exhibited a U shape. An examination of the ROCs showed that the dual-process model could account for the observed ROCs, but that models based solely on either threshold or signal-detection processes failed to provide a sufficient account of the data. Furthermore, an examination of subjects' introspective reports using the remember/know procedure showed that subjects were aware of recollection and familiarity and were able to consistently report on their occurrence. The remember/know data were used to accurately predict the shapes of the ROCs, and estimates of recollection and familiarity derived from the ROC data mirrored the subjective reports of these processes.     

Signal detection and threshold models of source memory

Recently, several authors claimed that the curvilinear shape of rating-based source memory receiver operating characteristics (ROCs) refutes threshold models. However, rating-based ROCs are not diagnostic to disprove threshold models. Furthermore, source memory ROC-analyses ignore influences of other processes like old-new-detection and old-new-response-tendencies, so direct estimation of parameters is preferable. Five source monitoring experiments with different response bias manipulations and materials were conducted. We fitted the Two High Threshold Multinomial Model of Source Monitoring by Bayen, Murnane, and Erdfelder (1996) and the Multivariate Signal Detection Model for Selection DeCarlo (2003). We also included rating-based ROCs. The results suggest that both models are at least equally valid as measurement tools which capture bias processes in the corresponding parameters, and they can perhaps be integrated theoretically.     

Dimensions in appreciation of car interior design

We report two studies in which the interplay between stimulus properties and perceiver characteristics in the appreciation car interiors was investigated. In Experiment 1 three design components, complexity, curvature and innovativeness, which are all thought to affect design appreciation were combined in a fully factorial design. All dimensions were confirmed to affect ratings, and curvature and innovativeness particularly affected the attractiveness ratings. Curved and non‐innovative designs were generally preferred. Moreover, participants who were particularly interested in art were more sensitive to curvature and innovativeness. In Experiment 2 two dimensions of Experiment 1 were replicated using similar stimuli. Moreover, the specific effects of a design knowledge treatment were investigated. Results replicated the preference for curved and non‐innovative (rather classic) designs. The treatment had only small effects, which support a general rather than dimension‐specific effects of cognitive pre‐information. Copyright © 2005 John Wiley & Sons, Ltd.     

Recognition memory for faces: When familiarity supports associative recognition judgments

Recognition memory for single items can be dissociated from recognition memory for the associations between items. For example, recognition tests for single words produce curvilinear receiver operating characteristics (ROCs), but associative recognition tests for word pairs produce linear ROCs. These dissociations are consistent with dual-process theories of recognition and suggest that associative recognition relies on recollection but that item recognition relies on a combination of recollection and assessments of familiarity. In the present study, we examined associative recognition ROCs for facial stimuli by manipulating the central and external features, in order to determine whether linear ROCs would be observed for stimuli other than arbitrary word pairs. When the faces were presented upright, familiarity estimates were significantly above zero, and the associative ROCs were curvilinear, suggesting that familiarity contributed to associative judgments. However, presenting the faces upside down effectively eliminated the contribution of familiarity to associative recognition, and the ROCs were linear. The results suggest that familiarity can support associative recognition judgments, if the associated components are encoded as a coherent gestalt, as in upright faces.