Support for a continuous (single-process) model of recognition memory and source memory

Does memory retrieval occur in a continuous or an all-or-none manner? The shape of the receiver operating characteristic (ROC) has been used to answer this question, with curvilinear and linear memory ROCs indicating continuous and all-or-none retrieval processes, respectively. Signal detection models (e.g., the unequal variance model) correspond to a continuous retrieval process, whereas threshold models (including the multinomial model and the recollection component of the dual-process model) correspond to an all-or-none process. In studies of source memory, Slotnick et al. (2000) and others have observed curvilinear ROCs (supporting the unequal variance model), whereas Yonelinas (1999) observed linear ROCs (supporting the dual-process model). We resolve these seemingly inconsistent results, showing that source memory ROCs are naturally curvilinear but can appear linear when nondiagnostic source information is included in the analysis. Furthermore, the unequal variance model accounted for both recognition memory and source memory ROCs, supporting a continuous process of memory retrieval.     

The influence of averaging and noisy decision strategies on the recognition memory ROC

Many single- and dual-process models of recognition memory predict that the ratings and remember-know receiver operating characteristics (ROCs) are the same, but Rotello, Macmillan, and Reeder (2004) reported that the slopes of the remember-know and ratingsz-transformed ROCs (zROCs) are different. The authors show that averaging introduces nonlinearities to the form of thezROC and that ratings and remember-knowzROCs are indistinguishable when constructed in a conventional manner. The authors show, further, that some nonoptimal decision strategies have a distinctive, nonlinear effect on the form of the single-process continuous-statezROC. The conclusion is that many factors having nothing to do with the nature of recognition memory can affect the shape ofzROCs, and that therefore, the shape of thezROC does not, alone, characterize different memory models.     

The belief bias effect is aptly named: a reply to Klauer and Kellen (2011)

In "Assessing the Belief Bias Effect With ROCs: It's a Response Bias Effect," Dube, Rotello, and Heit (2010) examined the form of receiver operating characteristic (ROC) curves for reasoning and the effects of belief bias on measurement indices that differ in whether they imply a curved or linear ROC function. We concluded that the ROC data are in fact curved and that analyses using statistics that assume a linear ROC are likely to produce Type I errors. Importantly, we showed that the interaction between logic and belief that has inspired much of the theoretical work on belief bias is in fact an error stemming from inappropriate reliance on a contrast (hit rate-false alarm rate) that implies linear ROCs. Dube et al. advanced a new model of belief bias, which, in light of their data, is currently the only plausible account of the effect. Klauer and Kellen (2011) disputed these conclusions, largely on the basis of speculation about the data collection method used by Dube et al. to construct the ROCs. New data and model-based analyses are presented that refute the speculations made by Klauer and Kellen. We also show that new modeling results presented by Klauer and Kellen actually support the conclusions advanced by Dube et al. Together, these data show that the methods used by Dube et al. are valid and that the belief bias effect is simply a response bias effect.     

ROC in animals: Uncovering the neural substrates of recollection and familiarity in episodic recognition memory

It is a consensus that familiarity and recollection contribute to episodic recognition memory. However, it remains controversial whether familiarity and recollection are qualitatively distinct processes supported by different brain regions, or whether they reflect different strengths of the same process and share the same support. In this review, I discuss how adapting standard human recognition memory paradigms to rats, performing circumscribed brain lesions and using receiver operating characteristic (ROC) methods contributed to solve this controversy. First, I describe the validation of the animal ROC paradigms and report evidence that familiarity and recollection are distinct processes in intact rats. Second, I report results from rats with hippocampal dysfunction which confirm this finding and lead to the conclusion that the hippocampus supports recollection but not familiarity. Finally, I describe a recent study focusing on the medial entorhinal cortex (MEC) that investigates the contribution of areas upstream of the hippocampus to recollection and familiarity.     

The contribution of recollection and familiarity to recognition and source-memory judgments: a formal dual-process model and an analysis of receiver operating characteristics

A formal dual-process model that assumes that memory judgments can be based on a threshold recollection process and a signal-detection-based familiarity process is proposed to account for both recognition and source-memory performance. The model was tested in 4 experiments by examining recognition and source-memory receiver operating characteristics (ROCs). In agreement with the predictions of the model, recognition and source memory dissociated in certain conditions. Recognition ROCs were curvilinear in probability space and relatively linear in z-space, as expected if recollection and familiarity contributed to performance. In contrast, source ROCs typically were linear and exhibited a pronounced U shape in z-space, as expected if performance primarily relied on recollection. However, in conditions in which familiarity was clearly indicative of an item's source, the source ROC became curvilinear, suggesting that participants could use familiarity as a basis for source judgments. Several alternative models, including the unequal-variance signal-detection model, were found to be inconsistent with the ROC data.     

Recollection is a continuous process: Evidence from plurality memory receiver operating characteristics

Is recollection a continuous/graded process or a threshold/all-or-none process? Receiver operating characteristic (ROC) analysis can answer this question as the continuous model and the threshold model predict curved and linear recollection ROCs, respectively. As memory for plurality, an item's previous singular or plural form, is assumed to rely on recollection, the nature of recollection can be investigated by evaluating plurality memory ROCs. The present study consisted of four experiments. During encoding, words (singular or plural) or objects (single/singular or duplicate/plural) were presented. During retrieval, old items with the same plurality or different plurality were presented. For each item, participants made a confidence rating ranging from “very sure old”, which was correct for same plurality items, to “very sure new”, which was correct for different plurality items. Each plurality memory ROC was the proportion of same versus different plurality items classified as “old” (i.e., hits versus false alarms). Chi-squared analysis revealed that all of the plurality memory ROCs were adequately fit by the continuous unequal variance model, whereas none of the ROCs were adequately fit by the two-high threshold model. These plurality memory ROC results indicate recollection is a continuous process, which complements previous source memory and associative memory ROC findings.     

Searching for the sequential line-up advantage: A distinctiveness explanation

Signal Detection models as well as the Two-High-Threshold model (2HTM) have been used successfully as measurement models in recognition tasks to disentangle memory performance and response biases. A popular method in recognition memory is to elicit confidence judgements about the presumed old/new status of an item, allowing for the easy construction of ROCs. Since the 2HTM assumes fewer latent memory states than response options are available in confidence ratings, the 2HTM has to be extended by a mapping function which models individual rating scale usage. Unpublished data from 2 experiments in Bröder and Schütz (2009) validate the core memory parameters of the model, and 3 new experiments show that the response mapping parameters are selectively affected by manipulations intended to affect rating scale use, and this is independent of overall old/new bias. Comparisons with SDT show that both models behave similarly, a case that highlights the notion that both modelling approaches can be valuable (and complementary) elements in a researcher's toolbox.     

Signal detection and threshold models of source memory

Recently, several authors claimed that the curvilinear shape of rating-based source memory receiver operating characteristics (ROCs) refutes threshold models. However, rating-based ROCs are not diagnostic to disprove threshold models. Furthermore, source memory ROC-analyses ignore influences of other processes like old-new-detection and old-new-response-tendencies, so direct estimation of parameters is preferable. Five source monitoring experiments with different response bias manipulations and materials were conducted. We fitted the Two High Threshold Multinomial Model of Source Monitoring by Bayen, Murnane, and Erdfelder (1996) and the Multivariate Signal Detection Model for Selection DeCarlo (2003). We also included rating-based ROCs. The results suggest that both models are at least equally valid as measurement tools which capture bias processes in the corresponding parameters, and they can perhaps be integrated theoretically.     

Signal-Detection, Threshold, and Dual-Process Models of Recognition Memory: ROCs and Conscious Recollection

Threshold- and signal-detection-based models have dominated theorizing about recognition memory. Building upon these theoretical frameworks, we have argued for a dual-process model in which conscious recollection (a threshold process) and familiarity (a signal-detection process) contribute to memory performance. In the current paper we assessed several memory models by examining the effects of levels of processing and the number of presentations on recognition memory receiver operating characteristics (ROCs). In general, when the ROCs were plotted in probability space they exhibited an inverted U shape; however, when they were plotted inzspace they exhibited a U shape. An examination of the ROCs showed that the dual-process model could account for the observed ROCs, but that models based solely on either threshold or signal-detection processes failed to provide a sufficient account of the data. Furthermore, an examination of subjects' introspective reports using the remember/know procedure showed that subjects were aware of recollection and familiarity and were able to consistently report on their occurrence. The remember/know data were used to accurately predict the shapes of the ROCs, and estimates of recollection and familiarity derived from the ROC data mirrored the subjective reports of these processes.     

Traditional difference-score analyses of reasoning are flawed

Studies of the belief bias effect in syllogistic reasoning have relied on three traditional difference score measures: the logic index, belief index, and interaction index. Dube, Rotello, and Heit (2010, 2011) argued that the interaction index incorrectly assumes a linear receiver operating characteristic (ROC). Here, all three measures are addressed. Simulations indicated that traditional analyses of reasoning experiments are likely to lead to incorrect conclusions. Two new experiments examined the role of instructional manipulations on the belief bias effect. The form of the ROCs violated assumptions of traditional measures. In comparison, signal detection theory (SDT) model-based analyses were a better match for the form of the ROCs, and implied that belief bias and instructional manipulations are predominantly response bias effects. Finally, reanalyses of previous studies of conditional reasoning also showed non-linear ROCs, violating assumptions of traditional analyses. Overall, reasoning research using traditional measures is at risk of drawing incorrect conclusions.     

Binary ROCs in perception and recognition memory are curved

In recognition memory, a classic finding is that receiver operating characteristics (ROCs) are curvilinear. This has been taken to support the fundamental assumptions of signal detection theory (SDT) over discrete-state models such as the double high-threshold model (2HTM), which predicts linear ROCs. Recently, however, Br?der and Schütz (2009) challenged this argument by noting that most of the data on which support for SDT is based have involved confidence ratings. The authors argued that certain types of rating scale usage may result in curved ROCs even if the generating process is thresholded in nature. From this point of view, only ROCs constructed via experimental bias manipulations are useful for discriminating between the models. Br?der and Schütz conducted a meta-analysis and new experiments that compared SDT and the 2HTM using binary (yes-no) ROCs and found that many of these functions were linear, supporting 2HTM over SDT. We examine all the data reported by Br?der and Schütz, noting important limitations in their methodology, analyses, and conclusions. We report a new meta-analysis and 2 new experiments to examine the issue more closely while avoiding the limitations of Br?der and Schütz's study. These new data indicate that binary ROCs are curved in recognition, consistent with previous findings in perception and reasoning. Our results support classic arguments in favor of SDT and indicate that curvature in ratings ROCs is not task specific. We recommend the ratings procedure and suggest that analyses based on threshold models be treated with caution.     

Recognition memory for faces: When familiarity supports associative recognition judgments

Recognition memory for single items can be dissociated from recognition memory for the associations between items. For example, recognition tests for single words produce curvilinear receiver operating characteristics (ROCs), but associative recognition tests for word pairs produce linear ROCs. These dissociations are consistent with dual-process theories of recognition and suggest that associative recognition relies on recollection but that item recognition relies on a combination of recollection and assessments of familiarity. In the present study, we examined associative recognition ROCs for facial stimuli by manipulating the central and external features, in order to determine whether linear ROCs would be observed for stimuli other than arbitrary word pairs. When the faces were presented upright, familiarity estimates were significantly above zero, and the associative ROCs were curvilinear, suggesting that familiarity contributed to associative judgments. However, presenting the faces upside down effectively eliminated the contribution of familiarity to associative recognition, and the ROCs were linear. The results suggest that familiarity can support associative recognition judgments, if the associated components are encoded as a coherent gestalt, as in upright faces.     

Toward a complete decision model of item and source recognition

In a recognition memory test, subjects may be asked to decide whether a test item is old or new (item recognition) or to decide among alternative sources from which it might have been drawn for study (source recognition). Confidence-rating-based receiver operating characteristic (ROC) curves for these superficially similar tasks are quite different, leading to the inference of correspondingly different decision processes. A complete account of source and item recognition will require a single model that can be fit to the entire data set. We postulated a detection-theoretic decision space whose dimensions, in the style of Banks (2000), are item strength and the relative strengths of the two sources. A model that assumes decision boundaries of constant likelihood ratios, source guessing for unrecognized items, and nonoptimal allocation of attention can account for data from three canonical data sets without assuming any processes specifically devoted to recollection. Observed and predicted ROCs for one of these data sets are given in the article, and ROCs for the other two may be downloaded from the Psychonomic Society’s Archive of Norms, Stimuli, and Data, www.psychonomic.org/archive.     

Recognition memory ROCs for item and associative information: The contribution of recollection and familiarity

Receiver-operating characteristics (ROCs) were examined in three recognition memory experiments. ROCs for item information (i.e., was this word presented?) were found to be curvilinear. However, ROCs for associative information (i.e., were these two words presented together?) were found to be linear. The results are in agreement with the predictions of a dual-process model that assumes that recognition judgments are -based on familiarity and recollection. Familiarity reflects the assessment of a continuous strength dimension and is well described as a signal detection process, whereas recollection reflects the retrieval of qualitative information about the study episode and behaves like a discrete threshold process. The results showed that memory judgments about items relied on a combination of recollection and familiarity, but that judgments about associations relied primarily on recollection. Further examination of the associative ROCs suggested that subjects were able to recollect that old pairs of items were in the study list, and, under some conditions, that new pairs were not in the study list.     

Consciousness, control, and confidence: the 3 Cs of recognition memory

The contributions of recollection and familiarity to recognition memory performance were examined using the process dissociation, remember-know, and receiver operating characteristic (ROC) procedures. Under standard test conditions the 3 measurement procedures led to process estimates that were almost identical and to similar conclusions regarding the effects of different encoding manipulations. Dividing attention led to a large decrease in recollection and a smaller, sometimes nonsignificant, decrease in familiarity. Semantic compared with perceptual processing led to a large increase in recollection and a moderate increase in familiarity. Moreover, the results showed that familiarity was well described by classical signal-detection theory but that recollection reflected a threshold process. The convergence observed across the 3 measurement procedures shows that the 3 procedures tap similar underlying processes and that recollection and familiarity differ in terms of conscious awareness, intentional control, and the manner in which they contribute to the shape of response confidence ROCs.     

Item recognition memory and the receiver operating characteristic

Four experiments were conducted to investigate the effects of study time, study repetition, semantic and orthographic similarity, and category length on item recognition memory receiver operating characteristics (ROCs). Analyses of ROC shape rejected A. P. Yonelinas's (1994) dual-process model. The normal unequal variance signal-detection model provided a better account of the data, except for a small but consistent excess of high-confidence errors. It was found that z-transformed ROC slope was increased by similarity, category length, and study item repetition, rejecting R. Ratcliff, G. McKoon, and M. Tindall's (1994) "constancy-of-slopes" generalization for these variables, but slope was relatively unaffected by massed study time.     

Cross-task and cross-manipulation stability in shifting the decision criterion

In recognition memory experiments participants must discriminate between old and new items, a judgment influenced by response bias. Research has shown substantial individual differences in the extent to which people will strategically adjust their response bias to diagnostic cues such as the prior probability of an old item. Despite this significant between subject variability, shifts in bias have been found to be relatively predictive within individuals across memory tests. Experiment 1 sought to determine whether this predictability extends beyond memory. Results revealed that the amount a subject shifted response bias in a recognition memory task was significantly predictive of shifting in a visual perception task, suggesting that shifting can generalise outside of a specific testing domain. Experiment 2 sought to determine how predictive shifting would be across two manipulations well known to induce shifts in bias: a probability manipulation and a response payoff manipulation. A modest positive relationship between these two methods was observed, suggesting that shifting behaviour is relatively predictive across different manipulations of shifting. Overall, results from both experiments suggest that response bias shifting, like response bias setting, is a relatively stable behaviour within individuals despite changes in test domain and test manipulation.     

ROCs in Eyewitness Identification: Instructions versus Confidence Ratings

From the perspective of signal detection theory, different lineup instructions may induce different levels of response bias. If so, then collecting correct and false identification rates across different instructional conditions will trace out the receiver operating characteristic (ROC)—the same ROC that, theoretically, could also be traced out from a single instruction condition in which each eyewitness decision is accompanied by a confidence rating. We tested whether the two approaches do in fact yield the same ROC. Participants were assigned to a confidence rating condition or to an instructional biasing condition (liberal, neutral, unbiased, or conservative). After watching a video of a mock crime, participants were presented with instructions followed by a six‐person simultaneous photo lineup. The ROCs from both methods were similar, but they were not exactly the same. These findings have potentially important policy implications for how the legal system should go about controlling eyewitness response bias.Copyright © 2017 John Wiley & Sons, Ltd.     

Problematic effects of aggregation in z ROC analysis and a hierarchical modeling solution

The confidence-rating recognition memory paradigm has been used extensively for testing theories of memory. Current methods for analyzing confidence-rating data require that data be aggregated over participants, items, or both. We show that this aggregation threatens claims of curvature in zROCs and distorts estimates of signal-distribution variance—the very findings that are used to make fine-grained distinctions between competing models. We develop a hierarchical signal detection model that accounts for variability from the sampling of participants and items in addition to variability from the underlying psychological processes. The model provides for accurate signal detection parameter estimates. With accurate estimates, the validity of benchmark findings in recognition memory may be assessed.     

Receiver operating characteristics (ROCs) in recognition memory: a review

Receiver operating characteristic (ROC) analysis is being used increasingly to examine the memory processes underlying recognition memory. The authors discuss the methodological issues involved in conducting and analyzing ROC results, describe the various models that have been developed to account for these results, review the behavioral empirical literature, and assess the models in light of those results. The empirical literature includes studies of item recognition, relational recognition (e.g., source and associative tests), as well as exclusion and remember-know tasks. Nine empirical regularities are described, and a number of unresolved empirical issues are identified. The results indicate that several common classes of recognition models, such as pure threshold and pure signal detection models, are inadequate to account for recognition memory, whereas several hybrid models that incorporate a signal detection-based process and a threshold recollection or attention process are in better agreement with the results. The results indicate that there are at least 2 functionally distinct component/processes underlying recognition memory. In addition, the ROC results have various implications for how recognition memory performance should be measured.