ROC and z-ROC analysis in recognition memory: continuous, threshold, and asymmetric models

In this research, we applied the operating characteristic of the receiver (ROC) to evaluate item-recognition memory by means of open class words (CA) and closed class words (CC). The purpose was to evaluate memory processes and models, according to the type of stimuli used: words in Spanish, classified according to grammar characteristics. To register the data, a rating scale was applied to obtain the appropriate response levels. Statistically significant differences between the processing of both types of words were obtained. We evaluated models of threshold, continuous, and unequal variance. As a result, we could contribute to the resolution of the present controversy about the models adapted to evaluate the memory of item recognition. Lastly, the differentiation and integration of the models from an empirical point of view is discussed.     

ROC parameters in item and context recognition

Conflicting theories argue that recognition is achieved either by familiarity exclusively, or by a mixture of familiarity and recollection. We explore in three experiments the goodness of fit of both positions to experimental data in which context information is manipulated. In Experiments 1 and 2, we explore the availability of context information in recognition, testing the focus stimulus, its context, and their associative relation. In Experiment 3, participants were confronted with a plurality task in an attempt to force them to use the peripheral information in recognition. The results show that people acquire specific associative information, and although overall recognition performance was not affected by the use of context, receiver operating characteristic (ROC) analysis showed that people use a duality of processes in recognition.     

ROC curves and confidence judgements in recognition memory

Most models of recognition memory rely on a strength/familiarity-based signal detection account that assumes that the processes giving rise to a confidence judgment are the same as those giving rise to an old-new decision. Confidence is assumed to be scaled directly from the perceived familiarity of a probe. This assumption was tested in 2 experiments that examine the shape of confidence-based z receiver operating characteristic (zROC) curves under different levels of response bias induced by changing stimulus probabilities (Experiment 1) and payoffs (Experiment 2). Changes in the shape of the zROC curves with bias indicate that confidence is not scaled directly from perceived familiarity or likelihood. A model of information accumulation in recognition memory is proposed that can account for the observed effects.     

Context, remember-know recognition judgements, and ROC parameters

Recent work (e.g., Dunn, 2004; Heathcote, 2003) has questioned the necessity of postulating two processes to explain recognition memory. As part of this trend, strength theories of the remember-know methodology have gained in support. We present three experiments with pictorial material in which we force participants to use differential contextual information at test. Participants were required to give remember-know judgements and confidence ratings for each test stimulus. Hits, false alarms, remember-know data, and discrimination indices indicated systematic variations as a function of the availability and use of contextual information. Moreover, when we normalised the receiver operating characteristic data in terms of z-scores, the slopes were lower than 1 and slightly concave. Additionally, we computed the same set of statistical indices suggested by Wixted and Stretch (2004), with mixed results. Overall, we think that the data support a two-factor theory of remember-know and recognition, although many results fit well signal detection views of the task. Finally, the idea that remember and know responses are pure manifestations of recollection and familiarity seems difficult to sustain. We think that a productive use of the remember-know methodology involves the minimisation of the bias factors that may contaminate the responses, in addition to the introduction of the experimental manipulations needed to promote recollective and/or familiarity processes.     

Phantom ROC: Recollection rejection in a hybrid conjoint recognition signal detection model

The present experiments test predictions of dual process models with regards to recollection rejection using a model called phantom ROC. Participants studied pictures and then took a recognition test in which they were presented with targets (i.e., the exact picture presented), related lures (i.e., the same object but from a different angle), and unrelated lures (i.e., objects that had not been shown). For each item, participants answered both standard recognition and meaning recognition questions. In Experiment 1 participants studied pictures under either full attention or divided attention. In Experiment 2 some participants were told that no object was shown twice (i.e., mutual exclusivity instructions), while others were told that both camera angles might have been shown for some objects (no mutual exclusivity instructions). The present experiments provide three converging measures, all of which are consistent with dual process models that propose a recollection rejection mechanism.     

Showups versus lineups: An evaluation using ROC analysis

Showups (a one-person identification) were compared to both simultaneous and sequential lineups that varied in lineup fairness and the position of the suspect in the lineup. We reanalyzed data from a study by Gronlund, Carlson, Dailey, and Goodsell (2009), which included simultaneous and sequential lineups, and using the same stimuli and procedures, collected new data using showup identifications. Performance was compared using ROC analysis, which is superior to traditional measures such as correct and false identification rates, and probative value measures. ROC analysis showed that simultaneous lineups consistently produced more accurate identification evidence than showups, but sequential lineups were sometimes no more accurate than showups, and were never more accurate than simultaneous lineups. These results supported prior suppositions regarding the suggestiveness of showups, revealed a misconception about the superiority of sequential lineups, and demonstrated why eyewitness identification procedures need to be evaluated using ROC analyses.     

Cyclic changes in pain perception: An ROC analysis

The rating scale responses of three groups of men and women to thermal and painful radiant heat stimuli were studied over a 4-week period. Receiver operating characteristic curve parameters were computed for each of the phases of the menstrual cycle. Women with normal menstrual periods experienced a heightened sensitivity to pain during ovulation. This effect was absent in women using oral contraceptive pills and in men. The findings demonstrate that past reports of pain threshold changes in association with menstrual phase were basically sensory effects and not just a result of shifts in the observer’s criteria for reporting pain. In addition, sex differences in pain perception were found only when men were compared with women who were ovulating.     

Eyewitness identification discriminability: ROC analysis versus logistic regression

To reach conclusions regarding the respective accuracy of two conditions, eyewitness researchers evaluate correct and false identification rates computed across participants. Two approaches typically are employed. One approach relies on ratio-based probative value measures; but Wixted and Mickes (2012) and Gronlund, Wixted, and Mickes (2014) showed that these measures fail to disentangle an assessment of accuracy (i.e., discriminability between guilty and innocent suspects) from response bias (i.e., a willingness to make a response). Our focus is on a second approach, logistic regression analyses of the correct and of the false identification rates. Logistic regression also fails to disentangle discriminability from bias. Therefore, it only can denote the most accurate condition in limited circumstances. The best approach for reaching the proper conclusion regarding which condition is most accurate is to use receiver operator characteristic (ROC) analysis. Simulated ROC data illustrate the problem with a reliance on logistic regression to assess accuracy.     

Six regularities of source recognition

In recent work, researchers have shown that source-recognition memory can be incorporated in an extended signal detection model that covers both it and item-recognition memory (A. Hilford, M. Glanzer, K. Kim, & L. T. DeCarlo, 2002). In 5 experiments, using learning variables that have an established effect on item recognition, the authors tested further implications of that extended model. The results establish 6 source-recognition regularities that support the model. An important regularity in that set is the linkage of source and item recognition: Any learning variable that increases item recognition increases source recognition.     

ROC in animals: Uncovering the neural substrates of recollection and familiarity in episodic recognition memory

It is a consensus that familiarity and recollection contribute to episodic recognition memory. However, it remains controversial whether familiarity and recollection are qualitatively distinct processes supported by different brain regions, or whether they reflect different strengths of the same process and share the same support. In this review, I discuss how adapting standard human recognition memory paradigms to rats, performing circumscribed brain lesions and using receiver operating characteristic (ROC) methods contributed to solve this controversy. First, I describe the validation of the animal ROC paradigms and report evidence that familiarity and recollection are distinct processes in intact rats. Second, I report results from rats with hippocampal dysfunction which confirm this finding and lead to the conclusion that the hippocampus supports recollection but not familiarity. Finally, I describe a recent study focusing on the medial entorhinal cortex (MEC) that investigates the contribution of areas upstream of the hippocampus to recollection and familiarity.     

Isolating the contributions of familiarity and source information to item recognition: a time course analysis.

Recognition memory may be mediated by the retrieval of distinct types of information, notably, a general assessment of familiarity and the recovery of specific source information. A response-signal speed-accuracy trade-off variant of an exclusion procedure was used to isolate the retrieval time course for familiarity and source information. In 2 experiments, participants studied spoken and read lists (with various numbers of presentations) and then performed an exclusion task, judging an item as old only if it was in the heard list. Dual-process fits of the time course data indicated that familiarity information typically is retrieved before source information. The implications that these data have for models of recognition, including dual-process and global memory models, are discussed.     

Using ROC curves to test models of recognition memory: The relationship between presentation duration and slope

Many theories of memory predict that the slope of the z-transformed receiver-operating characteristic (ROC) curve should decrease with increases in the memory strength of old items. While several prior studies have failed to demonstrate this effect, the results of two experiments demonstrate that increasing presentation duration can reduce the slope of the z-transformed ROC curve. These results raise questions about the generality of prior results and of dual-process theories designed to accommodate those results. We close by emphasizing that determining the experimental circumstances that affect whether or not the slope of the z-transformed ROC curve decreases will provide important constraints on theories of recognition memory.     

The influence of averaging and noisy decision strategies on the recognition memory ROC

Many single- and dual-process models of recognition memory predict that the ratings and remember-know receiver operating characteristics (ROCs) are the same, but Rotello, Macmillan, and Reeder (2004) reported that the slopes of the remember-know and ratingsz-transformed ROCs (zROCs) are different. The authors show that averaging introduces nonlinearities to the form of thezROC and that ratings and remember-knowzROCs are indistinguishable when constructed in a conventional manner. The authors show, further, that some nonoptimal decision strategies have a distinctive, nonlinear effect on the form of the single-process continuous-statezROC. The conclusion is that many factors having nothing to do with the nature of recognition memory can affect the shape ofzROCs, and that therefore, the shape of thezROC does not, alone, characterize different memory models.     

Variations in recollection: the effects of complexity on source recognition

Whether recollection is a threshold or signal detection process is highly controversial, and the controversy has centered in part on the shape of receiver operating characteristics (ROCs) and z-transformed ROCs (zROCs). U-shaped zROCs observed in tests thought to rely heavily on recollection, such as source memory tests, have provided evidence in favor of the threshold assumption, but zROCs are not always as U-shaped as threshold theory predicts. Source zROCs have been shown to become more linear when the contribution of familiarity to source discriminations is increased, and this may account for the existing results. However, another way in which source zROCs may become more linear is if the recollection threshold begins to break down and recollection becomes more graded and Gaussian. We tested the "graded recollection" account in the current study. We found that increasing stimulus complexity (i.e., changing from single words to sentences) or increasing source complexity (i.e., changing the sources from audio to videos of speakers) resulted in flatter source zROCs. In addition, conditions expected to reduce recollection (i.e., divided attention and amnesia) had comparable effects on source memory in simple and complex conditions, suggesting that differences between simple and complex conditions were due to differences in the nature of recollection, rather than differences in the utility of familiarity. The results suggest that under conditions of high complexity, recollection can appear more graded, and it can produce curved ROCs. The results have implications for measurement models and for current theories of recognition memory.     

Cross-modal source information and spoken word recognition

In a cross-modal matching task, participants were asked to match visual and auditory displays of speech based on the identity of the speaker. The present investigation used this task with acoustically transformed speech to examine the properties of sound that can convey cross-modal information. Word recognition performance was also measured under the same transformations. The authors found that cross-modal matching was only possible under transformations that preserved the relative spectral and temporal patterns of formant frequencies. In addition, cross-modal matching was only possible under the same conditions that yielded robust word recognition performance. The results are consistent with the hypothesis that acoustic and optical displays of speech simultaneously carry articulatory information about both the underlying linguistic message and indexical properties of the talker.     

False recognition occurs more frequently during source identification than during old-new recognition

A number of recent reports have investigated false memories using variants of the Deese-Roediger-McDermott (DRM) paradigm. Because these false memories have been difficult to eliminate, this study investigated whether false recognition could be reduced by incorporating source-monitoring criteria into decision processes. Making claims about the manner in which items were learned should require more careful scrutiny of memories, and therefore false recognition should be minimized with source instructions as compared with old-new recognition instructions. In 3 experiments that varied the combination of sources, false recognition was increased rather than reduced by applying source-monitoring processes. The theoretical implications of these counterintuitive results are discussed in terms of the old-new detection component of source judgments.     

A note on ROC analysis and non-parametric estimate of sensitivity

In the signal detection paradigm, the non-parametric index of sensitivity A′, as first introduced by Pollack and Norman (1964), is a popular alternative to the more traditional d′ measure of sensitivity. Smith (1995) clarified a confusion about the interpretation of A′ in relation to the area beneath proper receiver operating characteristic (ROC) curves, and provided a formula (which he called A′′) for this commonly held interpretation. However, he made an error in his calculations. Here, we rectify this error by providing the correct formula (which we call A) and compare the discrepancy that would have resulted. The corresponding measure for bias b is also provided. Since all such calculations apply to “proper” ROC curves with non-decreasing slopes, we also prove, as a separate result, the slope-monotonicity of ROC curves generated by likelihood-ratio criterion.This revised article was published online in August 2005 with the PDF paginated correctly.     

Modes of cognitive control in recognition and source memory: Depth of retrieval

Recognition memory is usually regarded as a judgment based on trace strength or familiarity. But recognition may also be accomplished by constraining retrieval so that only sought after information comes to mind (source-constrained retrieval). We introduce amemory-for-foils paradigm that provides evidence for source-constrained retrieval in recognition memory (Experiment 1) and source memory (Experiment 2). In this paradigm, subjects studied words under deep or shallow encoding conditions and were given a memory test (recognition or source) that required them to discriminate between new items (foils) and either deep or shallow targets. A final recognition test was used to examine memory for the foils. In both experiments, foil memory was superior when subjects attempted to retrieve deep rather than shallow targets on the earlier test. These findings support a sourceconstrained retrieval view of cognitive control by demonstrating qualitative differences in the basis for memory performance.