On the form of ROCs constructed from confidence ratings

A classical question for memory researchers is whether memories vary in an all-or-nothing, discrete manner (e.g., stored vs. not stored, recalled vs. not recalled), or whether they vary along a continuous dimension (e.g., strength, similarity, or familiarity). For yes-no classification tasks, continuous- and discrete-state models predict nonlinear and linear receiver operating characteristics (ROCs), respectively (D. M. Green & J. A. Swets, 1966; N. A. Macmillan & C. D. Creelman, 1991). Recently, several authors have assumed that these predictions are generalizable to confidence ratings tasks (J. Qin, C. L. Raye, M. K. Johnson, & K. J. Mitchell, 2001; S. D. Slotnick, S. A. Klein, C. S. Dodson, & A. P. Shimamura, 2000, and A. P. Yonelinas, 1999). This assumption is shown to be unwarranted by showing that discrete-state ratings models predict both linear and nonlinear ROCs. The critical factor determining the form of the discrete-state ROC is the response strategy adopted by the classifier.     

Long-term memory in amnesia: cued recall, recognition memory, and confidence ratings

We explored what kind of information is acquired when amnesic patients are able to exhibit significant retention on tests of cued recall and recognition memory. Amnesic patients and control subjects attempted to learn sets of sentences. Memory for the last word in each sentence was tested after 1 hr in the case of the amnesic patients, or after 1 to 2 weeks in the case of (delayed) control subjects. Amnesic patients and (delayed) control subjects performed at similar levels on tests of cued recall and recognition memory. Amnesic patients were just as confident of their correct answers as were control subjects. However, amnesic patients were no more disadvantaged than control subjects when they were cued indirectly by presenting paraphrases of the original sentences. These findings demonstrate that the residual knowledge retained by amnesic patients can be as flexible, as accessible to indirect cues, and as available to awareness as the knowledge retained by (delayed) control subjects.     

Selecting accurate statements from the cognitive interview using confidence ratings

Participants viewed a videotape of a simulated murder, and their recall (and confidence) was tested 1 week later with the cognitive interview. Results indicated that (a) the subset of statements assigned high confidence was more accurate than the full set of statements; (b) the accuracy benefit was limited to information that forensic experts considered relevant to an investigation, whereas peripheral information showed the opposite pattern; (c) the confidence-accuracy relationship was higher for relevant than for peripheral information; (d) the focused-retrieval phase was associated with a greater proportion of peripheral and a lesser proportion of relevant information than the other phases; and (e) only about 50% of the relevant information was elicited, and most of this was elicited in Phase 1.     

Organization revealed by recall orders and confirmed by pauses

The technique introduced here induces the organization of information in memory from systematic inspection of regularities in free recall. The form of the representation of this organization is an “ordered tree.” The technique has the advantage of being based on a theory of the way in which the data were generated and can be shown to produce a unique structure that captures all the kinds of regularities the theory of recall prescribes. Also presented is a collateral technique for measuring the amount of organization evidenced in a struture, as well as a procedure for identifying errors. The experimental work shows the technique's ability to recover the details of an organization presented to subjects and provides converging evidence for the particular structures induced from the pattern of recall pauses. In addition, the application of the technique to structures unknown a priori produced organizations that were easy to interpret and a second set of pauses that further confirmed the details of the induced structures.     

A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings

How should we measure metacognitive (“type 2”) sensitivity, i.e. the efficacy with which observers’ confidence ratings discriminate between their own correct and incorrect stimulus classifications? We argue that currently available methods are inadequate because they are influenced by factors such as response bias and type 1 sensitivity (i.e. ability to distinguish stimuli). Extending the signal detection theory (SDT) approach of Galvin, Podd, Drga, and Whitmore (2003), we propose a method of measuring type 2 sensitivity that is free from these confounds. We call our measure meta-d′, which reflects how much information, in signal-to-noise units, is available for metacognition. Applying this novel method in a 2-interval forced choice visual task, we found that subjects’ metacognitive sensitivity was close to, but significantly below, optimality. We discuss the theoretical implications of these findings, as well as related computational issues of the method. We also provide free Matlab code for implementing the analysis.     

Temporal associative processes revealed by intrusions in paired-associate recall

Although much is known about the factors that influence the acquisition and retention of individual paired associates, the existence of temporally defined associations spanning multiple pairs has not been demonstrated. We report two experiments in which subjects studied randomly paired nouns for a subsequent cued recall test. When subjects recalled nontarget items, their intrusions tended to come from nearby pairs. This across-pair contiguity effect was graded, spanning noncontiguously studied word pairs. The existence of such long-range temporally defined associations lends further support to contextual-retrieval models of episodic association.     

A comparison of type II operating characteristics derived from confidence ratings and from latencies

On each of a series of trials, S judged two stimuli to be the same or different. S then made a confidence rating of his judgment. The difficulty of discrimination and feedback of response information (latency and correctness) were manipulated. Feedback had no effect. Latencies conditional on confidence rating decreased with increasing confidence and remained relatively constant with changes in discrimination difficulty. Type II operating characteristics were derived from latencies and from confidence ratings; both appropriately reflected differences in task difficulty.     

The ROC Toolbox: A toolbox for analyzing receiver-operating characteristics derived from confidence ratings

Signal-detection theory, and the analysis of receiver-operating characteristics (ROCs), has played a critical role in the development of theories of episodic memory and perception. The purpose of the current paper is to present the ROC Toolbox. This toolbox is a set of functions written in the Matlab programming language that can be used to fit various common signal detection models to ROC data obtained from confidence rating experiments. The goals for developing the ROC Toolbox were to create a tool (1) that is easy to use and easy for researchers to implement with their own data, (2) that can flexibly define models based on varying study parameters, such as the number of response options (e.g., confidence ratings) and experimental conditions, and (3) that provides optimal routines (e.g., Maximum Likelihood estimation) to obtain parameter estimates and numerous goodness-of-fit measures.The ROC toolbox allows for various different confidence scales and currently includes the models commonly used in recognition memory and perception: (1) the unequal variance signal detection (UVSD) model, (2) the dual process signal detection (DPSD) model, and (3) the mixture signal detection (MSD) model. For each model fit to a given data set the ROC toolbox plots summary information about the best fitting model parameters and various goodness-of-fit measures. Here, we present an overview of the ROC Toolbox, illustrate how it can be used to input and analyse real data, and finish with a brief discussion on features that can be added to the toolbox.     

Implicit learning revealed by the method of opposition

Complex information, such as that required for motor skills, can be learned implicitly, without awareness. Much debate has centered on the appropriate methods for proving that implicit learning is not influenced by explicit awareness. A recent study by Destrebecqz and Cleeremans has provided compelling evidence for implicit sequence learning without awareness by using the ‘method of opposition’.     

Matching bias in syllogistic reasoning: Evidence for a dual-process account from response times and confidence ratings

We examined matching bias in syllogistic reasoning by analysing response times, confidence ratings, and individual differences. Roberts’ (2005) “negations paradigm” was used to generate conflict between the surface features of problems and the logical status of conclusions. The experiment replicated matching bias effects in conclusion evaluation (Stupple & Waterhouse, 2009), revealing increased processing times for matching/logic “conflict problems”. Results paralleled chronometric evidence from the belief bias paradigm indicating that logic/belief conflict problems take longer to process than non-conflict problems (Stupple, Ball, Evans, & Kamal-Smith, 2011). Individuals’ response times for conflict problems also showed patterns of association with the degree of overall normative responding. Acceptance rates, response times, metacognitive confidence judgements, and individual differences all converged in supporting dual-process theory. This is noteworthy because dual-process predictions about heuristic/analytic conflict in syllogistic reasoning generalised from the belief bias paradigm to a situation where matching features of conclusions, rather than beliefs, were set in opposition to logic.     

Using ecphoric confidence ratings to discriminate seen from unseen faces: The effects of retention interval and distinctiveness

Theories of confidence processing for recognition judgments suggest that confidence indexes the degree of match between a presented stimulus and an image in memory (ecphoric similarity). Recent research has demonstrated that having participants rate their confidence that a face had been previously seen provides an equivalent or a better index of the stimulus's status than does eliciting a simple binary response (Sauer, Brewer, & Weber, Journal of Experimental Psychology: General 137: 528-547, 2008). Using a face recognition paradigm, we manipulated retention interval and stimulus distinctiveness to directly test the suggestion that confidence indexes ecphoric similarity and to probe the boundary conditions for using confidence ratings to discriminate seen from unseen faces. Consistent with the proposed ecphoric basis for confidence ratings, mean confidence was higher for previously seen than for unseen faces, and conditions conducive to the formation of strong memories improved confidence-based discrimination. In all conditions, after the application of a classification algorithm, confidence ratings provided a more sensitive index of face status (i.e., seen or unseen) than did binary responses.     

Impaired social learning in patients with major depressive disorder revealed by a reinforcement learning model

Background/objectivePatients with major depressive disorder (MDD) have altered learning rates for rewards and losses in non-social learning paradigms. However, it is not well understood whether the ability to learn from social interactions is altered in MDD patients. Using reinforcement learning during the repeated Trust Game (rTG), we investigated how MDD patients learn to trust newly-met partners in MDD patients.MethodSixty-eight MDD patients and fifty-four controls each played as ‘investor’ and interacted with ten different partners. We manipulated both the level of trustworthiness by varying the chance of reciprocity (10, 30, 50, 70 and 90%) and reputation disclosure, where partners’ reputation was either pre-disclosed or hidden.ResultsOur reinforcement learning model revealed that MDD patients had significantly higher learning rates for losses than the controls in both the reputation disclosure and non-disclosure condition. The difference was larger when reputation was not disclosed than disclosed. We observed no difference in learning rates for gains in either condition.ConclusionsOur findings highlight that abnormal learning for losses underlies the social learning process in MDD patients. This abnormality is higher when situational unpredictability is high versus low. Our findings provide novel insights into social rehabilitation of MDD.     

Sequential priming is not constrained by the shape of long-term learning curves

When multiple stimulus-to-response (S-R) mappings are randomly intermixed and repeated in a block of trials, immediate repetitions of an aspect of a stimulus and/or a response can facilitate stimulus detection, classification, and/or response selection--known as sequential priming. In addition to these short-term effects, response times (RT) for almost any task diminish with extended practice, improvements can occur over many days, and RT learning curves typically assume exponential or power functions. We investigated whether short-term sequential priming and long-term practice modulate RT through a common mechanism, using a variant of the additive factors method. We tracked how various priming effects, presumably affecting different processing stages (e.g, stimulus selection, stimulus identification/classification, and S-R mapping), varied over training sessions as RT diminished. All the priming effects either were not reduced or reduced approximately linearly at rates much slower than those predicted by the shapes of the coresponding RT learning curves. The overall results suggest that short term sequential priming and long-term practice modulate RT through relatively separate mechanisms, even though they appear to affect a common set of behaviorally defined processing stages.     

Allocation of study time and recall by learning disabled and nondisabled children of different ages

Study time and recall by learning-disabled and nondisabled children of five different ages were examined in a task requiring recall of digits that were presented at the child's own rate. Recall increased with age and was significantly higher by nondisabled than disabled children, particularly at older ages. As additional digits of each sequence were presented, study time by 8-year-old disabled and nondisabled groups were relatively constant, increased in older disabled and nondisabled children, but increased more in older nondisabled children than older learning disabled children. Instructions in hierarchical grouping of digits increased recall by all groups to a similar degree, but the increase by younger children and learning disabled children was associated with longer study times. The results suggest that allocation of study time and recall are developmentally delayed in learning disabled children.     

Dissociations in perceptual learning revealed by adult age differences in adaptation to time-compressed speech

When presented with several time-compressed sentences, young adults' performance improves with practice. Such adaptation has not been studied in older adults. To study age-related changes in perceptual learning, the authors tested young and older adults' ability to adapt to degraded speech. First, the authors showed that older adults, when equated for starting accuracy with young adults, adapted at a rate and magnitude comparable to young adults. However, unlike young adults, older adults failed to transfer this learning to a different speech rate and did not show additional benefit when practice exceeded 20 sentences. Listeners did not adapt to speech degraded by noise, indicating that adaptation to time-compressed speech was not attributable to task familiarity. Finally, both young and older adults adapted to spectrally shifted noise-vocoded speech. The authors conclude that initial perceptual learning is comparable in young and older adults but maintenance and transfer of this learning decline with age.     

Individual differences in the functional neuroanatomy of verbal discrimination learning revealed by positron emission tomography

Why do some people have better memory abilities than others? This issue has been of long-standing interest to scientists and lay people. However, using purely behavioral methods, psychologists have made little progress in illuminating it. Now that functional brain imaging techniques have become available to study mind/brain relations, there is a new promise of understanding individual differences in learning and memory in terms of corresponding differences in brain activity. In this paper, we will present a positron emission tomography (PET) study designed to examine individual differences in learning and memory abilities. The basic assumption is that different patterns of brain activity serve as strong predictors of memory performance. Two specific questions were addressed in this study: (i) Can PET illuminate the relations between memory processes and their neuroanatomical correlates among individual learners and rememberers? and (ii) if so, how are these relations affected by the stage of practice on a given memory task? Our PET study examined individual differences in the neuroanatomical correlates of multi-trial verbal discrimination learning in 16 young healthy subjects. The results identified patterns of brain regions in which blood flow correlated with subjects' retrieval performance. However, these regions did not correlate with performance during all learning trials. Instead, a gradual shift was observed from one pattern of brain regions to another over the course of learning. These results suggest that individual differences in memory performance are related to differences in neural activity within specific brain circuits.     

Event and story structure recall by children with specific learning disabilities,language impairments,and normally achieving children

This study investigated the ability of children with specific learning disabilities (SLD), children with language impairments (LI), and children who are normally achieving (NA) to recall the events and story structures of a narrative and an expository text. Effects of group, verbal age, text structure, and order of presentation on recall as measured through listening comprehension were studied. Sixty students who were matched on verbal age served as subjects. Results suggested differences between the LI and SLD groups on text recall. Differences were also evident for text type, with recall of narrative text typically being superior to recall of expository text. In general, the performance of the group with SLD was similar to that of the NA group.