The testing effect as a function of explicit testing instructions and judgments of learning

During study, people monitor their learning; the output of this monitoring is captured in so-called judgments of learning (JOLs). JOLs predict later recall better if they are made after a slight delay, instead of immediately after study (the delayed JOL effect). According to the self-fulfilling prophecy (SFP) hypothesis delayed JOLs are based on covert retrieval attempts from long-term memory, and successful retrieval attempts in themselves enhance learning (the testing effect). We compared memory for 40 Swahili-Swedish paired associates after a week as a function of three different learning conditions, namely study plus (i) explicitly instructed self-testing, (ii) delayed JOLs, or (iii) less self-testing. We showed that repeated delayed JOLs lead to a memory improvement that does not differ significantly from a comparable condition where the participants are explicitly testing memory, and both the latter groups performed reliably better than a group that self-tested less. The results suggest that delayed JOLs improve long-term retention as efficiently as explicit memory testing and lend support to the SFP hypothesis.     

A model for stochastic drift in memory strength to account for judgments of learning

Previous research has shown that judgments of learning (JOLs) made immediately after encoding have a low correlation with actual cued-recall performance, whereas the correlation is high for delayed judgments. In this article, the authors propose a formal theory describing the stochastic drift of memory strength over the retention interval to account for the delayed-JOL effect. This is done by first decomposing the aggregated memory strength into exponential functions with slow and fast memory traces. The mean aggregated memory strength shows power-function forgetting curves. The drift of the memory strength is large for immediate JOLs (causing a low predictability) and weak for delayed JOLs (causing a high predictability). Consistent with empirical data, the model makes a novel prediction of JOL asymmetry, or that immediate weak JOLs are more predictive of future performance than are immediate strong JOLs. The JOL distributions for immediate and delayed JOLs are also accounted for.     

The role of memory for past test in the underconfidence with practice effect

According to the Memory for Past Test (MPT) heuristic, judgments of learning (JOLs) may be based, in part, on memory for the correctness of answers on a previous test. The authors explored MPT as the source of the underconfidence with practice effect (UWP; A. Koriat, L. Sheffer, & H. Ma'ayan, 2002), whereby Trial 1 overconfidence switches to underconfidence by Trial 2. Immediate and delayed JOLs were contrasted because only immediate JOLs demonstrate UWP. Consistent with MPT for immediate JOLs, Trial 1 test performance better predicted Trial 2 JOLs than did Trial 2 test performance. Delayed JOLs showed the reverse. Furthermore, items forgotten on Trial 1 but remembered on Trial 2 contributed disproportionately to UWP, but only with immediate JOLs.     

延迟学习判断研究的发展与展望

延迟学习判断是学习判断的一种形式,是指在材料学习完以后间隔一段时间才发生的学习判断。在与即时学习判断的对比研究中发现,延迟学习判断具有较高的相对准确性,这种现象被称为延迟学习判断效应。研究者进行了大量的研究并提出了多种理论来解释这种延迟学习判断效应。随着研究的不断深入,延迟学习判断的研究从研究指标、研究方法甚至是研究的理论基础都在不断更新。延迟学习判断的研究进展,包括主要理论和相关实验,以及最新研究成果将被介绍。最后,文章梳理了延迟JOL的研究进程,并指出了未来的研究方向。     

Familiarity and retrieval processes in delayed judgments of learning

Two processes are postulated to underlie delayed judgments of learning (JOLs)--cue familiarity and target retrievability. The two processes are distinguishable because the familiarity-based judgments are thought to be faster than the retrieval-based processes, because only retrieval-based JOLs should enhance the relative accuracy of the correlations between the JOLs and criterion test performance, and because only retrieval-based judgments should enhance memory. To test these predictions, in three experiments, the authors either speeded people's JOLs or allowed them to be unspeeded. The relative accuracy of the JOLs in predicting performance on the criterion test was higher for the unspeeded JOLs than for the speeded JOLs, as predicted. The unspeeded JOL conditions showed enhanced memory as compared with the speeded JOL conditions, as predicted. Finally, the unspeeded JOLs were sensitive to manipulations that modified recallability of the target, whereas the speeded JOLs were selectively sensitive to experimental variations in the familiarity of the cues. Thus, all three of the predictions about the consequences of the two processes potentially underlying delayed JOLs were borne out. A model of the processes underlying delayed JOLs based on these and earlier results is presented.     

Multiple bases for young and older adults' judgments of learning in multitrial learning

Prior work has suggested that participants use a memory-for-past-tests (MPT) heuristic for judgments of learning (JOLs) in a multitrial learning scenario. That is, when learning the same material in multiple sessions, previous memory performance can be used as a basis for later memory predictions. We explored this issue by evaluating the impact of healthy aging on the use of MPT across trials. Young adults and healthy older adults learned pairs of words, made JOLs, and received a memory test in three study-test trials on the same material. Results indicated that both young and older adults relied on MPT as a basis for JOLs and changes in MPT across trials were nominal. Further, only the most-recent past test influenced JOLs, whereas earlier tests were unrelated to later judgments. JOLs were also influenced by prior-trial JOLs and were related to subsequent memory performance on the same trial. We suggest that these data support both indirect- and direct-memory mechanisms as the bases for the MPT heuristic. Further, in a multitrial learning scenario, in which the same information was being learned, young and older adults used the same bases for their JOLs.     

Are judgments of learning made after correct responses during retrieval practice sensitive to lag and criterion level effects?

Although successful retrieval practice is beneficial for memory, various factors (e.g., lag and criterion level) moderate this benefit. Accordingly, the efficacy of retrieval practice depends on how students use retrieval practice during learning, which in turn depends on accurate metacognitive monitoring. The present experiments evaluated the extent to which judgments of learning (JOLs) made after correct responses are sensitive to factors (i.e., lag and criterion level) that moderate retrieval practice effects, as well as which cues influence JOLs under these conditions. Participants completed retrieval practice for word pairs with either short or long lags between practice trials until items were correctly recalled 1, 3, 6, or 9 times. After the criterion trial for an item, participants judged the likelihood of recalling that item on the final test 1 week later. JOLs showed correct directional sensitivity to criterion level, with both final test performance and JOLs increasing as criterion level increased. However, JOLs showed incorrect directional sensitivity to lag, with greater performance but lower JOLs for longer versus shorter lags. Additionally, results indicated that retrieval fluency and metacognitive beliefs about criterion level--but not lag--influenced JOLs.     

Effects of testing on subsequent re‐encoding and long‐term forgetting of action‐relevant materials: On the influence of recall type

Testing one's memory of previously studied information reduces the rate of forgetting, compared to restudy. However, little is known about how this direct testing effect applies to action phrases (e.g., “wash the car”) – a learning material relevant to everyday memory. As action phrases consist of two different components, a verb (e.g., “wash”) and a noun (e.g., “car”), testing can either be implemented as noun‐cued recall of verbs or verb‐cued recall of nouns, which may differently affect later memory performance. In the present study, we investigated the effect of testing for these two recall types, using verbally encoded action phrases as learning materials. Results showed that repeated study–test practice, compared to repeated study–restudy practice, decreased the forgetting rate across 1 week to a similar degree for both noun‐cued and verb‐cued recall types. However, noun‐cued recall of verbs initiated more new subsequent learning during the first restudy, compared to verb‐cued recall of nouns. The study provides evidence that testing has benefits on both subsequent restudy and long‐term retention of action‐relevant materials, but that these benefits are differently expressed with testing via noun‐cued versus verb‐cued recall.     

Monitoring memory errors: The influence of the veracity of retrieved information on the accuracy of judgements of learning

The current study examined the degree to which predictions of memory performance made immediately or at a delay are sensitive to confidently held memory illusions. Participants studied unrelated pairs of words and made judgements of learning (JOLs) for each item, either immediately or after a delay. Half of the unrelated pairs (deceptive items; e.g., nurse–dollar) had a semantically related competitor (e.g., doctor) that was easily accessible when given a test cue (e.g., nurse–do_ _ _r) and half had no semantically related competitor (control items; e.g., subject–dollar). Following the study phase, participants were administered a cued recall test. Results from Experiment 1 showed that memory performance was less accurate for deceptive compared with control items. In addition, delaying judgement improved the relative accuracy of JOLs for control items but not for deceptive items. Subsequent experiments explored the degree to which the relative accuracy of delayed JOLs for deceptive items improved as a result of a warning to ensure that retrieved memories were accurate (Experiment 2) and corrective feedback regarding the veracity of information retrieved prior to making a JOL (Experiment 3). In all, these data suggest that delayed JOLs may be largely insensitive to memory errors unless participants are provided with feedback regarding memory accuracy.     

学习和测验经验对学习判断中UWP效应的影响

针对重复学习判断中出现的练习伴随低估效应（UWP效应）,目前存在多种理论解释。本研究基于过去测验记忆假说,在学习阶段和测验阶段引入两种判断：学习判断准确性的判断和回溯性信心判断,通过两个实验考察学习经验和测验经验对UWP效应的影响。结果发现：在学习判断中学习判断准确性的判断和回溯性信心判断均消除了UWP效应,间接证明了学习和测验经验均影响到UWP效应的出现。     

The sensitivity of judgment-of-learning resolution to past test performance,new learning,and forgetting

When people judge their learning of items across study–test trials, their accuracy in discriminating between learned and unlearned items improves on the second trial. We examined the source of this improvement by estimating the contribution of three factors—memory for past test performance (MPT), new learning, and forgetting—to accuracy on trial 2. In Experiment 1, during an initial trial, participants studied paired associates, made a judgment of learning (JOL) for each one, and were tested. During the second trial, we manipulated two variables: when the JOL was made (either immediately before or after studying an item) and whether participants were told the outcome of the initial recall attempt on trial 1. In Experiment 2, the same procedure was used with a 1-week retention interval between study and test on trial 2. In both experiments, JOL resolution was higher on trial 2 than on trial 1. Fine-grained analyses of JOL magnitude and decomposition of resolution supported several conclusions. First, MPT contributed the most to boosts in JOL magnitude and improvements in resolution across trials. Second, JOLs and subsequent resolution were sensitive to new learning and forgetting, but only when participants’ judgments were made after study. Thus, JOLs appear to integrate information from multiple factors, and these factors jointly contribute to JOL resolution.     

No evidence that alcohol intoxication impairs judgments of learning in face recognition

Alcohol use is frequently involved in crime, making it crucial to understand the role of alcohol in facial recognition to maximize correct perpetrator identifications. Although the majority of the alcohol and face recognition research has investigated recognition with retrospective confidence judgments, we examined the effects of alcohol intoxication on face recognition with prospective metacognitive judgments. Participants (N = 54 university students without a history of hazardous alcohol/substance use) consumed either alcohol (mean breath alcohol concentration of 0.06 at pretest and 0.07 at post‐test) or a non‐alcoholic placebo drink. Participants then studied unfamiliar male and female faces and made judgments of learning (JOLs) for each face (i.e., predicted the likelihood of recognizing that face on a future memory test). After a brief distractor task, participants completed an old–new recognition test on which they attempted to distinguish the studied faces from new faces. It was found that the alcohol manipulation had minimal effect on face recognition performance or judgments of learning. Our results suggest that theory‐based cues about the effects of alcohol might play a greater role in retrospective judgments than prospective judgments. Although not a primary focus of the study, face recognition was better for male faces than female faces, and this occurred for both female and male participants. Limitations and implications of the research are discussed.     

Effects of Problem Solving after Worked Example Study on Primary School Children's Monitoring Accuracy

Research on expository text has shown that the accuracy of students' judgments of learning (JOLs) can be improved by instructional interventions that allow students to test their knowledge of the text. The present study extends this research, investigating whether allowing students to test the knowledge they acquired from studying a worked example by means of solving an identical problem, either immediately or delayed, would enhance JOL accuracy. Fifth grade children (i) gave an immediate JOL, (ii) a delayed JOL, (iii) solved a problem immediately and then gave a JOL, (iv) solved a problem immediately and gave a delayed JOL, or (v) solved a problem at a delay and then gave a JOL. Results show that problem solving after example study improved children's JOL accuracy (i.e., overestimation decreased). However, no differences in the accuracy of restudy indications were found. Results are discussed in relation to cue utilization when making JOLs. Copyright © 2014 John Wiley & Sons, Ltd.     

People use the memory for past-test heuristic as an explicit cue for judgments of learning

When people estimate their memory for to-be-learned material over multiple study–test trials, they tend to base their judgments of learning (JOLs) on their test performance for those materials on the previous trial. Their use of this information—known as the memory for past-test (MPT) heuristic—is believed to be responsible for improvements in the relative accuracy (resolution) of people’s JOLs across learning trials. Although participants seem to use past-test information as a major basis for their JOLs, little is known about how learners translate this information into a judgment of learning. Toward this end, in two experiments, we examined whether participants factored past-test performance into their JOLs in either an explicit, theory-based way or an implicit way. To do so, we had one group of participants (learners) study paired associates, make JOLs, and take a test on two study–test trials. Other participants (observers) viewed learners’ protocols and made JOLs for the learners. Presumably, observers could only use theory-based information to make JOLs for the learners, which allowed us to estimate the contribution of explicit and implicit information to learners’ JOLs. Our analyses suggest that all participants factored simple past-test performance into their JOLs in an explicit, theory-based way but that this information made limited contributions to improvements in relative accuracy across trials. In contrast, learners also used other privileged, implicit information about their learning to inform their judgments (that observers had no access to) that allowed them to achieve further improvements in relative accuracy across trials.     

Do metacognitive judgments alter memory performance beyond the benefits of retrieval practice? A comment on and replication attempt of Dougherty,Scheck, Nelson,and Narens (2005)

A central question in the metacognitive literature concerns whether the act of making a metacognitive judgment alters one’s memory for the information about which the judgment was made. Dougherty, Scheck, Nelson, and Narens (2005, Memory & Cognition, 33(6), 1096–1115) attempted to address this question by having participants make either retrospective confidence judgments (RCJs; i.e., evaluations of past retrieval success), judgments of learning (JOLs; i.e., predictions of future retrieval success), or no explicit judgments. When comparing final retrieval accuracy they found that accuracy was greater for items where participants had made JOLs compared with items that received RCJs or no judgment, suggesting that simply making a JOL can improve later memory performance. The present article presents results from four separate replication attempts that fail to duplicate this finding. Combined results provide compelling evidence that making a metacognitive judgment, regardless of the type, has no impact on later memory performance above and beyond retrieval practice.     

How crucial is the response format for the testing effect?

Combining study and test trials during learning is more beneficial for long-term retention than repeated study without testing (i.e., the testing effect). Less is known about the relative efficacy of different response formats during testing. We tested the hypothesis that overt testing (typing responses on a keyboard) during a practice phase benefits later memory more than covert testing (only pressing a button to indicate successful retrieval). In Experiment 1, three groups learned 40 word pairs either by repeatedly studying them, by studying and overtly testing them, or by studying and covertly testing them. In Experiment 2, only the two testing conditions were manipulated in a within-subjects design. In both experiments, participants received cued recall tests after a short (~19 min) and a long (1 week) retention interval. In Experiment 1, all groups performed equally well at the short retention interval. The overt testing group reliably outperformed the repeated study group after 1 week, whereas the covert testing group performed insignificantly different from both these groups. Hence, the testing effect was demonstrated for overt, but failed to show for covert testing. In Experiment 2, overtly tested items were better and more quickly retrieved than those covertly tested. Further, this does not seem to be due to any differences in retrieval effort during learning. To conclude, overt testing was more beneficial for later retention than covert testing, but the effect size was small. Possible explanations are discussed.     

Metacognitive monitoring during criterion learning: When and why are judgments accurate?

Research on metacognitive judgment accuracy during retrieval practice has increased in recent years. However, prior work had not systematically evaluated item-level judgment accuracy and the underlying bases of judgment accuracy in a criterion-learning paradigm (in which items are practiced until correctly recalled during encoding). Understanding these relationships during criterion learning has important theoretical implications for self-regulated learning frameworks, and also has applied implications for student learning: If the factors that influence metacognitive judgments are not predictive of subsequent test performance, students may make poor decisions during self-regulated learning. In the present experiments, participants engaged in test–restudy practice until items were recalled correctly. Once a given item reached criterion, participants made an immediate or delayed judgment of learning (JOL) for the item. A final cued-recall test occurred 30 min later. We examined judgment accuracy (the relationship between JOLs and test performance) and the underlying bases of judgment accuracy by evaluating cue utilization (the relationship between cues and JOLs) and cue diagnosticity (the relationship between cues and test performance). Immediate JOLs were only modestly related to subsequent test performance, and further analyses revealed that the cues related to JOLs were only weakly predictive of test accuracy. However, delaying JOLs improved both the accuracy of the JOLs and the diagnosticity of the cues that influenced judgments.     

Test-enhanced learning: taking memory tests improves long-term retention

Taking a memory test not only assesses what one knows, but also enhances later retention, a phenomenon known as the testing effect. We studied this effect with educationally relevant materials and investigated whether testing facilitates learning only because tests offer an opportunity to restudy material. In two experiments, students studied prose passages and took one or three immediate free-recall tests, without feedback, or restudied the material the same number of times as the students who received tests. Students then took a final retention test 5 min, 2 days, or 1 week later. When the final test was given after 5 min, repeated studying improved recall relative to repeated testing. However, on the delayed tests, prior testing produced substantially greater retention than studying, even though repeated studying increased students' confidence in their ability to remember the material. Testing is a powerful means of improving learning, not just assessing it.     

Monitoring memory errors: the influence of the veracity of retrieved information on the accuracy of judgements of learning

The current study examined the degree to which predictions of memory performance made immediately or at a delay are sensitive to confidently held memory illusions. Participants studied unrelated pairs of words and made judgements of learning (JOLs) for each item, either immediately or after a delay. Half of the unrelated pairs (deceptive items; e.g., nurse-dollar) had a semantically related competitor (e.g., doctor) that was easily accessible when given a test cue (e.g., nurse-do_ _ _r) and half had no semantically related competitor (control items; e.g., subject-dollar). Following the study phase, participants were administered a cued recall test. Results from Experiment 1 showed that memory performance was less accurate for deceptive compared with control items. In addition, delaying judgement improved the relative accuracy of JOLs for control items but not for deceptive items. Subsequent experiments explored the degree to which the relative accuracy of delayed JOLs for deceptive items improved as a result of a warning to ensure that retrieved memories were accurate (Experiment 2) and corrective feedback regarding the veracity of information retrieved prior to making a JOL (Experiment 3). In all, these data suggest that delayed JOLs may be largely insensitive to memory errors unless participants are provided with feedback regarding memory accuracy.     

Recollection-based prospective metamemory judgments are more accurate than those based on confidence: judgments of remembering and knowing (JORKS)

Five experiments were conducted to examine whether the nature of the information that is monitored during prospective metamemory judgments affected the relative accuracy of those judgments. We compared item-by-item judgments of learning (JOLs), which involved participants determining how confident they were that they would remember studied items, with judgments of remembering and knowing (JORKs), which involved participants determining whether studied items would later be accompanied by contextual details (i.e., remembering) or would not (i.e., knowing). JORKs were more accurate than JOLs when remember-know or confidence judgments were made at test and when cued recall was the outcome measure, but not for yes-no recognition. We conclude that the accuracy of metamemory judgments depends on the nature of the information monitored during study and test and that metamemory monitoring can be improved if participants are asked to base their judgments on contextual details rather than on confidence. These data support the contention that metamemory decisions can be based on qualitatively distinct cues, rather than an overall memory strength signal.