Multiple study trials and judgments of learning

We compared judgments of learning (JOLs) that were made either (a) after 1 study trial, (b) 2 study trials, or (c) in-between the 1st and 2nd study trials. In regard to the absolute accuracy of JOLs at predicting subsequent recall, we replicated previous findings of an underconfidence-with-practice effect for immediate JOLs and report for the first time a new finding of an underconfidence-with-practice effect for delayed JOLs (i.e., delayed JOLs after one trial overestimated the likelihood of subsequent recall, whereas delayed JOLs after two trials underestimated that likelihood). Also, although delayed JOLs always had a greater relative accuracy than did immediate JOLs, the relative accuracy of immediate and delayed JOLs was approximately the same after 1 versus 2 study trials. These results demonstrate that additional study trials affect the absolute accuracy of all JOLs but not the relative accuracy of any JOLs. Thus an increase in the number of study trials produced an increasing bias to be underconfident about the subsequent likelihood of recall but did not affect people's ordering of which items had been more (versus less) well-learned.     

A revised methodology for research on metamemory: Pre-judgment Recall and Monitoring (PRAM)

A revised methodology is described for research on metacognitive monitoring, especially judgments of learning (JOLs), to investigate psychological processing that previously has been only hypothetical and unobservable. During data collection a new stage of recall occurs just prior to the JOL, so that during data analysis the items can be partitioned into subcategories to measure the degree of JOL accuracy in ways that are more analytic than was previously possible. A weighted-average combinatorial rule allows the component measures of JOL accuracy to be combined into the usual overall measure of metacognitive accuracy. An example using the revised methodology offers a new explanation for the delayed-JOL effect, in which delayed JOLs are more accurate than immediate JOLs for predicting recall.     

Aging and monitoring associative learning: is monitoring accuracy spared or impaired?

Mixed lists of associatively related and unrelated paired associates were used to study monitoring of associative learning. Older and younger adults produced above-chance levels of relative accuracy, as measured by intraindividual correlations (gamma) of judgments of learning (JOLs) with item recall. JOLs were strongly influenced by relatedness, and this effect was greater for older adults. Relative accuracy was higher for unrelated than for related pairs. Correlations of JOLs with item recall for a randomly yoked learner indicated that access to one's own encoding experiences increased relative accuracy. Both age groups manifested a contrast effect (lower JOLs for unrelated items when mixed with related items). Aging appears to spare monitoring of encoding, even though it adversely affects associative learning.     

Metacognitive monitoring during criterion learning: When and why are judgments accurate?

Research on metacognitive judgment accuracy during retrieval practice has increased in recent years. However, prior work had not systematically evaluated item-level judgment accuracy and the underlying bases of judgment accuracy in a criterion-learning paradigm (in which items are practiced until correctly recalled during encoding). Understanding these relationships during criterion learning has important theoretical implications for self-regulated learning frameworks, and also has applied implications for student learning: If the factors that influence metacognitive judgments are not predictive of subsequent test performance, students may make poor decisions during self-regulated learning. In the present experiments, participants engaged in test–restudy practice until items were recalled correctly. Once a given item reached criterion, participants made an immediate or delayed judgment of learning (JOL) for the item. A final cued-recall test occurred 30 min later. We examined judgment accuracy (the relationship between JOLs and test performance) and the underlying bases of judgment accuracy by evaluating cue utilization (the relationship between cues and JOLs) and cue diagnosticity (the relationship between cues and test performance). Immediate JOLs were only modestly related to subsequent test performance, and further analyses revealed that the cues related to JOLs were only weakly predictive of test accuracy. However, delaying JOLs improved both the accuracy of the JOLs and the diagnosticity of the cues that influenced judgments.     

延迟学习判断的时间加工进程

学习判断是指人们在学习之后对自己学习效果所做的一种预测,是元记忆监测性判断的一种形式。学习判断按其发生时间的不同可以分为即时学习判断和延迟学习判断。与即时学习判断相比,延迟学习判断的相对准确性较高,被称为延迟学习判断效应。实验借助事件相关电位技术探究延迟学习判断的时间加工进程,发现高等级学习判断比低等级学习判断诱发了更大的时间窗为400-600ms的早期新旧效应和800-1200ms的晚期右侧额区新旧效应。上述结果表明,延迟学习判断包含不止一个加工过程,早期新旧效应反映了线索再认的过程,晚期右侧额区效应则反映了对提取之后的评估过程,该结果进一步验证了两阶段模型解释延迟学习判断效应的有效性。     

Does delaying judgments of learning really improve the efficacy of study decisions? Not so much

A widely held assumption in metamemory is that better, more accurate metamemory monitoring leads to better, more efficacious restudy decisions, reflected in better memory performance--we refer to this causal chain as the restudy selectivity hypothesis. In 3 sets of experiments, we tested this hypothesis by factorially manipulating metamemory monitoring accuracy and self-regulation of study. To manipulate monitoring accuracy, we compared judgments of learning (JOLs) made contemporaneously with a delayed retrieval attempt to JOLs either made at a delay without attempting retrieval or made immediately after study; in previous studies, delayed retrieval-based JOLs have robustly predicted recall with greater relative accuracy than have the other JOL types. To manipulate self-regulation of study, in Experiments 1A-1C and 2A-2C, we compared conditions in which participants' restudy selections were honored with conditions in which they were completely or randomly dishonored; in Experiments 3A-3C, we randomly honored or dishonored half of the restudy selections and half of the nonselections. Results revealed that the benefit of delayed, retrieval-based JOLs for final memory performance was due largely to the selection of more items for restudy rather than to better discriminations between items that would benefit more versus less from restudy. In most cases, gains in recall due to greater self-regulation of study did not increase with better monitoring accuracy; when they did, the effect was extremely small. The surprising conclusion was that restudy decisions were not very much more efficacious under conditions that yield greater monitoring accuracy than under those that do not.     

The influence of delaying judgments of learning on metacognitive accuracy: a meta-analytic review

Many studies have examined the accuracy of predictions of future memory performance solicited through judgments of learning (JOLs). Among the most robust findings in this literature is that delaying predictions serves to substantially increase the relative accuracy of JOLs compared with soliciting JOLs immediately after study, a finding termed the delayed JOL effect. The meta-analyses reported in the current study examined the predominant theoretical accounts as well as potential moderators of the delayed JOL effect. The first meta-analysis examined the relative accuracy of delayed compared with immediate JOLs across 4,554 participants (112 effect sizes) through gamma correlations between JOLs and memory accuracy. Those data showed that delaying JOLs leads to robust benefits to relative accuracy (g = 0.93). The second meta-analysis examined memory performance for delayed compared with immediate JOLs across 3,807 participants (98 effect sizes). Those data showed that delayed JOLs result in a modest but reliable benefit for memory performance relative to immediate JOLs (g = 0.08). Findings from these meta-analyses are well accommodated by theories suggesting that delayed JOL accuracy reflects access to more diagnostic information from long-term memory rather than being a by-product of a retrieval opportunity. However, these data also suggest that theories proposing that the delayed JOL effect results from a memorial benefit or the match between the cues available for JOLs and those available at test may also provide viable explanatory mechanisms necessary for a comprehensive account.     

延迟学习判断研究的发展与展望

延迟学习判断是学习判断的一种形式,是指在材料学习完以后间隔一段时间才发生的学习判断。在与即时学习判断的对比研究中发现,延迟学习判断具有较高的相对准确性,这种现象被称为延迟学习判断效应。研究者进行了大量的研究并提出了多种理论来解释这种延迟学习判断效应。随着研究的不断深入,延迟学习判断的研究从研究指标、研究方法甚至是研究的理论基础都在不断更新。延迟学习判断的研究进展,包括主要理论和相关实验,以及最新研究成果将被介绍。最后,文章梳理了延迟JOL的研究进程,并指出了未来的研究方向。     

Influence of cue word perceptual information on metamemory accuracy in judgement of learning

Previous studies have suggested that perceptual information regarding to-be-remembered words in the study phase affects the accuracy of judgement of learning (JOL). However, few have investigated whether the perceptual information in the JOL phase influences JOL accuracy. This study examined the influence of cue word perceptual information in the JOL phase on immediate and delayed JOL accuracy through changes in cue word font size. In Experiment 1, large-cue word pairs had significantly higher mean JOL magnitude than small-cue word pairs in immediate JOLs and higher relative accuracy than small-cue pairs in delayed JOLs, but font size had no influence on recall performance. Experiment 2 increased the JOL time, and mean JOL magnitude did not reliably differ for large-cue compared with small-cue pairs in immediate JOLs. However, the influence on relative accuracy still existed in delayed JOLs. Experiment 3 increased the familiarity of small-cue words in the delayed JOL phase by adding a lexical decision task. The results indicated that cue word font size no longer affected relative accuracy in delayed JOLs. The three experiments in our study indicated that the perceptual information regarding cue words in the JOL phase affects immediate and delayed JOLs in different ways.     

Evidence that judgments of learning are causally related to study choice

Three experiments investigated whether study choice was directly related to judgments of learning (JOLs) by examining people’s choices in cases in which JOLs were dissociated from recall. In Experiment 1, items were given either three repetitions or one repetition on Trial 1. Items given three repetitions received one on Trial 2, and those given one repetition received three on Trial 2—equating performance at the end of Trial 2, but yielding different immediate Trial 2 JOLs. Study choice followed the “illusory” JOLs. A delayed JOL condition in Experiment 2 did not show this JOL bias and neither did study choice. Finally, using a paradigm (Koriat & Bjork, 2005) in which similar JOLs are given to forward and backward associative pairs, despite much worse performance on the backward pairs, study choice again followed the mistaken JOLs. We concluded that JOLs—what people believe they know—directly influence people’s study choices.     

The role of memory for past test in the underconfidence with practice effect

According to the Memory for Past Test (MPT) heuristic, judgments of learning (JOLs) may be based, in part, on memory for the correctness of answers on a previous test. The authors explored MPT as the source of the underconfidence with practice effect (UWP; A. Koriat, L. Sheffer, & H. Ma'ayan, 2002), whereby Trial 1 overconfidence switches to underconfidence by Trial 2. Immediate and delayed JOLs were contrasted because only immediate JOLs demonstrate UWP. Consistent with MPT for immediate JOLs, Trial 1 test performance better predicted Trial 2 JOLs than did Trial 2 test performance. Delayed JOLs showed the reverse. Furthermore, items forgotten on Trial 1 but remembered on Trial 2 contributed disproportionately to UWP, but only with immediate JOLs.     

The effects of proactive interference (PI) and release from PI on judgments of learning

In three experiments, we investigated metacognitive monitoring in a variant of an A–B A–C learning paradigm in which the repetition of cues, but not targets, led to increasing proactive interference (PI) across trials. Judgments of learning (JOLs) correctly predicted decreases across trials in this paradigm but incorrectly continued to predict decreases on a final release trial in which new cues were introduced and performance consequently increased. Experience with the paradigm did not ameliorate this metacognitive failure (Experiment 3). In addition, JOLs decreased equally for pairs with repeated and with novel cue terms, even though recall of the latter group of items did not decrease across trials (Experiment 2). These results suggest that metacognizers’ naïve theories of remembering and forgetting include a role for global, but not cue-specific, interference.     

Lack of pervasiveness of the underconfidence-with-practice effect: boundary conditions and an explanation via anchoring

The authors investigated whether underconfidence in judgments of learning (JOLs) is pervasive across multiple study-test trials as suggested by A. Koriat, L. Sheffer, and H. Ma'ayan (2002) or whether underconfidence with practice (UWP) might be a kind of anchoring-and-adjustment effect, such that the occurrence or nonoccurrence of the UWP effect depends on whether recall is above a psychological anchor. Participants studied normatively difficult items or normatively easy items and made immediate JOLs or delayed JOLs. The UWP effect occurred for easy items, but for difficult items an overconfidence-with-practice (OWP) effect occurred for delayed JOLs and no bias occurred for immediate JOLs. The systematic occurrence of all 3 outcomes establishes boundary conditions for the UWP effect and confirms the hypothesis that underconfidence (or the lack thereof) may arise at least in part from an anchoring-and-adjustment mechanism.     

Monitoring memory errors: the influence of the veracity of retrieved information on the accuracy of judgements of learning

The current study examined the degree to which predictions of memory performance made immediately or at a delay are sensitive to confidently held memory illusions. Participants studied unrelated pairs of words and made judgements of learning (JOLs) for each item, either immediately or after a delay. Half of the unrelated pairs (deceptive items; e.g., nurse-dollar) had a semantically related competitor (e.g., doctor) that was easily accessible when given a test cue (e.g., nurse-do_ _ _r) and half had no semantically related competitor (control items; e.g., subject-dollar). Following the study phase, participants were administered a cued recall test. Results from Experiment 1 showed that memory performance was less accurate for deceptive compared with control items. In addition, delaying judgement improved the relative accuracy of JOLs for control items but not for deceptive items. Subsequent experiments explored the degree to which the relative accuracy of delayed JOLs for deceptive items improved as a result of a warning to ensure that retrieved memories were accurate (Experiment 2) and corrective feedback regarding the veracity of information retrieved prior to making a JOL (Experiment 3). In all, these data suggest that delayed JOLs may be largely insensitive to memory errors unless participants are provided with feedback regarding memory accuracy.     

Experience with proactive interference diminishes its effects: mechanisms of change

In three experiments, we examined the mechanisms by which prior experience with proactive interference (PI) diminished its effects. Cued recall tasks conforming to an A–B, A–D paradigm were used to induce PI effects. Experiment 1 showed that reduced PI was not due to a reduction in attention to the source of PI. Experiment 2 revealed that participants’ awareness of PI effects on memory performance increased with experience, resulting in a shift in encoding processes. Experiment 3 demonstrated that changes in encoding provided additional support for recollection that further enhanced participants’ ability to constrain their retrieval processing to the appropriate source of information at the time of test. These results can be interpreted as showing that experience with PI enhances awareness of its effects and allows individuals to adjust their learning and retrieval strategies to compensate for such effects.     

Developmental trends in children's memory monitoring: Evidence from a judgment-of-learning task

Two experiments were conducted to explore young children's memory monitoring abilities on a judgment-of-learning (JOL) task. Recent research on adults' JOLs has shown that predictions about subsequent recall for items that have been recently studied have never been very accurate immediately after learning but have been very accurate when judgments were delayed. One of the major goals of the present studies was to investigate whether the delayed-JOL effect could be observed in children of different ages. A secondary goal of the study was to compare individual-item JOLs with aggregate JOLs based on all items of a given list. If young children possess basic monitoring skills, both their delayed JOLs and their aggregate judgments should be comparably realistic. Our two experiments confirmed this assumption for all age groups involved (kindergartners, second and fourth graders). That is, JOLs were much more accurate when given after a delay of about 2 min than immediately after study, and overconfidence was typically larger for item-by-item JOLs than for aggregate-item JOLs. In fact, the pattern of findings for the older school children was very similar to that found for adults. Overall, these findings support the position that developmental trends in children's procedural metamemory are not due to differences in basic monitoring skills but attributable to developmental changes concerning the interplay between monitoring and self-regulation activities.     

Learning style,judgements of learning,and learning of verbal and visual information

The concept of learning style is immensely popular despite the lack of evidence showing that learning style influences performance. This study tested the hypothesis that the popularity of learning style is maintained because it is associated with subjective aspects of learning, such as judgements of learning (JOLs). Preference for verbal and visual information was assessed using the revised Verbalizer‐Visualizer Questionnaire (VVQ). Then, participants studied a list of word pairs and a list of picture pairs, making JOLs (immediate, delayed, and global) while studying each list. Learning was tested by cued recall. The results showed that higher VVQ verbalizer scores were associated with higher immediate JOLs for words, and higher VVQ visualizer scores were associated with higher immediate JOLs for pictures. There was no association between VVQ scores and recall or JOL accuracy. As predicted, learning style was associated with subjective aspects of learning but not objective aspects of learning.     

The effects of categorical relatedness on judgements of learning (JOLs)

On-line monitoring during study can be influenced by the relatedness shared between the cue and target of a paired associate. We examined the effects on people's judgements of learning (JOLs) of a different kind of relatedness, which occurs in a list organised into sets of categorically related words and unrelated words. In two experiments, participants studied a list of words organised into a series of sets of four categorically related words or four unrelated words. In Experiment 1, JOLs were made immediately after each word had been studied, and JOL magnitude was greater for related than unrelated words. In Experiment 2, JOLs were delayed after study and, as expected, they were substantially greater for related sets of words. Serial position effects (an increase in JOL magnitude across the words of a related set) were evident with immediate JOLs but not with delayed JOLs. The relatedness effect was not present early in the list for immediate JOLs but was present throughout the list for delayed JOLs. We conclude by discussing some preliminary explanations for these new phenomena.     

The effects of categorical relatedness on judgements of learning (JOLs)

On-line monitoring during study can be influenced by the relatedness shared between the cue and target of a paired associate. We examined the effects on people's judgements of learning (JOLs) of a different kind of relatedness, which occurs in a list organised into sets of categorically related words and unrelated words. In two experiments, participants studied a list of words organised into a series of sets of four categorically related words or four unrelated words. In Experiment 1, JOLs were made immediately after each word had been studied, and JOL magnitude was greater for related than unrelated words. In Experiment 2, JOLs were delayed after study and, as expected, they were substantially greater for related sets of words. Serial position effects (an increase in JOL magnitude across the words of a related set) were evident with immediate JOLs but not with delayed JOLs. The relatedness effect was not present early in the list for immediate JOLs but was present throughout the list for delayed JOLs. We conclude by discussing some preliminary explanations for these new phenomena.     

The sensitivity of judgment-of-learning resolution to past test performance,new learning,and forgetting

When people judge their learning of items across study–test trials, their accuracy in discriminating between learned and unlearned items improves on the second trial. We examined the source of this improvement by estimating the contribution of three factors—memory for past test performance (MPT), new learning, and forgetting—to accuracy on trial 2. In Experiment 1, during an initial trial, participants studied paired associates, made a judgment of learning (JOL) for each one, and were tested. During the second trial, we manipulated two variables: when the JOL was made (either immediately before or after studying an item) and whether participants were told the outcome of the initial recall attempt on trial 1. In Experiment 2, the same procedure was used with a 1-week retention interval between study and test on trial 2. In both experiments, JOL resolution was higher on trial 2 than on trial 1. Fine-grained analyses of JOL magnitude and decomposition of resolution supported several conclusions. First, MPT contributed the most to boosts in JOL magnitude and improvements in resolution across trials. Second, JOLs and subsequent resolution were sensitive to new learning and forgetting, but only when participants’ judgments were made after study. Thus, JOLs appear to integrate information from multiple factors, and these factors jointly contribute to JOL resolution.