Reward schedule effects in extinction: Intertrial interval, memory, and memory retrieval

Rats were trained in a runway such that partial reward occurred on Trial 1 of the day and consistent reward on subsequent massed trials (Group PRT1), or consistent reward occurred on Trial 1 of the day and partial reward on subsequent massed trials (Group PRTM). Under spaced (24-hr) extinction, Group PRT1 was more resistant to extinction than Group PRTM and under massed (1-min) extinction, Group PRTM was more resistant to extinction than Group PRT1. These findings suggest that (a) distinctive stimuli are associated with Trial 1 of the day and with subsequent massed trials, (b) these distinctive stimuli function as retrieval cues for memories, memory retrieval being independent of intertrial interval, and (c) behavior in extinction is controlled by a stimulus compound consisting of the memory of nonreward plus stimuli which accompany the memory of nonreward on rewarded acquisition trials.     

Odor production and utilization under conditions of nonreward and small reward

A series of five experiments was conducted to examine odor production and utilization under conditions of contrasting large and small reward. Using daily, eight-trial double-alternation sequences of reward (L)-nonreward (N) or small reward (S), i.e., LLNNLLNN and LLSSLLSS, appropriate patterning was shown only when an empty goalbox was initially encountered on N or S trials (Experiments 1–5). Manipulations designed to increase frustration, for example, preceding double-alternation training with a block of large-reward trials in Experiment 2, or increasing the L vs S ratio in Experiment 3, failed to yield patterning when small reward was presented at the start of the S-trial confinement period. Patterning was shown when small reward was presented at the end of the S-trial confinement (Experiments 4 and 5). Immediate vs delayed (30 sec) S or N trial goalbox removal failed to result in differences in patterning (Experiments 1 and 2). To distinguish odor production from odor use, subjects trained on an LLSSLLSS pattern preceded subjects trained on an LLNNLLNN pattern (Experiment 5). Because patterning failed to develop in the LLSSLLSS subjects and the first subject in the LLNNLLNN group, it was concluded that a discriminably different cue was not produced on S trials under the LLSSLLSS sequence when small reward was delivered at the beginning of the goalbox confinement period. Shifting small-reward delivery to the end of goalbox confinement resulted in the immediate display of patterning by the first LLNNLLNN subject, and the subsequent development of patterning by the LLSSLLSS subjects.     

Startbox-goalbox confinement durations as determinants of self-punitive behavior

Two levels of confinement duration (5 or 60 sec) in either startbox or goalbox were factorially combined with presence or absence of shock during extinction of a runway escape response in a self-punitive paradigm under a spaced-trials procedure. All groups were equated for amount and temporal spacing of handling before and after each trial. Shocked rats were more resistant to extinction than nonshocked rats only when startbox confinement was short. When startbox confinement was long, shock facilitated extinction. Long goalbox confinement enhanced running speed for shocked rats, but only in the presence of shock. It was suggested that duration of startbox confinement affects strength of conditioned fear, with long confinement leading to its extinction.     

The differential reinforcement of reward-produced and response-produced stimuli

In two experiments, groups received successive large-reward trials on odd-numbered days and successive small-reward trials on even-numbered days in the same gray alleyway. This produced a discrimination problem in which the memory of large reward (S^L) was reliably discriminative of large reward and the memory of small reward (S^S) was reliably discriminative of small reward. Intertrial interval (ITI) was varied both between and within groups. In the within-groups manipulation, ITI separating S+ trials differed from that separating S- trials. Experimental groups learned the discrimination, running slower to S^S (S- cue) than to S^L (S+ cue), and showed a negative contrast effect, running slower to S- than a small-reward control group. Discrimination was somewhat faster at massed than at spaced trials. The within-groups manipulation of ITI suggested that the effects of ITI were mediated by time-dependent changes in internal cues produced by reward events and by the instrumental response. The control exercised by internal cues was shown to be associative rather than nonassociative, e.g., motivational. Similarities with, and implications for, conventional brightness differential conditioning were discussed.     

Effects of preshift small magnitude of reward received during goalbox placements on the negative contrast effect

During preshift, one experimental group of rats was given a large magnitude of food reward following a traversal of a straight alley and during a goalbox placement, while the other experimental group was given a small reward during goalbox placement and a large reward following a run. During postshift, all experimental groups were given a small reward of food following a traversal down the runway and during a goalbox placement. A control group was maintained on small reward during placements and following a traversal throughout the study. Only the group who received preshift large reward during placement and following a runway response ran slower to small reward during postshift than the control group maintained on small reward (negative contrast effect).     

Extinction of a running response as a function of reinforcement magnitude

Four groups of rats were trained on different sucrose solutions in a straight runway. Terminal running speed was a monotonic function of reinforcement magnitude. After training each group was subdivided, one subgroup being extinguished under spaced, the other under massed conditions. In spaced extinction the animals trained on non extreme reward magnitudes showed most resistance to extinction. It was concluded that resistance to extinction is an inverted U-shaped function of reinforcement magnitude found in training. The massed extinction trials were conducted with a very short inter-trial interval. The animals showed an immediate drop in running speed followed by a gradual recovery and a subsequent decline. The number of trials taken to reach the peak recovery speed was a function of reinforcement magnitude found in training. Results on both massed and spaced extinction trials were interpreted in terms of the facilitatory and inhibitory effects of momentary and conditioned frustration.     

Grouping, chunking, memory, and learning

An example of grouping is dividing a long series of digits into two or more smaller units by, for example, pausing for a short time between items within groups but for a longer time between groups. Grouping, virtually neglected in animal learning, was examined in each of five rat investigations reported here in which a series of two or more runway trials was either grouped together with or apart from a terminal large reward trial. It was found that running speed on early small reward trials in the series was greater when prior trials were grouped together with rather than apart from the terminal large reward trial and this when the intertrial interval was short, 20 sec, or long, 20 to 40 min. Three possible explanations of the present findings were examined: a reward schedule view, a rule-learning view, and an anticipation view based on memory. The most feasible of these explanations, it was suggested, was the anticipation view. According to this view, when prior trials are grouped together with a terminal large reward, there is a tendency for the rat to anticipate the terminal large reward well before its scheduled occurrence, elevating running speed on earlier small reward trials. Such anticipation occurs, it was suggested, because when trials are grouped together the memory of each reward event in the series is retrieved on each subsequent trial, including the remote terminal large reward trial, where it becomes a signal for large reward. Thus the present results implicate not merely adjacent associations—associations between adjacent events—but also the highly controversial remote associations—associations between two events separated not only in time but by one or more intervening events as well.     

Does mediator use contribute to the spacing effect for cued recall? Critical tests of the mediator hypothesis

When study is spaced across sessions (versus massed within a single session), final performance is greater after spacing. This spacing effect may have multiple causes, and according to the mediator hypothesis, part of the effect can be explained by the use of mediator-based strategies. This hypothesis proposes that when study is spaced across sessions, rather than massed within a session, more mediators will be generated that are longer lasting and hence more mediators will be available to support criterion recall. In two experiments, participants were randomly assigned to study paired associates using either a spaced or massed schedule. They reported strategy use for each item during study trials and during the final test. Consistent with the mediator hypothesis, participants who had spaced (as compared to massed) practice reported using more mediators on the final test. This use of effective mediators also statistically accounted for some – but not all of – the spacing effect on final performance.     

Interaction between amount and pattern of training in the induction of intermediate- and long-term memory for sensitization in aplysia

In Aplysia, three distinct phases of memory for sensitization can be dissociated based on their temporal and molecular features. A single training trial induces short-term memory (STM, lasting <30 min), whereas five trials delivered at 15-min intervals induces both intermediate-term memory (ITM, lasting >90 min) and long-term memory (LTM, lasting >24 h). Here, we explore the interaction of amount and pattern of training in establishing ITM and LTM by examining memory for sensitization after different numbers of trials (each trial = one tail shock) and different patterns of training (massed vs. spaced). Under spaced training patterns, two trials produced STM exclusively, whereas four or five trials each produced both ITM and LTM. Three spaced trials failed to induce LTM but did produce an early decaying form of ITM (E-ITM) that was significantly shorter and weaker in magnitude than the late-decaying ITM (L-ITM) observed after four to five trials. In addition, E-ITM was induced after three trials with both massed and spaced patterns of training. However, L-ITM and LTM after four to five trials require spaced training: Four or five massed trials failed to induce LTM and produced only E-ITM. Collectively, our results indicate that in addition to three identified phases of memory for sensitization—STM, ITM, and LTM—a unique temporal profile of memory, E-ITM, is revealed by varying either the amount or pattern of training.     

Massed extinction trials produce better short-term but worse long-term loss of context conditioned fear responses than spaced trials

A series of experiments studied the effects of the interval between extinction trials on the loss of context conditioned freezing responses. Rats were shocked in one context (A) but not in another (B) and subjected to extinction trials in context A. In Experiment 1, massed trials produced more rapid loss than spaced trials. A shift from spaced to massed trials maintained this loss, but the shift from massed to spaced trials restored lost responses. Experiments 2-5 examined this effect of massed trials on responding across spaced trials. They provided evidence that (a) a single trial was as effective as multiple daily massed trials, (b) learning occurred on the first of the massed trials but not on later ones, and (c) the first trial reduced the amount learned across subsequent massed trials. Finally, alternating extinction trials in A and B produced more response loss across spaced trials than blocks of trials in A and B. The results were discussed in terms of the role accorded to self-generated priming in the models developed by A. R. Wagner (1978, 1981).     

难度与分值对高一学生选择停止、集中或分散学习的影响

选取50名高一学生,选用反义词对和人工词对两种学习材料,赋予学习材料不同的分值,探讨项目难度和分值对学习者集中学习或分散学习选择及其效果的影响,并通过问卷调查被试对不同学习方式的认识。结果发现:(1)只有44%的被试报告分散学习更有效,被试总体上也更多地选择了集中学习;(2)在困难项目上,分散学习比集中学习的效果更好;(3)被试对容易项目更多选择停止学习,对困难项目更少选择停止学习,特别是困难高分项目;对困难项目比对容易项目更多选择分散学习,而在困难与容易项目上,对集中学习的选择不存在显著差异。     

Trial spacing is a determinant of cue interaction

Four conditioned lick suppression experiments with rats examined the effect of trial spacing on cue interaction. Experiments 1 and 2 found overshadowing to be eliminated with massed compound stimulus-outcome pairings and the usual trial spacing effect to be reversed with compound acquisition trials. Experiment 3 found that whether acquisition compound-outcome pairings were massed or spaced determined the effect of posttraining extinction treatment. Extinction of the overshadowing cue reduced responding following massed training and increased responding following spaced training. Extinction of the context decreased responding following massed training. Experiment 4 found the conditioning and devaluation results to be associative and stimulus specific. These results are in accord with the extended comparator hypothesis (J. C. Denniston, H. I. Savastano, & R. R. Miller, 2001).     

Context Shift and Protein Synthesis Inhibition Disrupt Long-Term Habituation after Spaced,but Not Massed,Training in the CrabChasmagnathus

An opaque screen moving overhead elicits an escape response in the crabChasmagnathusthat after a few presentations habituates for a long period (long-term habituation, LTH). Previous results suggested that spaced (15 trials separated by 171 s) and massed training (300 trials without rest interval) were correlated with two different memory components of LTH. The present experiments were aimed at further studying the mechanisms subserving these components. Results indicate that LTH acquired by spaced but not by massed training is blocked either by a training-to-testing context shift or by cycloheximide (15–25 μg) pre- or posttraining injection and that LTH after spaced training persists for longer time (5 days) than after massed training (2 days). A model based on these results that distinguishes two LTH-memory components is proposed: a (context-signal) LTH yielded by spaced training, dependent of context, sensitive to cycloheximide (CYX), and long lasting; and a (signal) LTH yielded by massed training, dependent only on the signal invariance, insensitive to CYX, and shorter lasting.     

Effects of previous delay of reward,generalized effort,and deprivation on impulsiveness

Two experiments investigated the effects of learning and motivational variables upon rats' degree of impulsiveness (choice of a small, immediate reward versus a large, delayed reward). In Experiment 1, food was presented in one of four ways for 24 sessions: fixed-ratio lever pressing requirement, continuous-reinforcement lever-pressing requirement, long intervals between free food presentations, or short intervals between free food presentations. Exposure to the long intervals produced less subsequent impulsiveness in 48 choice trials than did the short intervals, there being no effect of the instrumental requirement. In Experiment 2, there were 48 choice trials in which two hunger levels were combined factorially with two delays of reward for the self-control alternative. Hunger level did not affect implusiveness when there was a short delay, but high hunger increased the impulsiveness when there was a long delay. These results indicate that impulsiveness is affected by prior learning and current motivation. Implications for Baum, Rachlin, and Green's matching formula and for frustration theory are discussed.     

Neurogenesis and the spacing effect: learning over time enhances memory and the survival of new neurons

Information that is spaced over time is better remembered than the same amount of information massed together. This phenomenon, known as the spacing effect, was explored with respect to its effect on learning and neurogenesis in the adult dentate gyrus of the hippocampal formation. Because the cells are generated over time and because learning enhances their survival, we hypothesized that training with spaced trials would rescue more new neurons from death than the same number of massed trials. In the first experiment, animals trained with spaced trials in the Morris water maze outperformed animals trained with massed trials, but there was not a direct effect of trial spacing on cell survival. Rather, animals that learned well retained more cells than animals that did not learn or learned poorly. Moreover, performance during acquisition correlated with the number of cells remaining in the dentate gyrus after training. In the second experiment, the time between blocks of trials was increased. Consequently, animals trained with spaced trials performed as well as those trained with massed, but remembered the location better two weeks later. The strength of that memory correlated with the number of new cells remaining in the hippocampus. Together, these data indicate that learning, and not mere exposure to training, enhances the survival of cells that are generated 1 wk before training. They also indicate that learning over an extended period of time induces a more persistent memory, which then relates to the number of cells that reside in the hippocampus.     

Effect of shifts in body weight on rats' straight alley performance as a function of reward magnitude

The performance of rats trained in a straight alley for 55 trials under 75% body weight and then shifted to 90% body weight was compared to that of rats trained only under 90% body weight. When training was with a small reward the speed of the shifted group decreased to the level of the 90% small reward control groups, but when training was with a large reward the speed of the shifted group dropped below the level of the 90% large reward control group. Thus, the effects on performance of shifting deprivation level following extended training depend on the reward magnitude employed.     

Studies in Spaced and Massed Learning: III. Paired-Associate and Serial Learning

An experiment on paired-associate learning was carried out to test Hovland's prediction that spaced practice should be more advantageous than massed if the order of presenting the pairs of nonsense syllables were fixed.

In the results, spaced and massed practice showed little difference in learning efficiency. Hovland's prediction was not confirmed.

In a previous experiment of this series of studies it had been found that spaced practice was more efficient than massed in serial learning when syllables of “low-meaningfulness” value were used; but that there was little difference in efficiency between the two forms of distribution when “high-meaningfulness” syllables were used. The results of the two experiments naturally suggested that, as regards the relative efficiency of spaced and massed practice, paired-associate learning of low-meaningfulness syllables differed from serial learning of similar syllables, but matched serial learning of high-meaningfulness syllables. It seems justifiable to draw the conclusion that the relative efficiency of spaced and massed practice is different in paired-associate and serial learning, not because remote associations can be formed in the one case and not in the other, as Hovland (1939a, 1939b) assumed; but because the organization of the material and the learning processes are different in the two forms of learning.     

Visual complexity and stimulus pacing in chicks

Two groups were tested for spontaneous alternation using massed and spaced trials but alternation rate was not above chance level. Two experiments then examined the use of visual stimuli as pacers for an approach response. The results indicate that subjects responded to complexity and that complexity may be a more effective pacer than novelty.     

Long-lasting and context-specific freezing preference is acquired after spaced repeated presentations of a danger stimulus in the crab Chasmagnathus

A visual danger stimulus elicits an escape response in the crab Chasmagnathus that declines after repeated presentations. Previous results report that such waning may be retained as context-signal memory (CSM) or signal memory (SM): CSM is long lasting, associative, and produced by spaced training, while SM is an intermediate memory, nonassociative, and produced by massed training. The performances of both spaced and massed trained crabs are here examined, using video analysis to determine topographic changes in the behavioral response during and after training. During spaced training, escape vanishes and is mainly replaced by freezing, while during massed training, escape decreases over trials without being replaced by any defensive response. After 24 h, the marked proclivity to freezing persists in spaced trained crabs, while a high level of escaping is shown by massed trained crabs. The long-lasting freezing preference of spaced trained crabs proves to be context-specific and apparent from the very first presentation of the danger stimulus at testing, though freezing is not triggered by the sole exposure to the context. We conclude (a) that freezing preference is the acquired response of the CSM process; (b) that CSM can be properly categorized as an instance of contextual conditioning and SM of classical habituation; (c) that CSM and SM are not two phases of a memory processing but two distinctly types of memory; and (d) that therefore, the temporal distribution of training trials has a drastic effect on crab's memory, more dramatic than that previously described. The possibility that massed and spaced presentations of the same stimulus may represent two different stimulus types is discussed.     

Massed vs. spaced exposure sessions in the treatment of agoraphobia

The differential effects of massed and spaced sessions of exposure in vivo were investigated with 11 agoraphobics in a crossover design. Clients were randomly assigned to one of the following two groups: (a) 10 daily sessions followed by 10 weekly sessions, or (b) 10 once-weekly sessions followed by 10 daily sessions. The results indicated that both massed and spaced sessions of exposure in vivo effected changes in avoidance behavior and subjective anxiety of agoraphobics, massed practice being superior especially with respect to avoidance. It was suggested that the superiority of massed practice is due to the lack of opportunity to engage in avoidance or escape during short inter-session intervals. The relationship between subjective anxiety and avoidance were considered and the differences between clinical and statistical significance were discussed.