Role of magnitude of reinforcement in spaced-trial instrumental learning in turtles (Geoclemys reevesii)

The runway performance of turtles (Geoclemys reevesii) was studied as a function of magnitude of reinforcement under spaced training conditions (one trial per day). A large reward magnitude (24 pellets) produced faster acquisition than a small magnitude (2 pellets). After a shift from the large to the small magnitude, latencies of the shifted animals continued to be significantly lower than the latencies of the non-shifted, small-magnitude controls, and similar to those of the non-shifted, large-magnitude controls. There was no evidence of the successive negative contrast effect. Extinction latencies were also significantly lower after training with the large magnitude than after training with the small magnitude; that is, there was no evidence of the magnitude of reinforcement extinction effect. The results are discussed in relation to the comparative analysis of the so-called paradoxical effects of reward, a family of learning phenomena that has not yet been found in a variety of experiments with fish, amphibians, and reptiles trained under widely spaced conditions.     

Reward schedule effects in extinction: Intertrial interval, memory, and memory retrieval

Rats were trained in a runway such that partial reward occurred on Trial 1 of the day and consistent reward on subsequent massed trials (Group PRT1), or consistent reward occurred on Trial 1 of the day and partial reward on subsequent massed trials (Group PRTM). Under spaced (24-hr) extinction, Group PRT1 was more resistant to extinction than Group PRTM and under massed (1-min) extinction, Group PRTM was more resistant to extinction than Group PRT1. These findings suggest that (a) distinctive stimuli are associated with Trial 1 of the day and with subsequent massed trials, (b) these distinctive stimuli function as retrieval cues for memories, memory retrieval being independent of intertrial interval, and (c) behavior in extinction is controlled by a stimulus compound consisting of the memory of nonreward plus stimuli which accompany the memory of nonreward on rewarded acquisition trials.     

Neurogenesis and the spacing effect: learning over time enhances memory and the survival of new neurons

Information that is spaced over time is better remembered than the same amount of information massed together. This phenomenon, known as the spacing effect, was explored with respect to its effect on learning and neurogenesis in the adult dentate gyrus of the hippocampal formation. Because the cells are generated over time and because learning enhances their survival, we hypothesized that training with spaced trials would rescue more new neurons from death than the same number of massed trials. In the first experiment, animals trained with spaced trials in the Morris water maze outperformed animals trained with massed trials, but there was not a direct effect of trial spacing on cell survival. Rather, animals that learned well retained more cells than animals that did not learn or learned poorly. Moreover, performance during acquisition correlated with the number of cells remaining in the dentate gyrus after training. In the second experiment, the time between blocks of trials was increased. Consequently, animals trained with spaced trials performed as well as those trained with massed, but remembered the location better two weeks later. The strength of that memory correlated with the number of new cells remaining in the hippocampus. Together, these data indicate that learning, and not mere exposure to training, enhances the survival of cells that are generated 1 wk before training. They also indicate that learning over an extended period of time induces a more persistent memory, which then relates to the number of cells that reside in the hippocampus.     

Trial spacing is a determinant of cue interaction

Four conditioned lick suppression experiments with rats examined the effect of trial spacing on cue interaction. Experiments 1 and 2 found overshadowing to be eliminated with massed compound stimulus-outcome pairings and the usual trial spacing effect to be reversed with compound acquisition trials. Experiment 3 found that whether acquisition compound-outcome pairings were massed or spaced determined the effect of posttraining extinction treatment. Extinction of the overshadowing cue reduced responding following massed training and increased responding following spaced training. Extinction of the context decreased responding following massed training. Experiment 4 found the conditioning and devaluation results to be associative and stimulus specific. These results are in accord with the extended comparator hypothesis (J. C. Denniston, H. I. Savastano, & R. R. Miller, 2001).     

Long-lasting and context-specific freezing preference is acquired after spaced repeated presentations of a danger stimulus in the crab Chasmagnathus

A visual danger stimulus elicits an escape response in the crab Chasmagnathus that declines after repeated presentations. Previous results report that such waning may be retained as context-signal memory (CSM) or signal memory (SM): CSM is long lasting, associative, and produced by spaced training, while SM is an intermediate memory, nonassociative, and produced by massed training. The performances of both spaced and massed trained crabs are here examined, using video analysis to determine topographic changes in the behavioral response during and after training. During spaced training, escape vanishes and is mainly replaced by freezing, while during massed training, escape decreases over trials without being replaced by any defensive response. After 24 h, the marked proclivity to freezing persists in spaced trained crabs, while a high level of escaping is shown by massed trained crabs. The long-lasting freezing preference of spaced trained crabs proves to be context-specific and apparent from the very first presentation of the danger stimulus at testing, though freezing is not triggered by the sole exposure to the context. We conclude (a) that freezing preference is the acquired response of the CSM process; (b) that CSM can be properly categorized as an instance of contextual conditioning and SM of classical habituation; (c) that CSM and SM are not two phases of a memory processing but two distinctly types of memory; and (d) that therefore, the temporal distribution of training trials has a drastic effect on crab's memory, more dramatic than that previously described. The possibility that massed and spaced presentations of the same stimulus may represent two different stimulus types is discussed.     

Massed extinction trials produce better short-term but worse long-term loss of context conditioned fear responses than spaced trials

A series of experiments studied the effects of the interval between extinction trials on the loss of context conditioned freezing responses. Rats were shocked in one context (A) but not in another (B) and subjected to extinction trials in context A. In Experiment 1, massed trials produced more rapid loss than spaced trials. A shift from spaced to massed trials maintained this loss, but the shift from massed to spaced trials restored lost responses. Experiments 2-5 examined this effect of massed trials on responding across spaced trials. They provided evidence that (a) a single trial was as effective as multiple daily massed trials, (b) learning occurred on the first of the massed trials but not on later ones, and (c) the first trial reduced the amount learned across subsequent massed trials. Finally, alternating extinction trials in A and B produced more response loss across spaced trials than blocks of trials in A and B. The results were discussed in terms of the role accorded to self-generated priming in the models developed by A. R. Wagner (1978, 1981).     

The partial punishment effect following minimal acquisition training: Sodium amobarbital and the stimulus properties of early punished trials

In a runway investigation, six groups of rats received limited runway training such that partial punishment, partial reinforcement, or continuous reinforcement was accompanied by sodium amobarbital or saline. Following an interpolated phase of continuous reinforcement without injections, all groups were given punished extinction. The entire experiment was conducted under widely spaced conditions (ITI = 24 hr). It was found that partial punishment increased resistance to punished extinction relative to partially and continuously reinforced controls when acquisition was given under saline. When partial punishment training was accompanied by amobarbital this effect was eliminated. The drug was observed to have no effect on the punished extinction performance of the partial reinforcement and continuous groups, respectively. Moreover, the partial reinforcement effect (PRE) did not generalize to punished extinction. These data provide information concerning the difference between the stimuli associated with the early trials of punishment and nonreward and indicate that the former but not the latter contain emotional elements.     

Opiates and homing

Beginning at 15 days of age. Long-Evans rat pups were trained to run toward their home cage in a T-maze task. Morphine (.5-1.0 mg/kg sc) slowed initial acquisition running times but did not change the number of trials required to learn the position habit. Morphine markedly impeded extinction of the homing behavior. Opiate-treated animals ran as accurately and as quickly toward home on the 12th day of extinction as on the first (10 trials given per day). Conversely, naloxone (1 mg/kg sc) reduced resistance to extinction. The morphine effect was not state-dependent since the drug also impeded extinction in animals that had acquired the task under saline. The morphine effect was blocked by naloxane, which indicates that the increased resistance to extinction was due to an opiate receptor effect. These results indicate that morphine has a strong capacity to sustain a social habit in the absence of reinforcement.     

Infantile stimulation and its effects on frustration- and fear-motivated behavior in rats

Infant rats that were either removed from the nest each day (handled) or left undisturbed (nonhandled) were, in adulthood, given 72 food-reinforced runway acquisition trials followed by 24 trials of extinction training with or without shock. Handled and nonhandled control animals were given runway training without food reinforcement. Reinforced rats ran faster than nonreinforced rats, and handled rats ran faster than nonhandled rats during the initial trials of runway acquisition irrespective of the reinforcement condition. Nonhadled rats stopped running sooner than handled rats when shock was introduced in the goalbox, but differences between handled and nonhandled rats given extinction training without shock were small. Results of a second experiment showed no differences between handled and non-handled rats in the magnitude of the depression effect after an incentive shift. It was concluded that infantile handling had little effect on frustration-motivated behavior, but did affect fear-motivated behavior.     

A comparator view of Pavlovian and differential inhibition

In 3 experiments using rats as subjects, the authors varied trial spacing to investigate the conditions under which Pavlovian and differential inhibition are observed. Experiment 1 compared Pavlovian and differential inhibition with spaced training trials. Spaced trials resulted in only the Pavlovian inhibitor passing both summation and retardation tests. Conversely, Experiment 2 compared these 2 types of inhibition with massed training trials. This training resulted in only the differential inhibitor passing both tests for conditioned inhibition. Finally, in Experiment 3 all subjects experienced Pavlovian inhibition training with massed trials. Although this training by itself did not result in behavior indicative of inhibition, subjects that also experienced posttraining extinction of the training context did pass both tests for inhibition. Overall, these results are anticipated by the extended comparator hypothesis (Denniston, Savastano, & Miller, 2001) but are problematic for most contemporary associative learning theories.     

Resistance to extinction and punishment following training with shock and non-reinforcement: Failure to obtain cross-tolerance

Two experiments investigated the phenomenon of cross-tolerance between the partial reinforcement extinction effect (PREE) and the partial punishment effect (PPE). Three groups of rats were trained in acquisition to run in a straight alley. The continuously reinforced (CRF) group received a reward on every trial. The partially reinforced (PRF) group was rewarded on a quasi-random 50% schedule. The partially punished (PP) group received food reward on every trial but, in addition, received foot shocks of gradually increasing intensity in the goal box on a random 50% of the trials. In the test stage, half of the animals in each training condition were tested in extinction, where no reward was given on any of the trials, and the other half were tested in punishment, with both food and shock presented on each trial. Experiment 1 used a 1-trial/day procedure; Experiment 2 used a multi-trial procedure. In both procedures, clear PREE and PPE were obtained. In the 1-trial/day procedure, no cross-tolerance was evident: animals trained on a PRF or PP schedule did not show increased resistance to punishment and extinction, respectively. In the multi-trial procedure, only weak cross-tolerance was obtained in animals trained on partial reinforcement and tested in punishment.     

Stress-enhanced fear learning in rats is resistant to the effects of immediate massed extinction

Enhanced fear learning occurs subsequent to traumatic or stressful events and is a persistent challenge to the treatment of post-traumatic stress disorder (PTSD). Facilitation of learning produced by prior stress can elicit an exaggerated fear response to a minimally aversive event or stimulus. Stress-enhanced fear learning (SEFL) is a rat model of PTSD; rats previously exposed to the SEFL 15 electrical shocks procedure exhibit several behavioral responses similar to those seen in patients with PTSD. However, past reports found that SEFL is not mitigated by extinction (a model of exposure therapy) when the spaced extinction began 24?h after stress. Recent studies found that extinction from 10?min to 1?h subsequent to fear conditioning "erased" learning, whereas later extinction, occurring from 24 to 72?h after conditioning did not. Other studies indicate that massed extinction is more effective than spaced procedures. Therefore, we examined the time-dependent nature of extinction on the stress-induced enhancement of fear learning using a massed trial's procedure. Experimental rats received 15 foot shocks and were given either no extinction or massed extinction 10?min or 72?h later. Our present data indicate that SEFL, following traumatic stress, is resistant to immediate massed extinction. Experimental rats showed exaggerated new fear learning regardless of when extinction training occurred. Thus, post-traumatic reactivity such as SEFL does not seem responsive to extinction treatments.     

Interaction between amount and pattern of training in the induction of intermediate- and long-term memory for sensitization in aplysia

In Aplysia, three distinct phases of memory for sensitization can be dissociated based on their temporal and molecular features. A single training trial induces short-term memory (STM, lasting <30 min), whereas five trials delivered at 15-min intervals induces both intermediate-term memory (ITM, lasting >90 min) and long-term memory (LTM, lasting >24 h). Here, we explore the interaction of amount and pattern of training in establishing ITM and LTM by examining memory for sensitization after different numbers of trials (each trial = one tail shock) and different patterns of training (massed vs. spaced). Under spaced training patterns, two trials produced STM exclusively, whereas four or five trials each produced both ITM and LTM. Three spaced trials failed to induce LTM but did produce an early decaying form of ITM (E-ITM) that was significantly shorter and weaker in magnitude than the late-decaying ITM (L-ITM) observed after four to five trials. In addition, E-ITM was induced after three trials with both massed and spaced patterns of training. However, L-ITM and LTM after four to five trials require spaced training: Four or five massed trials failed to induce LTM and produced only E-ITM. Collectively, our results indicate that in addition to three identified phases of memory for sensitization—STM, ITM, and LTM—a unique temporal profile of memory, E-ITM, is revealed by varying either the amount or pattern of training.     

Acquisition and extinction of signaled avoidance as a function of intermittent reinforcement

Signaled, shuttle-box avoidance responding in female rats of the Fischer₃₄₄ strain was examined as a function of four separate contingencies of intermittent reinforcement. In Experiment 1, when avoidance responses during acquisition were reinforced 25% of the time with prompt CS termination, animals responded equally often during acquisition and significantly more often during extinction than animals who received such reinforcement on a 100% schedule. Similar results were found under a trace procedure in Experiment 2 when avoidance responses were reinforced 25% of the time with informational feedback stimuli. In contrast, during Experiment 3, when animals were shocked on only 25% of the trials on which they failed to respond, the level of avoidance responding during both acquisition and extinction was significantly less than it was when animals were shocked on a 100% schedule. Comparable results were found in Experiment 4 when avoidance responses during acquisition averted shock on only 25% of the trials. Thus, intermittent reinforcement contingencies involving response-contingent feedback stimuli and shock have differential effects on avoidance responding during both acquisition and extinction trials under the signaled avoidance procedure.     

Effects of medial and lateral septal lesions on the partial reinforcement extinction effect at short inter-trial intervals

Rats sustained electrolytic lesions either in the medial septal (MS) area (of a kind known to eliminate the hippocampal theta rhythm) or in the dorso-lateral septal (LS) area (of a kind known to spare theta) and were compared to sham-operated controls in three experiments in the straight alley with food reward on continuous (CRF) or partial (PRF) reinforcement and inter-trial intervals of 3-8 min. With 6 acquisition trials MS lesions increased resistance to extinction and enhanced the partial reinforcement extinction effect (PREE). With 48 acquisition trials MS lesions did not alter resistance to extinction after either CRF or PRF training, but LS lesions abolished the PREE by increasing resistance to extinction in rats trained with CRF and decreasing it in rats trained with PRF. With 96 acquisition trials LS lesions were without effect on resistance to extinction after either CRF or PRF training, as previously reported by Henke (1974) using total septal lesions. Thus the impairment in the PREE previously shown after large septal lesions is due to damage to the lateral, not the medial, septal area.     

Massed trial Overtraining Effects on Extinction and Reversal Performance in Turtles (Geoclemys reevesii)

Two experiments with turtles (Geoclemys reevesii) examined the overtraining extinction effect (OEE) and the overtraining reversal effect (ORE), under massed training conditions. In Experiment 1, three groups of turtles received 7, 14, or 21 sessions of training in a runway situation for food reinforcement, followed by 15 sessions of extinction. Extinction was faster, the greater the number of acquisition sessions. In Experiment 2, the two groups of turtles learned a spatial discrimination for food reinforcement either to a criterion (19 of 20 correct responses), or to that criterion plus 100 additional trials of overtraining. When the validity of the positions was reversed, learning was faster for the overtrained group than for the group trained to a criterion. This evidence of the OEE and the ORE is the first yet reported for a reptile. The results are discussed in the framework of comparative research on rewardschedule effects.     

Temporally massed CS presentations generate more fear extinction than spaced presentations

Rodent fear conditioning models both excitatory learning and the pathogenesis of human anxiety, whereas extinction of conditional fear is a paradigm of inhibitory learning and the explicit model for behavior therapy. Many studies support a general learning rule for acquisition: Temporally spaced training is more effective than massed training. The authors asked whether this rule applies to extinction of conditional fear in mice. The authors find that both short- and long-term fear extinction are greater with temporally massed presentations of the conditional stimulus (CS). The data also indicate that once CS presentations are sufficiently massed to initiate, or "induce," extinction learning, further CS presentations are more effective when spaced.     

Factors influencing inhibitory stimulus control: discrimination training and prior non-differential reinforcement

In Exp. I, shallow U-shaped gradients of inhibition in the line-orientation dimension were obtained from birds that had a vertical (0°) line on a green surround correlated with extinction and a blank green surround correlated with reinforcement. Birds that had massed extinction in the presence of the 0° line showed flat gradients. Thus, discrimination training, but not massed extinction, appears to generate inhibitory control. In Exp. II, as in studies of control by a stimulus correlated with punishment, non-differential training across the line-orientation dimension preceded further sessions. Steep inverted gradients about the 0° line were obtained after discrimination training with the 0° line correlated with extinction. Gradients obtained after massed extinction tended to be flat. Again, discrimination training was critical in obtaining negative gradients of stimulus control.     

Contrasting reduced overshadowing and blocking

Preexposure of a cue without an outcome (X-) prior to compound pairings with the outcome (XZ-->O) can reduce overshadowing of a target cue (Z). Moreover, pairing a cue with an outcome (X-->O) before compound training can enhance its ability to compete with another cue (i.e., blocking). Four experiments were conducted in a conditioned bar-press suppression preparation with rats to determine whether spacing of the X- or X-->O trials would differentially affect reduced overshadowing and blocking. Experiment 1a showed that reduced overshadowing was larger with massed trials than with spaced trials. Experiment 1b found that blocking was larger with spaced trials than with massed trials. Experiments 2a and 2b indicated that these effects of trial spacing were both mediated by the associative status of the context at test. The results are interpreted in the framework of contemporary learning theories.     

Some experiments on the organization of a class of imitative behaviors

A retarded child was taught to imitate diverse demonstrations made by an experimenter, until new demonstrations were imitated correctly upon first presentation without direct training. These imitations could be maintained without reinforcement, when they were distributed among other reinforced imitations. Factors responsible for the continued performance of these unreinforced imitations were examined. When subjected to massed extinction trials, unreinforced imitations eventually disappeared; they reappeared when again interspersed among reinforced imitations. In addition, the stimulus function of "similarity of response between subject and experimenter" was examined. The subject was taught a set of non-imitative responses, through discriminative stimuli controlled by the experimenter, and a comparable imitative set. Unreinforced non-imitations, like reinforced imitations, were maintained only when interspersed among reinforced imitations. When all reinforcement was discontinued, all responses extinguished similarly, indicating that reinforcement was necessary to maintain the response-class organization, but not confirming an essential role for "similarity" as such.