Magnitude of negative reinforcement and resistance to extinction

In the first experiment rats experienced large or small magnitude of negative reinforcement (shock reduction) in a straight alley. Half of the subjects in each magnitude group received continuous reinforcement, and the other half received a 50% partial reinforcement schedule (nonreinforcement consisting of no shock reduction in the goal box). In extinction the groups were ordered: large partial > small-partial > small-continuous > large-continuous. In the second experiment rats received large, small and nonreinforcement in various sequences using the runway-negative reinforcement procedure and were ordered: SNL>LNL>SNL>LNS in resistance to extinction (letters represent the magnitudes in the sequence experienced in acquisition). The results of these experiments indicate a commality between positive and negative reinforcement with respect to behavioral phenomena and theoretical accounts of those phenomena.     

Partial reinforcement effects in instrumental escape conditioning

In Experiment I acquisition and extinction of instrumental escape conditioning with rats (N = 64) were studied as a function of reinforcement magnitude under conditions of partial and continuous reinforcement. In Experiment II the effects of partial and continuous reinforcement were studied in rats (N = 96) during acquisition followed by small, medium, and large reductions in reinforcement magnitude. A water-tank escape apparatus was used with temperature as the relevant variable. It was found that (1) with large reinforcement magnitude a continuously reinforced group was superior in acquisition to one that was partially reinforced; there were no differences with small reinforcement; (2) disruptive effects of a nonreinforced trial (a) appear early in learning, (b) are quite strong after each nonreinforced trial, and (c) persist through several succeeding reinforced trials; (3) major competing behaviors persist throughout acquisition for small reinforcement magnitude regardless of schedule, decline with large reinforcement (more so with continuous than with partial), and return to a high level in extinction for all conditions; (4) the partial reinforcement extinction effect occurs after large reinforcement but not after small, and it appears only with large reductions in reinforcement magnitude which approach extinction conditions. Only the first part of the last finding appears to be consistent with the appetitive conditioning literature.     

Positive contrast obtained in reacquisition following interpolation of nonreinforced or partially reinforced trials

Two experiments investigated the effects of shifting from either nonreinforcement or partial reinforcement (PRF) to continuous reinforcement conditions (CRF). In the first experiment, three groups of rats were given food reinforcement under CRF conditions in a runway followed by regular extinction trials (RE), extinction trials where Ss were delayed for 30 sec before entering the empty goal box (DE), or CRF trials where Ss were delayed for 30 sec before entering the baited goal box (DF). Then all Ss were run on the delayed reinforcement condition (DF). In the final delayed reinforcement condition, group DE ran significantly faster than group DF, reflecting positive contrast. In the second experiment, four groups of rats were trained in a runway to receive either 4% or 18% sucrose reinforcers under either PRF or CRF conditions. Then all Ss were transferred to a Skinner box and bar presses were continuously reinforced, with each S continuing to receive the same sucrose concentration as before. The former PRF Ss, regardless of the reinforcer, bar pressed at a significantly higher rate in the Skinner box than the former CRF Ss. The evidence seemed to favor the view that the effectiveness of a reinforcer is not an absolute, unchanging quantity but rather depends on the historical context in which the reinforcer occurs.     

The partial punishment effect following minimal acquisition training: Sodium amobarbital and the stimulus properties of early punished trials

In a runway investigation, six groups of rats received limited runway training such that partial punishment, partial reinforcement, or continuous reinforcement was accompanied by sodium amobarbital or saline. Following an interpolated phase of continuous reinforcement without injections, all groups were given punished extinction. The entire experiment was conducted under widely spaced conditions (ITI = 24 hr). It was found that partial punishment increased resistance to punished extinction relative to partially and continuously reinforced controls when acquisition was given under saline. When partial punishment training was accompanied by amobarbital this effect was eliminated. The drug was observed to have no effect on the punished extinction performance of the partial reinforcement and continuous groups, respectively. Moreover, the partial reinforcement effect (PRE) did not generalize to punished extinction. These data provide information concerning the difference between the stimuli associated with the early trials of punishment and nonreward and indicate that the former but not the latter contain emotional elements.     

Changes Accompanying Practice upon Successive Samples of Verbal Material

Three groups of rats received either continuous, partial, or zero reinforcement in a first acquisition phase which was followed by an extended extinction phase. Then all Ss were given a reacquisition phase under continuous reinforcement conditions followed by a second extinction phase. While the usual partial reinforcement extinction effect (PREE) was found during the major part of the first extinction phase, it disappeared during the last few trials of that phase. No PREE was obtained during the second extinction phase in any of the three sections of the runway. The abolition of the PREE was attributed to the extinction of the r_F-s_F mechanism in the partial reinforcement group.     

Partial reinforcement effects and type of reward

In two experiments 68 rats were trained to bar press or run down a straight runway for food or for water under conditions of either continuous reinforcement or partial reinforcement. In both experiments, there was greater persistence of behavior which had been reinforced with food than with water. In Expt 2, the partial reinforcement extinction effect (PREE) was observed with food reward, but not with water.Within the context of the experimental procedures used, it can be concluded that the rat has mechanisms for developing persistence which are dependent on the specific motivational system involved. This conclusion is related to theories of partial reinforcement effects and to possible biological origins of the mechanisms.     

Transfer of approach responding between punishment and frustrative nonreward sustained through continuous reinforcement

A block of continuously reinforced nonpunished trials was interpolated between acquisition of a runway response with either partial reinforcement or intermittent punishment and subsequent tests for resistance of that response to the suppressive effects of either extinction or continuous punishment. As previous investigations have shown, both the partial reinforcement effect (PRE) and the intermittent punishment effect (IPE) were sustained through the block of continuously reinforced nonpunished trials. Furthermore, the increased resistance to extinction following intermittent punishment and the increased resistance to punishment following partial reinforcement were also sustained through the interpolated continuous reinforcement. These results support a hypothesized similarity of punishment and frustrative nonreward and were interpreted in an extension of Amsel's conditioning model theory of the role of nonreinforcement in the PRE.     

Perseveration of the partial reinforcement effect in extinction with rats over two phases of extinction and two stages of continuous reinforcement

One group of 10 male albino rats was given partial reinforcement while the other 10 rats received continuous reinforcement in a straight alley. Subjects then experienced five consecutive stages of Extinction 1, Continuous Reinforcement 1, Extinction 2, Continuous Reinforcement 2, and finally, Extinction 3. Analysis showed the partial reinforcement effect in extinction was sustained over two stages of extinction and two stages of continuous reinforcement, since subjects receiving partial reinforcement ran faster than rats given continuous reinforcement throughout all three of the extinction periods. The results seem to support those of Amsel's (1967) and Cabpaldi's (1967) theoretical formulations of the partial reinforcement effect in extinction.     

Effects of schedule and magnitude of reinforcement under conditions of thirst motivation

In contrast to a recent finding (Macdonald, G. E., & De Toledo, L. Learning and Motivation, 1974, 5, 288–298.) the results of three experiments investigating various partial reinforcement (PRF) manipulations under conditions of thirst motivation demonstrated strong similarity to analogous manipulations involving food reward. Specifically, for animals receiving water reinforcement, PRF was shown to generate greater resistance to extinction than continuous reinforcement (Expt 1 & 3), the schedule of reinforcement was shown to interact with level of acquisition (Expt 1 and 2), and the magnitude of the partial reinforcement extinction effect was shown to be a function of reward magnitude (Expt 3). These results provide strong evidence that mechanisms which operate in partial reinforcement situations are highly similar, regardless of the type of appetitive reinforcement.     

More rapid associative change with retraining than with initial training

Retraining was compared with initial acquisition in 4 magazine approach experiments with rats and 1 autoshaping experiment with pigeons. The levels of performance were matched prior to reinforcement and nonreinforcement for stimuli with a history of both training and extinction and stimuli with only a minimal history of training. Under these conditions, a previously extinguished stimulus was more vulnerable both to the incremental effects of reinforcement and to the decremental effects of nonreinforcement compared with a stimulus that had only been minimally trained.     

The order of continuous,partial and nonreward trials and resistance to extinction

In the first experiment rats were given either 16 or 48 nonrewarded or continuously rewarded trials prior to 24 continuously or partially rewarded trials, followed by extinction. Increased resistance to extinction was found for increasing numbers of nonrewarded trials when they were followed by partial reward, but not when followed by continuous reward. Similarly, more continuously rewarded trials followed by partial reward tended to increase resistance to extinction. Because of the theoretical importance of the effect of continuous reward followed by partial, a second experiment was performed where the range of the number of continuously rewarded trials was extended to 0, 48, and 96. Contrary to many theoretical predictions, resistance to extinction increased as a function of increasing amounts of continuous reward.     

A partial reinforcement extinction effect despite equal rates of reinforcement during Pavlovian conditioning

In 4 experiments rats received appetitive Pavlovian conditioning followed by extinction. Food accompanied every trial with the conditioned stimulus (CS) for the continuously reinforced groups and only half of the trials for the partially reinforced groups. In contrast to previous experiments that have compared the effects of partial and continuous reinforcement, the rate at which food was delivered during the CS was the same for both groups. The strength of the conditioned response during extinction weakened more rapidly in the continuously than in the partially reinforced groups. The results demonstrate that the partial reinforcement extinction effect is a consequence of the nonreinforced trials with the CS, rather than the rate at which the unconditioned stimulus is delivered during the CS.     

Appetitive conditioning in Octopus cyanea

The performance of Octopus cyanea was studied in 3 appetitive conditioning situations. In Experiment 1, 2 groups were trained in a runway; a large reward produced faster acquisition when reinforcement was consistent and better subsequent performance on a partial schedule than did a small reward. In Experiment 2, activity in the vicinity of a feeder was measured, and in Experiment 3, latency and probability of response were measured in an automated version of a traditional conditioned attack situation (Boycott & Young, 1950). There was evidence of acquisition with continuous reinforcement in both experiments but in neither with partial reinforcement. All of the results can be understood in terms of growth and decline in the strength of stimulus-reinforcer associations with reinforcement and nonreinforcement.     

Survival of the partial reinforcement extinction effect after contextual shifts

The effects of contextual shifts on the partial reinforcement extinction effect (PREE) were studied in autoshaping with rats. Experiment 1 established that the two contexts used subsequently were easily discriminable and equally salient. In Experiment 2, independent groups of rats received acquisition training under partial reinforcement (PRF) or continuous reinforcement (CRF). Significant PREEs were observed whether extinction occurred in the same or in a different context. In Experiment 3, after PRF or CRF acquisition training, each group received extinction of the same conditioned stimulus in both the same and different contexts. PREEs were observed under both conditions, although extinction was accelerated after a contextual shift. These results were discussed in relation to the role of mediational states in the PREE.     

Between-subject PREE and within-subject reversed PREE in spaced-trial extinction with pigeons

Three experiments explored the partial reinforcement extinction effect (PREE; greater resistance to extinction after partial, rather than continuous, reinforcement training), in a spaced-trial situation with pigeons. Experiments 1 and 2 report conventional PREEs with 24-h intertrial intervals and between-subject designs. The corresponding outcome (food reinforcement or nonreinforcement) was delivered after satisfaction of a fixed-ratio 10 (Experiment 1) or a fixed-ratio 1 (Experiment 2). Experiment 3 reports a reversed PREE in a within-subject design with a fixed-ratio 10 requirement. Extinction occurred faster for the response paired with 50% partial reinforcement than for the response paired with continuous reinforcement. A third response paired with a small reinforcer (1 pellet/trial) in 100% of the trials extinguished faster than a response paired with a large reinforcer (15 pellets/trial). These results are discussed in the context of spaced-trial instrumental performance (key pecking and running), in pigeons.     

Partial Reinforcement in Appetitive Pavlovian Conditioning with Rats

Four experiments examined the effects of a partial reinforcement schedule on extinction using appetitive Pavlovian conditioning. Extinction was slower after partial than after continuous reinforcement when the schedules were administered to different groups (Experiment 1). The opposite result was found in Experiments 2 and 3 when both schedules were presented to the same group in the same context. When the schedules were presented to the same group in different contexts, then extinction was again slower after partial reinforcement (Experiment 3). Experiment 4 demonstrated that a change of context facilitates extinction to a greater extent after conditioning with a partial reinforcement schedule than with a continuous one. The results are explained by assuming that the nonreinforced trials of a partial reinforcement schedule create an internal state that serves as a contextual cue.     

Differential involvement of the central amygdala in appetitive versus aversive learning

Understanding the function of the distinct amygdaloid nuclei in learning comprises a major challenge. In the two studies described herein, we used c-Fos immunolabeling to compare the engagement of various nuclei of the amygdala in appetitive and aversive instrumental training procedures. In the first experiment, rats that had already acquired a bar-pressing response to a partial food reinforcement were further trained to learn that an acoustic stimulus signaled either continuous food reinforcement (appetitive training) or a footshock (aversive training). The first training session of the presentation of the acoustic stimulus resulted in significant increases of c-Fos immunolabeling throughout the amygdala; however, the pattern of activation of the nuclei of the amygdala differed according to the valence of motivation. The medial part of the central amygdala (CE) responded, surprisingly, to the appetitive conditioning selectively. The second experiment was designed to extend the aversive versus appetitive conditioning to mice, trained either for place preference or place avoidance in an automated learning system (INTELLICAGE). Again, much more intense c-Fos expression was observed in the medial part of the CE after the appetitive training as compared to the aversive training. These data, obtained in two species and by means of novel experimental approaches balancing appetitive versus aversive conditioning, support the hypothesis that the central nucleus of the amygdala is particularly involved in appetitively motivated learning processes.     

Extinction of the context and latent inhibition

The hypothesis that latent inhibition could be reduced by extinguishing the experimental context, that is, exposing the rats to the context between exposure to the conditional stimulus (CS) and conditioning, was tested. All experiments used the conditioned emotional response procedure. In Experiment 1, extinction was not effective when the animals were exposed to the clicker 40 times off the baseline of responding for food and when the clicker CS was partially reinforced with shocks during the test phase. In Experiments 2 and 3, latent inhibition could be reduced by extinction if the animals were exposed to the CS 24 or 16 times on-baseline, and if continuous reinforcement was used during the test. In Experiments 4, 5, and 6, we attempted to determine which variable was responsible for the discrepant results. In Experiment 4, extinction was effective with 20 or 40 on-baseline exposures to the CS, using continuous reinforcement during the test. In Experiment 5, extinction was not effective with exposure on- or off-baseline, using 24 exposures and partial reinforcement. Finally, in Experiment 6, extinction reduced latent inhibition using continuous, but not partial, reinforcement with 40 exposures off-baseline. From these results, we concluded that Wagner's model of habituation was not sufficient to account for latent inhibition and that a hybrid model, using both associative and cognitive representational processes, was necessary.     

Recovery of the US representation over time during extinction

Four experiments used a conditioned suppression procedure in rats to explore changes in the US representation over time during the course of extinction. They employed two previously reported effects: reinstatement of responding to an extinguished CS by separate US presentation, and the erasure of that effect by interposed nonreinforcement of a second excitatory CS. These effects have been interpreted as enhancing and depressing the US representation, respectively. Experiment 1 found the erasing effect to decrease but still to remain substantial after over a 4-day period, suggesting a partial recovery with time of a deliberately depressed US representation. Experiment 2 implicated this change as a contributor to the phenomenon of spontaneous recovery by showing that recovery to be sensitive to erasure effects. Experiments 3 and 4 found evidence for an interaction between the state of the US representation and the amount of associative change which results from nonreinforcement of an excitatory CS. When the US representation was strong, either because of reinstatement or the passage of time, nonreinforcement of a CS was especially effective in producing associative change. When the US representation had been depressed by erasure, those nonreinforcements produced relatively less associative loss. Moreover, these effects upon associations were reasonably stable in the sense that they left asymptotic differences in the strength of associations after extinction. Together with previous findings, these results point to an important role for the US representation in the performance and learning which occurs during extinction.     

Partial reinforcement effects in classical aversive conditioning in rabbits and human beings.

The partial reinforcement extinction effect (the PREE) in classical aversive conditioning was investigated in 2 experiments. In the first, the nictitating membrane responses of 120 rabbits were conditioned at a 250-msec. interstimulus interval (ISI) under continuous reinforcement, partial reinforcement with the unconditioned stimulus (US) omitted (Group PO), or partial reinforcement with the US delayed to 1,500 msec. (Group PD). These 3 groups were factorially extinguished under US-Omitted, US-Unpaired, or US-Delayed extinction regimens. A significant PREE was obtained, but only for PO training and US-Omitted extinction. The second experiment, employing human subjects in a masked eye blink conditioning task, produced parallel results. A general discrimination view of the classical PREE seems applicable, but one in which neither cognitive factors nor intertrial conditioning of reinforcement aftereffects play a significant role.