Partial reinforcement and extinction in vasomotor conditioning

In order to investigate cognitive versus traditional accounts of responding in extinction and the discrimination hypothesis for the partial reinforcement effect, 40 human subjects were randomly divided into two groups and were treated according to thermal vasomotor conditioning procedures using either 25 trials of continuous reinforcement or 100 trials of 25% partial reinforcement. At the onset of extinction, half of each group was given traditional noninformed extinction, while the other (informed) half had the thermal stimulator removed. The usual greater resistance to extinction was obtained after partial reinforcement than after continuous reinforcement in the two noninformed groups; however, immediate extinction of responding was obtained from the first extinction trial in the two informed groups. These results are consistent both with the discrimination hypothesis for the partial reinforcement extinction effect and with cognitive explanations of responding in extinction. Consequences for the behavioral therapies are discussed.     

Effects of reinforcement schedules on rats' choice behavior in extinction.

The effects of reinforcement schedules on rats' choice behavior in extinction were studied. In a free-operant chamber equipped with two retractable bars, the experimental animals were trained to press the bars separately for a food reward. One bar delivered the reward on a continuous reinforcement (CRF) schedule, and the other delivered the reward on a partial reinforcement (PRF) schedule. Control animals earned the reward from both bars with the same reinforcement schedule, either a CRF or a PRF. When both bars were simultaneously available during extinction, the experimental animals responded more frequently to the CRF than to the PRF alternative, demonstrating a reversed within-subjects partial reinforcement extinction effect (PREE). A conventional between-subjects PREE was replicated in the control subjects. The results of this study were inconsistent with both Amsel's (1962, 1967) frustration hypothesis and Capaldi's (1966, 1967) sequential hypotheses.     

Transfer of the partial reinforcement extinction effect between escape (shock) and appetitive (food) conditioning

In the first experiment rats were given partial reinforcement or continuous reinforcement in either an escape or an appetitive paradigm. Subsequently, the rats received continuous reinforcement training under motivational conditions opposite those experienced earlier. Finally, responses were extinguished according to the motivational conditions experienced in the second phase. The results indicated that partial reinforcement in the initial phase operated to increase resistance to extinction in the last. In a second experiment this intermotivational partial reinforcement extinction effect was shown to survive interpolated experiences with extinction, a 1-week rest period, and continuous reinforcement reacquisition. A third experiment examined the influence of intramodal versus intermodal nonreinforcement-reinforcement sequences on the intermotivational partial reinforcement extinction effect. Interactive effects between similarity of aversive outcome (escape nonreinforcement, appetitive nonreinforcement) and reinforcement type (negative, positive) were found. The theoretical implications of the data from all three experiments are discussed.     

Survival of the partial reinforcement extinction effect after contextual shifts

The effects of contextual shifts on the partial reinforcement extinction effect (PREE) were studied in autoshaping with rats. Experiment 1 established that the two contexts used subsequently were easily discriminable and equally salient. In Experiment 2, independent groups of rats received acquisition training under partial reinforcement (PRF) or continuous reinforcement (CRF). Significant PREEs were observed whether extinction occurred in the same or in a different context. In Experiment 3, after PRF or CRF acquisition training, each group received extinction of the same conditioned stimulus in both the same and different contexts. PREEs were observed under both conditions, although extinction was accelerated after a contextual shift. These results were discussed in relation to the role of mediational states in the PREE.     

Between-subject PREE and within-subject reversed PREE in spaced-trial extinction with pigeons

Three experiments explored the partial reinforcement extinction effect (PREE; greater resistance to extinction after partial, rather than continuous, reinforcement training), in a spaced-trial situation with pigeons. Experiments 1 and 2 report conventional PREEs with 24-h intertrial intervals and between-subject designs. The corresponding outcome (food reinforcement or nonreinforcement) was delivered after satisfaction of a fixed-ratio 10 (Experiment 1) or a fixed-ratio 1 (Experiment 2). Experiment 3 reports a reversed PREE in a within-subject design with a fixed-ratio 10 requirement. Extinction occurred faster for the response paired with 50% partial reinforcement than for the response paired with continuous reinforcement. A third response paired with a small reinforcer (1 pellet/trial) in 100% of the trials extinguished faster than a response paired with a large reinforcer (15 pellets/trial). These results are discussed in the context of spaced-trial instrumental performance (key pecking and running), in pigeons.     

The effects of early handling on the partial reinforcement extinction effect and the partial punishment effect in male and female rats

Handled (Day 1-22) and non-handled infantile Wistar rats were tested in maturity for the partial reinforcement extinction effect (PREE) and the partial punishment effect (PPE). In Experiments 1 and 2, mature male and female rats were trained to run in an alley for food reward on a 1-trial/day schedule. In the PREE paradigm (Experiment 1), the partially reinforced group (PRF) received reinforcement on a quasi-random 50% schedule, while the continuously reinforced group (CRF) received reinforcement on every trial. In the test stage, both groups were given extinction training. In the PPE paradigm (Experiment 2), the partially punished (PP) group received, together with continuous reinforcement, shocks on a quasirandom 50% schedule, while the continuously reinforced group was reinforced on every trial. In test, all animals were given both reinforcement and shock on every trial. In Experiment 1, PREE—i.e. increased resistance to extinction in the PRF as compared to the CRF group—was more pronounced in the handled animals. More specifically, no PREE was obtained in the non-handled males, and in the non-handled females the PREE was reduced compared to the handled females. The results of Experiment 2 revealed no effect of handling or sex on PPE, that is, increased resistance to punishment in the PP group as compared to the CRF group was evident in all four conditions. In Experiment 3, handled and non-handled male rats were tested for the PREE using a multi-trial procedure in an operant chamber. PREE was obtained in the handled but not in the non-handled animals. The implications of these results for the differential effects of handling on male and female rats and the distinction between the PREE and PPE paradigms are discussed.     

A partial reinforcement extinction effect despite equal rates of reinforcement during Pavlovian conditioning

In 4 experiments rats received appetitive Pavlovian conditioning followed by extinction. Food accompanied every trial with the conditioned stimulus (CS) for the continuously reinforced groups and only half of the trials for the partially reinforced groups. In contrast to previous experiments that have compared the effects of partial and continuous reinforcement, the rate at which food was delivered during the CS was the same for both groups. The strength of the conditioned response during extinction weakened more rapidly in the continuously than in the partially reinforced groups. The results demonstrate that the partial reinforcement extinction effect is a consequence of the nonreinforced trials with the CS, rather than the rate at which the unconditioned stimulus is delivered during the CS.     

Magnitude of negative reinforcement and resistance to extinction

In the first experiment rats experienced large or small magnitude of negative reinforcement (shock reduction) in a straight alley. Half of the subjects in each magnitude group received continuous reinforcement, and the other half received a 50% partial reinforcement schedule (nonreinforcement consisting of no shock reduction in the goal box). In extinction the groups were ordered: large partial > small-partial > small-continuous > large-continuous. In the second experiment rats received large, small and nonreinforcement in various sequences using the runway-negative reinforcement procedure and were ordered: SNL>LNL>SNL>LNS in resistance to extinction (letters represent the magnitudes in the sequence experienced in acquisition). The results of these experiments indicate a commality between positive and negative reinforcement with respect to behavioral phenomena and theoretical accounts of those phenomena.     

Partial reinforcement effects in instrumental escape conditioning

In Experiment I acquisition and extinction of instrumental escape conditioning with rats (N = 64) were studied as a function of reinforcement magnitude under conditions of partial and continuous reinforcement. In Experiment II the effects of partial and continuous reinforcement were studied in rats (N = 96) during acquisition followed by small, medium, and large reductions in reinforcement magnitude. A water-tank escape apparatus was used with temperature as the relevant variable. It was found that (1) with large reinforcement magnitude a continuously reinforced group was superior in acquisition to one that was partially reinforced; there were no differences with small reinforcement; (2) disruptive effects of a nonreinforced trial (a) appear early in learning, (b) are quite strong after each nonreinforced trial, and (c) persist through several succeeding reinforced trials; (3) major competing behaviors persist throughout acquisition for small reinforcement magnitude regardless of schedule, decline with large reinforcement (more so with continuous than with partial), and return to a high level in extinction for all conditions; (4) the partial reinforcement extinction effect occurs after large reinforcement but not after small, and it appears only with large reductions in reinforcement magnitude which approach extinction conditions. Only the first part of the last finding appears to be consistent with the appetitive conditioning literature.     

Within-subject Partial Reinforcement Extinction Effect in Autoshaping

Three autoshaping experiments explored the extinction of various keylight stimuli conditioned with partial and continuous reinforcement within the same pigeon subjects. Extinction of autoshaped keypecking proceeded more slowly in a stimulus trained with 50% (Experiment 1) and 25% (Experiment 2) reinforcement compared with a stimulus given 100% reinforcement. Experiment 3 found conditioning with 75% reinforcement to generate more rapid extinction than that with 25% reinforcement. These results have both theoretical and methodological implications for the study of extinction.     

A sequential analysis of the partial reinforcement extinction effect in children*

In a series of studies employing children between the ages of 7.6 and 10.6 years of age guessing on a modified Humphreys’s board, extinction training was administered following either continuous reinforcement or various schedules of partial reinforcement training. Besides the observation of a partial reinforcement extinction effect, it was found that resistance to extinction appeared to be regulated by those sequential variables specified by extensions of Capaldi’s theory of instrumental learning.     

Early extinction effects following intermittent reinforcement: Little evidence of extinction bursts

The occurrence of extinction bursts—transient increases in response rate in excess of those observed in baseline during the period immediately following discontinuation of reinforcement of a response—was examined. In Experiment 1, key pecking of pigeons was reinforced according to a multiple schedule in which a variable-ratio schedule alternated with an interval schedule in which the reinforcers were yoked to the preceding variable-ratio component. In Experiments 2 and 3, rats were screened such that the lever-press response rates of different rats maintained by variable-interval schedules were either relatively high or relatively low. Following these baseline conditions, in Experiments 1 and 2 responding was extinguished by eliminating the food reinforcer and in Experiment 3 by removing the response–reinforcer dependency. Responses immediately following extinction implementation were examined. Response increases relative to baseline during the first 20 min of a 324.75-min extinction session (Experiment 1) or during the first 30-min extinction session (Experiments 2 and 3) were rare and unsystematic. The results (a) reinforce earlier meta-analyses concluding that extinction bursts may be a less ubiquitous early effect of extinction than has been suggested and (b) invite further experimentation to establish their generality as a function of preceding reinforcement conditions.     

Effects of medial and lateral septal lesions on the partial reinforcement extinction effect at short inter-trial intervals

Rats sustained electrolytic lesions either in the medial septal (MS) area (of a kind known to eliminate the hippocampal theta rhythm) or in the dorso-lateral septal (LS) area (of a kind known to spare theta) and were compared to sham-operated controls in three experiments in the straight alley with food reward on continuous (CRF) or partial (PRF) reinforcement and inter-trial intervals of 3-8 min. With 6 acquisition trials MS lesions increased resistance to extinction and enhanced the partial reinforcement extinction effect (PREE). With 48 acquisition trials MS lesions did not alter resistance to extinction after either CRF or PRF training, but LS lesions abolished the PREE by increasing resistance to extinction in rats trained with CRF and decreasing it in rats trained with PRF. With 96 acquisition trials LS lesions were without effect on resistance to extinction after either CRF or PRF training, as previously reported by Henke (1974) using total septal lesions. Thus the impairment in the PREE previously shown after large septal lesions is due to damage to the lateral, not the medial, septal area.     

Punishment persistence and sequence: Some effects of P-length and N-length in the rat

In a runway investigation, two groups of rats received partial punishment training with P-lengths of 1, 2, and 3 (Group P123) and P-lengths of 1 (Group P1), respectively. Two additional groups received partial reinforcement with N-lengths of 1, 2, and 3 (Group N123) and N-length of 1 (Group N1). An additional group was given unpunished continuous reinforcement in the runway, but received control shocks in a separate apparatus. Following training all subjects received punished extinction (shock plus nonreward). The results indicated that P-length increased resistance to punished extinction, however N-length did not have the corresponding effect. In addition, partial reinforcement did not increase resistance to punished extinction relative to continuous reinforcement. These results were interpreted within a sequential-theoretical framework.     

Effect of schedule and magnitude of reinforcement on instrumental learning in the toad, Bufo arenarum

Extinction after training with continuous (CR) or 50% partial (PR) reinforcement, and with different magnitudes of reward, was studied in the amphibian Bufo arenarum, in a runway situation. In Experiment 1, a group of toads received massed-trial, CR training with access to water as the reward. Performance improved during acquisition, including an improvement on the first trial of each session. Extinction was rapid and there was evidence for spontaneous recovery of the running response. In Experiment 2, groups of toads received PR or CR training at a rate of one trial per day. PR impaired acquisition and resulted in poor responding during extinction, compared to CR. Experiment 3 factorially studied the effects of schedule (PR vs CR) and distribution of practice (15 s vs 300 s intertrial interval). Acquisition was impaired by PR training but had little effect on extinction performance. Different magnitudes of water reinforcement were used in Experiment 4 in a one-trial-per-day situation. Terminal acquisition performance was a monotonic function of reward magnitude, but there were no differences in extinction performance across groups. The results are discussed in relation to comparative and developmental data on the paradoxical effects of reward.     

Resistance to continuous delay of reinforcement or extinction following partial delay or partial reinforcement in acquisition: A direct comparison

Rats received either partial reward (PR) or partial delay (PD) in acquisition with one, two, or three delay or nonreward trials followed by an immediately rewarded trial or one delay or nonreward trial followed by an immediately rewarded trial. These four groups were then split in half and given either continuous delay or continuous nonreward (extinction) in a “response persistence” phase. In addition, two continuously reinforced groups, one experiencing continuous delay, and the other experiencing extinction were included. The results showed that response persistence was greater when PD groups were given continuous delay rather than extinction, but the opposite was true for PR groups. The “length” (1, 2, or 3 versus 1 nonreward or delay trial) also transferred to the response persistence phase with the length 1, 2, and 3 conditions being more persistent than the 1 length conditions. The results were discussed with respect to theoretical notions of response persistence.     

Partial reinforcement effects and type of reward

In two experiments 68 rats were trained to bar press or run down a straight runway for food or for water under conditions of either continuous reinforcement or partial reinforcement. In both experiments, there was greater persistence of behavior which had been reinforced with food than with water. In Expt 2, the partial reinforcement extinction effect (PREE) was observed with food reward, but not with water.Within the context of the experimental procedures used, it can be concluded that the rat has mechanisms for developing persistence which are dependent on the specific motivational system involved. This conclusion is related to theories of partial reinforcement effects and to possible biological origins of the mechanisms.     

Overlearning and instructional control of extinction of vasomotor responding

In a previous study by the authors, immediate extinction of conditioned vasomotor responding was obtained, under conditions of both continuous and partial reinforcement, when the UCS delivery apparatus was removed and subjects were informed that there would be no further UCS presentations. The present study varied the number of continuous reinforcement trials using the same conditioning procedure. Forty human subjects were randomly divided into two groups, given thermal vasomotor conditioning procedures on either 25 or 100 continuous reinforcement trials. At the onset of extinction half of each group was given traditional noninformed extinction procedures, while the other (informed) half had the thermal stimulator removed. Immediate extinction was obtained in informed subjects given 25 conditioning trials. However, there was no significant reduction of responding in informed subjects given 100 conditioning trials. Consequences for behavior theories and therapies are discussed.     

Determinants of instrumental extinction in terrestrial toads (Bufo arenarum)

Previous research in a water-reinforced instrumental training situation with toads (Bufo arenarum) has shown that performance in both acquisition and extinction is poorer after partial, rather than continuous reinforcement training. In Experiment 1, the performance of a group receiving 24 trials on a 50% partial reinforcement schedule was poorer in acquisition and extinction than that of continuously reinforced groups matched for trials or reinforcements. However, partially reinforced toads extinguished at the same rapid rate as a continuously reinforced group that received training only on the days in which the partial toads received water reinforcement. In Experiment 2, extinction was faster after 10 reinforced acquisition trials than after 30 trials. This evidence suggests that the deleterious effects of partial reinforcement in toads can be explained by a combination of two factors, namely, the distribution of reinforced trials across days and the total number of reinforcements.     

Partial Reinforcement in Appetitive Pavlovian Conditioning with Rats

Four experiments examined the effects of a partial reinforcement schedule on extinction using appetitive Pavlovian conditioning. Extinction was slower after partial than after continuous reinforcement when the schedules were administered to different groups (Experiment 1). The opposite result was found in Experiments 2 and 3 when both schedules were presented to the same group in the same context. When the schedules were presented to the same group in different contexts, then extinction was again slower after partial reinforcement (Experiment 3). Experiment 4 demonstrated that a change of context facilitates extinction to a greater extent after conditioning with a partial reinforcement schedule than with a continuous one. The results are explained by assuming that the nonreinforced trials of a partial reinforcement schedule create an internal state that serves as a contextual cue.