The effects of early handling on the partial reinforcement extinction effect and the partial punishment effect in male and female rats

Handled (Day 1-22) and non-handled infantile Wistar rats were tested in maturity for the partial reinforcement extinction effect (PREE) and the partial punishment effect (PPE). In Experiments 1 and 2, mature male and female rats were trained to run in an alley for food reward on a 1-trial/day schedule. In the PREE paradigm (Experiment 1), the partially reinforced group (PRF) received reinforcement on a quasi-random 50% schedule, while the continuously reinforced group (CRF) received reinforcement on every trial. In the test stage, both groups were given extinction training. In the PPE paradigm (Experiment 2), the partially punished (PP) group received, together with continuous reinforcement, shocks on a quasirandom 50% schedule, while the continuously reinforced group was reinforced on every trial. In test, all animals were given both reinforcement and shock on every trial. In Experiment 1, PREE—i.e. increased resistance to extinction in the PRF as compared to the CRF group—was more pronounced in the handled animals. More specifically, no PREE was obtained in the non-handled males, and in the non-handled females the PREE was reduced compared to the handled females. The results of Experiment 2 revealed no effect of handling or sex on PPE, that is, increased resistance to punishment in the PP group as compared to the CRF group was evident in all four conditions. In Experiment 3, handled and non-handled male rats were tested for the PREE using a multi-trial procedure in an operant chamber. PREE was obtained in the handled but not in the non-handled animals. The implications of these results for the differential effects of handling on male and female rats and the distinction between the PREE and PPE paradigms are discussed.     

Persistent behaviour in extinction after partial deprivation in training

Four groups of rats were trained to run an alleyway with one trial per day. Two groups were always deprived when trained while the other two received a partial deprivation schedule. One group of each pair received a continuous reward in the goal box while the other received partial reward. A partial reinforcement effect was found during extinction. The partially deprived groups also showed persistence in extinction. This result extends parallels between the effects of satiation and nonreward upon behaviour.     

Effects of reinforcement schedules on rats' choice behavior in extinction.

The effects of reinforcement schedules on rats' choice behavior in extinction were studied. In a free-operant chamber equipped with two retractable bars, the experimental animals were trained to press the bars separately for a food reward. One bar delivered the reward on a continuous reinforcement (CRF) schedule, and the other delivered the reward on a partial reinforcement (PRF) schedule. Control animals earned the reward from both bars with the same reinforcement schedule, either a CRF or a PRF. When both bars were simultaneously available during extinction, the experimental animals responded more frequently to the CRF than to the PRF alternative, demonstrating a reversed within-subjects partial reinforcement extinction effect (PREE). A conventional between-subjects PREE was replicated in the control subjects. The results of this study were inconsistent with both Amsel's (1962, 1967) frustration hypothesis and Capaldi's (1966, 1967) sequential hypotheses.     

Acquisition and extinction of signaled avoidance as a function of intermittent reinforcement

Signaled, shuttle-box avoidance responding in female rats of the Fischer₃₄₄ strain was examined as a function of four separate contingencies of intermittent reinforcement. In Experiment 1, when avoidance responses during acquisition were reinforced 25% of the time with prompt CS termination, animals responded equally often during acquisition and significantly more often during extinction than animals who received such reinforcement on a 100% schedule. Similar results were found under a trace procedure in Experiment 2 when avoidance responses were reinforced 25% of the time with informational feedback stimuli. In contrast, during Experiment 3, when animals were shocked on only 25% of the trials on which they failed to respond, the level of avoidance responding during both acquisition and extinction was significantly less than it was when animals were shocked on a 100% schedule. Comparable results were found in Experiment 4 when avoidance responses during acquisition averted shock on only 25% of the trials. Thus, intermittent reinforcement contingencies involving response-contingent feedback stimuli and shock have differential effects on avoidance responding during both acquisition and extinction trials under the signaled avoidance procedure.     

Sodium amylobarbitone and responses to nonreward

Three experiments are reported testing two alternative hypotheses concerning the behavioural effects of sodium amylobarbitone (SA): (1) that it blocks the after-effect of nonreward; (2) that it blocks conditioned frustration, elicited by stimuli associated with nonreward. In support of (2) Experiment I showed that SA given in acquisition abolished the partial reinforcement extinction effect (PREE) when rats were run at one trial a day in an alley for food reward on a continuous (CRF) or partial (PRF) reinforcement schedule. Experiment II showed that, in the goal section, the effect of the drug on the PREE was due to its presence during acquisition and was not due to state dependency; but the effect of the drug in the start section was consistent with state dependency of the PREE. In Experiment III, in opposition to (1) and again in support of (2), SA given to rats trained to show patterned running for water reward on a single alternation schedule blocked patterning by increasing running speeds on nonreward trials, not by decreasing running speeds on rewarded trials.     

Effect of schedule and magnitude of reinforcement on instrumental learning in the toad, Bufo arenarum

Extinction after training with continuous (CR) or 50% partial (PR) reinforcement, and with different magnitudes of reward, was studied in the amphibian Bufo arenarum, in a runway situation. In Experiment 1, a group of toads received massed-trial, CR training with access to water as the reward. Performance improved during acquisition, including an improvement on the first trial of each session. Extinction was rapid and there was evidence for spontaneous recovery of the running response. In Experiment 2, groups of toads received PR or CR training at a rate of one trial per day. PR impaired acquisition and resulted in poor responding during extinction, compared to CR. Experiment 3 factorially studied the effects of schedule (PR vs CR) and distribution of practice (15 s vs 300 s intertrial interval). Acquisition was impaired by PR training but had little effect on extinction performance. Different magnitudes of water reinforcement were used in Experiment 4 in a one-trial-per-day situation. Terminal acquisition performance was a monotonic function of reward magnitude, but there were no differences in extinction performance across groups. The results are discussed in relation to comparative and developmental data on the paradoxical effects of reward.     

Determinants of instrumental extinction in terrestrial toads (Bufo arenarum)

Previous research in a water-reinforced instrumental training situation with toads (Bufo arenarum) has shown that performance in both acquisition and extinction is poorer after partial, rather than continuous reinforcement training. In Experiment 1, the performance of a group receiving 24 trials on a 50% partial reinforcement schedule was poorer in acquisition and extinction than that of continuously reinforced groups matched for trials or reinforcements. However, partially reinforced toads extinguished at the same rapid rate as a continuously reinforced group that received training only on the days in which the partial toads received water reinforcement. In Experiment 2, extinction was faster after 10 reinforced acquisition trials than after 30 trials. This evidence suggests that the deleterious effects of partial reinforcement in toads can be explained by a combination of two factors, namely, the distribution of reinforced trials across days and the total number of reinforcements.     

Punishment of an extinguishing shock-avoidance response by time-out from positive reinforcement

Two experiments were conducted to determine the effects of punishment by time-out from positive reinforcement on the extinction of discriminated shock-avoidance responding. Subjects were trained initially to bar press for food on an intermittent schedule of reinforcement and, concurrently, to avoid shock at the onset of a warning signal. Experiment I compared avoidance extinction performance under no punishment and when avoidance responding resulted in a 30-sec TO from reinforced appetitive responding. In Exp II, the contingent use of TO punishment was compared with its random, or noncontingent use. The results of both experiments showed that in the absence of punishment, avoidance extinction was characterized by short latencies and nearly 100% avoidance responding. Avoidance responding in extinction was little affected by noncontingent TO punishment. When TO was made contingent upon avoidance responding, however, avoidance latencies immediately increased and the frequency of avoidance responses subsequently decreased to zero.     

Transfer of approach responding between punishment, frustrative nonreward, and the combination of punishment and nonreward

Rats trained to make an approach response with either partial reward, intermittent punishment, or a combination of partial reward and intermittent punishment, were tested for persistence to extinction, punishment with reward, or punishment during extinction. Partial reward, alone or with punishment, produced greates resistance to extinction, while intermittent punishment, alone or with partial reward, produced greatest persistence to punishment with reward. Transfer of persistence from partial reward to punishment with reward and intermittent punishment to extinction was also demonstrated. However, partial reward alone did not increase persistence to punishment during extinction, whereas intermittent punishment and partial reward combined with intermittent punishment did increase such persistence. These results were interpreted in Amsel's (1958, 1962) conditioning-model theory by extending the hypothesized similarity of frustrative nonreward and punishment.     

Response persistence in the rat following intermittent punishment and partial reward: Effects of trial sequence

Thirty-six rats were given 16 days of partial reward training in a runway. During the final 12 days each of the animals received one foot-shock experience each day. One group received shock on an N trial preceding an R trial (P-R), a second group was shocked on N trials not followed by an R trial (R-P), and the third group received shock after completing all daily trials (Control). Following acquisition the rats were split within each group (one half received 24 trials of unpunished extinction and one half continued to receive partial reward but were punished on every trial). During consistent punishment the P-R animals were more persistent than the R-P or Control rats and during unpunished extinction the P-R and Control animals were equal in persistence but both were superior to the R-P animals. The results were discussed in terms of Capaldi's sequential trial theory.     

Positive behavioral contrast in 3-month-old infants on multiple conjugate reinforcement schedules.

Positive behavioral contrast was assessed in two experiments with young infants using multiple conjugate reinforcement schedules. Reinforcement was produced by footkicks which activated the objects of an overhead crib mobile in a manner proportional to the vigor and rate of responding. Distinctive color/pattern cues on the sides of the objects served as discriminative stimuli for components of the multiple schedule. In Experiment 1, infants were trained with one cue (S+) only before insertion of S+ into a multiple schedule with an extinction component. A control group received S+ throughout all sessions. In Experiment 2, a multiple schedule was introduced at the outset, and responses in both components were reinforced before the introduction of extinction in the second component. In a final phase, reinforcement was reintroduced into the second component. Positive behavioral contrast occurred in both experiments. Response reduction in the extinction component was seen only in individual relative response curves. In both experiments, negative emotional behaviors accompanied the extinction component, and in Experiment 1, cooing accompanied presentations of S+.     

Partial reinforcement effects in instrumental escape conditioning

In Experiment I acquisition and extinction of instrumental escape conditioning with rats (N = 64) were studied as a function of reinforcement magnitude under conditions of partial and continuous reinforcement. In Experiment II the effects of partial and continuous reinforcement were studied in rats (N = 96) during acquisition followed by small, medium, and large reductions in reinforcement magnitude. A water-tank escape apparatus was used with temperature as the relevant variable. It was found that (1) with large reinforcement magnitude a continuously reinforced group was superior in acquisition to one that was partially reinforced; there were no differences with small reinforcement; (2) disruptive effects of a nonreinforced trial (a) appear early in learning, (b) are quite strong after each nonreinforced trial, and (c) persist through several succeeding reinforced trials; (3) major competing behaviors persist throughout acquisition for small reinforcement magnitude regardless of schedule, decline with large reinforcement (more so with continuous than with partial), and return to a high level in extinction for all conditions; (4) the partial reinforcement extinction effect occurs after large reinforcement but not after small, and it appears only with large reductions in reinforcement magnitude which approach extinction conditions. Only the first part of the last finding appears to be consistent with the appetitive conditioning literature.     

The partial punishment effect following minimal acquisition training: Sodium amobarbital and the stimulus properties of early punished trials

In a runway investigation, six groups of rats received limited runway training such that partial punishment, partial reinforcement, or continuous reinforcement was accompanied by sodium amobarbital or saline. Following an interpolated phase of continuous reinforcement without injections, all groups were given punished extinction. The entire experiment was conducted under widely spaced conditions (ITI = 24 hr). It was found that partial punishment increased resistance to punished extinction relative to partially and continuously reinforced controls when acquisition was given under saline. When partial punishment training was accompanied by amobarbital this effect was eliminated. The drug was observed to have no effect on the punished extinction performance of the partial reinforcement and continuous groups, respectively. Moreover, the partial reinforcement effect (PRE) did not generalize to punished extinction. These data provide information concerning the difference between the stimuli associated with the early trials of punishment and nonreward and indicate that the former but not the latter contain emotional elements.     

Extinction of the context and latent inhibition

The hypothesis that latent inhibition could be reduced by extinguishing the experimental context, that is, exposing the rats to the context between exposure to the conditional stimulus (CS) and conditioning, was tested. All experiments used the conditioned emotional response procedure. In Experiment 1, extinction was not effective when the animals were exposed to the clicker 40 times off the baseline of responding for food and when the clicker CS was partially reinforced with shocks during the test phase. In Experiments 2 and 3, latent inhibition could be reduced by extinction if the animals were exposed to the CS 24 or 16 times on-baseline, and if continuous reinforcement was used during the test. In Experiments 4, 5, and 6, we attempted to determine which variable was responsible for the discrepant results. In Experiment 4, extinction was effective with 20 or 40 on-baseline exposures to the CS, using continuous reinforcement during the test. In Experiment 5, extinction was not effective with exposure on- or off-baseline, using 24 exposures and partial reinforcement. Finally, in Experiment 6, extinction reduced latent inhibition using continuous, but not partial, reinforcement with 40 exposures off-baseline. From these results, we concluded that Wagner's model of habituation was not sufficient to account for latent inhibition and that a hybrid model, using both associative and cognitive representational processes, was necessary.     

Occasional reinforced trials during extinction can slow the rate of rapid reacquisition

Two appetitive conditioning experiments with rats examined reacquisition after conditioned responding was eliminated by either extinction or by a partial reinforcement procedure in which reinforced trials were occasionally presented among many nonreinforced trials. In Experiment 1, reacquisition to a conditional stimulus (CS) that had been conditioned and extinguished was more rapid than acquisition in a group that had received no prior conditioning. However, the addition of occasional reinforced trials to extinction slowed this rapid reacquisition effect. Experiment 2 replicated the result and showed that a procedure in which the CS and the unconditional stimulus (US) were unpaired in extinction interfered even further with reacquisition. The results suggest that rapid reacquisition is ordinarily produced when reinforced trials provide a contextual cue that can renew responding by signaling other acquisition trials (Ricker & Bouton, 1996). The effects of partial reinforcement in extinction are surprising from several theoretical perspectives and have useful clinical implications.     

Appetitive conditioning in Octopus cyanea

The performance of Octopus cyanea was studied in 3 appetitive conditioning situations. In Experiment 1, 2 groups were trained in a runway; a large reward produced faster acquisition when reinforcement was consistent and better subsequent performance on a partial schedule than did a small reward. In Experiment 2, activity in the vicinity of a feeder was measured, and in Experiment 3, latency and probability of response were measured in an automated version of a traditional conditioned attack situation (Boycott & Young, 1950). There was evidence of acquisition with continuous reinforcement in both experiments but in neither with partial reinforcement. All of the results can be understood in terms of growth and decline in the strength of stimulus-reinforcer associations with reinforcement and nonreinforcement.     

Effects of anxiolytic drugs on the partial reinforcement extinction effect in runway and skinner box

The partial reinforcement extinction effect (PREE) in the runway is reduced by anxiolytics in a non-state-dependent manner when a 24 h inter-trial interval is used, but there is some doubt as to the nature of the drug effects when shorter intervals are used. Experiment 1 repeated a study by Gray (1969), in which ambiguous results were obtained using eight trials/day. It demonstrated that the anxiolytic barbiturate, sodium amylobarbitone, given both in acquisition and extinction does not reduce the PREE. It confirmed Gray's observation that the PREE is abolished if the drug is given in acquisition but not in extinction. This suggests that a 24 h inter-trial interval is one critical factor in non-state-dependent reduction of the PREE. Experiments 2 and 3 tested the effects of the anxiolytic benzodiazepine, chlordiazepoxide, on the PREE with a 24 h inter-trial interval in the Skinner box. The basic task was a single FR5 sequence terminating in delivery of 10 (Experiment 2) or 20 (Experiment 3) reward pellets each day. There were 10 acquisition trials and partially reinforced rats received either three (Experiment 2) or four (Experiment 3) non-rewarded trials. The drug abolished the PREE in Experiment 2 and effectively reversed it in Experiment 3. These results confirm previous work with this drug in the runway (Feldon and Gray, 1981) and extend them to a very different experimental situation. These results support the idea that the PREE depends on different processes with different acquisition parameters; and that when a 24 h inter-trial interval is used the PREE is largely produced by some general process, probably the counterconditioning of conditioned frustration, which is sensitive to anxiolytic drugs. They also demonstrate very clearly the paradoxical effects of the anxiolytic drugs when given in both acquisition and extinction: they generally increase resistance to extinction in continuously reinforced animals, but block the increase in resistance (the PREE) produced by behavioural schedules.     

Successive discrimination training with equated reinforcement frequencies: failure to obtain behavioral contrast

In two experiments, pigeons were trained on two-component multiple schedules in which responding in one component (S₁) was always maintained by a variable-interval schedule. In Experiment I, low response rates were reinforced in the second (S₂) component for six master subjects. This schedule was adjusted to equate reinforcement frequencies in the two components. These subjects were compared to yoked partners, for which reinforcement in the S₂ component was made available on a variable-interval schedule whose value was determined by the master subjects. A similar procedure was used in Experiment II, where the S₂ schedule for master subjects made reinforcers contingent on the absence of responding. No evidence was found in either experiment for a behavioral contrast effect in the S₁ component attributable to response reduction in the S₂ component. A reliable contrast effect was obtained from a group of pigeons given extinction conditions in the S₂ component, which was compared to a group maintained throughout on a multiple variable-interval schedule. The results suggest that previous indications of behavioral contrast in similar situations were probably caused by uneven reinforcement distributions or reflect uncontrolled fluctuations in response rates.     

Context extinction following conditioning with delayed reward enhances subsequent instrumental responding

In Experiment 1, delayed reward generated low response rates relative to immediate reward delivered with the same frequency. Lister rats exposed to delayed reward subsequently responded at a higher rate in extinction if they received nonreinforced exposure to the conditioning context after instrumental training and prior to test, compared with animals that received home cage exposure. In Experiment 2, a signaled delay of reinforcement resulted in higher rates than an unsignaled delay. Nonreinforced exposure to the conditioning context elevated response rate for subjects in the unsignaled condition relative to a home cage group, but had no effect on response rates for subjects that had received the signaled delay. In Experiment 3, following an unsignaled reinforcement delay, groups receiving either no event or signaled food in the context responded faster in extinction than groups receiving no context exposure or unsignaled food.     

Effects of thinning the rate at which the alternative behavior is reinforced on resurgence of an extinguished instrumental response

Three experiments with rats examined the effects of thinning the rate of reinforcement for the alternative behavior in the resurgence paradigm. In all experiments, pressing one lever (L1) was first reinforced and then extinguished while pressing a second alternative lever (L2) was then reinforced. When L2 responding was then extinguished, L1 responses "resurged." Resurgence was always observed when L2 was reinforced on an unchanging reinforcement schedule during Phase 2. However, other rats received systematic decreases in the rate of L2 reinforcement before extinction of L2 began. Such a "thinning" procedure was predicted to reduce final resurgence by associating L1 extinction with longer and longer periods without a reinforcer. The procedure did reduce the resurgence effect observed when L2 was put on extinction (Experiment 3). However, in each experiment, thinned groups also returned to L1 responding, and continued to make L1 responses, while the reinforcement schedule for L2 was being thinned. Fine-grained analysis of behavior in time suggested that this early resurgence was not due to adventitious reinforcement of L1, occasion setting of L1 by reinforcer presentation, or the entrainment of L1 as a schedule-induced interim behavior. The results are overall consistent with the hypothesis that resurgence is a renewal effect in which extinguished L1 responding recovers when the context provided by the L2 reinforcement schedule is changed. Challenges for this view are also discussed.