Order of partial and consistent reward,reward magnitude shift,and resistance to extinction

In a separate-phase runway experiment with rats, four schedules involving partial (P) and consistent (C) reward (CC, CP, PC, and PP) were crossed with three reward magnitude shift conditions (upshift, nonshift, and downshift). The data revealed three major findings: (a) Reward magnitude downshift generally led to rapid extinction; (b) consistent reward prior to partial reward (CP) resulted in slower extinction than the reverse order (PC) under conditions of reward magnitude shift (particularly downshift); and (c) the relative performance of PC and CP under conditions of reward magnitude shift was reversed from postshift to extinction. On the basis of these data it was suggested that processes not presently identified by reinforcement level theory and stimulus analyzer theory influence extinction following separate-phase acquisition. A modification of reward level theory was presented to provide an account of extinction performance following separate-phase reward reduction.     

Children's acquisition and reversal behavior in a probability learning situation as a function of programed instruction,internal-external control,and schedules of reinforcement

A three-choice, contingent task was used with 192 fifth and sixth graders in a 2 × 2 × 3 factorial design combining instruction (programed instruction on selected probability concepts vs no programed instruction), locus of control (internal vs external), and schedules of reinforcement (33, 66, and 100%). The dependent measures were the percentage of correct acquisition responses, of correct reversal responses, and of pattern responses, as well as posttests on probability concepts. The major findings of the study were associated with schedule of reinforcement. In acquisition and reversal, Ss under 100% reinforcement during acquisition tended to maximize the greatest, followed by the 66 and 33% conditions, in that order. The ordinal relationship among schedules was the exact reverse of the maximizing approach for the pattern responses. A partial reinforcement effect was obtained in reversal. Evidence indicated that programed instruction and locus of control affected maximizing behavior, patterning behavior, and resistance to extinction (though these variables did not interact with reinforcement schedule in the predicted direction). Finally the posttest data showed that instructed Ss did learn more relative to noninstructed Ss.     

Partial reward training level and reward magnitude: Effects on acquisition and extinction

Two groups of 10 rats each received either a large magnitude (0.90 g) or a small magnitude (0.18 g) of partial reward (PR) and seven successive acquisition-extinction sessions in the runway. The large magnitude PR group ran much faster than the small magnitude PR group in the early acquisition sessions with differences between the groups declining over sessions. In the early extinction sessions, the large magnitude PR group showed greater resistance to extinction than the small magnitude PR group, but in the late extinction sessions this relation reversed itself. Finally, resistance to extinction decreased over sessions, this decrease being greater under a large than under a small magnitude of PR.     

Effects of schedule and magnitude of reinforcement under conditions of thirst motivation

In contrast to a recent finding (Macdonald, G. E., & De Toledo, L. Learning and Motivation, 1974, 5, 288–298.) the results of three experiments investigating various partial reinforcement (PRF) manipulations under conditions of thirst motivation demonstrated strong similarity to analogous manipulations involving food reward. Specifically, for animals receiving water reinforcement, PRF was shown to generate greater resistance to extinction than continuous reinforcement (Expt 1 & 3), the schedule of reinforcement was shown to interact with level of acquisition (Expt 1 and 2), and the magnitude of the partial reinforcement extinction effect was shown to be a function of reward magnitude (Expt 3). These results provide strong evidence that mechanisms which operate in partial reinforcement situations are highly similar, regardless of the type of appetitive reinforcement.     

Effect of schedule and magnitude of reinforcement on instrumental learning in the toad, Bufo arenarum

Extinction after training with continuous (CR) or 50% partial (PR) reinforcement, and with different magnitudes of reward, was studied in the amphibian Bufo arenarum, in a runway situation. In Experiment 1, a group of toads received massed-trial, CR training with access to water as the reward. Performance improved during acquisition, including an improvement on the first trial of each session. Extinction was rapid and there was evidence for spontaneous recovery of the running response. In Experiment 2, groups of toads received PR or CR training at a rate of one trial per day. PR impaired acquisition and resulted in poor responding during extinction, compared to CR. Experiment 3 factorially studied the effects of schedule (PR vs CR) and distribution of practice (15 s vs 300 s intertrial interval). Acquisition was impaired by PR training but had little effect on extinction performance. Different magnitudes of water reinforcement were used in Experiment 4 in a one-trial-per-day situation. Terminal acquisition performance was a monotonic function of reward magnitude, but there were no differences in extinction performance across groups. The results are discussed in relation to comparative and developmental data on the paradoxical effects of reward.     

How transitions from nonrewarded to rewarded trials regulate responding in Pavlovian and instrumental learning following extensive acquisition training

In both discrimination learning and partial reinforcement, transitions may occur from nonrewarded to rewarded trials (NR transition). In discrimination learning, NR transitions may occur in two different stimulus alternatives (NR different transitions). In partial reward, NR transitions may occur in a single stimulus alternative (NR same transitions). Available instrumental learning data indicate that resistance to extinction is increased by both types of NR transitions following limited acquisition training. Following more extensive acquisition training, resistance to extinction appears to be increased by NR same transitions but not by NR different transitions. In Experiment 1, it was shown for the first time that following extensive acquisition training the effects of the two types of transitions are the same in the Pavlovian situation as in the instrumental situation. This finding indicates that on the current trial the rat remembered the conditioned stimulus (CS) and the unconditioned stimulus (US) from the prior trial and those memories along with the CS on the current trial become the signal for the current US. Experiments 2 and 3, which tested hypotheses about instrumental learning, identified why NR different transitions lose their capacity to promote vigorous responding following extensive acquisition training. This is because cues occurring on the rewarded trials of NR different transitions more validly signal reward than other situational cues and thus overshadow them. Finally, some implications of the present findings for understanding the role of NR different transitions in discrimination learning situations were discussed.     

Effects of anxiolytic drugs on the partial reinforcement extinction effect in runway and skinner box

The partial reinforcement extinction effect (PREE) in the runway is reduced by anxiolytics in a non-state-dependent manner when a 24 h inter-trial interval is used, but there is some doubt as to the nature of the drug effects when shorter intervals are used. Experiment 1 repeated a study by Gray (1969), in which ambiguous results were obtained using eight trials/day. It demonstrated that the anxiolytic barbiturate, sodium amylobarbitone, given both in acquisition and extinction does not reduce the PREE. It confirmed Gray's observation that the PREE is abolished if the drug is given in acquisition but not in extinction. This suggests that a 24 h inter-trial interval is one critical factor in non-state-dependent reduction of the PREE. Experiments 2 and 3 tested the effects of the anxiolytic benzodiazepine, chlordiazepoxide, on the PREE with a 24 h inter-trial interval in the Skinner box. The basic task was a single FR5 sequence terminating in delivery of 10 (Experiment 2) or 20 (Experiment 3) reward pellets each day. There were 10 acquisition trials and partially reinforced rats received either three (Experiment 2) or four (Experiment 3) non-rewarded trials. The drug abolished the PREE in Experiment 2 and effectively reversed it in Experiment 3. These results confirm previous work with this drug in the runway (Feldon and Gray, 1981) and extend them to a very different experimental situation. These results support the idea that the PREE depends on different processes with different acquisition parameters; and that when a 24 h inter-trial interval is used the PREE is largely produced by some general process, probably the counterconditioning of conditioned frustration, which is sensitive to anxiolytic drugs. They also demonstrate very clearly the paradoxical effects of the anxiolytic drugs when given in both acquisition and extinction: they generally increase resistance to extinction in continuously reinforced animals, but block the increase in resistance (the PREE) produced by behavioural schedules.     

An analysis of sequential variables in Pavlovian conditioning employing extended and limited acquisition training

Sequential theory’s memory model of learning has been successfully applied in response contingent instrumental conditioning experiments (Capaldi, 1966, Capaldi, 1967, Capaldi, 1994 and Capaldi and Miller, 2003). However, it has not been systematically tested in nonresponse contingent Pavlovian conditioning experiments. The present experiments attempted to determine if several sequential variables affect responding in Pavlovian situations as they do in instrumental ones. Of primary concern here were the effects on extinction of number of NR transitions (the number of times a nonreinforced trial is followed by a reinforced trial), N-length (the number of successive nonreinforced trials that precede a reinforced trial), and percentage of reinforcement (50 versus 100%) following either extended acquisition training (Experiment 1, 720 trials) or limited acquisition training (Experiment 3, 24 trials). In agreement with a sequential analysis, N-length increased resistance to extinction more than number of NR transitions following extensive training with the opposite occurring following limited training. In Experiment 1, greater resistance to extinction was associated with 50% than with 100% reinforcement, a partial reinforcement extinction effect (PREE). Experiment 2 examined an anomalous finding obtained in Experiment 1. A major theoretical difference between instrumental and Pavlovian conditioning has been held to be the greater ease of producing a PREE in instrumental than in Pavlovian conditioning (Kimble, 1961 and Mackintosh, 1974). However, the findings obtained here suggest that the probability of obtaining a PREE and other Pavlovian extinction effects, as in instrumental conditioning, increases along with the effectiveness of the sequential variables employed.     

Resistance to continuous delay of reinforcement or extinction following partial delay or partial reinforcement in acquisition: A direct comparison

Rats received either partial reward (PR) or partial delay (PD) in acquisition with one, two, or three delay or nonreward trials followed by an immediately rewarded trial or one delay or nonreward trial followed by an immediately rewarded trial. These four groups were then split in half and given either continuous delay or continuous nonreward (extinction) in a “response persistence” phase. In addition, two continuously reinforced groups, one experiencing continuous delay, and the other experiencing extinction were included. The results showed that response persistence was greater when PD groups were given continuous delay rather than extinction, but the opposite was true for PR groups. The “length” (1, 2, or 3 versus 1 nonreward or delay trial) also transferred to the response persistence phase with the length 1, 2, and 3 conditions being more persistent than the 1 length conditions. The results were discussed with respect to theoretical notions of response persistence.     

Effect of interpolated extinction after partial delay of reward training on subsequent reacquisition and extinction

Four groups of albino rats were run four trials a day in a straight runway for 44 days. On the first 15 days, two groups were given continuous immediate reward (IR) and two groups a 50 per cent, schedule of 30-sec. partial delay of reward (PDR). On the next 15 days, one IR group and one PDR group were extinguished, while the other IR and PDR groups remained on their original schedules. In the third phase, all groups received 8 days of training on IR. Finally, all groups were given 6 days of extinction training. In the first extinction, PDR produced greater resistance to extinction than IR. In the second extinction period, the PDR group which had previously been given extinction and the two IR groups extinguished relatively rapidly and at approximately the same rates, while the PDR group which had not been extinguished was significantly more resistant to extinction than the other three groups.     

Effects of reward on delayed reaction time task performance of hyperactive children

This experiment was designed to investigate the effects of continuous, partial, and noncontingent schedules of reward, as well as the withdrawal of rewards, on the performance of hyperactive and normal control children on a delayed reaction time task. Although noncontingent reward resulted in faster reaction times for control subjects, performance of hyperactives deteriorated under noncontingent reward and improved when it was withdrawn. Also, reaction times of controls during extinction remained superior to baseline, whereas performance of hyperactives returned to baseline level. It was suggested that these and other findings reviewed point to an unusual sensitivity to rewards in hyperactive children.     

Resistance to satiation of consummatory and instrumental performance

In three experiments rats received training in a straight alley under high hunger and then were tested satiated. Both eating and running continued to occur under satiation, but the two responses were not completely correlated, and continued running did not depend upon continued eating. Further, groups differed in their eating behavior, although all experienced the same satiation procedure, suggesting that eating under satiation is not just a reflection of incomplete satiation. Resistance to satiation of the running response was greater following partial reward than following consistant reward and tended to be greater following small reward training than large reward training, regardless of schedule of reward. Eating during satiation was greater following partial than following consistent reward and was greater if the same reward magnitude was given in satiation as in acquisition than if a different reward magnitude was given. It was suggested that resistance to satiation is an associative phenomenon. Eating and running occur during satiation because the stimuli present during satiation continue to elicit them. The differences between results using rewarded satiation and results using high drive extinction as measures of persistence were attributed to satiation being nonfrustrating.     

Effects of UCS intensity and postpeak acquisition trials on classical conditioning of the SCR

The present study examined the effects of UCS intensity and number of postpeak acquisition trials on classical conditioning and extinction of the SCR. A 2 × 3 design was employed in which subjects received either a 1, 2, or 4 mA shock UCS and either two or 16 acquisition trials beyond the peak CR. While conditioning was demonstrated during acquisition, there was no relationship between strength of conditioning and intensity of UCS. The phenomenon of stronger resistance to extinction following fewer acquisition trials (e.g., two past the peak CR) than with many (e.g., 16 past the peak CR) was demonstrated only for the groups that were conditioned with the 4 mA UCS. Resistance to extinction varied positively with UCS intensity, but only for the subjects who received two postpeak acquisition trials. Sixteen trials beyond the peak CR resulted in the UCS intensity having little or no effect on resistance to extinction.     

Duration of extinction trials as a determinant of instrumental extinction in terrestrial toads (<Emphasis Type="Italic">Rhinella arenarum</Emphasis>)

Instrumental learning guides behavior toward resources. When such resources are no longer available, approach to previously reinforced locations is reduced, a process called extinction. The present experiments are concerned with factors affecting the extinction of acquired behaviors in toads. In previous experiments, total reward magnitude in acquisition and duration of extinction trials were confounded. The present experiments were designed to test the effects of these factors in factorial designs. Experiment 1 varied reward magnitude (900, 300, or 100 s of water access per trial) and amount of acquisition training (5 or 15 daily trials). With total amount of water access equated in acquisition, extinction with large rewards was faster (longer latencies in 900/5 than 300/15), but with total amount of training equated, extinction with small rewards was faster (longer latencies in 100/15 than 300/15). Experiment 2 varied reward magnitude (1200 or 120 s of water access per trial) while holding constant the number of acquisition trials (5 daily trials) and the duration of extinction trials (300 s). Extinction performance was lower with small, rather than large reward magnitude (longer latencies in 120/300 than in 1200/300). Thus, instrumental extinction depends upon the amount of time toads are exposed to the empty goal compartment during extinction trials.     

Purkinje cell loss by OX7-saporin impairs acquisition and extinction of eyeblink conditioning

The current study examined the effects of globally depleting Purkinje cells in the cerebellar cortex with the immunotoxin OX7-saporin on acquisition and extinction of delay eyeblink conditioning in rats. Rats were given OX7-saporin or saline 2 wk before the start of eyeblink conditioning. The rats that reached a performance criterion of two consecutive days with 80% or greater conditioned responses were given 5 d of extinction training followed by 2 d of reacquisition training. Rats that received infusions of OX7-saporin had 77.2%-97.9% Purkinje cell loss and exhibited impaired acquisition and extinction. The amount of Purkinje cell loss was correlated with the magnitude of the acquisition and extinction impairments. The highest correlations between Purkinje cell number and the rate of acquisition were in lobule HVI and the anterior lobe. The highest negative correlation between Purkinje cell number and the percentage of conditioned responses during extinction was in the anterior lobe. The results indicate that cerebellar Purkinje cells, particularly in the anterior lobe and lobule HVI, play significant roles in acquisition and extinction of eyeblink conditioning.     

Effects of reward and nonreward on frustration and attention in attention deficit disorder

The effects of reward schedule (100%, 50%, and 30%) and termination of rewards (extinction) on 30 attention deficit disorder with hyperactivity (ADD-H) and 30 normal children were studied using measures of frustration (speed/strength of lever pulling) and attention (reaction time to a light signal). ADD-Hs pulled harder on the lever than controls during extinction and on the lowest (30%) partial schedule, providing empirical evidence that they respond with greater frustration than normals when expected rewards fail to appear. The groups did not differ on the attentional measure on 100% reward. However, the partial schedules appeared to have an alerting or motivating effect on the controls, so that they responded more quickly and consistently than ADD-Hs on the partial schedules. Findings are discussed with reference to opposing theories regarding the nature of the abnormal response of ADD-Hs to reward.This research was supported by grant MA 11252 from the Medical Research Council of Canada.     

Partial reinforcement effects in instrumental escape conditioning

In Experiment I acquisition and extinction of instrumental escape conditioning with rats (N = 64) were studied as a function of reinforcement magnitude under conditions of partial and continuous reinforcement. In Experiment II the effects of partial and continuous reinforcement were studied in rats (N = 96) during acquisition followed by small, medium, and large reductions in reinforcement magnitude. A water-tank escape apparatus was used with temperature as the relevant variable. It was found that (1) with large reinforcement magnitude a continuously reinforced group was superior in acquisition to one that was partially reinforced; there were no differences with small reinforcement; (2) disruptive effects of a nonreinforced trial (a) appear early in learning, (b) are quite strong after each nonreinforced trial, and (c) persist through several succeeding reinforced trials; (3) major competing behaviors persist throughout acquisition for small reinforcement magnitude regardless of schedule, decline with large reinforcement (more so with continuous than with partial), and return to a high level in extinction for all conditions; (4) the partial reinforcement extinction effect occurs after large reinforcement but not after small, and it appears only with large reductions in reinforcement magnitude which approach extinction conditions. Only the first part of the last finding appears to be consistent with the appetitive conditioning literature.     

Effects of reinforcement schedules on rats' choice behavior in extinction.

The effects of reinforcement schedules on rats' choice behavior in extinction were studied. In a free-operant chamber equipped with two retractable bars, the experimental animals were trained to press the bars separately for a food reward. One bar delivered the reward on a continuous reinforcement (CRF) schedule, and the other delivered the reward on a partial reinforcement (PRF) schedule. Control animals earned the reward from both bars with the same reinforcement schedule, either a CRF or a PRF. When both bars were simultaneously available during extinction, the experimental animals responded more frequently to the CRF than to the PRF alternative, demonstrating a reversed within-subjects partial reinforcement extinction effect (PREE). A conventional between-subjects PREE was replicated in the control subjects. The results of this study were inconsistent with both Amsel's (1962, 1967) frustration hypothesis and Capaldi's (1966, 1967) sequential hypotheses.     

Aging affects acquisition and reversal of reward-based associative learning

Reward-based associative learning is mediated by a distributed network of brain regions that are dependent on the dopaminergic system. Age-related changes in key regions of this system, the striatum and the prefrontal cortex, may adversely affect the ability to use reward information for the guidance of behavior. The present study investigated the effects of healthy aging on different components of reward learning, such as acquisition, reversal, effects of reward magnitude, and transfer of learning. A group of 30 young (mean age = 24.2 yr) and a group of 30 older subjects (mean age = 64.1 yr) completed two probabilistic reward-based stimulus association learning tasks. Older subjects showed poorer overall acquisition and impaired reversal learning, as well as deficits in transfer learning. When only those subjects who showed evidence of significant learning were considered, younger subjects showed equivalently fast learning irrespective of reward magnitude, while learning curves in older subjects were steeper for high compared to low reward magnitudes. Acquired equivalence learning, which requires generalization across stimuli and transfer of learned contingencies to new stimuli, was mildly impaired in older subjects.     

The order of continuous,partial and nonreward trials and resistance to extinction

In the first experiment rats were given either 16 or 48 nonrewarded or continuously rewarded trials prior to 24 continuously or partially rewarded trials, followed by extinction. Increased resistance to extinction was found for increasing numbers of nonrewarded trials when they were followed by partial reward, but not when followed by continuous reward. Similarly, more continuously rewarded trials followed by partial reward tended to increase resistance to extinction. Because of the theoretical importance of the effect of continuous reward followed by partial, a second experiment was performed where the range of the number of continuously rewarded trials was extended to 0, 48, and 96. Contrary to many theoretical predictions, resistance to extinction increased as a function of increasing amounts of continuous reward.