Effects of reinforcement schedules on rats' choice behavior in extinction.

The effects of reinforcement schedules on rats' choice behavior in extinction were studied. In a free-operant chamber equipped with two retractable bars, the experimental animals were trained to press the bars separately for a food reward. One bar delivered the reward on a continuous reinforcement (CRF) schedule, and the other delivered the reward on a partial reinforcement (PRF) schedule. Control animals earned the reward from both bars with the same reinforcement schedule, either a CRF or a PRF. When both bars were simultaneously available during extinction, the experimental animals responded more frequently to the CRF than to the PRF alternative, demonstrating a reversed within-subjects partial reinforcement extinction effect (PREE). A conventional between-subjects PREE was replicated in the control subjects. The results of this study were inconsistent with both Amsel's (1962, 1967) frustration hypothesis and Capaldi's (1966, 1967) sequential hypotheses.     

Effect of interpolated extinction after partial delay of reward training on subsequent reacquisition and extinction

Four groups of albino rats were run four trials a day in a straight runway for 44 days. On the first 15 days, two groups were given continuous immediate reward (IR) and two groups a 50 per cent, schedule of 30-sec. partial delay of reward (PDR). On the next 15 days, one IR group and one PDR group were extinguished, while the other IR and PDR groups remained on their original schedules. In the third phase, all groups received 8 days of training on IR. Finally, all groups were given 6 days of extinction training. In the first extinction, PDR produced greater resistance to extinction than IR. In the second extinction period, the PDR group which had previously been given extinction and the two IR groups extinguished relatively rapidly and at approximately the same rates, while the PDR group which had not been extinguished was significantly more resistant to extinction than the other three groups.     

Memories of reward and nonreward regulate the extinction and reacquisition of both excitatory and inhibitory associations

The memories of the unconditioned stimulus (US) and its absence (No US), symbolized as S^R and S^N, respectively, may be retrieved on US or No US trials giving rise to four types of associations, S^R → US, S^R → No US, S^N → US, and S^N → No US. Here, following acquisition under partial reward (PRF), rats were shifted either to different schedules of PRF (Experiment 1) or extinction (Experiments 1 and 2). Inhibitory S^R → No US associations formed in acquisition survived extinction and shifts to one, but not another type of PRF schedule (Experiments 1 and 2). Excitatory S^R → US associations also survived extinction (Experiment 2). These findings, as well as the acquisition findings of Experiment 2, are consistent with the sequential model but not with the only other two theories said to be able to explain PRF findings, the frustration hypothesis of Amsel and the attention hypothesis of Mackintosh [see Haselgrove, M., Aydin, A., & Pearce, J. M. (2004). A partial reinforcement extinction effect despite equal rates of reinforcement during Pavlovian conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 30, 240–250]. Also, the reacquisition findings obtained here are inconsistent with two views applied to several learning phenomena, the rule-learning view and the position-item view.     

Runway acquisition and extinction as a joint function of magnitude of reward and percentage of rewarded acquisition trials

To determine the joint effects of partial reward and reward magnitude on acquisition and extinction rates, and on acquisition and extinction asymptotes, 215 Wistar albino rats were trained in a Hunter straight runway. The experimental design was a 4 × 4 × 2 factorial combining four reward magnitudes, four reward percentages, and two experimenters. The data revealed that the acquisition rate was an increasing function of both percentage and magnitude of reward and that neither reward magnitude nor percentage of reward significantly affected acquisition asymptote. For extinction, it was found that, for continuous schedules, the larger the reward magnitude the less the resistance to extinction and, for partial schedules, the larger the reward magnitude the greater the resistance to extinction. These results were interpreted within the framework of the sequential effects hypothesis (Capaldi, 1966).     

Effects of reinforcement on concept identification in hyperactive children

The effects of continuous and partial reward on the performance of hyperactive and normal children on a concept identification task were compared. Because reduction in information feedback is usually associated with partial reward schedules, the study was designed to yield information regarding the contribution of reduced feeback to performance in the partial reward condition. Previous findings of a performance deficit in hyperactives under partial reward were replicated. The findings help rule out an information feedback explanation for this deficit. The authors suggest that a motivational factor involving the elicitation of frustration when expected rewards fail to appear may be responsible for the poor performance of hyperactives on the partial schedule.This research was supported by Grant No. 943-03-34 from the Social Sciences Research Fund, McGill University.     

Response–reinforcer dependency and resistance to change

The effects of the response–reinforcer dependency on resistance to change were studied in three experiments with rats. In Experiment 1, lever pressing produced reinforcers at similar rates after variable interreinforcer intervals in each component of a two‐component multiple schedule. Across conditions, in the fixed component, all reinforcers were response‐dependent; in the alternative component, the percentage of response‐dependent reinforcers was 100, 50 (i.e., 50% response‐dependent and 50% response‐independent) or 10% (i.e., 10% response‐dependent and 90% response‐independent). Resistance to extinction was greater in the alternative than in the fixed component when the dependency in the former was 10%, but was similar between components when this dependency was 100 or 50%. In Experiment 2, a three‐component multiple schedule was used. The dependency was 100% in one component and 10% in the other two. The 10% components differed on how reinforcers were programmed. In one component, as in Experiment 1, a reinforcer had to be collected before the scheduling of other response‐dependent or independent reinforcers. In the other component, response‐dependent and ‐independent reinforcers were programmed by superimposing a variable‐time schedule on an independent variable‐interval schedule. Regardless of the procedure used to program the dependency, resistance to extinction was greater in the 10% components than in the 100% component. These results were replicated in Experiment 3 in which, instead of extinction, VT schedules replaced the baseline schedules in each multiple‐schedule component during the test. We argue that the relative change in dependency from Baseline to Test, which is greater when baseline dependencies are high rather than low, could account for the differential resistance to change in the present experiments. The inconsistencies in results across the present and previous experiments suggest that the effects of dependency on resistance to change are not well understood. Additional systematic analyses are important to further understand the effects of the response–reinforcer relation on resistance to change and to the development of a more comprehensive theory of behavioral persistence.     

Order of partial and consistent reward,reward magnitude shift,and resistance to extinction

In a separate-phase runway experiment with rats, four schedules involving partial (P) and consistent (C) reward (CC, CP, PC, and PP) were crossed with three reward magnitude shift conditions (upshift, nonshift, and downshift). The data revealed three major findings: (a) Reward magnitude downshift generally led to rapid extinction; (b) consistent reward prior to partial reward (CP) resulted in slower extinction than the reverse order (PC) under conditions of reward magnitude shift (particularly downshift); and (c) the relative performance of PC and CP under conditions of reward magnitude shift was reversed from postshift to extinction. On the basis of these data it was suggested that processes not presently identified by reinforcement level theory and stimulus analyzer theory influence extinction following separate-phase acquisition. A modification of reward level theory was presented to provide an account of extinction performance following separate-phase reward reduction.     

Interactions in multiple schedules: negative induction with squirrel monkeys

In Experiment I, lever pressing by squirrel monkeys was maintained under a sequence of variable-interval, multiple variable-interval variable-interval, and multiple variable-interval extinction schedules of food presentation. Negative induction (decreased responding in the unchanged component) occurred when one component of the multiple variable-interval variable-interval schedule was changed to extinction. Negative induction was transient over sessions; responding in the unchanged component usually recovered to a rate similar to that under the multiple variable-interval variable-interval schedule. Negative induction was not accompanied by consistent changes in the patterns of local responding within the unchanged component, and did not depend on whether component schedules were associated with localized (lever lights) or diffuse visual stimuli (houselights), or on whether the unchanged component was a 60- or 180-sec variable-interval schedule. In Experiment II, responding was maintained under a sequence of variable-interval and multiple variable-interval timeout schedules of food presentation. Negative induction occurred when responding declined gradually in the timeout component but not when responding declined abruptly. The nature of interactions in multiple schedules may depend on the species; negative induction was observed with squirrel monkeys under conditions similar to those that produce positive contrast with pigeons.     

Partial Reinforcement in Appetitive Pavlovian Conditioning with Rats

Four experiments examined the effects of a partial reinforcement schedule on extinction using appetitive Pavlovian conditioning. Extinction was slower after partial than after continuous reinforcement when the schedules were administered to different groups (Experiment 1). The opposite result was found in Experiments 2 and 3 when both schedules were presented to the same group in the same context. When the schedules were presented to the same group in different contexts, then extinction was again slower after partial reinforcement (Experiment 3). Experiment 4 demonstrated that a change of context facilitates extinction to a greater extent after conditioning with a partial reinforcement schedule than with a continuous one. The results are explained by assuming that the nonreinforced trials of a partial reinforcement schedule create an internal state that serves as a contextual cue.     

Anticipation of reward omission as a cue for choice behavior

A within-subject partial reinforcement extinction effect was obtained in a conditional discriminative choice task. In choice acquisition, one group had learned a task in which one choice alternative was continuously reinforced (CRF) and the other alternative was partially reinforced (PRF). Most errors in the acquisition phase were the choosing of the CRF lever when the conditional stimulus signaled the PRF lever; in extinction this pattern of choice errors reversed. In extinction, most errors were choosing of the PRF lever when the conditional stimulus signaled the CRF lever. The predictions of frustration theory and sequential theory were compared with the choice data. The results were interpreted as consistent with the anticipatory frustration construct within an associative mediational theory.     

Extinction of responding maintained by timeout from avoidance

The resistance to extinction of lever pressing maintained by timeout from avoidance was examined. Rats were trained under a concurrent schedule in which responses on one lever postponed shock on a free-operant avoidance (Sidman) schedule (response-shock interval = 30 s) and responses on another lever produced 2 min of signaled timeout from avoidance on a variable-ratio 15 schedule. Following extended training (106 to 363 2-hr sessions), two experiments were conducted. In Experiment 1 two different methods of extinction were compared. In one session, all shocks were omitted, and there was some weakening of avoidance but little change in timeout responding. In another session, responding on the timeout lever was ineffective, and under these conditions timeout responding showed rapid extinction. The within-session patterns produced by extinction manipulations were different than the effects of drugs such as morphine, which also reduces timeout responding. In Experiment 2 shock was omitted for many consecutive sessions. Response rates on the avoidance lever declined relatively rapidly, with noticeable reductions within 5 to 10 sessions. Extinction of the timeout lever response was much slower than extinction of avoidance in all 4 rats, and 2 rats continued responding at baseline levels for more than 20 extinction sessions. These results show that lever pressing maintained by negative reinforcement can be highly resistant to extinction. The persistence of responding on the timeout lever after avoidance extinction is not readily explained by current theories.     

Comparing the effects of static and dynamic signals during multiple schedules

The extent to which multiple schedules are an effective schedule thinning method following functional communication training (FCT) relies on the control the schedule-correlated stimuli exert over behavior. Thus, the stimuli used to signal the schedule in place (e.g., reinforcement and extinction) in a multiple schedule arrangement require special attention. To date, the majority of the research on multiple schedules has evaluated the use of different arbitrary signals as schedule-correlated stimuli (e.g., poster boards). These signals are considered static as they lack movement. More recently, some studies have successfully used dynamic signals, which include movement or animation, within multiple schedule arrangements. However, the extent to which one type of signal may result in faster stimulus control over behavior has not been evaluated. Thus, the purpose of this study was to compare the use of static and dynamic signals as schedule-correlated stimuli in multiple schedules used within the context of FCT. Four children diagnosed with autism spectrum disorder participated in the study. The results suggest that no differences in discriminated manding were observed for three out of four participants. Only dynamic signals resulted in discriminated manding for one participant.     

Schedules of reward shift in the double runway

Eighty food deprived rats received 62 trials in a double runway. On Trials 1-30, reward in the first goal box (GB1) was either always two food pellets or always zero pellets. All subjects received two pellets in the second goal box (GB2). On Trials 31-62 subjects in each preshift group (GB1 reward or GB1 nonreward) were shifted to the opposite GB1 reward level on 0, 25, 50, 75 or 100% of occasions. GB2 reward remained unaltered in all cases. For subjects experiencing reward decrease, second runway (A2) run and goal speeds after nonreward were generally enhanced, both within-group and in comparison with never rewarded controls. No such effect was evident on A2 start speed, nor was there any evidence to suggest that A2 performance after decreased reward was a function of the schedule of decrease. Increased GB1 reward resulted in general within-group impairment of A2 start and run speeds, with no effect on A2 goal performance. However, comparisons of speeds after increased reward with those of always rewarded controls revealed no difference on A2 start or run but indicated impairment of A2 goal performance. With the 50% schedule of reward increase, A2 run speeds after nonreward (the training level) exceeded those of never rewarded controls. Results are discussed with reference to McHose's contrast account of double runway phenomena and Amsel's frustration theory.     

Behavioral momentum: the effects of the temporal separation of rates of reinforcement.

In Part 1 of the experiment, rats responded under a variable-interval (VI) 30-s schedule and a VI 120-s schedule, with each in effect for a block of consecutive sessions. That is, the two VI schedules were presented in successive conditions. In Part 2 the VI schedules alternated each day, and in Part 3 the schedules alternated within the session as a multiple schedule. For half of the rats in Parts 1 and 2, the VI schedule alternated every few minutes within the session with a stimulus that signaled extinction. For each part, once response rates had stabilized, resistance to change was measured by prefeeding and extinction. When the schedules were examined in successive conditions (Part 1), resistance to extinction was greater under the VI 120-s schedule of reinforcement than under the VI 30-s schedule, but no consistent differences in resistance to prefeeding were observed between the two VI schedules. When the VI schedules alternated each day (Part 2), resistance to extinction was greater under the VI 120-s schedule. However, no consistent differences in resistance to prefeeding were observed between the VI schedules without extinction in Group A, but resistance to prefeeding was greater under the VI 30-s schedule for rats with the added extinction component in Group B. When the VI schedules alternated within the session as a multiple schedule (Part 3), resistance to extinction and resistance to prefeeding were greater under the VI 30-s schedule. The data suggest that different rates of reinforcement, and their accompanying discriminative stimuli, must be compared within the same session (or at least on alternate days) to produce data consistent with the behavioral momentum model.     

Variable-interval schedules of timeout from avoidance

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other occasionally produced a 2-min timeout during which the shock-postponement schedule was suspended and its correlated stimuli were removed. Throughout, the shock-postponement schedule maintained proficient levels of avoidance. Nevertheless, in Experiment 1 responding on the timeout lever was established rapidly, was maintained at stable levels on variable-interval schedules, was extinguished by withholding timeout, was reestablished when timeout was reintroduced, and was brought under discriminative control with a multiple variable-interval extinction schedule of timeout. These results are in contrast with Verhave's (1962) conclusion that timeout is an ineffective reinforcer when presented to rats on intermittent schedules. In Experiment 2 the consequence of responding on the timeout lever was altered so that the shock-postponement schedule remained in effect even though the stimulus conditions associated with timeout were produced for 2 min. Responding extinguished, indicating that suspension of the shock-postponement schedule, not stimulus change, was the source of reinforcement. By establishing the reinforcing efficacy of timeout with standard variable-interval schedules, these experiments illustrate a procedure for studying negative reinforcement in the same way as positive reinforcement.     

Sodium amylobarbitone and responses to nonreward

Three experiments are reported testing two alternative hypotheses concerning the behavioural effects of sodium amylobarbitone (SA): (1) that it blocks the after-effect of nonreward; (2) that it blocks conditioned frustration, elicited by stimuli associated with nonreward. In support of (2) Experiment I showed that SA given in acquisition abolished the partial reinforcement extinction effect (PREE) when rats were run at one trial a day in an alley for food reward on a continuous (CRF) or partial (PRF) reinforcement schedule. Experiment II showed that, in the goal section, the effect of the drug on the PREE was due to its presence during acquisition and was not due to state dependency; but the effect of the drug in the start section was consistent with state dependency of the PREE. In Experiment III, in opposition to (1) and again in support of (2), SA given to rats trained to show patterned running for water reward on a single alternation schedule blocked patterning by increasing running speeds on nonreward trials, not by decreasing running speeds on rewarded trials.     

Some effects of response independent reinforcers in multiple schedules

In Exp. I, rats' lever presses were conditioned on multiple variable-interval variable-interval schedules. Changing one of the multiple schedule components to variable time reduced responding in that component. Further reductions in responding occurred in both components when the schedule was changed to multiple variable-time variable-time. After reinstating the multiple variable-interval variable-time schedule, lower response rates were maintained in the variable-time component during a series of stimulus reversals. In Exp. II, replacement of extinction components of multiple variable-interval extinction or multiple extinction extinction with variable-time schedules for single sessions (probe) resulted in response rate increments in those components. In the former schedule these increases were concomitant with response decreases during the variable-interval components. Response increases in the variable-time probes were related to conditioning history and, as a result, to response probability at the time of the probe.     

RESPONDING MAINTAINED BY INTERMITTENT REINFORCEMENT: IMPLICATIONS FOR THE USE OF EXTINCTION WITH PROBLEM BEHAVIOR IN CLINICAL SETTINGS

Results of basic research have demonstrated that behavior maintained on an intermittent schedule of reinforcement (INT) will be extinguished more slowly than behavior maintained on a continuous schedule (CRF). Although these findings suggest that problem behaviors may be difficult to treat with extinction if they have been maintained on INT rather than on CRF schedules, few applied studies have examined this phenomenon with human behavior in clinical settings. The purpose of this study was to determine whether problem behavior maintained on CRF schedules would be extinguished more rapidly than behavior maintained on INT schedules. Three individuals diagnosed with profound mental retardation participated after results of pretreatment functional analyses had identified the sources of reinforcement that were maintaining their self-injury, aggression, or disruption. Subjects were exposed to extinction following baseline conditions with CRF or INT schedules alternated within reversal or multielement designs. Results suggested that problem behaviors may not be more difficult to treat with extinction if they have been maintained on INT rather than CRF schedules. However, switching from an INT to a CRF schedule prior to extinction may lower the baseline response rate as well as the total number of responses exhibited during extinction.     

Reductions in shock frequency and response effort as factors in reinforcement by timeout from avoidance

Rats' presses on one lever canceled shocks programmed after variable cycles, while presses on a second lever occasionally produced a 2-min timout during which the shock-delection schedule was suspended and its correlated stimuli removed. These concurrent schedules of avoidance and timeout were embedded in a multiple schedule whose components differed, within and across conditions, in terms of the programmed shock rate associated with the shock-deletion schedule. Analyses based on the generalized matching law suggest that the reduction in the response requirement correlated with termination of the avoidance schedule was a more important factor in the reinforcing effectiveness of timeout than was shock-frequency reduction, at least in 2 of 3 rats. After training in each condition, responding on the timeout lever was extinguished by withholding timeouts in both components over seven sessions. Resistance to extinction varied directly with the rates of both shock-frequency reduction and avoidance-response reduction experienced during training. Although reduction in response effort appeared to dominate shock-frequency reduction in the maintenance of responding, neither factor had a clear advantage in predicting the course of extinction.     

Resistance to extinction and punishment following training with shock and non-reinforcement: Failure to obtain cross-tolerance

Two experiments investigated the phenomenon of cross-tolerance between the partial reinforcement extinction effect (PREE) and the partial punishment effect (PPE). Three groups of rats were trained in acquisition to run in a straight alley. The continuously reinforced (CRF) group received a reward on every trial. The partially reinforced (PRF) group was rewarded on a quasi-random 50% schedule. The partially punished (PP) group received food reward on every trial but, in addition, received foot shocks of gradually increasing intensity in the goal box on a random 50% of the trials. In the test stage, half of the animals in each training condition were tested in extinction, where no reward was given on any of the trials, and the other half were tested in punishment, with both food and shock presented on each trial. Experiment 1 used a 1-trial/day procedure; Experiment 2 used a multi-trial procedure. In both procedures, clear PREE and PPE were obtained. In the 1-trial/day procedure, no cross-tolerance was evident: animals trained on a PRF or PP schedule did not show increased resistance to punishment and extinction, respectively. In the multi-trial procedure, only weak cross-tolerance was obtained in animals trained on partial reinforcement and tested in punishment.