Key pecking during extinction after intermittent or continuous reinforcement as a function of the number of reinforcers delivered during training.

Key pecking by 7 pigeons was established and maintained on a multiple variable-ratio variable-ratio (VR) schedule of food presentation. The schedule in one of the components was then changed to fixed-ratio (FR) 1 for a predetermined number of reinforcers. Both components were then changed to extinction (i.e., multiple extinction, extinction). This sequence was repeated a different number of times for each pigeon to determine the relation between the number of reinforcers delivered during each component of the multiple VR FR 1 schedule and the number of responses during extinction. For most pigeons, there were fewer responses during extinction in the presence of a stimulus recently correlated with FR 1, regardless of the number of reinforcers received. The ratio of the total responses in extinction in the former VR component to the total responses in the former FR 1 component increased as the number of reinforcers delivered during each component of the multiple schedule increased. Within-subject replications of the partial-reinforcement extinction effect generally occurred, and there were no overall reductions in the number of responses in extinction with repeated exposures to extinction.     

Early extinction effects following intermittent reinforcement: Little evidence of extinction bursts

The occurrence of extinction bursts—transient increases in response rate in excess of those observed in baseline during the period immediately following discontinuation of reinforcement of a response—was examined. In Experiment 1, key pecking of pigeons was reinforced according to a multiple schedule in which a variable-ratio schedule alternated with an interval schedule in which the reinforcers were yoked to the preceding variable-ratio component. In Experiments 2 and 3, rats were screened such that the lever-press response rates of different rats maintained by variable-interval schedules were either relatively high or relatively low. Following these baseline conditions, in Experiments 1 and 2 responding was extinguished by eliminating the food reinforcer and in Experiment 3 by removing the response–reinforcer dependency. Responses immediately following extinction implementation were examined. Response increases relative to baseline during the first 20 min of a 324.75-min extinction session (Experiment 1) or during the first 30-min extinction session (Experiments 2 and 3) were rare and unsystematic. The results (a) reinforce earlier meta-analyses concluding that extinction bursts may be a less ubiquitous early effect of extinction than has been suggested and (b) invite further experimentation to establish their generality as a function of preceding reinforcement conditions.     

Determinants of instrumental extinction in terrestrial toads (Bufo arenarum)

Previous research in a water-reinforced instrumental training situation with toads (Bufo arenarum) has shown that performance in both acquisition and extinction is poorer after partial, rather than continuous reinforcement training. In Experiment 1, the performance of a group receiving 24 trials on a 50% partial reinforcement schedule was poorer in acquisition and extinction than that of continuously reinforced groups matched for trials or reinforcements. However, partially reinforced toads extinguished at the same rapid rate as a continuously reinforced group that received training only on the days in which the partial toads received water reinforcement. In Experiment 2, extinction was faster after 10 reinforced acquisition trials than after 30 trials. This evidence suggests that the deleterious effects of partial reinforcement in toads can be explained by a combination of two factors, namely, the distribution of reinforced trials across days and the total number of reinforcements.     

The effects of extinction,noncontingent reinforcement and differential reinforcement of other behavior as control procedures

Several techniques have been used in applied research as controls for the introduction of a reinforcement contingency, including extinction, noncontingent reinforcement (NCR), and differential reinforcement of other behavior (DRO). Little research, however, has examined the relative strengths and limitations of these "reversal" controls. We compared the effects of extinction with those of NCR and DRO in both multi-element and reversal designs, with respect to (a) rate and amount of response decrement, (b) rate of response recovery following reintroduction of reinforcement, and (c) any positive or negative side effects associated with transitions. Results indicated that extinction generally produced the most consistent and rapid reversal effects, with few observed negative side effects.     

Extinction in multiple contexts does not necessarily make extinction less vulnerable to relapse

Three fear-conditioning experiments with rat subjects examined the effects of extinction in multiple contexts on a final relapse (renewal) effect that occurred when the extinguished fear cue was tested in a new context (Experiments 1 and 3) or in the context in which fear conditioning had first occurred (Experiment 2). Rats that received extinction in three contexts demonstrated more fear during extinction than rats that received the same number and temporal distribution of extinction trials in one context; extinction was partially lost with each context switch. Although extinction in multiple contexts thus had an impact on extinction behavior, it did not reduce the size of the final renewal effect. Fear during extinction was occasionally positively correlated with fear during final testing, but the two were never negatively correlated. The results suggest that extinction in multiple contexts does not necessarily weaken fear renewal, and that extinction procedures that generate high levels of responding in extinction do not necessarily make extinction learning less context-specific.     

Effects of noncontingent reinforcement on problem behavior and stimulus engagement: the role of satiation, extinction, and alternative reinforcement

This study examined the effects of noncontingent reinforcement (NCR) with and without extinction on problem behavior and stimulus engagement (consumption of reinforcement) of 4 participants. Reductions in problem behavior using NCR have frequently been attributed to both satiation of the reinforcer and extinction. In the current study, aspects of the NCR treatment effects were difficult to explain based solely on either a satiation or an extinction account. Specifically, it was found that stimulus engagement remained high throughout the NCR treatment analysis, and that problem behavior was reduced to near-zero levels during NCR without extinction. The implications of these findings are discussed with respect to the satiation and extinction hypotheses frequently described in the applied literature. Findings from basic studies examining the effects of response-independent schedules are presented, and are used as the basis for a matching theory account of NCR-related effects. It is proposed that reductions in problem behavior observed during NCR interventions may be a function of the availability of alternative sources of reinforcement.     

On the relative contributions of noncontingent reinforcement and escape extinction in the treatment of food refusal

In the current investigation, we evaluated the relative effects of noncontingent reinforcement (NCR), escape extinction, and a combination of NCR and escape extinction as treatment for the feeding problems exhibited by 4 children. For each participant, consumption increased only when escape extinction was implemented, independent of whether NCR was present or absent. These results were consistent with prior research suggesting that positive reinforcement alone is insufficient for increasing consumption, and that escape extinction often is necessary to increase and maintain food acceptance. However, NCR appeared to decrease inappropriate behavior for some participants.     

Experience with dynamic reinforcement rates decreases resistance to extinction

The ability of organisms to detect reinforcer‐rate changes in choice preparations is positively related to two factors: the magnitude of the change in rate and the frequency with which rates change. Gallistel (2012) suggested similar rate‐detection processes are responsible for decreases in responding during operant extinction. Although effects of magnitude of change in reinforcer rate on resistance to extinction are well known (e.g., the partial‐reinforcement‐extinction effect), effects of frequency of changes in rate prior to extinction are unknown. Thus, the present experiments examined whether frequency of changes in baseline reinforcer rates impacts resistance to extinction. Pigeons pecked keys for variable‐interval food under conditions where reinforcer rates were stable and where they changed within and between sessions. Overall reinforcer rates between conditions were controlled. In Experiment 1, resistance to extinction was lower following exposure to dynamic reinforcement schedules than to static schedules. Experiment 2 showed that resistance to presession feeding, a disruptor that should not involve change‐detection processes, was unaffected by baseline‐schedule dynamics. These findings are consistent with the suggestion that change detection contributes to extinction. We discuss implications of change‐detection processes for extinction of simple and discriminated operant behavior and relate these processes to the behavioral‐momentum based approach to understanding extinction.     

Persistence during and resurgence following noncontingent reinforcement implemented with and without extinction

Noncontingent reinforcement (NCR) is typically implemented with extinction (EXT) for destructive behavior reinforced by social consequences and without EXT for destructive behavior reinforced by sensory consequences. Behavioral momentum theory (BMT) predicts that responding will be more persistent, and treatment relapse in the form of response resurgence more likely, when NCR is implemented without EXT due to the greater overall rate of reinforcement associated with this intervention. We used an analogue arrangement to test these predictions of BMT by comparing NCR implemented with and without EXT. For two of three participants, we observed more immediate reductions in responding during NCR without EXT. However, for all participants, NCR without EXT produced greater resurgence than NCR with EXT when we discontinued all reinforcers during an EXT Only phase, although there was variability in response patterns across and within participants. Implications for treatment of destructive behavior using NCR are discussed.     

Occasional reinforced responses during extinction can slow the rate of reacquisition of an operant response

Three experiments with rats examined reacquisition of an operant response after either extinction or a response-elimination procedure that included occasional reinforced responses during extinction. In each experiment, reacquisition was slower when response elimination had included occasional reinforced responses, although the effect was especially evident when responding was examined immediately following each response-reinforcer pairing during reacquisition (Experiments 2 and 3). An extinction procedure with added noncontingent reinforcers also slowed reacquisition (Experiment 3). The results are consistent with research in classical conditioning (Bouton, M. E., Woods, A. M., & Pineño, O. (2004). Occasional reinforced trials during extinction can slow the rate of rapid reacquisition. Learning & Motivation, 35, 371-390) and suggest that rapid reacquisition after extinction is analogous to a renewal effect that occurs when reinforced responses signal a return to the conditioning context. Clinical implications are also discussed.     

On the relative contributions of positive reinforcement and escape extinction in the treatment of food refusal

We compared the effects of positive reinforcement alone, escape extinction alone, and positive reinforcement with escape extinction in the treatment of the food and fluid refusal of 4 children who had been diagnosed with a pediatric feeding disorder. Consumption did not increase when positive reinforcement was implemented alone. By contrast, consumption increased for all participants when escape extinction was implemented, independent of the presence or absence of positive reinforcement. However, the addition of positive reinforcement to escape extinction was associated with beneficial effects (e.g., greater decreases in negative vocalizations and inappropriate behavior) for some participants.     

Multiple schedules,off‐baseline reinforcement shifts,and resistance to extinction

Resistance to extinction in a target multiple‐schedule component varies inversely with the rate of reinforcement arranged in an alternative component during baseline. The present experiment asked whether changing the reinforcer rate in an alternative component would impact extinction of target component responding if those changes occurred in an off‐baseline phase during which the target component was never experienced. Pigeons' key pecking was studied in three types of conditions, and each condition consisted of three phases. In Phase 1, pecking produced food in the target and alternative components of a multiple schedule according to variable‐interval 60‐s schedules. In Phase 2, the alternative‐component stimulus was presented alone in a single schedule. Pecking during this phase produced the same reinforcer rate as in baseline in the Control condition, a higher rate of food (variable‐interval 15 s) in the High‐Rate condition, or was extinguished in the Extinction condition. Extinction of target‐ and alternative‐component key pecking then was assessed in a multiple schedule during the final phase of each condition. Resistance to extinction of target‐component key pecking was the same between the Control and High‐Rate conditions but lower in the Extinction condition. These findings are discussed in terms of discrimination and generalization processes.     

An investigation of differential reinforcement of alternative behavior without extinction

We manipulated relative reinforcement for problem behavior and appropriate behavior using differential reinforcement of alternative behavior (DRA) without an extinction component. Seven children with developmental disabilities participated. We manipulated duration (Experiment 1), quality (Experiment 2), delay (Experiment 3), or a combination of each (Experiment 4), such that reinforcement favored appropriate behavior rather than problem behavior even though problem behavior still produced reinforcement. Results of Experiments 1 to 3 showed that behavior was often sensitive to manipulations of duration, quality, and delay in isolation, but the largest and most consistent behavior change was observed when several dimensions of reinforcement were combined to favor appropriate behavior (Experiment 4). Results suggest strategies for reducing problem behavior and increasing appropriate behavior without extinction.     

An evaluation of two differential reinforcement procedures with escape extinction to treat food refusal

Consumption of solids and liquids occurs as a chain of behaviors that may include accepting, swallowing, and retaining the food or drink. In the current investigation, we evaluated the relative effectiveness of differential reinforcement of the first behavior in the chain (acceptance) versus differential reinforcement for the terminal behavior in the chain (mouth clean). Three children who had been diagnosed with a feeding disorder participated. Acceptance remained at zero when differential reinforcement contingencies were implemented for acceptance or mouth clean. Acceptance and mouth clean increased for all 3 participants once escape extinction was added to the differential reinforcement procedures, independent of whether reinforcement was provided for acceptance or for mouth clean. Maintenance was observed in 2 children when escape extinction was removed from the treatment package. The mechanism by which consumption increased is discussed in relation to positive and negative reinforcement contingencies.     

A comparison of various forms of reinforcement with and without extinction as treatment for escape-maintained problem behavior

The present investigation compared the effects of reinforcing compliance with either positive or negative reinforcement for a participant who displayed escape-maintained problem behavior. The results indicated that positive reinforcement in the form of a highly preferred edible or leisure item produced higher levels of compliance and lower levels of problem behavior when compared to negative reinforcement in the form of escape from demands. In addition, an extinction procedure was unnecessary to achieve high levels of compliance.     

A method for identifying satiation versus extinction effects under noncontingent reinforcement schedules

We evaluated one method for determining whether response suppression under noncontingent reinforcement (NCR) is a function of satiation or extinction. Three individuals with developmental disabilities who engaged in self-injurious behavior (SIB) or aggression participated. Results of functional analyses indicated that their problem behavior was maintained by social-positive reinforcement. NCR procedures, individualized for each participant, were implemented in a multiple baseline across subjects design and were associated with decreases in all participants' problem behavior. Identification of the mechanism by which NCR produced these effects was based on examination of cumulative records showing response patterns during and immediately following each NCR session. Satiation during NCR should lead to a temporary increase in responding during the post-NCR (extinction) period due to a transition from the availability to the unavailability of reinforcement (satiation to deprivation). Alternatively, extinction during NCR should reveal no increase in responding during the extinction period because the contingency for the problem behavior would remain unchanged and the transition from satiation to deprivation conditions would be irrelevant. Results suggested that the operative mechanisms of NCR were idiosyncratic across the 3 participants and appeared to change during treatment for 1 of the participants.     

A comparison of dense-to-lean and fixed lean schedules of alternative reinforcement and extinction

Behavior-reduction interventions typically employ dense schedules of alternative reinforcement in conjunction with operant extinction for problem behavior. After problem behavior is reduced in the initial treatment stages, schedule thinning is routinely conducted to make the intervention more practical in natural environments. In the current investigation, two methods for thinning alternative reinforcement schedules were compared for 3 clients who exhibited severe problem behavior. In the dense-to-lean (DTL) condition, reinforcement was delivered on relatively dense schedules (using noncontingent reinforcement for 1 participant and functional communication training for 2 participants), followed by systematic schedule thinning to progressively leaner schedules. During the fixed lean (FL) condition, reinforcement was delivered on lean schedules (equivalent to the terminal schedule of the DTL condition). The FL condition produced a quicker attainment of individual treatment goals for 2 of the 3 participants. The results are discussed in terms of the potential utility of using relatively lean schedules at treatment outset.     

A partial reinforcement extinction effect despite equal rates of reinforcement during Pavlovian conditioning

In 4 experiments rats received appetitive Pavlovian conditioning followed by extinction. Food accompanied every trial with the conditioned stimulus (CS) for the continuously reinforced groups and only half of the trials for the partially reinforced groups. In contrast to previous experiments that have compared the effects of partial and continuous reinforcement, the rate at which food was delivered during the CS was the same for both groups. The strength of the conditioned response during extinction weakened more rapidly in the continuously than in the partially reinforced groups. The results demonstrate that the partial reinforcement extinction effect is a consequence of the nonreinforced trials with the CS, rather than the rate at which the unconditioned stimulus is delivered during the CS.     

Unpaired shocks during extinction weaken the contextual renewal of a conditioned discrimination

Extinction is generally more fragile than conditioning, as illustrated by the contextual renewal effect. The traditional extinction procedure entails isolated presentations of the conditioned stimulus. Extinction may be boosted by adding isolated presentations of the unconditioned stimulus, as this should augment breaking the contingency between the two stimuli. In a human conditioning experiment with on-line expectancy ratings and electrodermal responding as dependent variables, 32 participants were differentially conditioned to two neutral figures using electric shock. After a change of context, one group received normal extinction treatment whereas another group received explicitly unpaired presentations of the figures and shock. At test, the two figures were presented in the original context again. For both measures, only the group that received normal extinction showed renewal of the conditioned discrimination. These results suggest that unpaired shocks during extinction strengthen the extinction learning.     

A sequential analysis of the partial reinforcement extinction effect in children*

In a series of studies employing children between the ages of 7.6 and 10.6 years of age guessing on a modified Humphreys’s board, extinction training was administered following either continuous reinforcement or various schedules of partial reinforcement training. Besides the observation of a partial reinforcement extinction effect, it was found that resistance to extinction appeared to be regulated by those sequential variables specified by extensions of Capaldi’s theory of instrumental learning.