Early extinction effects following intermittent reinforcement: Little evidence of extinction bursts

The occurrence of extinction bursts—transient increases in response rate in excess of those observed in baseline during the period immediately following discontinuation of reinforcement of a response—was examined. In Experiment 1, key pecking of pigeons was reinforced according to a multiple schedule in which a variable-ratio schedule alternated with an interval schedule in which the reinforcers were yoked to the preceding variable-ratio component. In Experiments 2 and 3, rats were screened such that the lever-press response rates of different rats maintained by variable-interval schedules were either relatively high or relatively low. Following these baseline conditions, in Experiments 1 and 2 responding was extinguished by eliminating the food reinforcer and in Experiment 3 by removing the response–reinforcer dependency. Responses immediately following extinction implementation were examined. Response increases relative to baseline during the first 20 min of a 324.75-min extinction session (Experiment 1) or during the first 30-min extinction session (Experiments 2 and 3) were rare and unsystematic. The results (a) reinforce earlier meta-analyses concluding that extinction bursts may be a less ubiquitous early effect of extinction than has been suggested and (b) invite further experimentation to establish their generality as a function of preceding reinforcement conditions.     

Reinforcement magnitude and responding during treatment with differential reinforcement

Basic findings indicate that the amount or magnitude of reinforcement can influence free-operant responding prior to and during extinction. In this study, the relation between reinforcement magnitude and adaptive behavior was evaluated with 3 children as part of treatment with differential reinforcement. In the first experiment, a communicative response was shaped and maintained by the same reinforcer that was found to maintain problem behavior. Two reinforcement magnitudes (20-s or 60-s access to toys or escape from demands) were compared and found to be associated with similar levels of resistance to extinction. The relation between reinforcement magnitude and response maintenance was further evaluated in the second experiment by exposing the communicative response to 20-s or 300-s access to toys or escape. Results for 2 participants suggested that this factor may alter the duration of postreinforcement pauses.     

Resurgence of previously taught academic responses

Resurgence is often discussed in relation to the relapse of undesirable behavior. However, resurgence may also describe the recurrence of socially appropriate behavior, including academic responding. The recurrence of academic responses following periods of extinction may aid in the solution of novel problems. The aims of this study were to evaluate the resurgence of complex, desirable behavior related to college-level instruction and to explore problem form as an aspect of environmental context. Each participant was taught 2 response chains to solve quadratic equations across experimental phases, followed by a phase in which neither chain resulted in the correct solution (extinction). During Experiment 1, the equations presented during extinction resembled those presented during reinforcement of the alternative response. Of the 8 participants in Experiment 1, 4 attempted to use the first-taught chain to solve an equation in the extinction phase. During Experiment 2, the equations presented during extinction resembled those presented during reinforcement of the target response. Of the 8 participants in Experiment 2, 6 attempted to use the first-taught chain to solve an equation in the extinction phase. Results demonstrate the resurgence of academic responses and suggest that the form of the problem may constitute a context that affects resurgence.     

Baseline response rates affect resistance to change

The effect of response rates on resistance to change, measured as resistance to extinction, was examined in two experiments. In Experiment 1, responding in transition from a variable‐ratio schedule and its yoked‐interval counterpart to extinction was compared with pigeons. Following training on a multiple variable‐ratio yoked‐interval schedule of reinforcement, in which response rates were higher in the former component, reinforcement was removed from both components during a single extended extinction session. Resistance to extinction in the yoked‐interval component was always either greater or equal to that in the variable‐ratio component. In Experiment 2, resistance to extinction was compared for two groups of rats that exhibited either high or low response rates when maintained on identical variable‐interval schedules. Resistance to extinction was greater for the lower‐response‐rate group. These results suggest that baseline response rate can contribute to resistance to change. Such effects, however, can only be revealed when baseline response rate and reinforcement rate are disentangled (Experiments 1 and 2) from the more usual circumstance where the two covary. Furthermore, they are more cleanly revealed when the programmed contingencies controlling high and low response rates are identical, as in Experiment 2.     

Stimulus–reinforcer relations established during training determine resistance to extinction and relapse via reinstatement

The baseline rate of a reinforced target response decreases with the availability of response‐independent sources of alternative reinforcement; however, resistance to disruption and relapse increases. Because many behavioral treatments for problem behavior include response‐dependent reinforcement of alternative behavior, the present study assessed whether response‐dependent alternative reinforcement also decreases baseline response rates but increases resistance to extinction and relapse. We reinforced target responding at equal rates across two components of a multiple schedule with pigeons. We compared resistance to extinction and relapse via reinstatement of (1) a target response trained concurrently with a reinforced alternative response in one component with (2) a target response trained either concurrently or in separate components from the alternative response across conditions. Target response rates trained alone in baseline were higher but resistance to extinction and relapse via reinstatement tests were greater after training concurrently with the alternative response. In another assessment, training target and alternative responding together, but separating them during extinction and reinstatement tests, produced equal resistance to extinction and relapse. Together, these findings are consistent with behavioral momentum theory—operant response–reinforcer relations determined baseline response rates but Pavlovian stimulus–reinforcer relations established during training determined resistance to extinction and relapse. These findings imply that reinforcing alternative behavior to treat problem behavior could initially reduce rates but increase persistence.     

Promoting response variability and stimulus generalization in martial arts training

The effects of reinforcement and extinction on response variability and stimulus generalization in the punching and kicking techniques of 2 martial arts students were evaluated across drill and sparring conditions. During both conditions, the students were asked to demonstrate different techniques in response to an instructor's punching attack. During baseline, the students received no feedback on their responses in either condition. During the intervention phase, the students received differential reinforcement in the form of instructor feedback for each different punching or kicking technique they performed during a session of the drill condition, but no reinforcement was provided for techniques in the sparring condition. Results showed that both students increased the number of different techniques they performed when reinforcement and extinction procedures were conducted during the drill condition, and that this increase in response variability generalized to the sparring condition.     

The role of response force on the persistence and structure of behavior during extinction

Behavior Momentum Theory has emerged as a prominent account of resistance to change in both basic and applied research. Although laboratory studies often define precise, repeatable responses, application research often deals with response classes that may vary widely along a number of dimensions. In general, Behavior Momentum Theory has not addressed how response dimensions impact resistance to change, providing an opportunity to expand the model in new directions. Four rats pressed a force transducer under a multiple variable interval (VI) 60‐s VI 60‐s schedule of reinforcement. In one component, responses satisfied the schedule only if the response force fell within a “low” force band requirement; responses in the other schedule were required to satisfy a “high” force band. Once responding stabilized, extinction was programmed for three sessions. Then, the procedures were replicated. The results showed that response force came under discriminative control, but force requirements had no impact on resistance to extinction. In a follow‐up condition, the schedule was changed to a multiple VI 30‐s VI 120‐s schedule and the low‐force band operated in both components. The results showed that behavior maintained by the VI 30‐s schedule was generally more resistant to extinction. A secondary analysis showed that force distributions created under baseline maintained during extinction. Overall, the results suggest that differential response force requirements prevailing in steady state do not affect the course of extinction.     

Response production during extinction training is not sufficient for extinction of evaluative conditioning

Two high-powered experiments examined the role of evaluative response production in the extinction of evaluative conditioning (EC) by positioning EC in the procedural and conceptual framework of classical conditioning (CC). According to Rescorla's response inhibition hypothesis, more frequent responding during extinction training results in larger extinction during testing. Experiment 1 used three extinction conditions following response acquisition in an EC procedure: evaluative responses were measured only after extinction; after acquisition and after extinction; or were continuously measured after acquisition, during extinction and after extinction. Based on Rescorla's response inhibition hypothesis, we predicted that extinction of EC would be the highest in the third condition. Experiment 2 was aimed at further facilitating extinction of EC by encouraging participants to experience that their evaluation may change over the course of the experiment. To this end, half of the participants completed pre- and post-acquisition ratings prior to practicing continuous response expression in the extinction phase. Contrary to our predictions, no extinction of EC was observed in either of these experiments. We conclude that Rescorla's inhibition response hypothesis may not apply to EC and discuss the theoretical implications of this finding.     

Timeout postponement without increased reinforcement frequency

Three experiments were conducted to examine pigeons' postponement of signaled extinction periods (timeouts) from a schedule of food reinforcement when such responding neither decreased overall timeout frequency nor increased the overall frequency of food reinforcement. A discrete-trial procedure was used in which a response during the first 5 s of a trial postponed an otherwise immediate 60-s timeout to a later part of that same trial but had no effect on whether the timeout occurred. During time-in periods, responses on a second key produced food according to a random-interval 20-s schedule. In Experiment 1, the response-timeout interval was 45 s under postponement conditions and 0 s under extinction conditions (responses were ineffective in postponing timeouts). The percentage of trials with a response was consistently high when the timeout-postponement contingency was in effect and decreased to low levels when it was discontinued under extinction conditions. In Experiment 2, the response-timeout interval was also 45 s but postponement responses increased the duration of the timeout, which varied from 60 s to 105 s across conditions. Postponement responding was maintained, generally at high levels, at all timeout durations, despite sometimes large decreases in the overall frequency of food reinforcement. In Experiment 3, timeout duration was held constant at 60 s while the response-timeout interval was varied systematically across conditions from 0 s to 45 s. Postponement responding was maintained under all conditions in which the response-timeout interval exceeded 0 s (the timeout interval in the absence of a response). In some conditions of Experiment 3, which were designed to control for the immediacy of food reinforcement and food-correlated (time-in) stimuli, responding postponed timeout but the timeout was delayed whether a response occurred or not. Responding was maintained for 2 of 3 subjects, suggesting that behavior was negatively reinforced by timeout postponement rather than positively reinforced by the more immediate presentation of food or food-correlated (time-in) stimuli.     

Additional free reinforcers increase persistence of problem behavior in a clinical context: A partial replication of laboratory findings

Behavioral momentum theory is a quantitative framework used to characterize the persistence of behavior during response disruptors as a function of baseline stimulus–reinforcer relations. Results of several investigations have shown that alternative reinforcement can increase the resistance to change of a target response during extinction. In the present study, concomitant variable‐interval fixed‐time schedules of reinforcement for problem behavior were employed to simulate naturalistic situations involving the superimposition of response‐independent reinforcers on a baseline schedule of reinforcement for problem behavior, as in the common use of noncontingent reinforcement treatments. Resistance to change of problem behavior was assessed during postsession periods of extinction by comparing response rates in extinction following sessions with and without additional reinforcer deliveries arranged by fixed‐time schedules. For 2 out of 3 participants, problem behavior tended to be more resistant to extinction following periods in which additional fixed‐time reinforcers were delivered. These results are discussed in terms of potential effects of noncontingent reinforcement on problem behavior when the intervention is discontinued or implemented without good treatment integrity.     

Resistance to extinction following variable-interval reinforcement: reinforcer rate and amount

Rats obtained food-pellet reinforcers by nose poking a lighted key. Experiment 1 examined resistance to extinction following single-schedule training with different variable-interval schedules, ranging from a mean interval of 16 min to 0.25 min. That is, for each schedule, the rats received 20 consecutive daily baseline sessions and then a session of extinction (i.e., no reinforcers). Resistance to extinction (decline in response rate relative to baseline) was negatively related to the rate of reinforcers obtained during baseline, a relation analogous to the partial-reinforcement-extinction effect. A positive relation between these variables emerged, however, when the unit of extinction was taken as the mean interreinforcer interval that had been in effect during training (i.e., as an omitted reinforcer during extinction). In a second experiment, rats received blocks of training sessions, all with the same variable-interval schedule but with a reinforcer of four pellets for some blocks and one pellet for others. Resistance to extinction was greater following training with the larger (four pellets) than with the smaller (one pellet) reinforcer. Taken together, these results support the principle that greater reinforcement during training (e.g., higher rate or larger amount) engenders greater resistance to extinction even when the different conditions of reinforcement are varied between blocks of sessions.     

Persistence and relapse of reinforced behavioral variability

The present study examined persistence and relapse of reinforced behavioral variability in pigeons. Pigeons emitted four‐response sequences across two keys. Sequences produced food according to a lag schedule, in which a response sequence was followed by food if it differed from a certain number of previous sequences. In Experiment 1, food was delivered for sequences that satisfied a lag schedule in both components of a multiple schedule. When reinforcement was removed for one component (i.e., extinction), levels of behavioral variability decreased for only that component. In Experiment 2, food was delivered for sequences satisfying a lag schedule in one component of a multiple schedule. In the other component, food was delivered at the same rate, but without the lag variability requirement (i.e., yoked). Following extinction, levels of behavioral variability returned to baseline for both components after response‐independent food delivery (i.e., reinstatement). In Experiment 3, one group of pigeons responded on a lag variability schedule, and the other group responded on a lag repetition schedule. For both groups, levels of behavioral variability increased when alternative reinforcement was suspended (i.e., resurgence). In each experiment, we observed some evidence for extinction‐induced response variability and for variability as an operant dimension of behavior.     

Resurgence following differential reinforcement of alternative behavior implemented with and without extinction

In the clinic, differential reinforcement of alternative behavior (DRA) often involves programming extinction for destructive behavior while reinforcing an alternative form of communication (e.g., a functional communication response); however, implementing extinction can be unsafe or impractical under some circumstances. Quantitative theories of resurgence (i.e., Behavioral Momentum Theory and Resurgence as Choice) predict differences in the efficacy of treatments that do and do not involve extinction of target responding when reinforcement conditions maintaining alternative responding worsen. We tested these predictions by examining resurgence following two DRA conditions in which we equated rates of reinforcement. In DRA without extinction, target and alternative behavior produced reinforcement. In DRA with extinction plus noncontingent reinforcement, only alternative behavior produced reinforcement. We conducted this study in a reverse-translation sequence, first with participants who engaged in destructive behavior (Experiment 1) and then in a laboratory setting with rats (Experiment 2). Across both experiments, we observed proportionally lower levels of target responding during and following the DRA condition that arranged extinction for the target response. However, levels of resurgence were similar following both arrangements.     

Intolerance of uncertainty and novelty facilitated extinction: The impact of reinforcement schedule

Individuals who score high in intolerance of uncertainty (IU) display reduced threat extinction. Recently, it was shown that replacing threat associations with novel associations during extinction learning (i.e., presenting a novel tone 100% of the time) can promote threat extinction retention in individuals with high IU. This novelty facilitated extinction (NFE) effect could be driven by the tone's novelty or reliability. Here, we sought to address this question by adjusting the reliability of the novel tone (i.e., the reinforcement rate) during NFE. We measured skin conductance response during an associative learning task in which participants (n = 92) were assigned to one of three experimental groups: standard extinction, NFE 100% reinforcement, or NFE 50% reinforcement. For standard extinction, compared to NFE 100% and 50% reinforcement groups, we observed a trend for greater recovery of the conditioned response during extinction retention. Individuals with high IU relative to low IU in the standard extinction group demonstrated a larger recovery of the conditioned response during extinction retention. These findings tentatively suggest that NFE effects are driven by the novelty rather than the reliability of the new stimulus. The implications of these findings for translational and clinical research in anxiety disorder pathology are discussed.     

Extinction of responding maintained by timeout from avoidance

The resistance to extinction of lever pressing maintained by timeout from avoidance was examined. Rats were trained under a concurrent schedule in which responses on one lever postponed shock on a free-operant avoidance (Sidman) schedule (response-shock interval = 30 s) and responses on another lever produced 2 min of signaled timeout from avoidance on a variable-ratio 15 schedule. Following extended training (106 to 363 2-hr sessions), two experiments were conducted. In Experiment 1 two different methods of extinction were compared. In one session, all shocks were omitted, and there was some weakening of avoidance but little change in timeout responding. In another session, responding on the timeout lever was ineffective, and under these conditions timeout responding showed rapid extinction. The within-session patterns produced by extinction manipulations were different than the effects of drugs such as morphine, which also reduces timeout responding. In Experiment 2 shock was omitted for many consecutive sessions. Response rates on the avoidance lever declined relatively rapidly, with noticeable reductions within 5 to 10 sessions. Extinction of the timeout lever response was much slower than extinction of avoidance in all 4 rats, and 2 rats continued responding at baseline levels for more than 20 extinction sessions. These results show that lever pressing maintained by negative reinforcement can be highly resistant to extinction. The persistence of responding on the timeout lever after avoidance extinction is not readily explained by current theories.     

Behavioral momentum and resistance to extinction across repeated extinction tests

The present experiments assessed whether resistance to extinction of pigeons' key pecking decreased across repeated extinction tests. An additional impetus for this research was to determine how the quantitative framework provided by behavioral momentum theory might be used to describe any such changes across tests. Pigeons pecked keys in two‐component multiple schedules (one component associated with a higher reinforcer rate and the other with a lower rate) in which baseline and extinction conditions alternated. In Experiment 1, baseline and extinction conditions alternated every session, and, in Experiment 2, these conditions lasted for 10 and 7 sessions, respectively. Resistance to extinction decreased across successive extinction conditions in both experiments. Fits of the behavioral‐momentum based model of extinction to the data returned uncertain results in Experiment 1 but implicated both generalization decrement and response–reinforcer contingency termination as the possible mechanisms responsible for behavior change in Experiment 2. Thus, these data suggest that experimental manipulations that affect discrimination of changes in reinforcement contingencies may influence resistance to extinction by modulating the disruptive impacts of removing reinforcers from the experimental context and of suspending response–reinforcer contingencies.     

Alternative response training, differential reinforcement of other behavior, and extinction in squirrel monkeys (Saimiri sciureus)

In Experiment I, (a) extinction, (b) extinction plus reinforcement of a discrete alternative response, and (c) differential reinforcement of other behavior were each correlated with a different stimulus in a three-component multiple schedule. The alternative-response procedure more rapidly and completely suppressed behavior than did differential reinforcement of other behavior. Differential reinforcement of other behavior was slightly more effective than extinction alone. In Experiment II, reinforcement of specific alternative behavior during extinction and differential reinforcement of other behavior were used in two components, while one component continued to provide reinforcement for the original response. Once again, the alternative-response procedure was most effective in reducing responding as long as it remained in effect. However, the responding partially recovered when reinforcement for competing behavior was discontinued. In general, responding was less readily reduced by differential reinforcement of other behavior than by the specific alternative-response procedure.     

Treatment of resistance to change in children with autism

“Resistance to change” represents a core symptom of autism that we conceptualized and assessed as resulting in part due to factors known to govern free‐operant choice. During a free‐choice baseline, participants chose between problematic, resistive responses and an appropriate alternative response. During the asymmetrical‐choice condition, we delivered their most highly preferred item if the participant chose the alternative response (i.e., differential reinforcement of alternative behavior [DRA]). During the guided‐ (Experiment 1) and singular‐ (Experiment 2) choice conditions, we prompted participants to choose the alternative response and then delivered their most highly preferred item (i.e., DRA with escape extinction). All participants learned to tolerate (Experiment 1) or choose (Experiment 2) the alternative response when we combined DRA with escape extinction. After exposure to escape extinction, two participants showed strong maintenance effects with DRA alone. We discuss these finding relative to the effects of DRA and escape extinction on resistance to change.     

Survival of the partial reinforcement extinction effect after contextual shifts

The effects of contextual shifts on the partial reinforcement extinction effect (PREE) were studied in autoshaping with rats. Experiment 1 established that the two contexts used subsequently were easily discriminable and equally salient. In Experiment 2, independent groups of rats received acquisition training under partial reinforcement (PRF) or continuous reinforcement (CRF). Significant PREEs were observed whether extinction occurred in the same or in a different context. In Experiment 3, after PRF or CRF acquisition training, each group received extinction of the same conditioned stimulus in both the same and different contexts. PREEs were observed under both conditions, although extinction was accelerated after a contextual shift. These results were discussed in relation to the role of mediational states in the PREE.