Early extinction effects following intermittent reinforcement: Little evidence of extinction bursts

The occurrence of extinction bursts—transient increases in response rate in excess of those observed in baseline during the period immediately following discontinuation of reinforcement of a response—was examined. In Experiment 1, key pecking of pigeons was reinforced according to a multiple schedule in which a variable-ratio schedule alternated with an interval schedule in which the reinforcers were yoked to the preceding variable-ratio component. In Experiments 2 and 3, rats were screened such that the lever-press response rates of different rats maintained by variable-interval schedules were either relatively high or relatively low. Following these baseline conditions, in Experiments 1 and 2 responding was extinguished by eliminating the food reinforcer and in Experiment 3 by removing the response–reinforcer dependency. Responses immediately following extinction implementation were examined. Response increases relative to baseline during the first 20 min of a 324.75-min extinction session (Experiment 1) or during the first 30-min extinction session (Experiments 2 and 3) were rare and unsystematic. The results (a) reinforce earlier meta-analyses concluding that extinction bursts may be a less ubiquitous early effect of extinction than has been suggested and (b) invite further experimentation to establish their generality as a function of preceding reinforcement conditions.     

Changeover behavior and preference in concurrent schedules

Pigeons were trained on a multiple schedule of reinforcement in which separate concurrent schedules occurred in each of two components. Key pecking was reinforced with milo. During one component, a variable-interval 40-s schedule was concurrent with a variable-interval 20-s schedule; during the other component, a variable-interval 40-s schedule was concurrent with a variable-interval 80-s schedule. During probe tests, the stimuli correlated with the two variable-interval 40-s schedules were presented simultaneously to assess preference, measured by the relative response rates to the two stimuli. In Experiment 1, the concurrently available variable-interval 20-s schedule operated normally; that is, reinforcer availability was not signaled. Following this baseline training, relative response rate during the probes favored the variable-interval 40-s alternative that had been paired with the lower valued schedule (i.e., with the variable-interval 80-s schedule). In Experiment 2, a signal for reinforcer availability was added to the high-value alternative (i.e., to the variable-interval 20-s schedule), thus reducing the rate of key pecking maintained by that schedule but leaving the reinforcement rate unchanged. Following that baseline training, relative response rates during probes favored the variable-interval 40-s alternative that had been paired with the higher valued schedule. The reversal in the pattern of preference implies that the pattern of changeover behavior established during training, and not reinforcement rate, determined the preference patterns obtained on the probe tests.     

Multiple determinants of the effects of reinforcement magnitude on free-operant response rates

Four experiments examined the effects of increasing the number of food pellets given to hungry rats for a lever-press response. On a simple variable-interval 60-s schedule, increased number of pellets depressed response rates (Experiment 1). In Experiment 2, the decrease in response rate as a function of increased reinforcement magnitude was demonstrated on a variable-interval 30-s schedule, but enhanced rates of response were obtained with the same increase in reinforcement magnitude on a variable-ratio 30 schedule. In Experiment 3, higher rates of responding were maintained by the component of a concurrent variable-interval 60-s variable-interval 60-s schedule associated with a higher reinforcement magnitude. In Experiment 4, higher rates of response were produced in the component of a multiple variable-interval 60-s variable-interval 60-s schedule associated with the higher reinforcement magnitude. It is suggested that on simple schedules greater reinforcer magnitudes shape the reinforced pattern of responding more effectively than do smaller reinforcement magnitudes. This effect is, however, overridden by another process, such a contrast, when two magnitudes are presented within a single session on two-component schedules.     

Arousal, changeover responses, and preference in concurrent schedules

Pigeons were trained on multiple schedules that provided concurrent reinforcement in each of two components. In Experiment 1, one component consisted of a variable-interval (VI) 40-s schedule presented with a VI 20-s schedule, and the other a VI 40-s schedule presented with a VI 80-s schedule. After extended training, probe tests measured preference between the stimuli associated with the two 40-s schedules. Probe tests replicated the results of Belke (1992) that showed preference for the 40-s schedule that had been paired with the 80-s schedule. In a second condition, the overall reinforcer rate provided by the two components was equated by adding a signaled VI schedule to the component with the lower reinforcer rate. Probe results were unchanged. In Experiment 2, pigeons were trained on alternating concurrent VI 30-s VI 60-s schedules. One schedule provided 2-s access to food and the other provided 6-s access. The larger reinforcer magnitude produced higher response rates and was preferred on probe trials. Rate of changeover responding, however, did not differ as a function of reinforcer magnitude. The present results demonstrate that preference on probe trials is not a simple reflection of the pattern of changeover behavior established during training.     

Acquisition of a spatially defined operant with delayed reinforcement.

Two experiments investigated the role of an immediate, response-produced auditory stimulus during acquisition, via delayed reinforcement, of a response selected to control for possible unprogrammed, operandum-related sources of response feedback. Experimentally naive rats were exposed to a delayed-food reinforcement condition, specifically a tandem fixed-ratio 1 differential-reinforcement-of-other-behavior 30-s schedule. The response was defined as breaking a photocell beam located near the ceiling at the rear of the operant conditioning chamber. In Experiment 1, rates of photobeam breaking by each rat increased from near zero, regardless of the presence or absence of a tone that immediately followed the response initiating the delay interval. Though not essential, the tone facilitated response acquisition and resulted in more efficient response patterns at stability. Experiment 2 demonstrated that photobeam-breaking response rates under the delayed reinforcement contingency exceeded those in a preceding baseline condition in which no food was delivered. In addition, upon introduction of the delayed reinforcement procedure, correspondence between response patterns and the requirements of the reinforcement schedule increased over baseline levels in the absence of a food contingency. Together with a previous report of Lattal and Gleeson (1990), the present results suggest that response acquisition with delayed reinforcement is a robust phenomenon that may not depend on a mechanically defined response or an immediate external stimulus change to mediate the temporal gap between response and reinforcer.     

The effect of vicarious reinforcement on attentive behavior in the classroom

The effect of social reinforcement delivered to target subjects on the attentive behavior of adjacent peers was examined in a classroom setting. In a combined reversal and multiple baseline design, two pairs of mentally retarded children were sequentially exposed to three reinforcement phases. After baseline rates of attentive behavior were obtained, praise was delivered to the target subject in each subject pair for attentive behavior. After a reversal phase, praise was delivered contingently to target subjects for inattentive behavior. In a final phase, contingent praise for attentive behavior was reinstated for the target subjects. Throughout the study, nontarget subjects received no direct reinforcers. The results indicated a vicarious reinforcement effect. Reinforcing attentive behavior of target subjects increased this behavior in adjacent peers. However, reinforcing inattentive behavior of target subjects also increased the attentive behavior of adjacent peers. The effects obtained through vicarious reinforcement were considered to reflect the discriminative stimulus properties of reinforcement, which may serve as a cue for the performance of nonreinforced peers.     

THE EFFECTS OF VICARIOUS PROMPTING ON ATTENTIVE BEHAVIOR OF CHILDREN WITH BEHAVIOR DISORDERS

Vicarious prompting, reinforcing and labelling the appropriate behavior of a nontarget child contingent on inappropriate behavior of a target child, was assessed in two classrooms of children with behavior disorders. Vicarious prompts using social reinforcement alone were clearly effective in increasing the attentive behavior of one target child in Experiment 1. The children in Experiment 2, however, did not appera to be responsive to vicarious prompting. Comparison of the two groups suggests that vicarious prompting, a relatively subtle, indirect control technique, may be a low cost, positive, and effective alternative to other techniques under conditions in which the behavior of most of the group is attentive.     

Run length, visit duration, and reinforcers per visit in concurrent performance

The contingencies in each alternative of concurrent procedures consist of reinforcement for staying and reinforcement for switching. For the stay contingency, behavior directed at one alternative earns and obtains reinforcers. For the switch contingency, behavior directed at one alternative earns reinforcers but behavior directed at the other alternative obtains them. In Experiment 1, responses on the main lever, in S1, incremented stay and switch schedules and obtained a stay reinforcer when it became available. Responses on the switch lever changed S1 to S2 and obtained switch reinforcers when available. In S2, neither responses on the main lever nor on the switch lever were reinforced, but a switch response changed S2 to S1. Run lengths and visit durations were a function of the ratio of the scheduled probabilities of reinforcement (staying/switching). From run lengths and visit durations, traditional concurrent performance was synthesized, and that synthesized performance was consistent with the generalized matching law. Experiment 2 replicated and extended this analysis to concurrent variable-interval schedules. The synthesized results challenge any theory of matching that requires a comparison among the alternatives.     

Human observing: maintained by negative informative stimuli only if correlated with improvement in response efficiency.

Two experiments investigated the effect of observing responses that enabled college students to emit more efficient distributions of reinforced responses. In Experiment 1, the gains of response efficiency enabled by observing were minimized through use of identical low-effort response requirements in two alternating variable-interval schedules. These comprised a mixed schedule of reinforcement; they differed in the number of money-backed points per reinforcer. In each of three choices between two stimuli that varied in their correlation with the variable-interval schedules, the results showed that subjects preferred stimuli that were correlated with the larger average amount of reinforcement. This is consistent with a conditioned-reinforcement hypothesis. Negative informative stimuli--that is, stimuli correlated with the smaller of two rewards--did not maintain as much observing as stimuli that were uncorrelated with amount of reward. In Experiment 2, savings in effort made possible by producing S- were varied within subjects by alternately removing and reinstating the response-reinforcement contingency in a mixed variable-interval/extinction schedule of reinforcement. Preference for an uncorrelated stimulus compared to a negative informative stimulus (S-) decreased for each of six subjects, and usually reversed when observing permitted a more efficient temporal distribution of the responses required for reinforcement; in this case, the responses were pulls on a relatively high-effort plunger. When observing the S- could not improve response efficiency, subjects again chose the control stimulus. All of these results were inconsistent with the uncertainty-reduction hypothesis.     

Some effects of response-correlated increases in reinforcer magnitude on human behavior

After training under short or long fixed-interval schedules, humans responded under a modified fixed-interval schedule in which magnitude of reinforcement (X or 2X) was minimally correlated with response frequency. Response frequencies that equaled or exceeded a minimum response criterion were followed by the larger reinforcer at the end of the interval; otherwise, the smaller reinforcer was delivered. The modified schedule alternated with the baseline schedule across conditions. In a control condition, the reinforcer magnitudes produced by control subjects were yoked to those of experimental subjects. Experimental subjects, but not control subjects, showed increased responding. In addition to the baseline and modified fixed-interval schedules used in Experiment 1, subjects in Experiment 2 also responded under a second modified fixed-interval contingency in which increases in reinforcer magnitude were more highly correlated with response frequency. Experimental subjects, but not control subjects, showed increased responding under both procedures. Direct comparison of these two procedures showed that the high-correlation procedure produced greater increases in responding than did the low-correlation procedure.     

ALTERNATIVE REINFORCEMENT INCREASES RESISTANCE TO CHANGE: PAVLOVIAN OR OPERANT CONTINGENCIES?

Two multiple-schedule experiments with pigeons examined the effect of adding food reinforcement from an alternative source on the resistance of the reinforced response (target response) to the decremental effects of satiation and extinction. In Experiment 1, key pecks were reinforced by food in two components according to variable-interval schedules and, in some conditions, food was delivered according to variable-time schedules in one of the components. The rate of key pecking in a component was negatively related to the proportion of reinforcers from the alternative (variable-time) source. Resistance to satiation and extinction, in contrast, was positively related to the overall rate of reinforcement in the component. Experiment 2 was conceptually similar except that the alternative reinforcers were contingent on a specific concurrent response. Again, the rate of the target response varied as a function of its relative reinforcement, but its resistance to satiation and extinction varied directly with the overall rate of reinforcement in the component stimulus regardless of its relative reinforcement. Together the results of the two experiments suggest that the relative reinforcement of a response (the operant contingency) determines its rate, whereas the stimulus-reinforcement contingency (a Pavlovian contingency) determines its resistance to change.     

Resurgence and alternative‐reinforcer magnitude

Resurgence is defined as an increase in the frequency of a previously reinforced target response when an alternative source of reinforcement is suspended. Despite an extensive body of research examining factors that affect resurgence, the effects of alternative‐reinforcer magnitude have not been examined. Thus, the present experiments aimed to fill this gap in the literature. In Experiment 1, rats pressed levers for single‐pellet reinforcers during Phase 1. In Phase 2, target‐lever pressing was extinguished, and alternative‐lever pressing produced either five‐pellet, one‐pellet, or no alternative reinforcement. In Phase 3, alternative reinforcement was suspended to test for resurgence. Five‐pellet alternative reinforcement produced faster elimination and greater resurgence of target‐lever pressing than one‐pellet alternative reinforcement. In Experiment 2, effects of decreasing alternative‐reinforcer magnitude on resurgence were examined. Rats pressed levers and pulled chains for six‐pellet reinforcers during Phases 1 and 2, respectively. In Phase 3, alternative reinforcement was decreased to three pellets for one group, one pellet for a second group, and suspended altogether for a third group. Shifting from six‐pellet to one‐pellet alternative reinforcement produced as much resurgence as suspending alternative reinforcement altogether, while shifting from six pellets to three pellets did not produce resurgence. These results suggest that alternative‐reinforcer magnitude has effects on elimination and resurgence of target behavior that are similar to those of alternative‐reinforcer rate. Thus, both suppression of target behavior during alternative reinforcement and resurgence when conditions of alternative reinforcement are altered may be related to variables that affect the value of the alternative‐reinforcement source.     

Contingency-shaped and rule-governed behavior: instructional control of human loss avoidance

Instructions can override the influence of programmed schedules of reinforcement. Although this finding has been interpreted as a limitation of reinforcement schedule control in humans, an alternative approach considers instructional control, itself, as a phenomenon determined by subjects' reinforcement histories. This approach was supported in a series of experiments that studied instructional and schedule control when instructions either did or did not accord with the schedule of reinforcement. Experiment I demonstrated that accurate instructions control discriminative performances on multiple avoidance schedules, and that such control persists in a novel discrimination. Experiments II and III showed that elimination of instruction-following occurs when inaccurate instructions cause subjects to contact a monetary loss contingency. Experiment IV demonstrated the reinforcing properties of accurate instructions. Skinner's view of rule-governed behavior is consistent with these findings, and can be extended to account for many aspects of instructional control of human operant behavior.     

Aggression as a reinforcer: operant behavior in the mouse-killing rat

Two experiments examined mouse killing as a reinforcer of key pressing by rats that killed mice. In Experiment I, mouse-killing rats performed the key-pressing response when each press was reinforced with presentation of a mouse. Offered a choice between a key that yielded presentation of mice and one that did not, the rats preferred the key that yielded mice. When the contingency was reversed, the rats preferred the other key and continued to kill mice. In Experiment II, mouse-killing rats that did not kill rat pups performed a key-pressing response reinforced with presentation of mice on a variable-interval schedule. In tests for responding reinforced on that schedule with presentation of normal mice, anesthetized mice, dead mice, or rat pups, these rats that killed mice but not rat pups exhibited a decline in response rate when rat pups were the reinforcer. Altering the condition of the mice did not significantly affect performance.     

AGGRESSION AS POSITIVE REINFORCEMENT IN MICE UNDER VARIOUS RATIO‐ AND TIME‐BASED REINFORCEMENT SCHEDULES

There is evidence suggesting aggression may be a positive reinforcer in many species. However, only a few studies have examined the characteristics of aggression as a positive reinforcer in mice. Four types of reinforcement schedules were examined in the current experiment using male Swiss CFW albino mice in a resident—intruder model of aggression as a positive reinforcer. A nose poke response on an operant conditioning panel was reinforced under fixed‐ratio (FR 8), fixed‐interval (FI 5‐min), progressive ratio (PR 2), or differential reinforcement of low rate behavior reinforcement schedules (DRL 40‐s and DRL 80‐s). In the FR conditions, nose pokes were maintained by aggression and extinguished when the aggression contingency was removed. There were long postreinforcement pauses followed by bursts of responses with short interresponse times (IRTs). In the FI conditions, nose pokes were maintained by aggression, occurred more frequently as the interval elapsed, and extinguished when the contingency was removed. In the PR conditions, nose pokes were maintained by aggression, postreinforcement pauses increased as the ratio requirement increased, and responding was extinguished when the aggression contingency was removed. In the DRL conditions, the nose poke rate decreased, while the proportional distributions of IRTs and postreinforcement pauses shifted toward longer durations as the DRL interval increased. However, most responses occurred before the minimum IRT interval elapsed, suggesting weak temporal control of behavior. Overall, the findings suggest aggression can be a positive reinforcer for nose poke responses in mice on ratio‐ and time‐based reinforcement schedules.     

Temporal constraint on choice: Sensitivity and bias in multiple schedules

In multiple schedules of reinforcement, ratios of responses in successive components are relatively insensitive to ratios of obtained reinforcers. An analysis is proposed that attributes changes in absolute response rates to concurrent interactions between programmed reinforcement and extraneous reinforcement in other components. The analysis predicts that ratios of responses in successive components vary with reinforcer ratios, qualified by a term describing the reinforcement context, that is, programmed and extraneous reinforcers. Two main predictions from the analysis were confirmed in an experiment in which pigeons' responses were reinforced in the components of a multiple schedule and analog extraneous reinforcement was scheduled for an alternative response in each component. Sensitivity of response and time ratios to reinforcer ratios in the multiple schedules varied as a function of the rate of extraneous reinforcers. Bias towards responding in one component of the multiple schedule varied as an inverse function of the ratios of extraneous reinforcer rate in the two components. The data from this and previous studies of multiple-concurrent performance were accurately predicted by our analysis and supported our contention that the allocation of behavior in multiple-schedule components depends on the relative values of concurrently-available reinforcers within each component.     

Observing behavior: effects of rate and magnitude of primary reinforcement

Four experiments examined the free-operant observing behavior of rats. In Experiment 1, observing was a bitonic function of random-ratio schedule requirements for the primary reinforcer. In Experiment 2, decreases in the magnitude of the primary reinforcer decreased observing. Experiment 3 examined observing when a random-ratio schedule or a yoked random-time schedule of primary reinforcement was in effect across conditions. Removing the response requirement for the primary reinforcer increased observing, suggesting that the effects of the random-ratio schedule in Experiment 1 likely were due to an interaction between observing and responding for the primary reinforcer. In Experiment 4, decreasing the rate of primary reinforcement by increasing the duration of a random-time schedule decreased observing monotonically. Overall, these results suggest that observing decreases with decreases in the rate or magnitude of the primary reinforcer, but that behavior related to the primary reinforcer can affect observing and potentially affect measurement of conditioned reinforcing value.     

Choice between response units: The rate constancy model

In a conjoint schedule, reinforcement is available simultaneously on two or more schedules for the same response. The present experiments provided food for key pecking on both a random-interval and a differential-reinforcement-of-low-rate (DRL) schedule. Experiment 1 involved ordinary DRL schedules; Experiment 2 added an external stimulus to indicate when the required interresponse time had elapsed. In both experiments, the potential reinforcer frequency from each component was varied by means of a second-order fixed-ratio schedule, and the DRL time parameter was changed as well. Response rates were described by a model stating that time allocation to each component matches the relative frequency of reinforcement for that component. When spending time in a given component, the subject is assumed to respond at the rate characteristic of baseline performance. This model appeared preferable to the absolute-rate version of the matching law. The model was shown to be applicable to multiple-response concurrent schedules as well as to conjoint schedules, and it described some of the necessary conditions for response matching, undermatching, and bias. In addition, the pigeons did not optimize reinforcer frequency.     

Varying response-reinforcer contiguity in a recycling conjunctive schedule

Three experiments describe the effects of manipulating the frequency of response-reinforcer contiguity in a recycling conjunctive schedule. The schedule arranged that a reinforcer was delivered after 30 s provided at least one response had occurred; otherwise the next cycle started immediately. In Experiment 1, this schedule produced the typical pause–respond–pause pattern, with most responses at mid-interval; and, when a limited number of contiguities between responses and food delivery were added, the pattern became more like the monotonic scallop seen on fixed-interval schedules. In Experiment 2, the schedule was initially presented with an additional contingency that allowed contiguity on every trial. Fixed-interval-like behavior occurred and tended to persist as contiguities were gradually eliminated. In Experiment 3, the recycling conjunctive schedule alternated with a condition in which a large number of contiguities occurred. The pause–respond–pause pattern and fixed-interval-like performance occurred with few or many contiguities, respectively. The results of all three experiments illustrate how contiguity interacts with a small number of other variables to determine performance on interval schedules and illuminate previous findings with fixed-interval and fixed-time schedules.     

The role of intermittent food in the induction of attack in pigeons

The present experiments evaluated whether transitions in reinforcer probability are necessary to induce attack in pigeons. In Experiment I, three of six pigeons exposed to response-contingent constant-probability food schedules and a photograph of a conspecific as a target exhibited sustained postreinforcement attack on the target. The postreinforcement pattern of attack developed over the course of the experiment and was accompanied by a reduction in the rate of postreinforcement key pecking and an increase in the postreinforcement pause in key pecking. These effects on key pecking resulted in unprogrammed variations in the probability of reinforcement which may have been responsible for the induction of attack. In Experiment II, the attack-inducing properties of a constant-probability response-independent food schedule were compared to a periodic food schedule matched for overall rate of food delivery and to a no-food condition. In addition to attack, the spatial location of the subjects was monitored during each interfood interval. The periodic and aperiodic food schedules generated very different patterns of spatial location. Postfood attack was induced by both food schedules, although the constant-probability schedule induced attack in fewer birds. The no-food condition was not effective in inducing attack in any birds. These experiments indicate that intermittent food schedules without reductions in reinforcer probability are sufficient to induce attack in some pigeons, although not as effective as schedules with transitions in reinforcer probability.