Instrumental performance on negative schedules

Three experiments examined the performance of rats pressing a lever for food reinforcement on a schedule in which high rates of response resulted in lowered rates of reinforcement (i.e. a schedule with a negative component). In Experiment 1, rats responded on a variable interval (VI) schedule with a conjoint component such that every 30 responses a reinforcement programmed by the VI schedule was cancelled. These subjects generally emitted a lower response rate than rats responding on a VI schedule yoked to the former subjects with respect to the delivery of reinforcement, although response rate differences were sometimes not large. Similar response-rate effects were obtained in Experiment 2 using a within-subject yoking procedure. In Experiment 3, reinforced interresponse times were matched on negative and VI schedules yoked in terms of reinforcement rate, and the response rate emitted in these conditions were similar. These results give support to theories of instrumental conditioning that stress the strengthening and shaping properties of reinforcement.     

Interlocking schedules: the relationship between response and time requirements.

Rats were exposed to an interlocking fixed-ratio 150 fixed-interval 5-minute schedule of food reinforcement and then to yoked variable-ratio schedules in which individual ratios corresponded exactly to the ratios of responses to reinforcement obtained on the interlocking schedule. After additional training with the interlocking schedule, the rats were exposed to yoked variable-interval schedules in which intervals corresponded to the intervals between successive reinforcements obtained on the second interlocking schedule. Response rates were highest in the yoked VR condition and lowest in the yoked VI, while intermediate rates characterized the interlocking schedule. Break-run patterns of responding were generated by the interlocking schedule for all subjects, while both the yoked VR and VI schedules produced comparatively stable local rates of responding. These results indicate that responding is sensitive to the interlocking schedule's inverse relationship between reinforcement frequency and responses per reinforcement.     

RESISTANCE TO EXTINCTION AND RELAPSE IN COMBINED STIMULUS CONTEXTS

Reinforcing an alternative response in the same context as a target response reduces the rate of occurrence but increases the persistence of that target response. Applied researchers who use such techniques to decrease the rate of a target problem behavior risk inadvertently increasing the persistence of the same problem behavior. Behavioral momentum theory asserts that the increased persistence is a function of the alternative reinforcement enhancing the Pavlovian relation between the target stimulus context and reinforcement. A method showing promise for reducing the persistence‐enhancing effects of alternative reinforcement is to train the alternative response in a separate stimulus context before combining with the target stimulus in extinction. The present study replicated previous findings using pigeons by showing that combining an “alternative” richer VI schedule (96 reinforcers/ hr) with a “target” leaner VI schedule (24 reinforcers/hr) reduced resistance to extinction of target responding compared with concurrent training of the alternative and target responses (totaling 120 reinforcers/hr). We also found less relapse with a reinstatement procedure following extinction with separate‐context training, supporting previous findings that training conditions similarly influence both resistance to extinction and relapse. Finally, combining the alternative stimulus context was less disruptive to target responding previously trained in the concurrent schedule, relative to combining with the target response trained alone. Overall, the present findings suggest the technique of combining stimulus contexts associated with alternative responses with those associated with target responses disrupts target responding. Furthermore, the effectiveness of this disruption is a function of training context of reinforcement for target responding, consistent with assertions of behavioral momentum theory.     

Some temporal parameters of non-contingent reinforcement

In each baseline session, pigeons were exposed to a multiple schedule in which each of five distinctive stimuli was correlated with a different frequency of reinforcement. In one component, responses were reinforced with a probability of 0.10 (random-ratio schedule); in the other four components, responses were reinforced with different scheduled temporal frequencies averaging 30 to 240 sec between reinforcements (random-interval schedules). For periods lasting 30 sessions, contingent reinforcement was discontinued and reinforcement was presented independent of responding at irregular intervals averaging 30, 60, or 120 sec, while the sequence of stimuli continued. After each such period, the baseline was reinstated for 30 sessions. The data indicated that: (1) The rate of responding in the presence of all stimuli decreased as exposure to the non-contingent reinforcement procedure was prolonged, at all the frequencies of reinforcement employed; (2) The rate under the random-ratio schedule declined faster than the rates under all the random-interval schedules, presumably because the decrease in reinforcement frequency under this stimulus condition was greatest; (3) The decline in rates of responding under the stimuli correlated with the random-interval schedules tended to be greatest for the stimuli paired with the lowest frequencies of reinforcement.     

Independence of concurrent responding maintained by interval schedules of reinforcement

A pigeon's responses were reinforced on a variable-interval schedule on one key; and, concurrently, either a multiple or a fixed-interval schedule of reinforcement was in effect on a second key. These concurrent schedules, conc VI 3 (mult VI 3 EXT) or conc VI 3 FI 6, were programmed with or without a changeover delay (COD). Because the COD provided that responses on one key could not be followed by reinforced responses on the other key, responding on one key was not likely to accidentally come under the control of the reinforcement schedule on the other. When the COD was used, the performances on each key were comparable to the performances maintained when these interval schedules are programmed separately. The VI schedule maintained a relatively constant rate of responding, even though the rate of responding on the second key varied in a manner appropriate to the schedule on the second key. The mult VI 3 EXT schedule maintained two separate rates of responding: a relatively high rate during the VI 3 component, and almost no responding during the EXT component. The FI schedule maintained the gradually increasing rate of responding within each interval that is characteristic of the performance maintained by this schedule. The concurrent performances, however, did include certain interactions involving the local characteristics of responding and the over-all rates of responding maintained by the various schedules. The relevance of the present findings to an inter-response time analysis of VI responding, a chaining account of FI responding, and the concept of the reflex reserve was discussed.     

Stimulus–reinforcer relations established during training determine resistance to extinction and relapse via reinstatement

The baseline rate of a reinforced target response decreases with the availability of response‐independent sources of alternative reinforcement; however, resistance to disruption and relapse increases. Because many behavioral treatments for problem behavior include response‐dependent reinforcement of alternative behavior, the present study assessed whether response‐dependent alternative reinforcement also decreases baseline response rates but increases resistance to extinction and relapse. We reinforced target responding at equal rates across two components of a multiple schedule with pigeons. We compared resistance to extinction and relapse via reinstatement of (1) a target response trained concurrently with a reinforced alternative response in one component with (2) a target response trained either concurrently or in separate components from the alternative response across conditions. Target response rates trained alone in baseline were higher but resistance to extinction and relapse via reinstatement tests were greater after training concurrently with the alternative response. In another assessment, training target and alternative responding together, but separating them during extinction and reinstatement tests, produced equal resistance to extinction and relapse. Together, these findings are consistent with behavioral momentum theory—operant response–reinforcer relations determined baseline response rates but Pavlovian stimulus–reinforcer relations established during training determined resistance to extinction and relapse. These findings imply that reinforcing alternative behavior to treat problem behavior could initially reduce rates but increase persistence.     

Positive interaction (induction) in multiple variable-interval, differential-reinforcement-of-high-rate schedules

The effect of increases in the rate of responding in one component of a multiple schedule upon the rate of responding in a second component was investigated. Pigeons were exposed to a multiple schedule where both components were initially variable-interval schedules having the same parameter value. After rate of key pecking stabilized, one component was changed to a schedule that differentially reinforced high rates of responding. Rate of reinforcement in this varied component was adjusted to remain equal to rate of reinforcement in the constant (variable-interval) component. Four of five pigeons showed a maintained increase in rate of responding during both the constant and varied components, even though rates of reinforcement did not change.     

Resurgence and downshifts in alternative reinforcement rate

Resurgence refers to an increase in a previously suppressed target behavior with a relative worsening of conditions for a more recently reinforced alternative behavior. This experiment examined the relation between resurgence and the magnitude of a reduction in the rate of reinforcement for the alternative behavior. Groups of both male and female rats initially pressed a target lever for food on a variable-interval (VI) 30-s schedule. In a second phase, responding to the target lever was extinguished for all groups and pressing an alternative lever was reinforced on a VI 10-s schedule. Next, the rate of reinforcement for alternative behavior was reduced differentially across groups by arranging extinction, VI 80-s, VI 40-s, VI 20-s, or continued VI 10-s reinforcement. Target responding increased as an exponential function of the magnitude of the reduction in alternative reinforcement rates. With the exception that males appeared to show higher rates of target responding in baseline and higher rates of alternative responding in other phases, the overall pattern of responding across phases was not meaningfully different between sexes. The pattern of both target and alternative response rates across sessions and phases was well described quantitatively by the Resurgence as Choice in Context model.     

Persistent behavior maintained by unavoidable shocks

Squirrel monkeys were trained on a multiple schedule in which 10-min periods on a continuous shock avoidance schedule, indicated by a yellow light, alternated with 10-min periods on a 1.5-min variable interval schedule of food reinforcement (VI 1.5). A white light indicated that VI 1.5 was in effect, except for the middle 2 min of the period on VI 1.5, in which a blue light appeared and terminated with the delivery of a 0.5-sec unavoidable shock. Stable response rates developed in the avoidance and VI 1.5 components. However, the highest response rates occurred in the blue, preshock stimulus. A series of experiments showed that responding in the blue stimulus persisted even when responding had been extinguished on both the VI schedule of food reinforcement and the shock avoidance schedule. Responding in the blue stimulus ceased when the blue stimulus terminated without shock or when it terminated with a response-contingent shock. Each time responding ceased, it was restored by terminating the blue stimulus with an unavoidable shock. When the blue stimulus was on throughout each session and unavoidable shocks were delivered at regular 10-min intervals, responding was well maintained. These results show that in monkeys that have been trained on a continuous avoidance schedule, unavoidable shocks can maintain responding even under conditions where responses have no programmed consequences.     

Effects of variable-interval value and amount of training on stimulus generalization.

In Experiment 1 pigeons pecked a key that was illuminated with a 501-nm light and obtained food by doing so according to a variable-interval (VI) schedule of reinforcement, the mean value of which differed across groups: either 30 s, 120 s, or 240 s. The pigeons in all three groups were trained for 10 50-min sessions. Generalization testing was conducted in extinction with different wavelengths of light. Absolute and relative generalization gradients were similar in shape for the three groups. Experiment 2 was a systematic replication of Experiment 1 using line orientation as the stimulus dimension and a mean VI value of either 30 s or 240 s. Again, gradients of generalization were similar for the two groups. In Experiment 3 pigeons pecked a key that was illuminated with a 501-nm light and obtained food reinforcers according to either a VI 30-s or a 240-s schedule. Training continued until response rates stabilized (> 30 sessions). For subjects trained with the 30-s schedule, generalization gradients were virtually identical regardless of whether training was for 10 sessions (Experiment 1) or until response rates stabilized. For subjects trained with the VI 240-s schedule, absolute generalization gradients for subjects trained to stability were displaced upward relative to gradients for subjects trained for only 10 sessions (Experiment 1), and relative generalization gradients were slightly flatter. These results indicate that the shape of a generalization gradient does not necessarily depend on the rate of reinforcement during 10-session single-stimulus training but that the effects of prolonged training on stimulus generalization may be schedule dependent.     

Adventitious reinforcement during long-duration DRO exposure

Differential reinforcement of other behavior (DRO) is a procedure often used to decrease problem behavior, but the processes responsible for behavior reduction are not well understood. This study assessed whether adventitious reinforcement of other behavior contributes to DRO effectiveness when, relative to previous research, DRO exposure is prolonged. Two response options were presented on a computer and target responding was reinforced on a variable-ratio schedule. Response rates were then compared during DRO versus yoked variable-time or extinction probes. Across 2 experiments, DRO decreased target responding and increased other responding more than control conditions. However, increases in other responding did not usually maintain despite target responding remaining at low levels. DRO might adventitiously reinforce other responses transiently but the decreases in target behavior could not be entirely explained by adventitious reinforcement of the other response. Instead, reductions in target responding likely depend on the discriminability of the DRO contingency.     

The role of adventitious reinforcement during differential reinforcement of other behavior: A systematic replication

Differential reinforcement of other behavior (DRO) is commonly used to decrease problem behavior by presenting reinforcers contingent upon the absence of a target response. Although it is well demonstrated that DROs decrease response rates, the processes producing these decreases are not well understood. The present study systematically replicated previous research assessing whether adventitious reinforcement of alternative behavior contributes to the effectiveness of DRO. We presented university students with two options on a computer and reinforced target responding on a variable-ratio schedule. Next, we compared decreases in target-response rates and any increases in alternative responding during DRO schedules versus yoked variable-time schedules or extinction probes. DRO schedules resulted in the lowest target-response rate and highest alternative-response rate. These findings generally provide some support for the adventitious reinforcement of “other” behavior.     

Solution of the intermediate size problem by pigeons

Pigeons learned to respond to the middle-sized member (S^D) of a set of three simultaneously presented stimuli with responses to the S^D reinforced on a VI 1 schedule. They were then tested for several days with other sets of three stimuli. One procedure presented reinforcements on a VI 1 schedule during the test independent of the stimulus chosen when a reinforcement was programmed. The tests were also given under extinction conditions. With the testing carried out with extinction, preference consistently was for the test stimulus most similar in physical size to the S^D. However, when the tests were with reinforcement, random responding resulted. Another effect of testing with reinforcement was an increase in incorrect responding with the training set. Such a test procedure was unsatisfactory for determining the effective aspect of the S^D. The conclusion, based on the data of the extinction series, was that pigeons learned the intermediate size problem on the basis of the discrimination of absolute stimulus properties.     

Interaction of methadone, reinforcement history, and variable-interval performance.

In the present study, we examined how a reinforcement schedule history that generated high or low rates of responding influenced the effects of acute (Experiment 1) and chronic (Experiment 2) methadone administration. Initially, key-peck responses of pigeons were maintained under a variable-interval 90-s schedule of food presentation, and a methadone dose-response curve was determined with doses of 0.6, 1.2, and 2.4 mg/kg. The pigeons were then exposed, for at least 40 sessions, to either a fixed-ratio 50 schedule or a differential-reinforcement-of-low-rate 10-s schedule, or were given continued exposure to the variable-interval schedule. The methadone dose-response curve was redetermined after all pigeons again were responding under the variable-interval schedule. The effects of two different daily methadone doses (9.0 and 12.0 mg/kg/day) and withdrawal precipitated by naloxone also were assessed. Experience with a fixed-ratio or differential reinforcement of low rate schedule did not result in significantly different response rates under the variable-interval schedule and, in general, the acute effects of methadone did not have differential effects correlated with schedule history. However, for 2 of 4 subjects the rate-decreasing effects of methadone on rates of key pecking were greater following a history of low-rate responding, suggesting a possible interaction between schedule history and effects of methadone. Daily methadone administration under the variable-interval schedule revealed that pigeons with experience under the differential reinforcement of low rate schedule developed more rapid and complete tolerance to the rate-decreasing effects of methadone. Three of the 4 subjects in this group showed rate increases above drug-free baselines during chronic methadone dosing. Pigeons with a history of fixed-ratio responding also developed tolerance to the rate-decreasing effects of methadone but without the subsequent rate increases seen by subjects with low-rate histories. No subjects with variable-interval histories showed complete recovery of drug-free baselines, suggesting that interpolated training under other schedules may attenuate the rate-altering effects of chronically administered drugs. Naloxone (1.0 mg/kg), administered during the chronic methadone phase, resulted in greater disruption of responding by pigeons with a history of low-rate responding, as compared to subjects in the other two groups.(ABSTRACT TRUNCATED AT 400 WORDS)     

Effects of response rate, reinforcement frequency, and the duration of a stimulus preceding response-independent food

Food-reinforced key pecking in the pigeon was maintained under a four-component multiple schedule. In two components, responding was maintained at high rates under a random-ratio schedule. In the other two components, responding was maintained at low rates under a schedule that specified a minimum interresponse time. For both high and low response rates, one of the schedule components was associated with a high reinforcement frequency and the other components with a lower reinforcement frequency. During performance under these schedules, a stimulus terminated by access to response-independent food was periodically presented. The duration of this pre-food stimulus was 5, 30, 60, or 120 sec. Changes in rate of key pecking during the pre-food stimulus were systematically related to baseline response rate and the duration of the stimulus. Both high and low response rates were increased during the 5-sec stimulus. At longer stimulus durations, low response rates were unaffected and high response rates were decreased during the stimulus. For two of three pigeons, high response rates maintained under a lower frequency of reinforcement tended to be decreased more than high response rates maintained under a higher reinforcement frequency. In general, the magnitude of decrease in high response rates was inversely related to the duration of the pre-food stimulus.     

Continuous, fixed-ratio, and fixed-interval reinforcement in honey bees

Bees learned to enter a Plexiglas tube and to suck small portions of sugar solution; every entry or every fifth entry was reinforced. During an extinction phase, the bees on the fixed-ratio schedule emitted twice as many responses as did those given continuous reinforcement. Bees on a fixed-interval schedule of reinforcement emitted lower response rates than did those given fixed-ratio reinforcement. By extending the conditioning procedure for several days, it was possible to maintain responding with fixed-ratio schedules requiring 30 responses per reinforcement and with fixed-interval values up to 90 sec. Under fixed-interval schedules, response rates did not increase toward the end of the reinforcement intervals.     

The role of the response-reinforcer relation in delay-of-reinforcement effects

The role of the response-reinforcer relation in maintaining operant behavior under conditions of delayed reinforcement was investigated by using a two-operandum (i.e., two-key) procedure with pigeons. Responding on one key was reinforced under a tandem variable-interval differential-reinforcement-of-other-behavior (tandem VI DRO) schedule. The schedule defined a resetting unsignaled delay-of-reinforcement procedure in that a response was required when the interfood interval of the VI schedule lapsed, but further responding during the DRO component on either key reset the time interval. This ensured a fixed delay duration between any response and reinforcement. Responding on another key, physically identical to the first one except for spatial location, otherwise was without consequence. The location of the key correlated with the delay-of-reinforcement procedure varied between sessions according to a semirandom sequence. Differences in response rates between the two keys were greater, with proportionally higher rates on the key correlated with the delay-of-reinforcement procedure, the longer the delay-of-reinforcement procedure remained correlated with the same key. Differences in responding on the two keys also increased within individual sessions. These results suggest that the response-reinforcer relation is the primary determinant of responding when responding is acquired and maintained with delayed reinforcement.     

Response-dependent point loss and response force as disrupting operations on behavioral resistance to change in humans

Behavioral momentum theory (BMT) provides a theoretical and methodological framework for understanding how differentially maintained operant responding resists disruption. A common way to test operant resistance involves contingencies with suppressive effects, such as extinction or prefeeding. Other contingencies with known suppressive effects, such as response-cost procedures arranged as point-loss or increases in response force, remain untested as disruptive events within the BMT framework. In the present set of three experiments, responding of humans was maintained by point accumulation programmed according to a multiple variable-interval (VI) VI schedule with different reinforcement rates in either of two components. Subsequently, subtracting a point following each response (Experiment 1) or increasing the force required for the response to be registered (Experiments 2 and 3 decreased response rates, but responding was less disrupted in the component associated with the higher reinforcement rate. The point-loss contingency and increased response force similarly affected response rates by suppressing responding and human persistence, replicating previous findings with humans and nonhuman animals when other types of disruptive events (e.g., extinction and prefeeding) were investigated. The present findings moreover extend the generality of the effects of reinforcement rate on persistence, and thus BMT, extending the analysis of resistance to two well-known manipulations used to reduce responding in the experimental analysis of behavior.     

A comparison of the key-peck and treadle-press operants in the pigeon: differential-reinforcement-of-low-rate schedule of reinforcement

Key pecking and treadle pressing in pigeons were compared under concurrent (key-treadle) and single-operant differential-reinforcement-of-low-rate schedules of food reinforcement ranging from 5 to 60 sec (concurrent procedure) or 5 to 120 sec (single-operant procedure). Under both procedures, the two operants followed the same general law: decreasing response rate and reinforcement rate and increasing number of responses per reinforcement as a function of increasing schedule interval. High correlations were found between key pecking and treadle pressing for the measures of response rate, reinforcement rate, and responses per reinforcement. Regression equations allowed the prediction of treadle pressing from key pecking. More bursting occurred in responding to the key, and key pecking showed a more precise temporal discrimination than treadle pressing. A test for sequential dependencies between key and treadle responses showed significant dependencies not only under the concurrent procedure but also in data created artificially by merging key and treadle sequences from different pigeons under the concurrent procedure and from the same pigeon under the single-operant procedure. It seems likely that the sequential dependencies found were due to the independent action of the schedule on each operant and that behavioral dependencies did not occur with the concurrent training procedure. The key-peck operant does not appear to have any special qualities that preclude its use in discovering general laws of behavior, at least under the differential-reinforcement-of-low-rate schedule. The usefulness of the key peck in other situations requires direct experimental study.