Discrimination of a response-independent component in a multiple schedule

Pigeons were trained to respond in non-differential reinforcement pre-discrimination training, with a multiple variable-interval 1-min variable-interval 1-min schedule. Each bird then received discrimination training with a multiple variable-interval 1-min variable-time 1-min schedule. Thus, discrimination training was between response-dependent (variable-interval) and response-independent (variable-time) schedules with the rate of reinforcement equated. In Experiment I, only three sessions of non-differential reinforcement preceded discrimination training and for half the birds, a 0° line was correlated with the response-dependent schedule; for the remaining birds the 0° line was correlated with the response-independent schedule. Post-discrimination gradients of excitatory stimulus control were obtained from the former group, while the latter group showed little evidence of post-discrimination stimulus control by the 0° line. Differential responding to the variable-time schedule was not accompanied by behavioral contrast to the variable-interval schedule. In Experiment II, 20 sessions of non-differential reinforcement preceded discrimination training and the 0° line was correlated with variable-time reinforcement for each bird. Differential responding to the 0° line was accompanied by negative induction to the variable-interval schedule and by inhibitory stimulus control about the 0° line during a post-discrimination generalization test.     

Information on response requirements compared with information on food density as a reinforcer of observing in pigeons

On a variable-interval schedule, pecking the key to the pigeon's right (observing response) produced red or green displays relating to the delivery of grain and its dependence on pecking the key to the left (food key). During various blocks of sessions, mixed (no stimulus change) schedules including the following pairs of components were temporarily converted by the observing response to their corresponding multiple (correlated stimuli) schedules: variable-interval 60-s, extinction; variable-interval 60-s, variable-time (response-independent) 60-s; extinction, variable-time 60-s. Differences in food delivery maintained substantial rates of responding on the observing key, without regard to pecking requirements on the food key. Although stimuli correlated with differences in the response requirement on the food key maintained higher observing rates than those maintained by uncorrelated stimuli, they were much lower than those based on food. The value of predictive stimuli as reinforcers is determined by the value of the events predicted. In particular, the cost of pecking appears to be low, and this may place limitations on the applicability of energy-based and economic models of behavior.     

Effects of concurrent response-independent reinforcement on fixed-interval schedule performance

In three experiments, behavior maintained by fixed-interval schedules changed when response-independent reinforcement was delivered concurrently according to fixed- or variable-time schedules. In Experiment I, a pattern of positively accelerated responding during fixed interval was changed to a linear pattern when response-independent reinforcement occurred under a variable-time schedule. Overall response rates (total responses/total time) decreased as the frequency of response-independent reinforcement increased. Experiment II showed that the response-rate changes in the first experiment were controlled by the response-reinforcer relation, but the changes in patterns of responding were similar whether concurrently available reinforcement at varying times was response-dependent or response-independent. In the final experiment, the addition of response-independent reinforcement at fixed times to a fixed-interval schedule resulted in changes in both local and overall response rates and in the occurrence of positively accelerated responding between reinforcements. These results suggest that the temporal distribution of reinforcers determines response patterns and that both the response-reinforcement dependency and the schedule of reinforcement determine overall response rates during concurrently scheduled response-dependent and response-independent reinforcement.     

Resistance to the response-decrementing effects of response-independent reinforcement produced by delay and non-delay schedules of reinforcement

In two experiments, animals were initially exposed to response-dependent schedules of food before exposure to response-independent reinforcement matched for overall rate and temporal distribution of reinforcers to the preceding condition. In Experiment I, response decrements during the response-independent phase were smaller after delayed reinforcement training than after a comparable immediate reinforcement schedule, for both doves and rats. In Experiment II variable-interval and variable-ratio schedules, both with either immediate or delayed reinforcement, were used with rats. Both the delayed reinforcement schedules produced resistance to subsequent response-independent reinforcement, but response decrements were larger after either of the immediate reinforcement conditions. It was concluded that the critical factor in response maintenance under response-independent reinforcement was the type of response-reinforcer contiguities permitted under the response-dependent schedule rather than perception of response-reinforcer “contingencies”. If the response-dependent schedule was arranged so that behaviours other than a designated operant (key pecking or lever pressing) could be contiguous with food, responding was maintained well under response-independent schedules.     

Within-session Response Patterns On Conjoint Variable-interval Variable-time Schedules

Operant responding often changes within sessions, even when factors such as rate of reinforcement remain constant. The present study was designed to determine whether within-session response patterns are determined by the total number of reinforcers delivered during the session or only by the reinforcers earned by the operant response. Four rats pressed a lever and 3 pigeons pecked a key for food reinforcers delivered by a conjoint variable-interval variable-time schedule. The overall rate of reinforcement of the conjoint schedule varied across conditions from 15 to 480 reinforcers per hour. When fewer than 120 reinforcers were delivered per hour, the within-session patterns of responding on conjoint schedules were similar to those previously observed when subjects received the same total number of reinforcers by responding on simple variable-interval schedules. Response patterns were less similar to those observed on simple variable-interval schedules when the overall rate of reinforcement exceeded 120 reinforcers per hour. These results suggest that response-independent reinforcers can affect the within-session pattern of responding on a response-dependent schedule. The results are incompatible with a response-based explanation of within-session changes in responding (e.g., fatigue), but are consistent with both reinforcer-based (e.g., satiation) and stimulus-based (e.g., habituation) explanations.     

Reponse-reinforcer independence and conventional extinction after fixed-interval and variable-interval schedules

After training three albino rats to bar press during a multiple fixed-interval variable-interval schedule, the response-reinforcer dependency was simultaneously removed from both components, converting the schedule to multiple fixed-time variable-time. Response rates were reduced in both components under these conditions but the fixed-time schedule maintained relatively higher response rates with each rat. After reinstating the response-reinforcer dependency in both components, responding was conventionally extinguished by rendering the pellet dispenser inoperative. Responding rapidly decreased to near-zero levels. Differences in fixed and variable-time schedules in sustaining behavior are discussed in terms of differences in response rates at the time of reinforcer delivery. Similarities and differences between conventional extinction and schedules delivering response-independent reinforcers are also discussed.     

Response-reinforcer dependence and independence in multiple and mixed schedules

Albino rats were conditioned to lever press during two-component multiple and mixed schedules in which response-dependent and response-independent reinforcers occurred in the different components. Relative response rates in the components associated with response-dependent reinforcers were higher (a) when different visual and auditory stimuli were associated with the two components and (b) when mixed schedule components were long in duration. These results illustrate the contribution of the response-reinforcer relation to stimulus control and schedule control of behavior. They also suggest that under some conditions, reinforcers need not be consistently associated with a particular response to ensure that the response is maintained at a relatively high rate.     

Stimulus control of schedule-induced activity in pigeons during multiple schedules

Stimulus control of schedule-induced general activity was demonstrated with pigeons using multiple schedules of response-independent food delivery. In Experiment 1, the introduction of food during a multiple variable-time 30-second variable-time 30-second schedule produced a tenfold increase in activity above the no-food baseline. Each pigeon developed stable differential activity rates during the components (correlated with red and green lights) of a multiple variable-time 30-second extinction schedule. Lengthening the extinction component from 1 to 7 minutes increased the rate differences and produced a reliable pattern of responding during S− (the stimulus correlated with extinction): Activity rate was high immediately following the change from S+ (the stimulus correlated with variable-time 30-second) to S−, then decreased abruptly and remained low throughout the middle of the interval, and subsequently showed a positively accelerated increase until the stimulus changed to S+. In Experiment 2, three pigeons were exposed to a mixed variable-time extinction schedule prior to a multiple variable-time extinction schedule. Auditory rather than visual stimuli were used to determine the generality of Experiment 1 results. The multiple- versus mixed-schedule results indicated that stimulus control of activity occurred for two of the birds, but rate differences between S+ and S− were much less than those demonstrated with visual stimuli. A direct comparison of visual and auditory stimulus control in Experiment 3 supported this conclusion. These parallels between the stimulus control of reinforced responding and that of schedule-induced activity suggest that the stimulus control of induced activity may be a factor in operant stimulus control.     

Relative persistence of avoidance and positively reinforced behavior

In one experiment with rats, and a second with pigeons, subjects were trained on a schedule in which identical response dependencies applied to intermittent receipt of positive reinforcement (PR) for one group and avoidance (AV) of shock for a second group. After obtaining comparable stable training performance for groups PR and AV, persistence tests were conducted with a traditional extinction (EXT) procedure vs. response-independent (FREE) delivery of positive reinforcers or shocks. In both experiments, response elimination was rapid in EXT for groups PR and AV, and responding tended to be maintained in groups PR and AV in the FREE persistence test. These results contradict the widely held assumption that avoidance behavior is unusually resistant to extinction, and they call for a re-examination of elimination of avoidance responding.     

Some effects of response-independent reinforcement on auditory generalization gradients.

Two groups of six rats received discrimination training with two auditory stimuli differing in intensity. During one stimulus, the schedule was variable interval; during the other, it was either variable time or extinction. Both the variable time and extinction schedules resulted in differential rates of responding in the presence of the two stimuli. Extinction resulted in an earlier and more stable difference. Stimulus generalization gradients obtained along the noise-intensity dimension revealed peak shift with both procedures. In addition, a secondary peak to stimuli in between the two training stimuli occurred with the variable-time schedule.     

Using visual reinforcement to establish stimulus control of responding of Siamese fighting fish (Betta splendens)

Stimulus control of ring swimming was studied with male Siamese fighting fish (Betta splendens) using 2-component multiple schedules in which the components were correlated with the presence or absence of air bubbles in the water. In Experiment 1, either response-independent mirror presentations or extinction was juxtaposed with immediate response-dependent mirror presentations. Rates of ring swimming generally were higher with immediate reinforcement than with either response-independent mirror presentations or extinction. In Experiment 2, different durations of response-dependent mirror presentations were juxtaposed. Generally, higher rates of ring swimming occurred with 15-s than with 0-, 1-, or 3-s durations. Results demonstrate that stimulus control of responding can be established with these fish under several conditions of differential reinforcement.     

Economic and biological influences on key pecking and treadle pressing in pigeons

Pigeons were studied on a two-component multiple schedule in which the required operant was, in different conditions, biologically relevant (i.e., key pecking) or nonbiologically relevant (i.e., treadle pressing). Responding was reinforced on a variable-interval (VI) 2-min schedule in both components. In separate phases, additional food was delivered on a variable-time (VT) 15-s schedule (response independent) or a VI 15-s schedule (response dependent) in one of the components. The addition of response-independent food had different effects on responding depending on the operant response and on the frequency with which the components alternated. When components alternated frequently (every 10 s), all pigeons keypecked at a much higher rate during the component with the additional food deliveries, whether response dependent or independent. In comparison, treadle pressing was elevated only when the additional food was response dependent; rate of treadling was lower when the additional food was response independent. When components alternated infrequently (every 20 min), pigeons key pecked at high rates at points of transition into the component with the additional food deliveries. Rate of key pecking decreased with time spent in the 20-min component when the additional food was response independent, whereas rate of pecking remained elevated in that component when the additional food was response dependent. Under otherwise identical test conditions, rate of treadle pressing varied only as a function of its relative rate of response-dependent reinforcement. Delivery of response-independent food thus had different, but predictable, effects on responding depending on which operant was being studied, suggesting that animal-learning procedures can be integrated with biological considerations without the need to propose constraints that limit general laws of learning.     

Response rate viewed as engagement bouts: resistance to extinction

Rats obtained food pellets by nose poking a lighted key, the illumination of which alternated every 50 s during a session between blinking and steady, signaling either a relatively rich (60 per hour) or relatively lean (15 per hour) rate of reinforcement. During one training condition, all the reinforcers in the presence of the rich-reinforcement signal were response dependent (i.e., a variable-interval schedule); during another condition only 25% were response dependent (i.e., a variable-time schedule operated concurrently with a variable-interval schedule). An extinction session followed each training block. For both kinds of training schedule, and consistent with prior results, response rate was more resistant to extinction in the presence of the rich-reinforcement signal than in the presence of the lean-reinforcement signal. Analysis of interresponse-time distributions from baseline showed that differential resistance to extinction was not related to baseline differences in the rate of initiating response bouts or in the length of bouts. Also, bout-initiation rate (like response rate) was most resistant to extinction in the presence of the rich-reinforcement signal. These results support the proposal of behavioral momentum theory (e.g., Nevin & Grace, 2000) that resistance to extinction in the presence of a discriminative stimulus is determined more by the stimulus-reinforcer (Pavlovian) than by the stimulus-response-reinforcer (operant) contingency.     

Combinations of response-reinforcer dependence and independence

The contribution of the response-reinforcer dependency to the control of behavior was investigated. Pigeons were trained to key peck under a variable-interval schedule of reinforcement. With the total number and temporal distribution of reinforcer deliveries in experimental sessions constant, the effects of varying the percentage of response-independent reinforcement were examined. At different times, 100%, 66%, 33%, 10%, or 0% of the scheduled reinforcers were delivered dependent upon key pecking and the remainder were delivered independently of responding. Response rates were related to the percentage of response-dependent reinforcement with lower response rates associated with smaller percentages of response-dependent reinforcement. The results suggest that the response-reinforcer relation exerts control over behavior in a manner similar to that exerted by other parameters of reinforcement.     

Extinction of responding maintained by timeout from avoidance

The resistance to extinction of lever pressing maintained by timeout from avoidance was examined. Rats were trained under a concurrent schedule in which responses on one lever postponed shock on a free-operant avoidance (Sidman) schedule (response-shock interval = 30 s) and responses on another lever produced 2 min of signaled timeout from avoidance on a variable-ratio 15 schedule. Following extended training (106 to 363 2-hr sessions), two experiments were conducted. In Experiment 1 two different methods of extinction were compared. In one session, all shocks were omitted, and there was some weakening of avoidance but little change in timeout responding. In another session, responding on the timeout lever was ineffective, and under these conditions timeout responding showed rapid extinction. The within-session patterns produced by extinction manipulations were different than the effects of drugs such as morphine, which also reduces timeout responding. In Experiment 2 shock was omitted for many consecutive sessions. Response rates on the avoidance lever declined relatively rapidly, with noticeable reductions within 5 to 10 sessions. Extinction of the timeout lever response was much slower than extinction of avoidance in all 4 rats, and 2 rats continued responding at baseline levels for more than 20 extinction sessions. These results show that lever pressing maintained by negative reinforcement can be highly resistant to extinction. The persistence of responding on the timeout lever after avoidance extinction is not readily explained by current theories.     

Responding under schedules combining response-dependent and response-independent shock delivery

Lever pressing of three squirrel monkeys with experience under continuous avoidance schedules was maintained by response-produced shock under a 5-minute variable-interval schedule. Responding decreased when half of the scheduled shocks were delivered independently of lever pressing and decreased further when all shocks were independent of lever pressing. Responding was lowest when all shocks were eliminated. When the proportion of response-dependent shocks increased, responding increased. This relation occurred even though the frequency and temporal distribution of shock delivery remained the same. Responding of two monkeys increased in a graded fashion as the frequency of shock was increased by arranging variable-time 5-minute, 2-minute, and 1-minute schedules jointly with the variable-interval 5-minute schedule. Thus, increasing the proportion of response-independent shocks decreased responding when the overall frequency of shocks stayed the same, but increased responding when the overall frequency of shock delivery increased.     

RESISTANCE TO EXTINCTION AND RELAPSE IN COMBINED STIMULUS CONTEXTS

Reinforcing an alternative response in the same context as a target response reduces the rate of occurrence but increases the persistence of that target response. Applied researchers who use such techniques to decrease the rate of a target problem behavior risk inadvertently increasing the persistence of the same problem behavior. Behavioral momentum theory asserts that the increased persistence is a function of the alternative reinforcement enhancing the Pavlovian relation between the target stimulus context and reinforcement. A method showing promise for reducing the persistence‐enhancing effects of alternative reinforcement is to train the alternative response in a separate stimulus context before combining with the target stimulus in extinction. The present study replicated previous findings using pigeons by showing that combining an “alternative” richer VI schedule (96 reinforcers/ hr) with a “target” leaner VI schedule (24 reinforcers/hr) reduced resistance to extinction of target responding compared with concurrent training of the alternative and target responses (totaling 120 reinforcers/hr). We also found less relapse with a reinstatement procedure following extinction with separate‐context training, supporting previous findings that training conditions similarly influence both resistance to extinction and relapse. Finally, combining the alternative stimulus context was less disruptive to target responding previously trained in the concurrent schedule, relative to combining with the target response trained alone. Overall, the present findings suggest the technique of combining stimulus contexts associated with alternative responses with those associated with target responses disrupts target responding. Furthermore, the effectiveness of this disruption is a function of training context of reinforcement for target responding, consistent with assertions of behavioral momentum theory.     

Some effects of response independent reinforcers in multiple schedules

In Exp. I, rats' lever presses were conditioned on multiple variable-interval variable-interval schedules. Changing one of the multiple schedule components to variable time reduced responding in that component. Further reductions in responding occurred in both components when the schedule was changed to multiple variable-time variable-time. After reinstating the multiple variable-interval variable-time schedule, lower response rates were maintained in the variable-time component during a series of stimulus reversals. In Exp. II, replacement of extinction components of multiple variable-interval extinction or multiple extinction extinction with variable-time schedules for single sessions (probe) resulted in response rate increments in those components. In the former schedule these increases were concomitant with response decreases during the variable-interval components. Response increases in the variable-time probes were related to conditioning history and, as a result, to response probability at the time of the probe.