Timeout postponement without increased reinforcement frequency

Three experiments were conducted to examine pigeons' postponement of signaled extinction periods (timeouts) from a schedule of food reinforcement when such responding neither decreased overall timeout frequency nor increased the overall frequency of food reinforcement. A discrete-trial procedure was used in which a response during the first 5 s of a trial postponed an otherwise immediate 60-s timeout to a later part of that same trial but had no effect on whether the timeout occurred. During time-in periods, responses on a second key produced food according to a random-interval 20-s schedule. In Experiment 1, the response-timeout interval was 45 s under postponement conditions and 0 s under extinction conditions (responses were ineffective in postponing timeouts). The percentage of trials with a response was consistently high when the timeout-postponement contingency was in effect and decreased to low levels when it was discontinued under extinction conditions. In Experiment 2, the response-timeout interval was also 45 s but postponement responses increased the duration of the timeout, which varied from 60 s to 105 s across conditions. Postponement responding was maintained, generally at high levels, at all timeout durations, despite sometimes large decreases in the overall frequency of food reinforcement. In Experiment 3, timeout duration was held constant at 60 s while the response-timeout interval was varied systematically across conditions from 0 s to 45 s. Postponement responding was maintained under all conditions in which the response-timeout interval exceeded 0 s (the timeout interval in the absence of a response). In some conditions of Experiment 3, which were designed to control for the immediacy of food reinforcement and food-correlated (time-in) stimuli, responding postponed timeout but the timeout was delayed whether a response occurred or not. Responding was maintained for 2 of 3 subjects, suggesting that behavior was negatively reinforced by timeout postponement rather than positively reinforced by the more immediate presentation of food or food-correlated (time-in) stimuli.     

Self-imposed timeouts under increasing response requirements

Self-imposed timeouts by pigeons working under a progressive-ratio food schedule were studied under different conditions. The main findings were (1) continued production of timeouts over an extended series of sessions, (2) more frequent responding on the key with the timeout consequence than on a key having no consequence, (3) an inverse relationship between number of timeouts and level of body weight, (4) production of timeouts when the timeout duration was brief, lengthy, or controlled by the pigeon, and (5) dependence of self-imposed timeouts on variables controlling responding under the progressive-ratio schedule. Under all experimental conditions, with the exception of performances at the high body weight, timeouts were more frequent during the longer progressive-ratio steps and usually were localized in the post-reinforcement pause or the early part of the step. The timeout behavior could be interpreted as either an escape from aversive stimuli generated by the progressive-ratio schedule or as a response reinforced by the consequent stimulus change.     

Punishment of observing by the negative discriminative stimulus

To determine the effect of a negative discriminative stimulus on the response producing it, two pigeons were each studied in a three-key conditioning chamber. During alternating periods of unpredictable duration, pecking the center (food) key either was reinforced with grain on a variable-interval schedule or was never reinforced. On equal but independent variable-interval schedules, pecking either of the side (observing) keys changed the color of all keys for 30 sec from yellow to either green or red. When the schedule on the center key was variable-interval reinforcement, the color was green (positive discriminative stimulus); when no reinforcements were scheduled, the color was red (negative discriminative stimulus). Since pecking the side keys did not affect grain deliveries, changes in the rate of pecking could not be ascribed to changes in the frequency of primary reinforcement. In subsequent sessions, red was withheld as one of the possible consequences of pecking a given side key. When red was omitted, the rate on that key increased, and when red was restored, the rate decreased. It was concluded that red illumination of the keys, the negative discriminative stimulus, had a suppressive effect on the response that produced it.     

Effects of d-amphetamine and pentobarbital under concurrent fixed-ratio schedules.

Pigeons were studied under a two-key concurrent fixed-ratio schedule of food presentation. During the first five sessions, the fixed-ratio requirements were 30 responses on one key (major key) and 120 responses on the other key (minor key): responding occurred almost exclusively on the major key. When the fixed-ratio requirements were then made equal at 30 responses on both keys, responding continued to predominate on the major key. The asymmetric distribution of responses persisted when the concurrent fixed-ratio fixed-ratio schedule was interrupted with periods during which the major key was associated with extinction while the other key remained associated with a fixed-ratio schedule. Additionally, in some subjects the fixed-ratio requirements were increased. These schedule modifications decreased the asymmetry in responding but did not eliminate it. d-Amphetamine decreased rates on both keys and slightly increased the asymmetric distribution of responses, while pentobarbital reversed the distribution of responses by increasing low rates and decreasing high rates. The pigeons maintained their original asymmetric distribution of responses during the 1 1/2-year-long study, despite schedule alterations and drug administrations.     

Control of responding by stimulus duration

Pigeons were trained on a procedure in which the key was white for 30 sec, alternating with periods of darkness, or timeout. In a nondifferential training procedure, timeout duration was held constant at either 9 or 21 sec for different animals, and pecks on the white key were reinforced on a variable-interval 36-sec schedule. After 30 sessions an extinction generalization test was conducted where the duration of the timeout was varied from 3 to 27 sec. This test showed no differences in responding following timeouts of different durations. In a differential training procedure, timeout durations of either 9 or 21 sec were randomly scheduled for each animal. The variable-internal schedule was in effect following the same timeout duration as in the prior nondifferential procedure. No pecks were reinforced after the other timeout duration. In 40 sessions, differences in response rates following the two durations gradually developed. A maintained generalization procedure was then imposed in which timeout durations were varied from 3 to 27 sec, with the variable-interval schedule in effect following only the same duration as in the previous procedures. The first maintained generalization session showed that the prior differential training had established control of the animals' behavior by the timeout duration. In continued training on the maintained generalization procedure, control by the timeout duration decreased.     

Drug discrimination under a concurrent fixed-interval fixed-interval schedule.

Pigeons were trained to discriminate 5.0 mg/kg pentobarbital from saline under a concurrent fixed-interval (FI) FI schedule of food presentation on which, after pentobarbital administration, responses on one key were reinforced with food under an FI 60-s component and responses on the other key were reinforced under an FI 240-s component. After saline administration, the schedule contingencies on the two keys were reversed. After both pentobarbital and saline, pigeons responded more frequently on the key on which responses had been programmed to produce the reinforcer under the FI 60 component of the concurrent schedule. The schedule was changed to concurrent FI 150 FI 150 s for drug-substitution tests. In each bird, increasing doses of pentobarbital, ethanol, and chlordiazepoxide produced increases in the proportion of responses on the key on which responses had been reinforced under the FI 60 component after pentobarbital administration during training sessions. The proportion of responses on that key was slightly lower for ethanol than for chlordiazepoxide and pentobarbital. At a dose of pentobarbital higher than the training dose, responding decreased on the key that had been reinforced under the FI 60 component during training sessions. Phencyclidine produced less responding on the key programmed under the FI 60-s component than did pentobarbital. Methamphetamine produced responding primarily on the key on which responses had been reinforced under the FI 60-s component after saline administration.     

Delayed reinforcement of fixed-ratio performance without mediating exteroceptive conditioned reinforcement

The performance of pigeons was studied under conditions in which the completion of a fixed-ratio requirement was not contiguous with the presentation of a reinforcer. Timein and timeout periods alternated throughout the experimental sessions. Responses made by an experimental bird during the timein period were accumulated, and when a fixed-ratio requirement had been met, grain was presented to the experimental bird and a yoked control following their first response in the next timein period. Across most manipulations of the fixed-ratio requirement and of the duration of the timeout period, the response rates of the experimental birds were considerably higher than those of their controls, suggesting that the response-reinforcer dependency controlled the behavior of the experimental bird in the absence of a close temporal association between responding on the ratio schedule and reinforcer presentations.     

The role of the response-reinforcer relation in delay-of-reinforcement effects

The role of the response-reinforcer relation in maintaining operant behavior under conditions of delayed reinforcement was investigated by using a two-operandum (i.e., two-key) procedure with pigeons. Responding on one key was reinforced under a tandem variable-interval differential-reinforcement-of-other-behavior (tandem VI DRO) schedule. The schedule defined a resetting unsignaled delay-of-reinforcement procedure in that a response was required when the interfood interval of the VI schedule lapsed, but further responding during the DRO component on either key reset the time interval. This ensured a fixed delay duration between any response and reinforcement. Responding on another key, physically identical to the first one except for spatial location, otherwise was without consequence. The location of the key correlated with the delay-of-reinforcement procedure varied between sessions according to a semirandom sequence. Differences in response rates between the two keys were greater, with proportionally higher rates on the key correlated with the delay-of-reinforcement procedure, the longer the delay-of-reinforcement procedure remained correlated with the same key. Differences in responding on the two keys also increased within individual sessions. These results suggest that the response-reinforcer relation is the primary determinant of responding when responding is acquired and maintained with delayed reinforcement.     

A ROLE FOR NEGATIVE REINFORCEMENT OF RESPONSE OMISSION IN PUNISHMENT?

This experiment attempted to disentangle response-rate reductions controlled by the direct suppressive effects of a punisher from those due to negative reinforcement of response omission. Key-peck responding of pigeons was maintained by a conjoint variable-interval 3-min schedule of food presentation variable-interval 30-s schedule of response-dependent electric shock presentation. Omission of responses for 5, 10, or 30 s resulted in the possibility of canceling a scheduled shock. Response rates were a function of required pause duration, with lower rates occurring when longer periods of response omission were required for shock cancellation. These results show that, with several parameters of punishment held constant, response rates were controlled by the negative reinforcement contingency. Such a finding argues for renewed consideration of the role of negative reinforcement in punishment contingencies.     

Food-paired stimuli as conditioned reinforcers: effects of d-amphetamine.

Seven pigeons were studied in two experiments in which key pecks were reinforced under a second-order schedule wherein satisfaction of variable-interval schedule requirements produced food or a brief stimulus. In the second part of each session, responses produced only the brief stimulus according to a variable-interval schedule (food extinction). For the 4 pigeons in Experiment 1, the response key was red throughout the session. In separate phases, the brief stimulus was either paired with food, not paired with food, or not presented during extinction. d-Amphetamine (0.3 to 10.0 mg/kg) dose-dependently reduced food-maintained responding during the first part of the session and, at intermediate dosages, increased responding during the extinction portion of the session. The magnitude of these increases, however, did not consistently depend on whether the brief stimulus was paired, not paired, or not presented. It was also true that under nondrug conditions, response rates during extinction did not differ reliably depending on pairing operations for the brief stimulus. In Experiment 2, 3 different pigeons responded under a procedure wherein the key was red in the component with food presentations and blue in the extinction component (i.e., multiple schedule). Again, d-amphetamine produced dose-related decreases in responding during the first part of a session and increases in responding in the second part of the session. These increases, however, were related to the pairing operations; larger increases were observed when the brief stimulus was paired with food than when it was not or when it was not presented at all. Under nondrug conditions, the paired brief stimulus controlled higher response rates during extinction than did a nonpaired stimulus or no stimulus. These findings suggest that d-amphetamine can enhance the efficacy of conditioned reinforcers, and that this effect may be more robust if conditioned reinforcers occur in the context of a signaled period of extinction.     

Response-produced timeouts under a progressive-ratio schedule with a punished reset option

Three behavioral options were available to food-deprived pigeons: (1) pecking one key resulted in food reinforcement according to a 50-response progressive-ratio schedule, (2) pecking a second key reset the progressive-ratio schedule to the initial progressive-ratio step, and (3) pecking a third key produced a 3-min timeout period. Pecks on the reset key were shocked. Under low and intermediate shock intensities, timeouts were not produced; under high shock levels, timeouts were produced regularly. Timeouts occurred during the initial period of a progressive-ratio step and were more frequent during the longer steps of the progressive-ratio schedule. Response-produced timeouts under these experimental conditions could be interpreted either as an escape from aversive behavioral options or as a low-probability behavior emerging when the food reinforcement schedule exerted weaker control.     

Reinforcer effectiveness as a function of reinforcer rate and magnitude: a comparison of concurrent performances

Pigeons' pecks on each of two concurrently available response keys were reinforced under a variable-interval schedule that sometimes allotted food-pellet deliveries to one key and sometimes to the other. The keys differed in the number of reinforcements assigned to each and in the number of pellets delivered during each reinforcement. When the total quantity of food associated with each key during a session was constant, the proportion of responses to a key depended on the particular combinations of reinforcer rate and reinforcer magnitude scheduled on each key. A given quantity of food generated more responding on a key when it was delivered frequently in small amounts than when it was delivered infrequently in large amounts.     

Drug effects on responding maintained by stimulus-reinforcer and response-reinforcer contingencies.

The effects of pentobarbital and d-amphetamine were assessed on key pecking by pigeons under conventional single-key multiple schedules and under two-key multiple schedules in which discriminative stimuli appeared on one key (stimulus key) while pecks on a second key (constant key) produced food. Pecks on the stimulus key had no scheduled consequences. A 60-second variable-interval schedule operated in one component of each multiple schedule: either extinction or a 60-second variable-time schedule operated in the alternate component. When the alternate-component schedule was extinction, a high rate of responding was maintained in the variable-interval component of the single-key schedule; responding on both keys was maintained in the variable-interval component of the two-key schedule. Pentobarbital increased responding in the variable-interval component of the single-key schedule and increased stimulus-key, but not constant-key responding in that component of the two-key schedule. When the alternate-component schedule was changed to variable time, responding declined in the variable-interval component of the single-key schedule; stimulus-key responding was no longer maintained under the two-key schedule. Pentobarbital decreased responding in the variable-interval component of both schedules. With an exception, d-amphetamine only decreased responding in the variable-interval component of the single- and two-key schedules both when the alternate-component schedule was extinction and when it was variable time. The results suggest that the effects of pentobarbital, but not d-amphetamine, depend on the nature of the contingency (stimulus-reinforcer, response-reinforcer) that maintains responding.     

A test of the negative discriminative stimulus as a reinforcer of observing

Five pigeons were used to test the hypothesis that the source of reinforcement for observing behavior is the information that it provides concerning the schedule of primary reinforcement. On a variable-interval schedule, pecking the left-hand key produced a 30-sec display of such information. During this 30-sec period, when pecking the right-hand key was reinforced on a random-interval schedule, both keys were green; when no reinforcement was scheduled (extinction) both keys were red. Later, this baseline procedure, in which both red and green were available, was replaced for blocks of sessions by procedures in which either (a) the red was eliminated and only the green could be produced; or (b) the green was eliminated and only the red could be produced. The results were that green maintained rates of pecking on the left key that were as high or higher than when both colors were available and that red maintained no responding. It was concluded that the reinforcing value of a stimulus depends on the positive or negative direction of its correlation with primary reinforcement, rather than upon the amount of information that it conveys.     

Spatiotemporal patterns of behavior produced by variable-interval schedules of reinforcement

The spatiotemporal patterns of behavior exhibited by two pigeons during a variable-interval 15-second schedule of food reinforcement, a variable-interval 5-minute schedule, and then extinction of key pecking were recorded using an apparatus that continuously tracked the position of the bird in the experimental chamber. The variable-interval 15-second schedule produced a close-to-key pattern between reinforcements with two types of regular excursions from the region of the key frequently occurring after reinforcement. Subsequent exposure to the variable-interval 5-minute schedule produced more extended and extremely regular patterns between responses. Reinstatement of the variable-interval 15-second schedule reestablished the close-to-key pattern with regular excursions frequently occurring after reinforcement. During extinction the spatiotemporal patterns that had developed during the variable-interval 5-minute schedule reappeared and gradually dissipated. These patterns may have been a form of superstitious behavior.     

Escape from rich‐to‐lean transitions: Stimulus change and timeout

Extended pausing during discriminable transitions from rich‐to‐lean conditions can be viewed as escape (i.e., rich‐to‐lean transitions function aversively). In the current experiments, pigeons’ key pecking was maintained by a multiple fixed‐ratio fixed‐ratio schedule of rich or lean reinforcers. Pigeons then were provided with another, explicit, mechanism of escape by changing the stimulus from the transition‐specific stimulus used in the multiple schedule to a mixed‐schedule stimulus (Experiment 1) or by producing a period of timeout in which the stimulus was turned off and the schedule was suspended (Experiment 2). Overall, escape was under joint control of past and upcoming reinforcer magnitudes, such that responses on the escape key were most likely during rich‐to‐lean transitions, and second‐most likely during lean‐to‐lean transitions. Even though pigeons pecked the escape key, they paused before doing so, and the latency to begin the fixed ratio (i.e., the pause) remained extended during rich‐to‐lean transitions. These findings suggest that although the stimulus associated with rich‐to‐lean transitions functioned aversively, pausing is more than simply escape responding from the stimulus.     

Second-order schedules: discrimination of components

Pigeons were exposed to a series of second-order schedules in which the completion of a fixed number of fixed-interval components produced food. In Experiment 1, brief (2 sec) stimulus presentations occurred as each fixed-interval component was completed. During the brief-stimulus presentation terminating the last fixed-interval component, a response was required on a second key, the brief-stimulus key, to produce food. Responses on the brief-stimulus key before the last brief-stimulus presentation had no scheduled consequences, but served as a measure of the extent to which the final component was discriminated from preceding components. Whether there were one, two, four, or eight fixed-interval components, responses on the brief-stimulus key occurred during virtually every brief-stimulus presentation. In Experiment 2, an attempt was made to punish unnecessary responses on the brief-stimulus key, i.e., responses on the brief-stimulus key that occurred before the last component. None of the pigeons learned to withhold these responses, even though they produced a 15-sec timeout and loss of primary reinforcement. In Experiment 3, different key colors were associated with each component of a second-order schedule (a chain schedule). In contrast to Experiment 1, brief-stimulus key responses were confined to the last component. It was concluded that pigeons do not discriminate well between components of second-order schedules unless a unique exteroceptive cue is provided for each component. The relative discriminability of the components may account for the observed differences in initial-component response rates between comparable brief-stimulus, tandem, and chain schedules.     

Short-component multiple schedules: effects of relative reinforcement duration

Pigeons were exposed to multiple variable-interval 2-min variable-interval 2-min schedules of food presentation in which relative duration of food presentation was manipulated. When components alternated every 5 sec and were scheduled on separate response keys, relative response rates closely matched relative reinforcement duration in three of four pigeons. On the other hand, relative response rates were insensitive to relative reinforcement duration when components scheduled on a single response key alternated every 5 sec, and when components scheduled on separate response keys alternated every 2 min. Thus, both rapid alternation and spatial separation of components were necessary to produce approximate matching of relative responding to relative reinforcement duration. This finding contrasts with previous findings that only rapid component alternation is necessary for matching when relative rate of reinforcement is manipulated.     

Concurrent performances: rate constancies without changeover delays

Pigeons' pecks on two keys were maintained, without changeover delays, by independent variable-interval schedules of food reinforcement. Four regularly cycling 2-min components scheduled reinforcement respectively for both keys, left key only, both keys, and right key only. Initially, reinforcement scheduled for one key alone produced more responding on that key than reinforcement scheduled concurrently for both keys. Continued sessions reduced this difference; response rate on a given key approached constancy, or invariance with respect to the performance on and schedule for the other key. When extinction replaced the reinforcement schedule on either key, responding on that key decreased more during components that scheduled reinforcement for the other key than during those that did not. This demonstration that responses on one key were not supported by reinforcers on the other key suggested that the alternation of concurrent responding and either-key-alone responding prevented concurrent superstitions from developing.