Responding maintained under fixed-interval and fixed-time schedules of electric shock presentation

Following initial histories under a schedule of electric shock postponement, lever pressing in squirrel monkeys was maintained under fixed-interval and fixed-time schedules of electric shock presentation. No difference in either rate or pattern of responding was obtained when these schedules were presented as components of a multiple schedule. When they were presented singly for long periods of time, the fixed-interval schedule consistently maintained a higher response rate than the fixed-time schedule. The pattern of responding under both schedules was similar, typically consisting of a pause at the beginning of each interval followed by either a steady or a positively accelerating rate of responding. The results suggest that the response-shock dependency is of critical importance in the maintenance of high rates of responding under schedules of electric shock presentation, and support the general view that such responding may be conceptualized as operant behavior under control of many of the same variables that control responding under comparable schedules of food or water reinforcement.     

Reponse-reinforcer independence and conventional extinction after fixed-interval and variable-interval schedules

After training three albino rats to bar press during a multiple fixed-interval variable-interval schedule, the response-reinforcer dependency was simultaneously removed from both components, converting the schedule to multiple fixed-time variable-time. Response rates were reduced in both components under these conditions but the fixed-time schedule maintained relatively higher response rates with each rat. After reinstating the response-reinforcer dependency in both components, responding was conventionally extinguished by rendering the pellet dispenser inoperative. Responding rapidly decreased to near-zero levels. Differences in fixed and variable-time schedules in sustaining behavior are discussed in terms of differences in response rates at the time of reinforcer delivery. Similarities and differences between conventional extinction and schedules delivering response-independent reinforcers are also discussed.     

Responding under fixed-ratio and multiple fixed-interval fixed-ratio schedules of electric shock presentation

Squirrel monkeys, initially trained under a schedule of electric shock postponement and then under fixed-interval schedules of electric shock presentation, were studied under multiple fixed-interval fixed-ratio and under fixed-ratio schedules of shock presentation. Under the fixed-interval (10-min) component of the multiple schedule, a pause was followed by a gradual increase in responding to a rate maintained until shock presentation; under the fixed-ratio (3-, 10-, or 30-response) component of the multiple schedule, a brief pause was typically followed by a relatively high and uniform rate of responding until shock was presented. When the 60-sec timeout periods, which usually followed shock presentation, were eliminated from the multiple schedule for one monkey, responding was only transiently affected. In the one monkey studied, responding was maintained under a fixed-ratio schedule alone (with timeout periods), but rates of responding were lower than under the fixed-ratio component of the multiple schedule. Characteristic patterns of responding, similar to those engendered under schedules of food presentation or shock termination, can be maintained under fixed-ratio schedules of shock presentation; further, patterns of responding can be controlled by discriminative stimuli in multiple schedules.     

The effects of terminal-link fixed-interval and variable-interval schedules on responding under concurrent chained schedules

Previous work using variable-interval schedules in the terminal links of concurrent chained schedules suggested that relative choice proportion in the initial links equalled relative rate of reinforcement in the terminal links. With fixed-interval terminal-link schedules, however, matching was not obtained. The present study held pairs of fixed-interval terminal-link schedules in a constant ratio but varied absolute sizes. Relative choice for the smaller terminal-link fixed-interval schedule was a negatively accelerated, increasing function of absolute size of the fixed-interval pairs. Matching was found only with the fixed-interval pair of 5 and 10 sec. When pairs of variable-interval schedules were arranged so that the harmonic mean of the intervals equalled the fixed-interval parameter values, relative choice functions were like those for fixed-interval schedules.     

Fixed-interval schedules of electric shock presentation: extinction and recovery of performance under different shock intensities and fixed-interval durations

In squirrel monkeys responding under a schedule in which responding postponed the delivery of electric shock, the presentation of response-dependent shock under a fixed-interval (FI) schedule increased the rate of responding. When the schedule of shock-postponement was eliminated, so that the only shocks delivered were those produced by responses under the FI schedule, a pattern of positively accelerated responding developed and was maintained over an extended period. When responses did not produce shocks (extinction), responding decreased. When shocks were again presented under the FI schedule, the previous pattern of responding quickly redeveloped. In general, response rates were directly related to the intensity of the shock presented, and inversely related to the duration of the fixed-interval. These results raise fundamental questions about the traditional classification of stimuli as reinforcers or punishers. The basic similarities among FI schedules of food presentation, shock termination, and shock presentation strengthen the conclusion that the schedule under which an event is presented and the characteristics of the behavior at the time the event is presented, are of overriding importance in determining the effect of that event on behavior.     

Synthetic variable-interval schedules of reinforcement

Three pigeons pecked for food on a synthetic variable-interval schedule of reinforcement that had two independent parts: a variable-interval schedule that arranged a distribution of interreinforcement intervals, and a device that randomly assigned each reinforcement to one of 10 classes of interresponse times. The frequencies of reinforcement for the 10 classes of interresponse times were systematically varied, while the overall frequency of reinforcement was held within a comparatively narrow range. The 10 classes extended either from 0.1 to 0.6 sec in 0.05-sec intervals, or from 1.0 to 6.0 sec in 0.5-sec intervals. In the former case, some control by reinforcement was obtained, but it was weak and no simple relationships were discernible. In the latter case, the relative frequency of an interresponse time was a generally increasing function of its relative frequency of reinforcement, and two simple controlling relationships were found. First, the function relating interresponse times per opportunity to reinforcements per opportunity was, over a restricted range, approximately linear with a slope of unity. Second, when all 10 classes of interresponse times were reinforced equally often, the relative frequency of an interresponse time approximately equalled the relative reciprocal of its length.     

Two-key concurrent paced variable-interval paced variable-interval schedules of reinforcement

Nine pigeons were used in two experiments in which a response was reinforced if a variable-interval schedule had assigned a reinforcement and if the response terminated an interresponse time within a certain interval, or class, of interresponse times. One such class was scheduled on one key, and a second class was scheduled on a second key. The procedure was, therefore, a two-key concurrent paced variable-interval paced variable-interval schedule. In Exp. I, the lengths of the two reinforced interresponse times were varied. The relative frequency of responding on a key approximately equalled the relative reciprocal of the length of the interresponse time reinforced on that key. In Exp. II, the relative frequency and relative magnitude of reinforcement were varied. The relative frequency of responding on the key for which the shorter interresponse time was reinforced was a monotonically increasing, negatively accelerated function of the relative frequency of reinforcement on that key. The relative frequency of responding depended on the relative magnitude of reinforcement in approximately the same way as it depended on the relative frequency of reinforcement. The relative frequency of responding on the key for which the shorter interresponse time was reinforced depended on the lengths of the two reinforced interresponse times and on the relative frequency and relative magnitude of reinforcement in the same way as the relative frequency of the shorter interresponse time depended on these variables in previous one-key concurrent schedules of reinforcement for two interresponse times.     

Performance of humans in variable-interval avoidance schedules programmed singly, and concurrently with variable-interval schedules of positive reinforcement

Performance maintained under single variable-interval avoidance schedules, single variable-interval schedules of positive reinforcement, and concurrent schedules consisting of a variable-interval avoidance component and a variable-interval positive reinforcement component, was studied in three human subjects, using points exchangeable for money as the reinforcer. Response rate in the single variable-interval avoidance schedules was an increasing function of the frequency of monetary loss avoidance. Response rate in the single variable-interval positive reinforcement schedules was an increasing function of the frequency of obtained monetary reinforcement. In the concurrent avoidance/reinforcement schedules, the rate of responding in the avoidance component increased, and the rate of responding in the positive reinforcement schedule decreased (with one exception) as a function of the frequency of loss avoidance in the avoidance component. The logarithms of the ratios of the response rates in the two components, and the logarithms of the ratios of the times spent in the two components, were linearly related to the logarithms of the ratios of the frequency of loss avoidance in the avoidance component to the frequency of reinforcement in the positive reinforcement component. All three subjects exhibited marked undermatching of response rate ratios to reinforcement frequency ratios. The results are discussed in the context of Herrnstein's quantitative model of operant performance.     

Preference for less frequent shock under fixed-interval schedules of electric-shock presentation.

Lever pressing by 2 squirrel monkeys was maintained under fixed-interval 6-min and fixed-interval 2-min schedules of electric-shock presentation. Preference for these schedules was assessed during three experimental phases. In all phases, responses on one lever produced shock according to one or the other fixed-interval schedule, and responses on a second, changeover, lever switched between schedules. The opportunity to change over was presented during separate choice periods (during which the fixed-interval schedules did not operate) that followed the first through fourth shocks in each schedule. If no changeover occurred during those choice periods, a changeover automatically occurred following the fifth shock. In Phase I, durations of the choice periods were fixed. In Phase II, the choice periods equaled a proportion of their respective fixed interval. During Phase III (completed with 1 monkey) a response on the changeover lever during a given choice period reinstated the most recent fixed interval, and a failure to respond resulted in a changeover. During each of these phases, distinct preferences developed for the 6-min schedule. These results suggest that the maintenance of lever pressing by fixed-interval presentation of electric shock may not be an example of positive reinforcement, and that the response-maintaining characteristics of shock presentation may derive from other properties of the schedule.     

Choice between fixed-interval schedules: Graded versus step-like choice functions

Pigeons chose between two fixed-interval schedules of food reinforcement. A single peck on one of two lighted keys started the fixed-interval schedule correlated with that key. The schedule had to be completed before the next choice opportunity. The durations of the fixed intervals were varied over conditions from 15 s to 40 s. To maximize the rate of reinforcement, the pigeons had to choose exclusively the shorter of the two schedules. Nevertheless, choice was not all-or-none. Instead, relative choice, and the rates of producing the fixed intervals, varied in a graded fashion with the disparity between the two schedules. Choice ratios under this procedure (single response to choose) were highly sensitive to the ratios of the fixed-interval schedules.     

Preference for fixed-interval schedules of reinforcement

Pigeons were trained on a two-link concurrent chain schedule in which responding on either of two keys in the initial link occasionally produced a terminal link, signaled by a change in the color of that key and a darkening of the other. Further responding on the lighted key was reinforced with food according to a fixed-interval schedule. For one of the keys, this fixed interval was always 20 sec, while for the other it was held at values of 5, 14, 30, or 60 sec for several weeks. In the initial link, all pigeons responded relatively more often on the key with the shorter fixed interval than was predicted by the matching hypothesis. Responding in the initial link showed a large negative recency effect: pigeons responded less frequently on the key that provided their last reinforcement than predicted from the overall response rates.     

Alternative fixed-ratio fixed-interval schedules of reinforcement

Five rats were trained under alternative fixed-ratio fixed-interval schedules, in which food reinforcement was provided for the completion of either a fixed-ratio or a fixed-interval requirement, whichever was met first. Overall response rate and running rate (the rate of responding after the postreinforcement pause) decreased for all subjects as the fixed-ratio value increased. As the proportion of reinforcements obtained from the fixed-ratio component increased and the alternative schedule approached a simple fixed ratio, overall response rate and running rate both increased; conversely, as the proportion of reinforcements obtained from the fixed-interval component increased and the alternative schedule approached a simple fixed interval, response rates decreased. Postreinforcement pause length increased linearly as the average time between reinforcements increased, regardless of the schedule parameters. A break-run pattern of responding was predominant at low- and medium-valued fixed ratios. All subjects displayed at least occasional positively accelerated responding within interreinforcement intervals at higher fixed-ratio values.     

Rates and patterns of responding with concurrent fixed-interval and variable-interval reinforcement

Pigeons were exposed to concurrent fixed-interval and variable-interval schedules of food reinforcement on two keys. The times between reinforcement were varied systematically on both keys. The overall relative frequency of responding on the fixed-interval key depended on the relative frequency of reinforcement, but did not match it. Instead, the ratio of responses on the fixed-interval key to responses on the variable-interval key was a power function of the ratio of reinforcements, with an exponent of 0.5. Patterns of responding between reinforcements on the fixed-interval key depended on both relative and absolute values of the reinforcement schedules. Similar overall relative responding was obtained at different absolute schedule values with equal relative reinforcement, despite some differences in patterns of responding.     

Chained and tandem fixed-interval schedules of punishment

Pigeons' key pecks were both punished with electric shock on four-component chained and tandem fixed-interval schedules and reinforced on a variable-interval schedule of food presentation. Pecking was suppressed less in the early components of the chained schedule than in the early components of the tandem schedule. Related multiple and mixed schedules of punishment were also presented; these schedules were identical to the chained and tandem schedules, respectively, except that components changed independently of responding. Similar effects were obtained, in that responding was suppressed in all components of the mixed schedule and only in the fourth component of the multiple schedule of punishment. The performances maintained on the chained and tandem schedules of punishment were generally symmetrical to those found in analogous chained and tandem schedules of food reinforcement.     

Responding under sequence schedules of electric shock presentation

Lever pressing by squirrel monkeys was examined under second-order schedules of electric shock presentation in which different discriminative stimuli were associated with consecutive components (sequence schedules). Components were always two-minute fixed-interval schedules, and three different overall schedules were studied. Under an overall eight-minute fixed-interval schedule, the first component completion after at least eight minutes had elapsed produced electric shock. The number of components actually completed ranged from one to four; thus, different discriminative stimuli were occasionally associated with electric shock presentation. Under an overall “yoked” variable-ratio schedule, electric shock was presented after completion of a variable number of components; the required number and the distribution of components were matched to those obtained under the overall eight-minute fixed-interval schedule. Under an overall fixed-ratio schedule, electric shock was presented after completion of four components (chained schedule). Under all three sequence schedules, responding in early components was characterized by a pause followed by a single response after the end of the two-minute interval; responding in later components was characterized by a shorter pause followed by positively accelerated responding. Manipulation of overall schedules of shock presentation in these complex behavioral situations produced changes in responding comparable to those ordinarily obtained after similar manipulation of dependencies under both single and second-order schedules of food presentation. These experiments extend the range of conditions and levels of complexity under which responding can be maintained by presentation of electric shock.     

Unsignalled delay of reinforcement in variable-interval schedules

Three pigeons responded on several tandem variable-interval fixed-time schedules in which the value of the fixed-time component was varied to assess the effects of different unsignalled delays of reinforcement. Actual (obtained) delays between the last key peck in an interval and reinforcement were consistently shorter than the nominal (programmed) delay. When nominal delays were relatively short, response rates were higher during the delay condition than during the corresponding nondelay condition. At longer nominal delay intervals, response rates decreased monotonically with increasing delays. The results were consistent with those obtained from delay-of-reinforcement procedures that impose either a stimulus change (signal) or a no-response requirement during the delay interval.     

Responding under schedules combining response-dependent and response-independent shock delivery

Lever pressing of three squirrel monkeys with experience under continuous avoidance schedules was maintained by response-produced shock under a 5-minute variable-interval schedule. Responding decreased when half of the scheduled shocks were delivered independently of lever pressing and decreased further when all shocks were independent of lever pressing. Responding was lowest when all shocks were eliminated. When the proportion of response-dependent shocks increased, responding increased. This relation occurred even though the frequency and temporal distribution of shock delivery remained the same. Responding of two monkeys increased in a graded fashion as the frequency of shock was increased by arranging variable-time 5-minute, 2-minute, and 1-minute schedules jointly with the variable-interval 5-minute schedule. Thus, increasing the proportion of response-independent shocks decreased responding when the overall frequency of shocks stayed the same, but increased responding when the overall frequency of shock delivery increased.     

Summation of responding maintained by fixed-interval schedules

A light and tone were separately correlated with responding maintained by fixed-interval schedules, in which the level of responding varied continuously throughout the duration of the interval. Responding during the presence of the single stimuli and their compound was compared during the successive segments of the interval. The following results were obtained: (1) more responses were emitted during compounding than were emitted during either stimulus alone in all segments of the interval; (2) increases in the number of responses across the interval during compounding paralleled increases during single-stimulus presentations. The sum of the responses emitted during the single stimuli was similar to the number of responses emitted during compounding, suggesting that the response tendencies correlated with the single stimuli combined in a summative or additive fashion.     

Punished and unpunished responding in multiple variable-interval schedules

The performance of rats trained on multiple variable-interval schedules was examined before, during, and after punishment. The same linear function related relative response rates to relative density of reinforcement both in the presence and absence of punishment. Equal relative suppression was seen in both the high and low reinforcement density components. The intercept value of the function was zero. Each component of the schedule was programmed on a separate lever: thus during any component, there was an opportunity for responses on the nonoperative lever (errors). The proportions of these errors declined to a near-zero value during punishment and did not regain their prepunishment values after punishment was removed, suggesting that some discrimination learning occurred during punishment. Recovery of response rate during punishment was seen only where a greater-than-zero probability of reinforcement was associated with the response.     

Strict and random alternation in concurrent variable-interval schedules

Six pigeons responded on pairs of concurrent variable-interval schedules with, in different parts, four different arrangements of alternation between schedules. Following a single switching-key response, alternation was either strict or random, and the alternative presented after a switch (the postswitch alternative) was either signaled by the location of the switching key or unsignaled. Generalized-matching analyses showed little difference in behavior among the different alternation arrangements, except the usual finding of lower sensitivity of response allocation than time allocation was eliminated by arranging random alternation. Patterns of interchangeover times were similar for all arrangements except signaled random alternation. Differences in behavior preceding the different postswitch alternatives were found in the signaled random alternation procedure. Preference was biased towards the color of the signaled postswitch alternative and showed increased sensitivity when the postswitch alternative was to be the one with the higher reinforcer rate. Interchangeover times were substantially shorter when the postswitch alternative was signaled to be different from the current alternative than when it was signaled to be the same. However, when separate reinforcer ratios were calculated for the different postswitch alternatives, those effects were eliminated or greatly reduced. We suggest that, although behavior is indeed influenced by the postswitch alternative, the mechanism is indirect. That is, the distributions of reinforcers between alternatives obtained before each postswitch alternative differ when those alternatives are signaled, and those distributions are discriminated, but the same relations between choice and relative reinforcement hold irrespective of which postswitch alternative is signaled.