Characteristics and response-displacement effects of shock-generated responding during negative reinforcement procedures: pre-shock responding and post-shock aggressive responding

Bar-pressing (Experiment I) or key-pressing (Experiments II and III) responses of monkeys were reinforced according to a fixed-interval schedule of negative reinforcement: the first response after a fixed interval of time terminated regularly spaced shocks for a fixed time designated as the reinforcement period. During extinction, shocks continued during the reinforcement period. That there were two types of responding generated by shock alone was indicated by (1) the level of responding maintained during extinction relative to conditions without shock, (2) the stability of two between-shock response patterns across reinforcement and extinction conditions, and (3) the development of these two between-shock patterns without a history of reinforcement. Subjects developed either a pre-shock or a post-shock response pattern when only the bar was available. However, when both a bite tube, an operandum requiring an aggressive topography, and a recessed key, an operandum that did not require an aggressive topography, were provided, the post-shock pattern was observed in tube biting and the pre-shock pattern was observed in key pressing. Removal of the bite tube produced post-shock key responding similar to that observed when only the bar was available. The displacement of post-shock, aggression-motivated responding confirmed the confounding effect of shock-generated responding in negative reinforcement procedures, and suggests that the use of concurrent response alternatives would reduce such confounding.     

The effects of reinforcement frequency and response requirements on the maintenance of behavior.

Six rats responded under fixed-interval and tandem fixed-interval fixed-ratio schedules of food reinforcement. Basic fixed-interval schedules alternated over experimental conditions with tandem fixed-interval fixed-ratio schedules with the same fixed-interval value. Fixed-interval length was varied within subjects over pairs of experimental conditions; the ratio requirement of the tandem schedules was varied across subjects. For both subjects with a ratio requirement of 10, overall response rates and running response rates typically were higher under the tandem schedules than under the corresponding basic fixed-interval schedules. For all subjects with ratio requirements of 30 or 60, overall response rates and running response rates were higher under the tandem schedules than under the corresponding basic fixed-interval schedules only with relatively short fixed intervals. At longer fixed intervals, higher overall response rates and running rates were maintained by the basic fixed-interval schedules than by the tandem schedules. These findings support Zeiler and Buchman's (1979) reinforcement-theory account of response strength as an increasing monotonic function of both the response requirement and reinforcement frequency. Small response requirements added in tandem to fixed-interval schedules have little effect on reinforcement frequency and so their net effect is to enhance responding. Larger response requirements reduce reinforcement frequency more substantially; therefore their net effect depends on the length of the fixed interval, which limits overall reinforcement frequency. At the longest fixed intervals studied in the present experiment, reinforcement frequency under the tandem schedules was sufficiently low that responding weakened or ceased altogether.     

Effects of pre-operant running and sucrose concentration on operant wheel running on a fixed interval schedule of reinforcement

Prior research proposed that temporal control over the pattern of operant wheel running on a fixed interval (FI) schedule of sucrose reinforcement is a function of automatic reinforcement generated by wheel running and the experimentally arranged sucrose reinforcement. Two experiments were conducted to assess this prediction. In the first experiment, rats ran for different durations (0, 30, 60, and 180 min) prior to a session of operant wheel running on a FI 120-s schedule. In the second experiment, the concentration of sucrose reinforcement on a FI 180-s schedule was varied across values of 0, 5, 15, and 25%. In Experiment 1, as the duration of pre-operant running increased, the postreinforcement pause before initiation of running lengthened while wheel revolutions in the latter part of the FI interval increased. In Experiment 2, wheel revolutions markedly increased then decreased to a plateau early in the FI interval. Neither manipulation increased temporal control of the pattern of wheel running. Instead, results indicate that operant wheel running is regulated by automatic reinforcement generated by wheel activity and an adjunctive pattern of running induced by the temporal presentation of sucrose. Furthermore, the findings question whether the sucrose contingency regulates wheel running as a reinforcing consequence.     

Preference for less frequent shock under fixed-interval schedules of electric-shock presentation.

Lever pressing by 2 squirrel monkeys was maintained under fixed-interval 6-min and fixed-interval 2-min schedules of electric-shock presentation. Preference for these schedules was assessed during three experimental phases. In all phases, responses on one lever produced shock according to one or the other fixed-interval schedule, and responses on a second, changeover, lever switched between schedules. The opportunity to change over was presented during separate choice periods (during which the fixed-interval schedules did not operate) that followed the first through fourth shocks in each schedule. If no changeover occurred during those choice periods, a changeover automatically occurred following the fifth shock. In Phase I, durations of the choice periods were fixed. In Phase II, the choice periods equaled a proportion of their respective fixed interval. During Phase III (completed with 1 monkey) a response on the changeover lever during a given choice period reinstated the most recent fixed interval, and a failure to respond resulted in a changeover. During each of these phases, distinct preferences developed for the 6-min schedule. These results suggest that the maintenance of lever pressing by fixed-interval presentation of electric shock may not be an example of positive reinforcement, and that the response-maintaining characteristics of shock presentation may derive from other properties of the schedule.     

Effects of reinforcement magnitude on interval and ratio schedules

Rats' lever pressing was studied on three schedules of reinforcement: fixed interval, response-initiated fixed interval, and fixed ratio. In testing, concentration of the milk reinforcer was varied within each session. On all schedules, duration of the postreinforcement pause was an increasing function of the concentration of the preceding reinforcer. The running rate (response rate calculated by excluding the postreinforcement pauses) increased linearly as a function of the preceding magnitude of reinforcement on fixed interval, showed slight increases for two of the three animals on response-initiated fixed interval, and did not change systematically on fixed ratio. In all cases, the overall response rate either declined or showed no effect of concentration. The major effect of increasing the reinforcement magnitude was in determining the duration of the following postreinforcement pause, and changes in the response rate reflected this main effect.     

Response elimination in rats with schedules of omission training, including yoked and response-independent reinforcement comparisons

After rats were trained to lever press, response elimination began with factorial combinations of fixed vs variable and adjusting vs constant omission training schedules. A time interval scheduling reinforcement remained the same in a constant schedule, and its length was increased as response rate declined in the adjusting schedule. A variable time (VT) response-independent reinforcement schedule followed response elimination to test the durability of response cessation. Experiment I included groups whose reinforcement was yoked to that received by the omission schedule groups. Rate of response elimination was faster with an adjusting than a constant schedule, and slightly faster with a variable than a fixed schedule. There were no significant differences in rate of response elimination between omission schedule and yoked groups. Shorter delay of reinforcement tended to increase rate of response elimination. In the subsequent VT durability test all groups displayed near-zero response rates. In Experiment II adjusting fixed and variable omission schedules, including yoked groups, were compared with fixed time (FT) and VT reinforcement schedules. Response elimination was slower in the FT and VT groups, and they responded more in a subsequent VT durability test. It was concluded that differential reinforcement of other behavior fails to account for these omission training effects, and suggestions were made for an analysis based on the correlation between response and reinforcement rate.     

Effects of concurrent response-independent reinforcement on fixed-interval schedule performance

In three experiments, behavior maintained by fixed-interval schedules changed when response-independent reinforcement was delivered concurrently according to fixed- or variable-time schedules. In Experiment I, a pattern of positively accelerated responding during fixed interval was changed to a linear pattern when response-independent reinforcement occurred under a variable-time schedule. Overall response rates (total responses/total time) decreased as the frequency of response-independent reinforcement increased. Experiment II showed that the response-rate changes in the first experiment were controlled by the response-reinforcer relation, but the changes in patterns of responding were similar whether concurrently available reinforcement at varying times was response-dependent or response-independent. In the final experiment, the addition of response-independent reinforcement at fixed times to a fixed-interval schedule resulted in changes in both local and overall response rates and in the occurrence of positively accelerated responding between reinforcements. These results suggest that the temporal distribution of reinforcers determines response patterns and that both the response-reinforcement dependency and the schedule of reinforcement determine overall response rates during concurrently scheduled response-dependent and response-independent reinforcement.     

Deprivation and satiation: The interrelations between food and wheel running

Two experiments were designed to assess whether depriving rats of food would increase the reinforcement effectiveness of wheel running (Experiment 1) and whether satiation for wheel running would decrease the reinforcement effectiveness of food (Experiment 2). In Experiment 1, a progressive-ratio schedule was used to measure the reinforcement effectiveness of wheel running when rats were deprived or not deprived of food. Completion of a fixed number of lever presses released a brake on a running wheel for 60 s, and the response requirement was systematically increased until the rat stopped pressing or until 8 hr had elapsed. The ratio value reached (and the total number of lever presses) was an inverted-U function of food deprivation (percentage body weight). In Experiment 2, when wheel running preceded test sessions, fewer food-reinforced lever presses were maintained by the progressive-ratio schedule, and responding occurred at a lower rate on a variable-interval schedule. An interpretation of these results is that deprivation or satiation with respect to one event (such as food) alters the reinforcement effectiveness of a different event (such as access to wheel running).     

AGGRESSION AS POSITIVE REINFORCEMENT IN MICE UNDER VARIOUS RATIO‐ AND TIME‐BASED REINFORCEMENT SCHEDULES

There is evidence suggesting aggression may be a positive reinforcer in many species. However, only a few studies have examined the characteristics of aggression as a positive reinforcer in mice. Four types of reinforcement schedules were examined in the current experiment using male Swiss CFW albino mice in a resident—intruder model of aggression as a positive reinforcer. A nose poke response on an operant conditioning panel was reinforced under fixed‐ratio (FR 8), fixed‐interval (FI 5‐min), progressive ratio (PR 2), or differential reinforcement of low rate behavior reinforcement schedules (DRL 40‐s and DRL 80‐s). In the FR conditions, nose pokes were maintained by aggression and extinguished when the aggression contingency was removed. There were long postreinforcement pauses followed by bursts of responses with short interresponse times (IRTs). In the FI conditions, nose pokes were maintained by aggression, occurred more frequently as the interval elapsed, and extinguished when the contingency was removed. In the PR conditions, nose pokes were maintained by aggression, postreinforcement pauses increased as the ratio requirement increased, and responding was extinguished when the aggression contingency was removed. In the DRL conditions, the nose poke rate decreased, while the proportional distributions of IRTs and postreinforcement pauses shifted toward longer durations as the DRL interval increased. However, most responses occurred before the minimum IRT interval elapsed, suggesting weak temporal control of behavior. Overall, the findings suggest aggression can be a positive reinforcer for nose poke responses in mice on ratio‐ and time‐based reinforcement schedules.     

Effects of fixed-time shocks and brief stimuli on food-maintained behavior of rats

When a fixed-time schedule of shocks was presented to rats lever pressing for food on a random-interval schedule, a pattern of behavior developed with a high rate of pressing after shock declining to near zero before the next shock was delivered. Once this pattern had stabilized, one-quarter of the shocks were replaced with brief auditory stimuli (tones) in a random sequence. Tone maintained behavior similar to shock, although tone was never paired with shock. Both tone and shocks elicited responding when presented at various times as probe stimuli, and responding was usually totally suppressed if neither stimulus occurred at the beginning of the fixed-time interval. When other stimuli were paired with tone and shock, only those paired with tone gained discriminative control and elicited responding. These findings suggest that stimuli that signal a shock-free, or safe, period will maintain the pattern of behavior generated by shock on a fixed-time schedule. There is a parallel between this phenomenon and the control of behavior on second-order schedules of positive reinforcement with nonpaired brief stimuli.     

Persistent shock-elicited responding engendered by a negative-reinforcement procedure

A procedure in which responses reduced intermittently presented electric shocks to one quarter of their originally scheduled intensity, effectively engendered and maintained lever pressing in hooded rats. This contingency also markedly increased the response rates of rats initially trained under an unsignaled avoidance procedure. The responding of all animals extinguished rapidly when shock was withdrawn. Subsequently, it was discovered that high response rates could be maintained solely through presentation of shocks that were not affected by responses. Variations in the interval between shocks and changes in shock intensity over a wide range did not attenuate responding. Terminal performance was characterized by a consistent pattern of shock-elicited responses. Responses were also elicited by a tone following repeated tone-shock pairings. Finally, responding that was maintained by response-independent shocks was quickly suppressed by response-contingent shocks of the same intensity.     

Fixed interval schedules and delay of reinforcement

In a fixed interval schedule of reinforcement the only responses to be reinforced are those made when a certain time interval has elapsed since the previous reinforcement. The behaviour of three rats on such a schedule was compared with their behaviour on a schedule where a response made at any time during the interval was reinforced by setting up a reward which was delivered when the interval had elapsed. Response rates were higher in the ordinary fixed interval schedule than in its modified version, and it is argued that this rules out attempts to explain the maintenance of fixed interval performance by delayed reinforcement. Despite the clear difference in response rates, there was considerable similarity between the post-reinforcement pauses developed in the two schedules, and this suggests that pausing is influenced more by temporal than by response contingencies.     

Effects of cycle length on performance on a temporally defined avoidance schedule

Three rats were trained on a temporally defined avoidance schedule logically similar to a fixed-interval, limited-hold positive reinforcement schedule. This avoidance schedule was composed of time periods during which responses had no scheduled consequences alternating with time periods during which a response precluded shock. As with fixed-interval length and response rate on positive reinforcement schedules, an inverse relationship was obtained between the length of the no-consequence interval and response rate during the no-consequence interval. An inverse relationship was also obtained between the length of the no-consequence interval and the per cent of shocks avoided. A rate increase within the no-consequence interval, similar to that typically produced by fixed-interval positive reinforcement procedures, was displayed by one of the rats where the no-consequence interval was at intermediate values and frequency of shock was relatively high. The introduction of a discriminative stimulus correlated with the avoidance interval produced typical discriminated avoidance behavior as well as alterations in temporal patterning of responses during the no-consequence interval in the two rats exposed to this procedure. These alterations in temporal patterning disappeared when the discriminative stimulus was removed. The results were consonant with those reported in the literature involving food reinforcement and fixed-interval, limited-hold schedules.     

The response-reinforcement dependency in fixed-interval schedules of reinforcement

Pigeons were exposed to four different schedules of food reinforcement that arranged a fixed minimum time interval between reinforcements (60 sec or 300 sec). The first was a standard fixed-interval schedule. The second was a schedule in which food was presented automatically at the end of the fixed time interval as long as a response had occurred earlier. The third and fourth schedules were identical to the first two except that the first response after reinforcement changed the color on the key. When the schedule required a peck after the interval elapsed, the response pattern consisted of a pause after reinforcement followed by responding at a high rate until reinforcement. When a response was not required after the termination of the interval, the pattern consisted of a pause after reinforcement, followed by responses and then by a subsequent pause until reinforcement. Having the first response after reinforcement change the color on the key had little effect on performance. Post-reinforcement pause duration varied with the minimum interreinforcement interval but was unaffected by whether or not a response was required after the interval elapsed.     

The Probability of Small Schedule Values and Preference for Random-Interval Schedules

Preference for working on variable schedules and temporal discrimination were simultaneously examined in two experiments using a discrete-trial, concurrent-chains arrangement with fixed interval (FI) and random interval (RI) terminal links. The random schedule was generated by first sampling a probability distribution after the programmed delay to reinforcement on the FI schedule had elapsed, and thus the RI never produced a component schedule value shorter than the FI and maintained a rate of reinforcement half that of the FI. Despite these features, the FI was not strongly preferred. The probability of obtaining the smallest programmed delay to reinforcement on the RI schedule was manipulated in Experiment 1, and the interaction of this probability and initial link length was examined in Experiment 2. As the probability of obtaining small values in the RI increased, preference for the schedule increased while the discriminated time of reinforcer availability in the terminal link decreased. Both of these effects were attenuated by lengthening the initial links. The results support the view that in addition to the delay to reinforcement, the probability of obtaining a short delay is an important choice-affecting variable that likely contributes to the robust preferences for variable, as opposed to fixed, schedules of reinforcement.     

The role of the response-reinforcer relation in delay-of-reinforcement effects

The role of the response-reinforcer relation in maintaining operant behavior under conditions of delayed reinforcement was investigated by using a two-operandum (i.e., two-key) procedure with pigeons. Responding on one key was reinforced under a tandem variable-interval differential-reinforcement-of-other-behavior (tandem VI DRO) schedule. The schedule defined a resetting unsignaled delay-of-reinforcement procedure in that a response was required when the interfood interval of the VI schedule lapsed, but further responding during the DRO component on either key reset the time interval. This ensured a fixed delay duration between any response and reinforcement. Responding on another key, physically identical to the first one except for spatial location, otherwise was without consequence. The location of the key correlated with the delay-of-reinforcement procedure varied between sessions according to a semirandom sequence. Differences in response rates between the two keys were greater, with proportionally higher rates on the key correlated with the delay-of-reinforcement procedure, the longer the delay-of-reinforcement procedure remained correlated with the same key. Differences in responding on the two keys also increased within individual sessions. These results suggest that the response-reinforcer relation is the primary determinant of responding when responding is acquired and maintained with delayed reinforcement.     

Suppression of operant behavior and schedule-induced licking in rats

The first experiment studied the effects of punishment on rats' lever pressing maintained by a fixed-interval schedule of food reinforcement and on the associated schedule-induced licking. When licking was followed by shock, licking was suppressed but lever pressing was largely unaffected. When lever pressing was followed by shock, lever pressing was suppressed but licking was unaffected. In both cases, the punished behavior recovered its previous unpunished level when the shocks were discontinued. In a second experiment, the rats' lever pressing was maintained by a variable-interval schedule of food reinforcement under which polydipsic licking also developed. Both lever pressing and licking were partially suppressed during a stimulus correlated with occasional unavoidable electric shocks. With a higher shock intensity, both behaviors were suppressed further. Both lever pressing and licking recovered their previous levels when shocks were discontinued. These results show that schedule-induced licking, which has been described as adjunctive behavior, can be suppressed by procedures that suppress reinforced lever pressing, an operant behavior.     

Alternative fixed-ratio fixed-interval schedules of reinforcement

Five rats were trained under alternative fixed-ratio fixed-interval schedules, in which food reinforcement was provided for the completion of either a fixed-ratio or a fixed-interval requirement, whichever was met first. Overall response rate and running rate (the rate of responding after the postreinforcement pause) decreased for all subjects as the fixed-ratio value increased. As the proportion of reinforcements obtained from the fixed-ratio component increased and the alternative schedule approached a simple fixed ratio, overall response rate and running rate both increased; conversely, as the proportion of reinforcements obtained from the fixed-interval component increased and the alternative schedule approached a simple fixed interval, response rates decreased. Postreinforcement pause length increased linearly as the average time between reinforcements increased, regardless of the schedule parameters. A break-run pattern of responding was predominant at low- and medium-valued fixed ratios. All subjects displayed at least occasional positively accelerated responding within interreinforcement intervals at higher fixed-ratio values.     

A quantitative analysis of the responding maintained by interval schedules of reinforcement

Interval schedules of reinforcement maintained pigeons' key-pecking in six experiments. Each schedule was specified in terms of mean interval, which determined the maximum rate of reinforcement possible, and distribution of intervals, which ranged from many-valued (variable-interval) to single-valued (fixed-interval). In Exp. 1, the relative durations of a sequence of intervals from an arithmetic progression were held constant while the mean interval was varied. Rate of responding was a monotonically increasing, negatively accelerated function of rate of reinforcement over a range from 8.4 to 300 reinforcements per hour. The rate of responding also increased as time passed within the individual intervals of a given schedule. In Exp. 2 and 3, several variable-interval schedules made up of different sequences of intervals were examined. In each schedule, the rate of responding at a particular time within an interval was shown to depend at least in part on the local rate of reinforcement at that time, derived from a measure of the probability of reinforcement at that time and the proximity of potential reinforcements at other times. The functional relationship between rate of responding and rate of reinforcement at different times within the intervals of a single schedule was similar to that obtained across different schedules in Exp. 1. Experiments 4, 5, and 6 examined fixed-interval and two-valued (mixed fixed-interval fixed-interval) schedules, and demonstrated that reinforcement at one time in an interval had substantial effects on responding maintained at other times. It was concluded that the rate of responding maintained by a given interval schedule depends not on the overall rate of reinforcement provided but rather on the summation of different local effects of reinforcement at different times within intervals.     

Schedule-induced drinking as a function of percentage reinforcement

Drinking was recorded in rats while lever pressing was maintained on a series of percentage reinforcement schedules in which the per cent of 1-min fixed intervals terminating with food was 100, 90, 30, 70, 10, 50, and 100%. Intervals in which a pellet was omitted were terminated by brief light flash and click stimuli that were also correlated with food presentations. Drinking failed to develop in five of six subjects following intervals in which the brief stimuli were presented regardless of percentage reinforcement. Postpellet drinking, which followed intervals terminated with pellet delivery, however, increased in both duration and amount ingested per interval as percentage reinforcement was systematically decreased. The increase in postpellet drinking above that produced by 100% reinforcement was interpreted as an analogue of the positive-contrast effect observed with food-reinforced operants.