The effects of reinforcement frequency and response requirements on the maintenance of behavior.

Six rats responded under fixed-interval and tandem fixed-interval fixed-ratio schedules of food reinforcement. Basic fixed-interval schedules alternated over experimental conditions with tandem fixed-interval fixed-ratio schedules with the same fixed-interval value. Fixed-interval length was varied within subjects over pairs of experimental conditions; the ratio requirement of the tandem schedules was varied across subjects. For both subjects with a ratio requirement of 10, overall response rates and running response rates typically were higher under the tandem schedules than under the corresponding basic fixed-interval schedules. For all subjects with ratio requirements of 30 or 60, overall response rates and running response rates were higher under the tandem schedules than under the corresponding basic fixed-interval schedules only with relatively short fixed intervals. At longer fixed intervals, higher overall response rates and running rates were maintained by the basic fixed-interval schedules than by the tandem schedules. These findings support Zeiler and Buchman's (1979) reinforcement-theory account of response strength as an increasing monotonic function of both the response requirement and reinforcement frequency. Small response requirements added in tandem to fixed-interval schedules have little effect on reinforcement frequency and so their net effect is to enhance responding. Larger response requirements reduce reinforcement frequency more substantially; therefore their net effect depends on the length of the fixed interval, which limits overall reinforcement frequency. At the longest fixed intervals studied in the present experiment, reinforcement frequency under the tandem schedules was sufficiently low that responding weakened or ceased altogether.     

The delay-reduction hypothesis of conditioned reinforcement and punishment: Observing behavior

Pigeons responded in an observing-response procedure in which three fixed-interval components alternated. Pecking one response key produced food reinforcement according to a mixed schedule. Pecking the second (observing) key occasionally replaced the mixed-schedule stimulus with the stimulus correlated with the fixed-interval component then in effect. In Experiment 1, observing was best maintained by stimuli correlated with a reduction in mean time to reinforcement. That finding was consistent with the conditioned-reinforcement hypothesis of observing behavior. However, low rates of observing were also maintained by stimuli not representing delay reduction. Experiment 2 assessed the role of sensory reinforcement. It showed that response rate was higher when maintained by stimuli uncorrelated with reinforcement delay than when the stimuli were correlated with a delay increase. This latter result supports a symmetrical version of the conditioned-reinforcement hypothesis that requires suppression by stimuli correlated with an increase in time to reinforcement. The results were inconsistent with hypotheses stressing the reinforcing potency of uncertainty reduction.     

Increased reinforcement when timeout from avoidance includes access to a safe place

Three experiments investigated the reinforcing value of access to a safe place during timeout from an avoidance schedule. Rats were trained on conjoint schedules in which responding both postponed shock on a free-operant avoidance schedule and produced periods of timeout on fixed-ratio schedules. In some conditions, a shelf was inserted into the operant chamber during timeout, enabling subjects to get off the grid floor. The combination of timeout and shelf maintained substantially higher response rates than the baseline avoidance schedule with ratio requirements as high as 90 (Experiment I). Adding the shelf to timeouts in one component of multiple fixed-ratio schedules of timeout resulted in higher response rates in the component where the shelf was included (Experiment II). When timeouts with and without the shelf were arranged on concurrent schedules, the shelf-timeout combination was preferred, even when of shorter duration than timeout alone (Experiment III). In all three experiments, subjects climbed on the shelf, although all shocks were cancelled during timeout periods. The results could not be accounted for solely in terms of the reinforcing properties of changes in shock rates, but required an interpretation that ascribed conditioned reinforcing value to stimuli associated with such changes.     

Concurrent fixed-interval variable-ratio schedules and the matching relation

Five rats responded under concurrent fixed-interval variable-ratio schedules of food reinforcement. Fixed-interval values ranged from 50-seconds to 300-seconds and variable-ratio values ranged from 30 to 360; a five-second changeover delay was in effect throughout the experiment. The relations between reinforcement ratios obtained from the two schedules and the ratios of responses and time spent on the schedules were described by Baum's (1974) generalized matching equation. All subjects undermatched both response and time ratios to reinforcement ratios, and all subjects displayed systematic bias in favor of the variable-ratio schedules. Response ratios undermatched reinforcement ratios less than did time ratios, but response ratios produced greater bias than did time ratios for every subject and for the group as a whole. Local rates of responding were generally higher on the variable-ratio than on the fixed-interval schedules. When responding was maintained by both schedules, a period of no responding on either schedule immediately after fixed-interval reinforcement typically was followed by high-rate responding on the variable-ratio schedule. At short fixed-interval values, when a changeover to the fixed-interval schedule was made, responding usually continued until fixed-interval reinforcement was obtained; at longer values, a changeover back to the variable-ratio schedule usually occurred when fixed-interval reinforcement was not forthcoming within a few seconds, and responding then alternated between the two schedules every few seconds until fixed-interval reinforcement finally was obtained.     

Conditioned reinforcement versus time to reinforcement in chain schedules.

Pigeons were trained on three-component chain schedules in which the initial component was either a fixed-interval or variable-interval schedule. The middle and terminal components were varied among fixed-interval fixed-interval, variable-interval variable-interval, and an interdependent variable-interval variable-interval schedule in which the sum of the durations of the two variable-interval components was always equal to the sum of the fixed-interval fixed-interval components. At issue was whether the response rate in the initial component was controlled by its time to primary reinforcement or by the temporal parameters of the stimulus correlated with the middle terminal link. The fixed-interval initial-link schedule maintained much lower response rates than the variable-interval initial-link schedule regardless of the schedules in the middle and terminal links. Nevertheless, the intervening schedules played some role: With fixed-interval schedules in the initial links, response rates were consistently highest with independent variable-interval schedules in the middle and terminal links and intermediate with the interdependent variable-interval schedules; these initial-link differences were predicted by the response rates in the middle link of the chain. With variable-interval schedules in the initial links, response rates were lowest with the fixed-interval fixed-interval schedules following the initial link and were not systematically different for the two types of variable-interval variable-interval schedules. The results suggest that time to reinforcement itself accounts for little if any variance in initial-link responding.     

Fixed-interval performance and self-control in children.

Operant responses of 16 children (mean age 6 years and 1 month) were reinforced according to different fixed-interval schedules (with interreinforcer intervals of 20, 30, or 40 s) in which the reinforcers were either 20-s or 40-s presentations of a cartoon. In another procedure, they received training on a self-control paradigm in which both reinforcer delay (0.5 s or 40 s) and reinforcer duration (20 s or 40 s of cartoons) varied, and subjects were offered a choice between various combinations of delay and duration. Individual differences in behavior under the self-control procedure were precisely mirrored by individual differences under the fixed-interval schedule. Children who chose the smaller immediate reinforcer on the self-control procedure (impulsive) produced short postreinforcement pauses and high response rates in the fixed-interval conditions, and both measures changed little with changes in fixed-interval value. Conversely, children who chose the larger delayed reinforcer in the self-control condition (the self-controlled subjects) exhibited lower response rates and long postreinforcement pauses, which changed systematically with changes in the interval, in their fixed-interval performances.     

Stimulus control of behavioral history.

Pigeons were exposed to two different reinforcement schedules under different stimulus conditions in each of two daily sessions separated by 6 hr (Experiments 1 and 2) or in a single session (Experiment 3). Following this, either a fixed-interval (Experiment 1) or a variable-interval schedule (Experiments 2 and 3) was effected in both stimulus conditions. In the first two experiments, exposure to fixed-ratio or differential-reinforcement-of-low-rate schedules led to response-rate, but not pattern, differences in subsequent performance on fixed- or variable-interval schedules that persisted for up to 60 sessions. The effects of reinforcement-schedule history on fixed-interval schedule performance generally were more persistent. In Experiment 3, a history of high and low response rates in different components of a multiple schedule resulted in subsequent response-rate differences under identical variable-interval schedules. Higher response rates initially occurred in the component previously correlated with high response rates. For 3 of 4 subjects, the differences persisted for 20 or more sessions. Previous demonstrations of behavioral history effects have been confined largely to between-subject comparisons. By contrast, the present results demonstrate strong behavioral effects of schedule histories under stimulus control within individual subjects.     

Food-deprivation level alters the effects of morphine on pigeons'' key pecking.

Four pigeons pecked response keys under a multiple fixed-ratio 30 fixed-interval 5-min schedule of food presentation. Components alternated separated by 15-s timeouts; each was presented six times. Pigeons were maintained at 70%, 85%, and greater than 90% of their free-feeding weights across experimental conditions. When response rates were stable, the effects of morphine (0.56 to 10.0 mg/kg) and saline were investigated. Morphine reduced response rates in a dose-dependent manner under the fixed-ratio schedule and at high doses under the fixed-interval schedule. In some cases, low doses of morphine increased rates under the fixed-interval schedule. When pigeons were less food deprived, reductions in pecking rates occurred at lower doses under both schedules for 3 of 4 birds compared to when they were more food deprived. When pigeons were more food deprived, low doses of morphine increased rates of pecking in the initial portions of fixed intervals by a greater magnitude. Thus, food-deprivation levels altered both the rate-decreasing and rate-increasing effects of morphine. These effects may share a common mechanism with increased locomotor activity produced by drugs and with increased drug self-administration under conditions of more severe food deprivation.     

Remote effects of aversive contingencies: Disruption of appetitive behavior by adjacent avoidance sessions

Disruption of ongoing appetitive behavior before and after daily avoidance sessions was examined. After baselines of appetitive responding were established under a fixed-interval 180-s schedule of food presentation, 4 rats were exposed to 40-min sessions of the appetitive schedule just prior to 100-min sessions of electric shock postponement, while another 4 rats received the 40-min appetitive sessions just following daily sessions of shock postponement. In all 8 subjects, fixed-interval response rates decreased relative to baseline levels, the effect being somewhat more pronounced when the avoidance sessions immediately followed. The disruption of fixed-interval responding was only partially reversed when avoidance sessions were discontinued. During the initial exposure to the avoidance sessions, patterns of responding under the fixed-interval schedule were differentially sensitive to disruption, with high baseline response rates generally more disturbed than low rates. These disruptions were not systematically related to changes in reinforcement frequency, which remained fairly high and invariant across all conditions of the experiment; they were also not systematically related to the response rates or to the shock rates of the adjacent avoidance sessions. The results, while qualitatively resembling patterns of conditioned suppression as typically studied, occurred on a greatly expanded time scale. As disruption of behavior extending over time, the present data suggest that some forms of conditioned suppression are perhaps best viewed within a larger temporal context.     

Instructions as discriminative stimuli

Four undergraduates were exposed to a fixed-ratio schedule under an instruction to respond slowly and to a differential-reinforcement-of-low-rate 5-s schedule under an instruction to respond rapidly. Following this, a fixed-interval schedule was in effect under those same two sets of instructions. For 3 of 4 subjects, response rates were higher with the instruction to respond slowly than with the instruction to respond rapidly during the fixed-interval schedule. For the remaining subject, low-rate responding with the instruction to respond rapidly continued during the first 17 reinforcements of the fixed-interval schedule. Such control by instructions was not observed for other subjects exposed only to a fixed-interval schedule, with or without instructions. The results demonstrate that the effect of instructions can be altered by contingencies and suggest that instructions can function as discriminative stimuli.     

Accuracy of performance on a matching-to-sample procedure under interval schedules

Correct matches on a matching-to-sample procedure were reinforced under fixed-interval, chained fixed-interval, and fixed-interval schedules with exteroceptive stimulus changes correlated with time since the last reinforcer (an added clock). For all four pigeons, accuracy changed within the fixed-interval and fixed-interval schedules with added clock, decreasing from the beginning of the interval to some point in the middle. The performance then became increasingly more accurate until the end of the interval. Under the chained schedules, accuracy also changed within the components. During the initial component, accuracy decreased from the beginning of the fixed interval to some point in the middle or at the end. During the middle component, the performance usually remained at an intermediate level of accuracy. During the terminal component, the initially inaccurate performance became increasingly more accurate throughout the interval. Systematic relationships between response rate and per cent error showed that all four pigeons performed most accurately at high rates. The accuracy of the performance at low rates was also quite high. These relationships held for all three types of schedules through an eight-fold variation in scheduled interreinforcement time.     

The puzzle of responding maintained by response-contingent shock.

Four squirrel monkeys were first exposed to a sequence of procedures that reliably generate responding maintained by brief response-contingent electric shocks arranged according to a fixed-interval schedule. After responding had become stable on the fixed-interval schedule, additional contingencies were added in tandem, whereby after completion of the interval, the spacing of responses affected shock delivery. In one procedure, responses had to be spaced more widely than their previous median value if shock were to be delivered. In the other procedure, responses had to be spaced more closely to produce shock. On the first of these procedures, decreased but stable responses rates would indicate that shock functioned as a positive reinforcer; on the second, increased response rates would indicate the positively reinforcing function. Instead, response rates accelerated on the procedure that targeted more widely spaced responses for shock delivery, and decelerated or ceased on the procedure that arranged for shocks to be produced by more closely spaced responses. Consistent with other recent findings, these results question the interpretation of performances maintained by response-contingent shock as engendered by positive reinforcement and are consistent with aversive-control interpretations. The details of that aversive control are not entirely clear, however, and these same procedures would be informative if applied to shock-maintained behavior that is generated in other ways.     

Effects of reinforcer duration on responding in two-link chained interval schedules

Each of five pigeons was exposed to two or more durations of access to mixed grains on two-link, chained, interval schedules in which both links were identical fixed-interval or variable-interval schedules. Response rates were an increasing function of reinforcer duration for both initial and terminal links. For both types of interval schedules, initial-link response rates were more sensitive to reinforcer duration than were terminal-link response rates. The present results, together with prior ones, suggest that chaining and choice procedures are each sufficient for demonstrating substantial sensitivity of responding to changes in reinforcer duration.     

Interval and ratio reinforcement of a complex sequential operant in pigeons

Pigeons were required to produce exactly four pecks on each of two keys in any order for reinforcement. Correct response sequences were reinforced on either fixed-interval two-minute or fixed-ratio four schedules, with each correct sequence treated as a single response. Each pigeon developed a particular dominant sequence that accounted for more than 80% of all sequences. Sequence stereotypy was relatively unaffected by the temporal properties of the fixed-interval and fixed-ratio schedules. Response time (time from the first response in each sequence to the last) was also relatively unaffected by the temporal properties of the schedules. In contrast, response latency (time from end of one sequence to the beginning of the next) was markedly affected by the schedules. Latencies were long early in the interreinforcement interval and got shorter as the interreinforcement interval progressed. These data suggest that stereotyped response sequences become functional behavioral units, resistant to disruption or alteration by reinforcement variables that ordinarily influence the temporal spacing of individual responses.     

A comparison of responding maintained under second-order schedules of intramuscular cocaine injection or food presentation in squirrel monkeys.

Key pressing by squirrel monkeys was maintained under second-order schedules of either intramuscular cocaine injection or food presentation. Under one schedule, each completion of a 10-response fixed-ratio unit produced a brief visual stimulus; the first fixed-ratio unit completed after 30 minutes elapsed produced the stimulus paired with either cocaine injection or food presentation. Generally, short pauses followed by high rates of responding were maintained within the fixed-ratio units, and responding was positively accelerated over the 30-minute interval. Under another schedule, each completion of a 3-minute fixed-interval unit produced the brief stimulus; completion of the 10th fixed-interval unit produced the stimulus paired with either cocaine injection or food presentation. Generally, short pauses followed by high rates of responding were maintained within the fixed-ratio units, and responding was positively accelerated over the 30-minute interval. Under another schedule, each completion of a 3-minute fixed-interval unit produced the brief stimulus; completion of the 10th fixed-interval unit produced the stimulus paired with either cocaine injection or food presentation. Rates of responding increased within the fixed-interval units, and to a greater extent over the entire 10 fixed-interval units. Patterns of responding depended more on the schedule of reinforcement than on whether cocaine or food maintained responding. Omitting the brief stimuli following all but the last fixed-ratio or fixed-interval units decreased average rates and altered the patterns of responding. Substituting a visual stimulus that was never paired with cocaine or food following all but the last fixed-ratio or fixed-interval units decreased response rates to a lesser extent and did not substantially alter patterns of responding. When the duration of the paired stimulus was varied from .3 to 30.0 seconds, the highest response rates occurred at intermediate durations (1.0 to 10.0 seconds). The manner in which the stimulus changes affected performances depended more on the schedule of reinforcement than on whether cocaine injection or food presentation maintained responding.     

Maintenance of responding by squirrel monkeys under a concurrent shock-postponement, fixed-interval shock-presentation schedule.

A chain-pulling response was initially developed under a shock-postponement (avoidance) schedule with two squirrel monkeys. Few responses occurred on a lever where responding initially had no scheduled consequence or, subsequently, when a 3-minute fixed-interval shock-presentation schedule was concurrently arranged for lever responses. Appropriate rates and patterns of lever responding developed and were later maintained under the fixed-interval 3-minute shock-presentation schedule alone when the chain and shock-postponement schedule were removed. When both the shock-postponement and shock-presentation schedules were again simultaneously in effect, steady rates of chain pulling were maintained by the shock-postponement schedule and positively accelerated rates and patterns were maintained on the lever by the shock-presentation schedule. Response rates under both schedules were directly related to shock intensity. A history of exposure to a shock-postponement schedule, even though with a topographically different response and manipulandum, was sufficient for the development and eventual maintenance of responding by the presentation of shock. Further, differential performances can be maintained simultaneously by the presentation and postponement of electric shock.     

Behavior controlled by scheduled injections of cocaine in squirrel and rhesus monkeys.

Rates and patterns of key-press responding maintained under schedules in which responding resulted in intravenous injections of cocaine were studied in squirrel monkeys and rhesus monkeys. Each injection was followed by a 60- or 100-sec timeout period. Schedule-controlled behavior was obtained at appropriate cocaine doses in each species. Under FR 10 or FR 30 schedules, performance was characterized by high rates of responding (usually more than one response per second) in each ratio. Under FI 5-min schedules, performance was characterized by an initial pause, followed by acceleration of responding to a final rate that was maintained until the end of the interval. Under multiple fixed-ratio fixed-interval schedules, rates and patterns of responding appropriate to each schedule component were maintained. Responding seldom occurred during timeout periods under any schedule studied. At doses of cocaine above or below those that maintained characteristic schedule-controlled behavior, rates of responding were relatively low and patterns of responding were irregular. Characteristic fixed-interval responding was maintained over a wider range of cocaine doses than characteristic fixed-ratio responding. Complex patterns of responding controlled by discriminative stimuli under fixed-ratio or fixed-interval schedules can be maintained by cocaine injections in squirrel monkeys and rhesus monkeys.     

Choice and delay of reinforcement: Effects of terminal-link stimulus and response conditions

In two experiments, pigeons were exposed to concurrent-chains schedules in which a single initial-link variable-interval schedule led to access to terminal links composed of fixed-interval or fixed-delay schedules. In Experiment 1, an 8-s (or 16-s) delay to reinforcement was associated with the standard key, while reinforcer delay values associated with the experimental key were varied from 4 to 32 s. The results of Experiment 1 showed undermatching of response ratios to delay ratios with terminal-link fixed-delay schedules, whereas in some pigeons matching or overmatching was evident with the fixed-interval schedules. In Experiment 2, one pair of reinforcer delay values, either 8 versus 16 s or 16 versus 32 s, was used. In the first condition of Experiment 2, different delays were associated with different keylight stimuli (cued condition). In the second condition, different terminal-link delays were associated with the same stimulus, either a blackout (uncued-blackout condition) or a white key (uncued-white condition). To examine the role of responses emitted during delays, the keys were retracted during a delay (key-absent condition) in the third condition and responses were required by a fixed-interval schedule in the fourth condition. Experiment 2 demonstrated that the choice proportions for the shorter delay were more extreme in the cued condition than in the uncued-blackout condition, and that the response requirement imposed by the fixed-interval schedules did not affect choice of the shorter delay, nor did the key-absent and key-present conditions. These results indicate that the keylight-stimulus conditions affected preference for the shorter of two delays and that the findings obtained in Experiment 1 depended mainly on the keylight-stimulus conditions of the terminal links (i.e., the conditioned reinforcing value of the terminal-link stimuli).     

Effects of differences in interreinforcer intervals between past and current schedules on fixed-interval responding

Undergraduates were exposed to a mixed fixed-ratio differential-reinforcement-of-low-rate schedule. Values of the schedule components were adjusted so that interreinforcer intervals in one component were longer than those in another component. Following this, a mixed fixed-interval 5-s fixed-interval 20-s schedule (Experiment 1) or six fixed-interval schedules in which the values ranged from 5 to 40 s (Experiment 2) were in effect. In both experiments, response rates under the fixed-interval schedules were higher when the interreinforcer intervals approximated those produced under the fixed-ratio schedule, whereas the rates were lower when the interreinforcer intervals approximated those produced under the different-reinforcement-of-low-rate schedule. The present results demonstrate that the effects of behavioral history were under control of the interreinforcer intervals as discriminative stimuli.     

Effects of chlorpromazine on fixed-ratio responding: modification by fixed-interval discriminative stimuli.

Effects of chlorpromazine (1 to 100 mg/kg) were assessed on two pigeons' responding under various modifications of a multiple schedule of food delivery. During a fixed-interval component, the first response after 5 min produced food; during the subsequent, fixed-ratio component, the 30th response produced food. Modifications of the schedule entailed changes in stimulus conditions imposed during the fixed-ratio component that did not systematically alter characteristics of performance under non-drug conditions. In the first phase of the experiment, distinctive visual stimuli were correlated with each schedule component (conventional multiple schedule); chlorpromazine produced small decreases in fixed-ratio responding (20% at 30 mg/kg). When each response during the fixed-ratio component produced the stimulus correlated with the fixed-interval schedule (fixed-interval discriminative stimulus) for 1.2 s, effects of chlorpromazine were not different from those under the conventional multiple schedule. Chlorpromazine produced greater decreases in fixed-ratio responding (55% at 30 mg/kg) when either the first response of each fixed ratio changed the stimulus correlated with the fixed-ratio schedule to the fixed-interval discriminative stimulus for the remainder of the fixed-ratio component, or when the fixed-interval discriminative stimulus was presented independently of responding according to a matched temporal sequence. When the fixed-interval discriminative stimulus was present continuously during the fixed-ratio component (mixed schedule), chlorpromazine produced even more substantial decreases in fixed-ratio responding (greater than 80% at 30 mg/kg). Effects of chlorpromazine on fixed-interval responding were also modified by the schedules of fixed-interval discriminative stimulus presentation. The effects of chlorpromazine were a joint function of the stimuli prevailing during the multiple schedule and the degree to which responding influenced these stimuli.