Changing the response unit from a single peck to a fixed number of pecks in fixed-interval schedules

Each of three pigeons was studied first under a standard fixed-interval schedule. With the fixed interval held constant, the schedule was changed to a second-order schedule in which the response unit was the behavior on a small fixed-ratio schedule (first a fixed-ratio 10 and then a fixed-ratio 20 schedule). That is, every completion of the fixed-ratio schedule produced a 0.7-sec darkening of the key and reset the response count to zero for the next ratio. The first fixed-ratio completed after the fixed-interval schedule elapsed produced the 0.7-sec blackout followed immediately by food. These manipulations were carried out under two different fixed-interval durations for each bird ranging from 3 min to 12 min. The standard fixed-interval schedules produced the typical pause after reinforcement followed by responding at a moderate rate until the next reinforcement. The second-order schedules also engendered a pause after reinforcement, but responding occurred in bursts separated by brief pauses after each blackout. For a particular fixed-interval duration, post-reinforcement pauses increased slightly as the number of pecks in the response unit increased despite large differences in the rate and pattern of key pecking. Post-reinforcement pause increased with the fixed-interval duration under all response units. These data confirm that the allocation of time between pausing and responding is relatively independent of the rate and topography of responding after the pause.     

Some effects of response-dependent clock stimuli in a fixed-interval schedule

Two experiments studied the effects of brief response-dependent clock stimuli in fixed-interval schedules of reinforcement. In the first experiment, two pigeons were exposed to a fixed-interval schedule. Two conditions were compared. In both conditions each peck on the key produced a brief stimulus. In one condition, pecks produced a different stimulus in successive sixths of the interval. This was the clock condition. In the other condition, the same stimulus was produced throughout the interval. Response rates were lower and the pause after reinforcement was longer in the clock condition. In the second experiment, a two-key optional clock procedure was used. Responding on the clock key produced one of three stimuli correlated with the three successive minutes of a fixed-interval schedule. A response on the other key produced grain at the end of the 3 min. When the final stimulus was removed from the situation and pecking produced nothing during the third minute, responding to the clock key declined to a very low rate. When the first two stimuli were removed and the third one replaced, responding to the clock key was resumed.     

Responding under sequence schedules of electric shock presentation

Lever pressing by squirrel monkeys was examined under second-order schedules of electric shock presentation in which different discriminative stimuli were associated with consecutive components (sequence schedules). Components were always two-minute fixed-interval schedules, and three different overall schedules were studied. Under an overall eight-minute fixed-interval schedule, the first component completion after at least eight minutes had elapsed produced electric shock. The number of components actually completed ranged from one to four; thus, different discriminative stimuli were occasionally associated with electric shock presentation. Under an overall “yoked” variable-ratio schedule, electric shock was presented after completion of a variable number of components; the required number and the distribution of components were matched to those obtained under the overall eight-minute fixed-interval schedule. Under an overall fixed-ratio schedule, electric shock was presented after completion of four components (chained schedule). Under all three sequence schedules, responding in early components was characterized by a pause followed by a single response after the end of the two-minute interval; responding in later components was characterized by a shorter pause followed by positively accelerated responding. Manipulation of overall schedules of shock presentation in these complex behavioral situations produced changes in responding comparable to those ordinarily obtained after similar manipulation of dependencies under both single and second-order schedules of food presentation. These experiments extend the range of conditions and levels of complexity under which responding can be maintained by presentation of electric shock.     

Effects of chlorpromazine on fixed-ratio responding: modification by fixed-interval discriminative stimuli.

Effects of chlorpromazine (1 to 100 mg/kg) were assessed on two pigeons' responding under various modifications of a multiple schedule of food delivery. During a fixed-interval component, the first response after 5 min produced food; during the subsequent, fixed-ratio component, the 30th response produced food. Modifications of the schedule entailed changes in stimulus conditions imposed during the fixed-ratio component that did not systematically alter characteristics of performance under non-drug conditions. In the first phase of the experiment, distinctive visual stimuli were correlated with each schedule component (conventional multiple schedule); chlorpromazine produced small decreases in fixed-ratio responding (20% at 30 mg/kg). When each response during the fixed-ratio component produced the stimulus correlated with the fixed-interval schedule (fixed-interval discriminative stimulus) for 1.2 s, effects of chlorpromazine were not different from those under the conventional multiple schedule. Chlorpromazine produced greater decreases in fixed-ratio responding (55% at 30 mg/kg) when either the first response of each fixed ratio changed the stimulus correlated with the fixed-ratio schedule to the fixed-interval discriminative stimulus for the remainder of the fixed-ratio component, or when the fixed-interval discriminative stimulus was presented independently of responding according to a matched temporal sequence. When the fixed-interval discriminative stimulus was present continuously during the fixed-ratio component (mixed schedule), chlorpromazine produced even more substantial decreases in fixed-ratio responding (greater than 80% at 30 mg/kg). Effects of chlorpromazine on fixed-interval responding were also modified by the schedules of fixed-interval discriminative stimulus presentation. The effects of chlorpromazine were a joint function of the stimuli prevailing during the multiple schedule and the degree to which responding influenced these stimuli.     

An analysis of reinforcement history effects

Four pigeons were exposed to two tandem variable-interval differential-reinforcement-of-low-rate schedules under different stimulus conditions. The values of the tandem schedules were adjusted so that reinforcement rates in one stimulus condition were higher than those in the other, even though response rates in the two conditions were nearly identical. Following this, a fixed-interval schedule of either shorter or longer values than, or equal to the baseline schedule, was introduced in the two stimulus conditions respectively. Response rates during those fixed-interval schedules typically were higher in the presence of the stimuli previously correlated with the lower reinforcement rates than were those in the presence of the stimuli previously correlated with the higher reinforcement rates. Such effects of the reinforcement history were most prominent when the value of the fixed-interval schedule was shorter. The results are consistent with both incentive contrast and response strength conceptualizations of related effects. They also suggest methods for disentangling the effects of reinforcement rate on subsequent responding, from the response rate with which it is confounded in many conventional schedules of reinforcement.     

The effects of reinforcement frequency and response requirements on the maintenance of behavior.

Six rats responded under fixed-interval and tandem fixed-interval fixed-ratio schedules of food reinforcement. Basic fixed-interval schedules alternated over experimental conditions with tandem fixed-interval fixed-ratio schedules with the same fixed-interval value. Fixed-interval length was varied within subjects over pairs of experimental conditions; the ratio requirement of the tandem schedules was varied across subjects. For both subjects with a ratio requirement of 10, overall response rates and running response rates typically were higher under the tandem schedules than under the corresponding basic fixed-interval schedules. For all subjects with ratio requirements of 30 or 60, overall response rates and running response rates were higher under the tandem schedules than under the corresponding basic fixed-interval schedules only with relatively short fixed intervals. At longer fixed intervals, higher overall response rates and running rates were maintained by the basic fixed-interval schedules than by the tandem schedules. These findings support Zeiler and Buchman's (1979) reinforcement-theory account of response strength as an increasing monotonic function of both the response requirement and reinforcement frequency. Small response requirements added in tandem to fixed-interval schedules have little effect on reinforcement frequency and so their net effect is to enhance responding. Larger response requirements reduce reinforcement frequency more substantially; therefore their net effect depends on the length of the fixed interval, which limits overall reinforcement frequency. At the longest fixed intervals studied in the present experiment, reinforcement frequency under the tandem schedules was sufficiently low that responding weakened or ceased altogether.     

Behavior controlled by scheduled injections of cocaine in squirrel and rhesus monkeys.

Rates and patterns of key-press responding maintained under schedules in which responding resulted in intravenous injections of cocaine were studied in squirrel monkeys and rhesus monkeys. Each injection was followed by a 60- or 100-sec timeout period. Schedule-controlled behavior was obtained at appropriate cocaine doses in each species. Under FR 10 or FR 30 schedules, performance was characterized by high rates of responding (usually more than one response per second) in each ratio. Under FI 5-min schedules, performance was characterized by an initial pause, followed by acceleration of responding to a final rate that was maintained until the end of the interval. Under multiple fixed-ratio fixed-interval schedules, rates and patterns of responding appropriate to each schedule component were maintained. Responding seldom occurred during timeout periods under any schedule studied. At doses of cocaine above or below those that maintained characteristic schedule-controlled behavior, rates of responding were relatively low and patterns of responding were irregular. Characteristic fixed-interval responding was maintained over a wider range of cocaine doses than characteristic fixed-ratio responding. Complex patterns of responding controlled by discriminative stimuli under fixed-ratio or fixed-interval schedules can be maintained by cocaine injections in squirrel monkeys and rhesus monkeys.     

Behavior simultaneously maintained by both presentation and termination of noxious stimuli

Lever pressing by two squirrel monkeys was maintained under a 3-minute variable-interval schedule of response-produced electric-shock presentation. At the same time, responding on a second lever was maintained under a 3-minute fixed-interval schedule of termination of the shock-presentation schedule and shock-correlated stimuli. Under the termination schedule, the first response after a 3-minute period produced a 1-minute timeout, during which no events occurred and responding had no scheduled consequence. Relatively high and constant rates of responding were maintained on the lever where responding produced shock. Lower rates and positively accelerated patterns of responding occurred on the lever where responding terminated the shock schedule. Thus, responding was simultaneously maintained by presentation of an event and by termination of a stimulus associated with that event. Rates and patterns of responding on each lever were reversed when the schedules arranged on each lever were reversed on two occasions. When shock intensity was increased from 0 to 10 mA, responding maintained both by presentation of shock and by termination of the shock schedule increased, but responding maintained by shock presentation increased to a greater extent. Positive and negative reinforcement, usually regarded as separate behavioral processes involving different events, can coexist when behavior is controlled by different contingencies involving the same event.     

“Level of Aspiration”: Preference for Time Versus Response Feedback and History Effects on the Setting of Personal Performance Standards

Human subjects responded on a computer keyboard and accumulated points according to fixed-ratio (FR) reinforcement schedules. In Experiment 1, subjects responded under a FR 500 schedule. Under baseline conditions satisfying the schedule requirement resulted in counter points and session termination. Subsequently, subjects could (a) choose to have a clock appear on the screen during the interreinforcement interval (IRI) as well as to enter a target time which they would attempt to better during that session, (b) choose to enter a target time or respond under baseline conditions, and (c) enter a target time and choose between having a clock appear throughout or at the end of experimental sessions. In Experiment 2, subjects responded under a FR 500 schedule, entered a target time each session, and could respond during the session to briefly produce either (a) clock feedback, or (b) the number of responses emitted by the subject. In Experiment 3, subjects responded under FR 250, 500, 1000, and 2000 schedule parameters, entered target times and responded for either clock or response feedback. Subjects (a) preferred responding under conditions in which target times were entered to responding under baseline conditions, (b) preferred to have the clock illuminated throughout rather than at the end of experimental sessions, (c) preferred response to clock feedback under all schedule parameters, (d) responded to having equaled or bettered a target time by lowering target time for the subsequent session, and (e) responded to having missed a target time by maintaining the same time during the subsequent session. The results were interpreted within the context of behavior analytic as opposed to more traditional personality theory.

     

Separating the effects of interreinforcement time and number of interreinforcement responses

The relative importance of interreinforcement time and interreinforcement responses was evaluated by varying each independently. To do this, a blackout was presented after each nonreinforced response under both fixed-ratio and fixed-interval schedules of reinforcement. Manipulating the blackout duration under the fixed-ratio schedule caused interreinforcement time to vary without affecting the number of interreinforcement responses. Pigeons' post-reinforcement and post-blackout response latencies were found to increase linearly with interreinforcement time. Under the fixed-interval schedule, the same blackout manipulations changed the number of interreinforcement responses without affecting interreinforcement time. Post-reinforcement and post-blackout response latencies under this condition were approximately constant. These results suggest that responding is controlled by interreinforcement time and is not influenced by the number of responses emitted between reinforcements.     

Conjunctive schedules of reinforcement: III. A fixed-interval adjusting fixed-ratio schedule

Key pecking of three pigeons was studied under a conjunctive schedule that specified both a fixed-interval and an adjusting fixed-ratio requirement. The fixed-interval schedule was 6 min for one pigeon and 3 min for the other two. The size of the ratio requirement was determined within each cycle of the fixed interval by the duration of the pause before responding began. The fixed-ratio value was at maximum at the start of each fixed interval and decreased linearly until the first response occurred (adjusting fixed-ratio schedule). A peck produced food when the number of responses remaining on the fixed-ratio schedule was completed and when the fixed interval had elapsed. If no response occurred during the interval, the fixed-ratio requirement decreased to one and a single response after the interval elapsed produced food. The initial value of the adjusting fixed-ratio schedule was studied over a range of 0 to 900. Increases in the adjusting fixed-ratio schedule to about 300 responses increased both pause duration and running response rate and also modified the pattern of responding from that obtained under the fixed-interval schedule. Higher values of the adjusting fixed ratio generally decreased pause duration and running response rate and also disrupted responding. Interreinforcement time under the conjunctive schedule was increased substantially when the adjusting fixed-ratio size exceeded 300 responses.     

Fixed-interval schedules of electric shock presentation: extinction and recovery of performance under different shock intensities and fixed-interval durations

In squirrel monkeys responding under a schedule in which responding postponed the delivery of electric shock, the presentation of response-dependent shock under a fixed-interval (FI) schedule increased the rate of responding. When the schedule of shock-postponement was eliminated, so that the only shocks delivered were those produced by responses under the FI schedule, a pattern of positively accelerated responding developed and was maintained over an extended period. When responses did not produce shocks (extinction), responding decreased. When shocks were again presented under the FI schedule, the previous pattern of responding quickly redeveloped. In general, response rates were directly related to the intensity of the shock presented, and inversely related to the duration of the fixed-interval. These results raise fundamental questions about the traditional classification of stimuli as reinforcers or punishers. The basic similarities among FI schedules of food presentation, shock termination, and shock presentation strengthen the conclusion that the schedule under which an event is presented and the characteristics of the behavior at the time the event is presented, are of overriding importance in determining the effect of that event on behavior.     

Some effects of response-correlated increases in reinforcer magnitude on human behavior

After training under short or long fixed-interval schedules, humans responded under a modified fixed-interval schedule in which magnitude of reinforcement (X or 2X) was minimally correlated with response frequency. Response frequencies that equaled or exceeded a minimum response criterion were followed by the larger reinforcer at the end of the interval; otherwise, the smaller reinforcer was delivered. The modified schedule alternated with the baseline schedule across conditions. In a control condition, the reinforcer magnitudes produced by control subjects were yoked to those of experimental subjects. Experimental subjects, but not control subjects, showed increased responding. In addition to the baseline and modified fixed-interval schedules used in Experiment 1, subjects in Experiment 2 also responded under a second modified fixed-interval contingency in which increases in reinforcer magnitude were more highly correlated with response frequency. Experimental subjects, but not control subjects, showed increased responding under both procedures. Direct comparison of these two procedures showed that the high-correlation procedure produced greater increases in responding than did the low-correlation procedure.     

Discriminative functions of schedule stimuli and memory: a combination of schedule and choice procedures

Pigeons responded under a combination brief-stimulus schedule and choice procedure. Normally, a fixed-interval schedule was in effect, where completion randomly produced either a brief stimulus or food. Intermittently, this schedule was interrupted by a choice arrangement. Two choice keys were lit, either a short or a long time since a prior event (food or stimulus). One choice response produced food if the time had been short, and the alternate response produced food if the time had been long. Across conditions, the duration of the fixed-interval schedule was varied, the stimuli that comprised the brief-stimulus operation were changed, and the stimuli were presented as paired and nonpaired with food. The focus of the study was the control of both schedule performance and choice responding across conditions. The results showed that choice accuracy was correlated with the degree of fixed-interval curvature, the response pattern of a pause followed by a gradually accelerated rate. As fixed-interval schedule duration was increased, both the degree of fixed-interval curvature and choice accuracy decreased. The particular brief stimulus used affected schedule and choice performance, with a more salient stimulus producing a greater degree of curvature and higher accuracy. Pairing and nonpairing operations produced striking differences in performance with the less salient brief stimulus, but not with the more salient stimulus. The results suggest that brief-stimulus schedule performance may be conceptualized in the context of memory research.     

The microanalysis of fixed-interval responding

The fixed-interval schedule of reinforcement is one of the more widely studied schedules in the experimental analysis of behavior and is also a common baseline for behavior pharmacology. Despite many intensive studies, the controlling variables and the pattern of behavior engendered are not well understood. The present study examined the microstructure and superstructure of the behavior engendered by a fixed-interval 5- and a fixed-interval 15-minute schedule of food reinforcement in the pigeon. Analysis of performance typical of fixed-interval responding indicated that the scalloped pattern does not result from smooth acceleration in responding, but, rather, from renewed pausing early in the interval. Individual interresponse-time (IRT) analyses provided no evidence of acceleration. There was a strong indication of alternation in shorter-longer IRTs, but these shorter-longer IRTs did not occur at random, reflecting instead a sequential dependency in successive IRTs. Furthermore, early in the interval there was a high relative frequency of short IRTs. Such a pattern of early pauses and short IRTs does not suggest behavior typical of reinforced responding as exemplified by the pattern found near the end of the interval. Thus, behavior from clearly scalloped performance can be classified into three states: postreinforcement pause, interim behavior, and terminal behavior.     

Response requirements as constraints on output

Two experiments studied how added response requirements affected fixed-interval schedule performance. Experiment 1 involved tandem fixed-interval fixed-ratio schedules, and Experiment 2 studied conjunctive fixed-interval fixed-ratio schedules. In both, pigeons' output, defined as overall response rate or as responses during the interval, first increased and then decreased as the ratio was raised. With small ratio requirements, the frequency of reinforcement in time either did not change or decreased slightly. With progressively larger ratios, reinforcement frequency decreased consistently. Alternative explanations were discussed. The first, a reinforcement theory account, was that response strength is an increasing monotonic function of both the response requirement and reinforcement frequency, and the bitonic output function represents interacting effects. Increases in the response requirement accompanied by small changes in reinforcement frequency enhance output, but further increases result in large enough decrements in reinforcement frequency so that output is lowered. The second explanation does not view reinforcement as a basic process but, instead, derives from concepts of economics and conservation. Organisms allocate their behavior among alternatives so as to maximize value, where value is a function of the responses that can occur in a given situation under the set of restrictions imposed by particular schedules. One form of this theory explicitly predicts that output is a bitonic function of ratio requirements in simple ratio schedules. However, it was not clear that this model could explain the present effects involving joint ratio and interval schedule restrictions.     

The effects of a variety of instructions on human fixed-interval performance

College students were instructed to press a button for points under a single reinforcement schedule or under a variety of reinforcement schedules. Instructions for a single schedule were either specific or minimal. Instructions on a variety of schedules involved specific instructions on eight different schedules of reinforcement. Subsequent to the varied training, responding under a fixed-interval schedule occurred at a low rate. Both the minimal and specific instruction training led to fixed-interval responding that was similar to the responding exhibited during training. These findings suggest that under certain conditions instructed behavior is sensitive to changes in contingencies.     

Discriminated time-out avoidance in pigeons

Performances of two pigeons were studied on a concurrent discriminated TO avoidance-VR schedule. Each avoidance response postponed a TO from a VR 140 for a specified RS interval. The warning stimulus on the TO avoidance schedule was a discontinuous clock which consisted of a series of discrete color changes that varied systematically with the RS interval. Experimental manipulations established that the avoidance behavior was under the control of the avoidance schedule and the discontinuous clock. Five-min TOs maintained higher avoidance rates than shorter TO durations; a 15-min TO maintained less avoidance responding than the 5-min TO. Chlorpromazine hydrochloride increased TO avoidance behavior and decreased concurrent VR behavior.     

Effects of concurrent response-independent reinforcement on fixed-interval schedule performance

In three experiments, behavior maintained by fixed-interval schedules changed when response-independent reinforcement was delivered concurrently according to fixed- or variable-time schedules. In Experiment I, a pattern of positively accelerated responding during fixed interval was changed to a linear pattern when response-independent reinforcement occurred under a variable-time schedule. Overall response rates (total responses/total time) decreased as the frequency of response-independent reinforcement increased. Experiment II showed that the response-rate changes in the first experiment were controlled by the response-reinforcer relation, but the changes in patterns of responding were similar whether concurrently available reinforcement at varying times was response-dependent or response-independent. In the final experiment, the addition of response-independent reinforcement at fixed times to a fixed-interval schedule resulted in changes in both local and overall response rates and in the occurrence of positively accelerated responding between reinforcements. These results suggest that the temporal distribution of reinforcers determines response patterns and that both the response-reinforcement dependency and the schedule of reinforcement determine overall response rates during concurrently scheduled response-dependent and response-independent reinforcement.     

Second-order schedules with paired auditory brief stimuli

Pigeons were exposed to multiple second-order schedules of paired and unpaired brief stimuli in which responding on the main key was reinforced according to a fixed-interval thirty-second schedule by a brief stimulus (a tone in the paired schedule) and advancement to the next segment of the second-order schedule. In Experiment 1, a response on the second key was required during the tone in its fourth and final presentation to produce food. Responses during earlier brief stimuli indicated the extent to which the final brief stimulus was discriminated from preceding ones. Responding was comparable during all tones, extending prior findings with visual paired brief stimuli and weakening explanations of subjects' failure to discriminate between brief-stimulus presentations in terms of elicited responding. In Experiment 2 the number of fixed-interval segments comprising the second-order schedules varied from one through eight. Although main-key response rates increased across segments in both experiments, they increased much less sharply with a variable number of segments. These results suggest that the increase in main-key response rates across segments is due primarily to a degree of temporal discrimination not reflected on the second key. Main-key response rates were higher on paired auditory brief-stimulus schedules than on unpaired visual brief-stimulus schedules, especially in Experiment 2, thus further extending findings with visual brief stimuli to second-order schedules with auditory brief stimuli.