Effects of signaling on temporal control of behavior in response‐initiated fixed intervals

Behavior and events distributed in time can serve as markers that signal delays to future events. The majority of timing research has focused on how behavior changes as the time to some event, usually food availability, decreases. The primary objective of the two experiments presented here was to assess how behavior changes as time passes between two time markers when the first time marker was manipulated but the second, food delivery, was held constant. Pigeons were exposed to fixed‐interval, response‐initiated fixed‐interval, and signaled response‐initiated fixed‐interval 15‐ and 30‐s schedules of reinforcement. In Experiment 1, first‐response latencies were systematically shorter in the signaled response‐initiated schedules than response‐initiated schedules, suggesting that the first response was a more effective time marker when it was signaled. In Experiment 2, responding in no‐food (i.e. “peak”) trials indicated that timing accuracy was equivalent in the three schedule types. Compared to fixed interval schedules, timing precision was reduced in the signaled response‐initiated schedules and was lowest in response‐initiated schedules. Results from Experiments 1 and 2 coupled with previous research suggest that the overall “informativeness” of a time marker relative to other events and behaviors in the environment may determine its efficacy.     

INFLUENCES ON COCAINE TOLERANCE ASSESSED UNDER A MULTIPLE CONJUNCTIVE SCHEDULE OF REINFORCEMENT

Under multiple schedules of reinforcement, previous research has generally observed tolerance to the rate‐decreasing effects of cocaine that has been dependent on schedule‐parameter size in the context of fixed‐ratio (FR) schedules, but not under the context of fixed‐interval (FI) schedules of reinforcement. The current experiment examined the effects of cocaine on key‐pecking responses of White Carneau pigeons maintained under a three‐component multiple conjunctive FI (10 s, 30 s, & 120 s) FR (5 responses) schedule of food presentation. Dose‐effect curves representing the effects of presession cocaine on responding were assessed in the context of (1) acute administration of cocaine (2) chronic administration of cocaine and (3) daily administration of saline. Chronic administration of cocaine generally resulted in tolerance to the response‐rate decreasing effects of cocaine, and that tolerance was generally independent of relative FI value, as measured by changes in ED50 values. Daily administration of saline decreased ED50 values to those observed when cocaine was administered acutely. The results show that adding a FR requirement to FI schedules is not sufficient to produce schedule‐parameter‐specific tolerance. Tolerance to cocaine was generally independent of FI‐parameter under the present conjunctive schedules, indicating that a ratio requirement, per se, is not sufficient for tolerance to be dependent on FI parameter.     

Time allocation on concurrent schedules with asymmetrical response requirements

Pigeons were trained on concurrent schedules in which key pecking was required by both schedules (concurrent variable-interval variable-interval schedules) and on concurrent schedules in which key pecking was required by only one of the schedules (concurrent variable-interval variable-time schedules). The distribution of reinforcements was systematically varied with both types of concurrent schedules. The distribution of time between the schedules depended on the reinforcement distribution and was independent of the symmetry of the response requirement. The relation between time and reinforcement distributions appears to be invariant over a wide range of manipulations of responding maintained by concurrent schedules.     

AGGRESSION AS POSITIVE REINFORCEMENT IN MICE UNDER VARIOUS RATIO‐ AND TIME‐BASED REINFORCEMENT SCHEDULES

There is evidence suggesting aggression may be a positive reinforcer in many species. However, only a few studies have examined the characteristics of aggression as a positive reinforcer in mice. Four types of reinforcement schedules were examined in the current experiment using male Swiss CFW albino mice in a resident—intruder model of aggression as a positive reinforcer. A nose poke response on an operant conditioning panel was reinforced under fixed‐ratio (FR 8), fixed‐interval (FI 5‐min), progressive ratio (PR 2), or differential reinforcement of low rate behavior reinforcement schedules (DRL 40‐s and DRL 80‐s). In the FR conditions, nose pokes were maintained by aggression and extinguished when the aggression contingency was removed. There were long postreinforcement pauses followed by bursts of responses with short interresponse times (IRTs). In the FI conditions, nose pokes were maintained by aggression, occurred more frequently as the interval elapsed, and extinguished when the contingency was removed. In the PR conditions, nose pokes were maintained by aggression, postreinforcement pauses increased as the ratio requirement increased, and responding was extinguished when the aggression contingency was removed. In the DRL conditions, the nose poke rate decreased, while the proportional distributions of IRTs and postreinforcement pauses shifted toward longer durations as the DRL interval increased. However, most responses occurred before the minimum IRT interval elapsed, suggesting weak temporal control of behavior. Overall, the findings suggest aggression can be a positive reinforcer for nose poke responses in mice on ratio‐ and time‐based reinforcement schedules.     

Shaping the location of a pigeon's peck: effect of rate and size of shaping steps.

For several pigeons, pecking at particular locations within a ten-inch-wide response area was reinforced by grain presentations. The reinforced locations changed systematically to "shape" response location back and forth across the area. The rate and size of these shifts in reinforced locations were varied in both between-subject and within-subject comparisons to evaluate the influence of these variables on the shaping process. Larger step sizes produced larger shifts in location for all sizes inspected, with all sizes from .5 to 3.0 inches effective in shaping behavior. More rapid steps were approximately as effective as slower steps for all rates of shift inspected from 25 reinforcers to 400 reinforcers per step. These data suggest that shaping peck location proceeds most efficiently with rapid, relatively large shifts in criterion performance.     

Concurrent variable‐interval variable‐ratio schedules in a dynamic choice environment

Most studies of operant choice have focused on presenting subjects with a fixed pair of schedules across many experimental sessions. Using these methods, studies of concurrent variable‐ interval variable‐ratio schedules helped to evaluate theories of choice. More recently, a growing literature has focused on dynamic choice behavior. Those dynamic choice studies have analyzed behavior on a number of different time scales using concurrent variable‐interval schedules. Following the dynamic choice approach, the present experiment examined performance on concurrent variable‐interval variable‐ratio schedules in a rapidly changing environment. Our objectives were to compare performance on concurrent variable‐interval variable‐ratio schedules with extant data on concurrent variable‐interval variable‐interval schedules using a dynamic choice procedure and to extend earlier work on concurrent variable‐interval variable‐ratio schedules. We analyzed performances at different time scales, finding strong similarities between concurrent variable‐interval variable‐interval and concurrent variable‐interval variable‐ ratio performance within dynamic choice procedures. Time‐based measures revealed almost identical performance in the two procedures compared with response‐based measures, supporting the view that choice is best understood as time allocation. Performance at the smaller time scale of visits accorded with the tendency seen in earlier research toward developing a pattern of strong preference for and long visits to the richer alternative paired with brief “samples” at the leaner alternative (“fix and sample”).     

Alternative reinforcement effects on fixed-interval performance

Pigeons' key pecks were reinforced with food on a fixed-interval schedule. Food also was available at variable time periods either independently of responding or for not key pecking (a differential-reinforcement-of-other-behavior schedule). The latter condition arranged reinforcement following the first pause of t seconds after it became available according to a variable-time schedule. This schedule allowed separation of the effects of pause requirements ≤ five-seconds and reinforcement frequency. The time spent pausing increased as the duration of the pause required for reinforcement increased from 0 to 30 seconds and as the frequency of reinforcement for pausing increased from 0 to 2 reinforcers per minute. Key pecking was more evenly distributed within each fixed interval with shorter required pauses and with more frequent reinforcement for pausing. The results complement those obtained with other concurrent schedules in which the same operant response was reinforced in both components.     

Cooperative responding in rats: II. Performance on fixed-ratio schedules of mutual reinforcement

Coordinated responses of 5 dyads of rats were investigated under fixed-ratio (FR) schedules of mutual water reinforcement. Coordinated responding was defined as 2 consecutive lever-presses, 1 from each of 2 rats, occurring <.5 s apart. In the FR schedules, each coordinated episode was defined as 1 response in the FR sequence. The size of FR schedules was parametrically manipulated assuming the values of FR 1, 6, 12, 18, 24, 30, 50, and 9, in this order. Each FR remained in effect until responding reached stability. Under all conditions, pairs of rats received access to water simultaneously (mutual reinforcement). Rates and proportions of coordinated responding showed a bitonic inverted U-shaped function of ratio size. Postreinforcement pauses increased systematically as the interreinforcement interval increased. Local rates and proportions increased as a function of response location within ratios. Results of a control condition with relaxed temporal constraints for mutual reinforcement showed decreases in rates and proportion of coordinated responses, suggesting that the coordinated responses were controlled by the mutual reinforcement contingencies. The present experiment showed that coordinated responding is quantitatively affected by 3 properties of FR schedules: response requirement, reinforcement rates, and proximity to reinforcement.     

CHANGES IN BLOOD PRESSURE AND HEART RATE DURING FIXED‐INTERVAL RESPONDING IN SQUIRREL MONKEYS

Episodic and sustained increases in heart rate and mean arterial blood pressure can occur with recurring patterns of schedule‐controlled behavior. Most previous studies were conducted under fixed‐ratio schedules, which maintained a consistent high rate of responding that alternated with periods of no responding during times when the schedule was not in operation. The present study examined changes in heart rate and blood pressure under fixed‐interval schedules which maintained a range of rates that varied from little or no responding at the beginning of the fixed interval to high rates at the end of the interval. The relations of cardiovascular function to rate of responding were examined. Squirrel monkeys prepared with arterial catheters were trained to respond under fixed‐interval schedules of electric‐shock presentation. The duration of the interval was varied across sessions and cardiovascular parameters were examined. Local rates of responding were typically near zero during timeout periods, low at the beginning of each fixed‐interval cycle, and then increased as the fixed interval progressed. At most schedule durations, arterial blood pressure and heart rate levels were lowest at the beginning of the interval cycles, increased as the rate of responding increased, and then decreased during the timeout periods. At all parameters studied, there was a direct relationship between changes in response rate within fixed‐interval cycles and changes in heart rate and blood pressure. The results suggest that a much closer concordance of these cardiovascular parameters and schedule‐controlled responding is obtained by examining ongoing behavior as it occurs within the contingencies by which it is maintained.     

The role of the peck-food contingency on fixed-interval schedules

Pigeons were trained to peck on a fixed-interval schedule of food reinforcement and then exposed to three schedules in which there was either no, or an indirect, relation between pecking and food delivery: (a) a conjunctive schedule in which food was delivered at fixed intervals, providing at least one peck was emitted in the interval; (b) a recycling version of the conjunctive schedule that essentially eliminated occasional peck-food contiguities (recycling conjunctive); (c) delivery of food at fixed intervals independently of the birds' behavior (fixed time). The rates and patterns of pecking sustained by these procedures depended on interfood interval and relative proximity of pecks to food.     

Elicited responding in chain schedules.

An omission procedure was employed to study elicited pecking in the first component of a two-component chain schedule. Both components were fixed-interval schedules correlated with colored keylights. The first response following the initial-link schedule produced a second fixed-interval schedule. We studied several fixed-interval lengths in two conditions: a standard response-dependent condition and an omission-contingent condition. The omission-contingent condition differed from the response-dependent condition in that responses during the initial fixed interval terminated the trial (omitting the terminal component and grain). If the terminal component was not omitted, a response following the terminal link's requirement produced 4-s access to grain. Pigeons responded during more than 70% of the initial links in the omission-contingent condition and responded during more than 90% of the initial links in the response-dependent condition. In general, rates of responding were consistent with the percentage data. The responding in the omission condition suggests that there may be elicited pecking, in chain schedules using pigeons, that is not the result of contingent conditioned reinforcement.     

Contingency tracking during unsignaled delayed reinforcement: Effects of delay duration and d‐amphetamine

In two experiments, the role of the response–reinforcer relation in maintaining low‐rate responding under unsignaled delay conditions was investigated. In both experiments pecking by pigeons on one response key, denoted the relevant key, was reinforced under an unsignaled delay‐of‐reinforcement procedure (defined as tandem variable‐interval (VI) differential‐reinforcement‐of‐other behavior [DRO] schedule). Responding on a second key, denoted the irrelevant key, had no programmed consequences. Between sessions, the location of the relevant key varied (after one, two, or three sessions) pseudorandomly. In Experiment 1, the delay (DRO) duration was manipulated parametrically. Overall, proportional relevant‐key response rates (relevant‐key response rates / [relevant‐key response rates + irrelevant key response rates]) increased across 3‐session sequences in which the relevant key remained in the same location and decreased as the DRO duration was changed systematically (2, 5, and 10 s). In Experiment 2, acute administration of d‐amphetamine increased proportional relevant‐key response rates during 1‐day sequences for only the DRO 5‐s duration, and results over 3‐day sequences, once a discrimination had already been established, were inconsistent. Results support that the response–reinforcer relation is the primary determinant of responding, and such discriminations are relatively resistant to disruption or potentiation by behaviorally active doses of d‐amphetamine.     

The effects of reinforcement frequency and response requirements on the maintenance of behavior.

Six rats responded under fixed-interval and tandem fixed-interval fixed-ratio schedules of food reinforcement. Basic fixed-interval schedules alternated over experimental conditions with tandem fixed-interval fixed-ratio schedules with the same fixed-interval value. Fixed-interval length was varied within subjects over pairs of experimental conditions; the ratio requirement of the tandem schedules was varied across subjects. For both subjects with a ratio requirement of 10, overall response rates and running response rates typically were higher under the tandem schedules than under the corresponding basic fixed-interval schedules. For all subjects with ratio requirements of 30 or 60, overall response rates and running response rates were higher under the tandem schedules than under the corresponding basic fixed-interval schedules only with relatively short fixed intervals. At longer fixed intervals, higher overall response rates and running rates were maintained by the basic fixed-interval schedules than by the tandem schedules. These findings support Zeiler and Buchman's (1979) reinforcement-theory account of response strength as an increasing monotonic function of both the response requirement and reinforcement frequency. Small response requirements added in tandem to fixed-interval schedules have little effect on reinforcement frequency and so their net effect is to enhance responding. Larger response requirements reduce reinforcement frequency more substantially; therefore their net effect depends on the length of the fixed interval, which limits overall reinforcement frequency. At the longest fixed intervals studied in the present experiment, reinforcement frequency under the tandem schedules was sufficiently low that responding weakened or ceased altogether.     

Conjunctive schedules of reinforcement: III. A fixed-interval adjusting fixed-ratio schedule

Key pecking of three pigeons was studied under a conjunctive schedule that specified both a fixed-interval and an adjusting fixed-ratio requirement. The fixed-interval schedule was 6 min for one pigeon and 3 min for the other two. The size of the ratio requirement was determined within each cycle of the fixed interval by the duration of the pause before responding began. The fixed-ratio value was at maximum at the start of each fixed interval and decreased linearly until the first response occurred (adjusting fixed-ratio schedule). A peck produced food when the number of responses remaining on the fixed-ratio schedule was completed and when the fixed interval had elapsed. If no response occurred during the interval, the fixed-ratio requirement decreased to one and a single response after the interval elapsed produced food. The initial value of the adjusting fixed-ratio schedule was studied over a range of 0 to 900. Increases in the adjusting fixed-ratio schedule to about 300 responses increased both pause duration and running response rate and also modified the pattern of responding from that obtained under the fixed-interval schedule. Higher values of the adjusting fixed ratio generally decreased pause duration and running response rate and also disrupted responding. Interreinforcement time under the conjunctive schedule was increased substantially when the adjusting fixed-ratio size exceeded 300 responses.     

Additional free reinforcers increase persistence of problem behavior in a clinical context: A partial replication of laboratory findings

Behavioral momentum theory is a quantitative framework used to characterize the persistence of behavior during response disruptors as a function of baseline stimulus–reinforcer relations. Results of several investigations have shown that alternative reinforcement can increase the resistance to change of a target response during extinction. In the present study, concomitant variable‐interval fixed‐time schedules of reinforcement for problem behavior were employed to simulate naturalistic situations involving the superimposition of response‐independent reinforcers on a baseline schedule of reinforcement for problem behavior, as in the common use of noncontingent reinforcement treatments. Resistance to change of problem behavior was assessed during postsession periods of extinction by comparing response rates in extinction following sessions with and without additional reinforcer deliveries arranged by fixed‐time schedules. For 2 out of 3 participants, problem behavior tended to be more resistant to extinction following periods in which additional fixed‐time reinforcers were delivered. These results are discussed in terms of potential effects of noncontingent reinforcement on problem behavior when the intervention is discontinued or implemented without good treatment integrity.     

The role of intermittent food in the induction of attack in pigeons

The present experiments evaluated whether transitions in reinforcer probability are necessary to induce attack in pigeons. In Experiment I, three of six pigeons exposed to response-contingent constant-probability food schedules and a photograph of a conspecific as a target exhibited sustained postreinforcement attack on the target. The postreinforcement pattern of attack developed over the course of the experiment and was accompanied by a reduction in the rate of postreinforcement key pecking and an increase in the postreinforcement pause in key pecking. These effects on key pecking resulted in unprogrammed variations in the probability of reinforcement which may have been responsible for the induction of attack. In Experiment II, the attack-inducing properties of a constant-probability response-independent food schedule were compared to a periodic food schedule matched for overall rate of food delivery and to a no-food condition. In addition to attack, the spatial location of the subjects was monitored during each interfood interval. The periodic and aperiodic food schedules generated very different patterns of spatial location. Postfood attack was induced by both food schedules, although the constant-probability schedule induced attack in fewer birds. The no-food condition was not effective in inducing attack in any birds. These experiments indicate that intermittent food schedules without reductions in reinforcer probability are sufficient to induce attack in some pigeons, although not as effective as schedules with transitions in reinforcer probability.     

Effects of reinforcement magnitude on interval and ratio schedules

Rats' lever pressing was studied on three schedules of reinforcement: fixed interval, response-initiated fixed interval, and fixed ratio. In testing, concentration of the milk reinforcer was varied within each session. On all schedules, duration of the postreinforcement pause was an increasing function of the concentration of the preceding reinforcer. The running rate (response rate calculated by excluding the postreinforcement pauses) increased linearly as a function of the preceding magnitude of reinforcement on fixed interval, showed slight increases for two of the three animals on response-initiated fixed interval, and did not change systematically on fixed ratio. In all cases, the overall response rate either declined or showed no effect of concentration. The major effect of increasing the reinforcement magnitude was in determining the duration of the following postreinforcement pause, and changes in the response rate reflected this main effect.     

Second-order schedules with fixed-ratio components: variation of component size

Key pecking by pigeons was reinforced with food under second-order schedules with fixed-ratio units. A constant total number of key pecks was required for reinforcement under each condition, but the size and, inversely, number of fixed-ratio components were varied. The total response requirement of 256 pecks was divided into fixed-ratio units of 128, 64, 32, 8, and 2 responses. A brief stimulus, which always preceded food reinforcement, was presented upon completion of each fixed-ratio unit. Under most conditions, the pattern of within-unit responding was typical of that under simple fixed-ratio schedules. Overall response rate was an inverted U-shaped function of component size. That is, response rates were highest under moderate sized units (fixed ratio 128 and 64). This relationship is consistent with previous determinations of rate as a function of fixed-ratio value for simple fixed-ratio schedules.     

The effects of different component response requirements in multiple and concurrent schedules

Six pigeons were trained on multiple and concurrent schedules. The reinforcement rates were varied systematically (a) when lever pressing was required in one component and key pecking in the successive component; (b) when lever pressing was required in both multiple components; (c) when key pecking was required in both multiple components; and (d) when key pecking was required on one schedule and lever pressing was required on the concurrently-available schedule. Only the absolute level of responding was changed by different response requirements. Analyzed by the generalized matching law, performance under different response requirements resulted in a bias toward key pecking, and the measured response bias was the same in multiple and concurrent schedule arrangements. The bias in time measures obtained from concurrent schedule performance was reliably smaller than the obtained response biases. The sensitivity to reinforcement-rate changes was ordered: concurrent key-lever; multiple key-key; multiple lever-key; and, the least sensitive, multiple lever-lever. The results confirm that requirements of different topographical responses can be handled by the generalized matching law mainly in the bias parameter, but problems for this type of analysis may be caused by the changing sensitivity to reinforcement in multiple schedule performance as response requirements are changed.     

Effects of methadone on alternative fixed-ratio fixed-interval performance: latent influences on schedule-controlled responding.

Effects of methadone on pigeons' key pecking were examined under four conditions selected to analyze the control of behavior under alternative fixed-ratio fixed-interval schedules. In Condition 1, pigeons pecked under one of three different alternative schedules (alternative fixed-ratio 50 fixed-interval 90 s, alternative fixed-ratio 75 fixed-interval 90 s and alternative fixed-ratio 200 fixed-interval 90 s) each week. In Condition 2, fixed-ratio 50 or fixed-ratio 75 schedules were in effect during baseline sessions, and alternative fixed-ratio 50 fixed-interval 90-s or alternative fixed-ratio 75 fixed-interval 90-s schedules were in effect during sessions in which methadone was administered. In Condition 3, effects of methadone on key pecking maintained under fixed-ratio 50 and fixed-ratio 75 schedules were examined, whereas in Condition 4 the effects of methadone on key pecking under a fixed-interval 90-s schedule as well as fixed-ratio 50 and fixed-ratio 75 schedules were investigated. Control by the fixed-interval contingency was assessed by computing the proportion of total session reinforcers delivered under the fixed-interval schedule. Methadone administration (0.5-4.0 mg/kg) shifted the predominant source of schedule control under the alternative schedule from the fixed-ratio schedule to the fixed-interval contingency. This shift was dependent on methadone dose and fixed-ratio size. Control by the fixed-interval contingency was greatest following extensive exposure to the interval component embedded within the alternative schedule (Condition 1), but was apparent to a lesser degree with even very limited exposure to the alternative fixed-ratio fixed-interval schedule (Condition 2). Interreinforcement intervals comparable to those under fixed-interval schedule were not observed under the fixed-ratio schedules presented alone (Condition 3). Repeated exposure to the fixed-interval contingency outside the context of the alternative fixed-ratio fixed-interval schedule did not engender performance changes under a fixed-ratio schedule which would mimic those of increased fixed-interval contingency control (Condition 4). These data suggest that drug administration can be used to unmask the influence of contingencies that are latent under baseline conditions and reveal influences of both past and present environmental variables.