Controlling human fixed-interval performance

Both high and relatively constant rates of responding without post-reinforcement pauses and lower rates with pauses after reinforcement are produced by human subjects under fixed-interval (FI) schedules. Such FI rates and patterns may be controlled when subjects are provided with different histories of conditioning and different conditions of response cost (reinforcement penalties per response). Subjects with a conditioning history under ratio schedules typically produce high and relatively constant rates of responding under FI schedules; this responding does not change systematically with changes in FI value. In contrast, subjects with a history under schedules which produce little or no responding between reforcements [such as differential-reinforcement-of-low-rate (DRL) schedules] tend to pause after reinforcement and respond at low rates under FI schedules, whether or not they also have ratio conditioning histories; cost increases the likelihood of this type of performance. For DRL-history subjects, post-reinforcement pauses increase and response rates decrease as FI values increase.     

Some effects of response cost upon human operant behavior

Three experiments are reported which investigated the effects of cost (point loss per response) upon human-observer responses maintained by VI and FI schedules of reinforcement (acquisition of points via critical-signal detections). (I) Cost attenuated VI response rates without substantially disturbing the constancy of responding, regardless of the presentation sequence of the no-cost and cost conditions. (II) FI scalloping appeared only under cost conditions. Under no cost, a constant rate of responding (similar to VI performance) characterized inter-reinforcement intervals. Exposure to cost did not prevent the recovery of previously established no-cost baselines. (III) FI irregularities, analogous to those commonly observed under FI reinforcement schedules, may be produced by different temporal presentations of the no-cost and cost conditions.

The results of all three experiments emphasize the importance of cost as a factor in the maintenance of human behavior on schedules of positive reinforcement.

     

Quasireinforcement: Control of behavior by second-order interval schedules

Pigeons were trained on a variable-interval 66-sec schedule of reinforcement that was segmented into either fixed- or variable-interval 10-sec components. Three-second access to food followed some components according to the overall VI 66-sec schedule, but 3-sec periods of nonreinforcement followed the other components. With both FI 10-sec and VI 10-sec segments, overall response rates were generally higher when the completion of unreinforced segments was signaled by a red key (never paired with food) than when it was unsignaled. Response rates during the red-key periods dropped to zero. Brief presentations of the red key engendered the distinctive (FI or VI) patterns of responding which would be expected if each segment were followed by food. These data demonstrate behavioral control by brief stimuli which are not paired with primary reinforcement and show that such control may develop even when the sequences of behavior required to produce food or brief stimuli are variable in duration.     

Multiple fixed-interval schedules: transient contrast and temporal inhibition

Pigeons were exposed to four cycles per session of a multiple schedule in which each cycle involved twelve 60-sec fixed intervals followed by four 180-sec intervals [(12 FI 60-sec)(4 FI 180-sec) schedule]. Post-reinforcement pauses were shorter during the first few short intervals of each cycle than during later short intervals, and increased over the four long intervals of each cycle (positive and negative transient contrast). A (12 FI 15-sec)(4 FI 45-sec) schedule showed similar results. These two schedules differed in some other respects indicating effects of absolute FI duration on stimulus control. Differences in contrast properties between both these procedures and multiple variable-interval schedules were related to the pause-producing property of reinforcement on FI (temporal inhibition). Behavior under two other multiple fixed-interval schedules—(2 FI 360-sec)(1 FI 720-sec) and (3 FI 360-sec)(1 FI 720-sec)—differed in certain respects from both the (12 FI x-sec)(4 FI 3x-sec) schedules. These differences may be related to differences in the number of successive fixed intervals within a component (run length).     

The effect of multiple S periods on responding on a fixed-interval schedule: IV. Effect of continuous S with only short S probes

Pigeons were studied under FI 500 sec in which an S^Δ was present throughout the interval except during the terminal 50-sec segment and one earlier 50-sec segment. Very little responding occurred during the presence of S^Δ. The rate of responding in the earlier 50-sec S^D segments was lower than in the terminal S^D segment. There was a clear trend for the rate of responding in the earlier S^D segment to be progressively higher the later it occurred in the course of the FI 500 sec. This trend was shown roughly to parallel the increasing rate of responding in a conventional FI 500 sec with no interruption by S^Δ. Since the changing tendency to respond through the FI survives massive disruption by S^Δ, it is concluded that the control of responding through the FI does not require continuous mediating behavior. It is suggested that it is the decaying retroactive influence of the reinforcer on responses that occurred longer and longer before the reinforcer occurred which produces the familiar scalloped pattern of responding under FI schedules.     

Response rate under varying frequency of non-contingent reinforcement

Two White Carneaux hen pigeons were exposed to a 60-sec random-interval baseline procedure. Six different exteroceptive stimuli were successively correlated, within a single session, with blocks of 10 reinforcement presentations. Following this training, a non-contingent reinforcement procedure was instated with inter-reinforcement intervals of 5, 15, 30, 60, 120, and 240 sec. Within a single session, each non-contingent frequency was correlated with one of the previously presented discriminative stimuli. After an initial increase in the rate of responding as the result of a high density of non-contingent reinforcements, the rate declined as exposure to each non-contingent frequency was prolonged.     

Role of nonreinforcement in the fixed-interval performance of pigeons

Fixed-interval (FI) schedules have been used extensively to study timing abilities. In FI schedules, animals typically show higher response rates immediately after nonreinforced (N) cycles rather than reinforced (R) cycles (the reinforcement-omission effect), and they exhibit the highest rate approximately at the time when the reinforcer is scheduled to occur (peak performance). The present experiments were designed to determine the extent to which factors other than timing contribute significantly to these two learning phenomena. Pigeons were trained in an FI 16-sec schedule in which half the cycles were R and half were N. When successive cycles were separated by a 2-sec interval, responding early in the FI interval was higher after an N cycle than after an R cycle. This reinforcement-omission effect was eliminated when the interval between cycles was increased to 12 sec, because of an increase in performance after R cycles. In addition, timing of the 16-sec interval was assessed by interpolating 32-sec test cycles (all N cycles) at two rates—either 1 test cycle every other session, or 25 test cycles per session. Peak performance, presumably indexing the animal’s ability to time the 16-sec interval, emerged only with 25 test cycles per day, but not with 1 test cycle every other day, despite extensive training with the target, 16-sec-long interval. These results suggest that transient demotivation and time-based discrimination contribute significantly to the reinforcement-omission effect and peak performance, respectively.     

Rate-dependent effect of methylphenidate(Ritalin) on fixed-interval behavior in rats

The behavioral effects of 1.0–18.0 mg/kg methylphenidate (Ritalin) injections on leverpressing reinforced under a fixed-interval (FI) 60-sec schedule for water in Mäll-Wistar rats were studied. Effets of methylphenidate were dose-dependent: it increased response rates early in the FI 60-sec adn reduced the higher response rates later in the FI, without changing the average response rate. There was no systematic effect of the drug upon locomotor activity. The local response-rate change during the FI 60-sec are in accord with the rate-dependency hypothesis (Dews, 1958). the bidirectional response-rate changes withing the FI explain why methylphenidate did not affect the average response rate.     

Behavior controlled by scheduled injections of cocaine in squirrel and rhesus monkeys.

Rates and patterns of key-press responding maintained under schedules in which responding resulted in intravenous injections of cocaine were studied in squirrel monkeys and rhesus monkeys. Each injection was followed by a 60- or 100-sec timeout period. Schedule-controlled behavior was obtained at appropriate cocaine doses in each species. Under FR 10 or FR 30 schedules, performance was characterized by high rates of responding (usually more than one response per second) in each ratio. Under FI 5-min schedules, performance was characterized by an initial pause, followed by acceleration of responding to a final rate that was maintained until the end of the interval. Under multiple fixed-ratio fixed-interval schedules, rates and patterns of responding appropriate to each schedule component were maintained. Responding seldom occurred during timeout periods under any schedule studied. At doses of cocaine above or below those that maintained characteristic schedule-controlled behavior, rates of responding were relatively low and patterns of responding were irregular. Characteristic fixed-interval responding was maintained over a wider range of cocaine doses than characteristic fixed-ratio responding. Complex patterns of responding controlled by discriminative stimuli under fixed-ratio or fixed-interval schedules can be maintained by cocaine injections in squirrel monkeys and rhesus monkeys.     

The long-term effect of high- and low-rate responding histories on fixed-interval responding in rats

Tell rats were given extended lever-press training on a fixed-interval (FI) 30-s food reinforcement schedule from the outset or following exposure to one or two previous reinforcement schedules. For 4 rats the previots schedule was either fixed-ratio 20, which generated high response rates, or differential-reinforcement-of-low-rate 20 s, which produced low response rates. For 4 additional rats the extended training on FI 30 s was preceded by experience with two schedules: fixed-ratio 20 followed by differential-reinforcement-of-low-rate 20 s; or the same two schedules in the reverse order. Fixed-interval response rates were initially affected by the immediately preceding schedule, but after 80 to 100 sessions, all traces of prior schedule history had disappeared. The results also showed no long-term effect of schedule history on the interfood-interval patterns of responding on the FI 30-s schedule. These results support one of the most central tenets of the experimental analysis of behavior: control by the immediate consequences of behavior.     

Conditioned reinforcement in second-order schedules

Pigeons responded under a schedule in which food was presented only after a fixed number of fixed-interval components were completed. Two such second-order schedules were studied: under one, 30 consecutive 2-min fixed-interval components were required; under the other, 15 consecutive 4-min fixed-interval components were required. Under both schedules, when a 0.7-sec stimulus light was presented at completion of each fixed interval, positively accelerated responding developed in each component. When no stimulus change occurred at completion of each fixed interval, relatively low and constant rates of responding prevailed in each component; a similar result was obtained when a 0.7-sec stimulus change occurred at completion of each fixed interval except the one which terminated with primary reinforcement. The 0.7-sec stimulus correlated with food delivery was an effective conditioned reinforcer in maintaining patterns of responding in fixed-interval components despite low average frequencies of food reinforcement.     

The role of context in intertrial interval effects in autoshaping

Two experiments evaluated the role of differential conditioning of the context in mediating the effect of intertrial interval (ITI) in autoshaping. In Experiment 1 pigeons were given acquisition with two keylights, each presented in a particular context. A given keylight/context combination had associated with it either a short (10-sec) or a long (2-min) ITI. Acquisition was more rapid with the long ITI. Tests with those keylights in a common third context indicated that the longer ITI had resulted in greater conditioning. On the other hand, pigeons trained on keylights with mixed ITIs in a third context evoked more responding when they were tested in the short ITI context compared with the long ITI context. That suggests that a context with a history of a short ITI enhances performance. In Experiment 2, two keylights were initially conditioned with mixed ITIs and then extinguished in different contexts under different ITI lengths. Extinction was more rapid for the keylight presented with a short ITI. That difference persisted when the keylights were tested with mixed ITIs in a common third context, suggesting a difference in associative strength of the keylights. The results are interpreted in terms of differential context conditioning resulting in differences in learning about the keylight.     

Intermittent punishment of human responding maintained by intermittent reinforcement

To determine the effects of variable-interval shock punishment on behavior maintained by variable-interval and variable-ratio reinforcement, human subjects' key-pressing behavior was reinforced with money on a four-component multiple schedule. Components 1 and 2 were variable-interval 30-sec, and Components 3 and 4 were variable-ratio 210. After responding was stabilized, response-contingent electric shock was scheduled on a variable-interval 10-sec schedule during the second and fourth components of each cycle. Subjects instructed as to the reinforcement contingencies showed gradually increasing suppression of variable-interval responding at increasing shock intensities and either very high or very low rates of variable-ratio responding at higher intensities. Minimally instructed subjects showed suppression at higher shock intensities, but no clear differential suppression as a function of reinforcement schedule. Recovery from initial suppression was observed within sessions.     

Low-response-rate conditioning history and fixed-interval responding in rats.

Bar presses by one group of rats were conditioned under a differential-reinforcement-of-low-rate reinforcement schedule immediately prior to conditioning under a fixed-interval schedule. In a second group of rats, bar presses were conditioned first under a differential-reinforcement-of-low-rate schedule and then under a fixed-ratio schedule prior to conditioning under a fixed-interval schedule. Low response rates occurred under the fixed-interval schedule only when it was immediately preceded by low-rate conditioning. Otherwise, fixed-interval responding was similar to responding under the fixed-ratio schedule. This finding suggests that responses of laboratory animals are sensitive to immediate history, and, unlike human responses, are relatively insensitive to a history of low-rate conditioning when it is followed by high-rate conditioning.     

Drugs and punished responding II: d-amphetamine-induced increases in punished responding

The effects of d-amphetamine on punished responding were studied in two experiments. In Experiment I, pigeons responded under a multiple fixed-ratio 30 response fixed-interval 5-min schedule of food presentation with 60-sec limited holds in both components. Each response was punished with electric shock, the intensity of which was varied systematically. In Experiment II, another group of pigeons responded under a multiple fixed-interval 5-min fixed-interval 5-min schedule of food presentation with 40-sec limited holds. Each response was punished with shock during one component, and every thirtieth response was punished in the other component. d-Amphetamine increased overall rates of punished responding only rarely under any of the punishment conditions; however, response rates within the fixed-interval when rates were low were increased by d-amphetamine when the shock intensity was low (Experiment I), or when responses produced shock intermittently (Experiment II). The data suggest that the effects of d-amphetamine on punished responding depend on the control rate of responding, the punishment intensity, the punishment frequency, and the schedule of food presentation.     

The effect of signaled reinforcement on rats' fixed-interval responding

Four experiments examined the effect on rats' response rate of presenting a brief (500 ms) stimulus simultaneously with the delivery of food on fixed-interval (FI) schedules. In Experiment 1, reinforcement signals that were spatially diffuse (both tones and lights) elevated rates of responding, but responding was attenuated by localized visual stimuli. The remaining experiments examined the signal-induced potentiation of responding. In Experiment 2, a tone reinforcement signal potentiated response rates on an FI schedule, but attenuated response rates on a variable-interval (VI) schedule. This difference was obtained even though the overall rate of responding was equated on the two schedules before the introduction of the signal. Signal-induced potentiation of responding occurred over a range of FI values employed in Experiment 3. In Experiment 4, presenting a reinforcement signal when high local rates of response had occurred immediately before reinforcement resulted in potentiated rates of responding on an FI schedule. The opposite effect on response rate occurred when the reinforcement signal followed only low local rates of response. These results indicate that a variety of factors influence the effects of a reinforcement signal. They imply, however, that the local rate of response at the time of reinforcement is a key factor in establishing the nature of the signaling effect.     

Failure to acquire an inhibitory task following seizure-induced brain damage

Male rats that had displayed limbic seizures following a single pair of systemic injections of lithium and pilocarpine were trained 2 months later on an operant schedule that required differential low rates of responding (DRL). The seizured rats never acquired schedules that required either 6-sec. or 12-sec. inhibition of responses following a reward; these rats displayed more perseverative responding and shorter interresponse times than controls. Histomorphology indicated severe brain damage primarily within the entorhinal cortices (and adjacent amygdala) and dorsal thalamus.     

Responding in the squirrel monkey under second-order schedules of shock delivery

Lever-pressing responses were maintained in the squirrel monkey when the only consequence of responding was the delivery of a response-produced electric shock, or alternatively, a brief visual stimulus that was occasionally followed by an electric shock. When shock was produced by the first response occurring after 8 min (8-min fixed-interval schedule), a period of no responding at the beginning of the interval was followed by a gradual increase in response rate during the interval. Similar rates and patterns of responding were maintained when a 1-sec visual stimulus was produced by the first response occurring after 8 min and shock delivery followed the brief stimulus. Subsequently, patterns of positively accelerated responding were engendered during individual fixed-interval components when the first response occurring after 4 min produced a 1-sec visual stimulus and shock delivery followed the second, and later the fourth, presentation of the 1-sec stimulus. When the duration of the brief stimulus was varied over a 100-fold range from 0.1 to 10.0 sec (1) mean response rates decreased monotonically as stimulus duration increased, and (2) patterns of positively accelerated responding were least variable and response rates during the initial part of each 4-min interval were lowest at a stimulus duration of 1 sec.     

Effects of real vs pretend shock on semantic differential judgments and the GSR in some classical conditioning paradigms

In two experiments subjects were shown different color 1-sec lights, one just before the end of a 4-sec UCS and the other coming 10 sec after UCS termination and 25 sec before the next UCS. For some groups the UCS was strong electric shock; for others it was a tone which subjects were instructed to imagine was a strong shock. The primary data were changes in semantic differential ratings of the lights before and after 12 “conditioning” trials. There were no differences among the groups. In a third experiment a 6-sec delay CS was used for GSR conditioning, some subjects receiving actual shock and others pretend shock. Only the former showed conditioning. The results are discussed in terms of a model of experimental demand.     

Comparable Rates of Responding and Reinforcement Do Not Eliminate the Differential Effects of Ethanol on Response Chain Acquisition and Performance

Ethanol and other drugs commonly disrupt responding under repeated acquisition and performance baselines, with responding in the former condition being more sensitive to such disruption than the latter. The present study was conducted to determine if differential drug effects would occur when baseline rates of responding were comparable in the two baseline conditions. The acute effects of ethanol (0, 0.4, and 0.8 g/kg) were examined in healthy adult volunteers responding under a multiple schedule of repeated acquisition and performance of 10-response sequences. A 1-sec delay occurred after each response to keep rates of responding comparable in the acquisition and performance conditions. This delay also reduced differences in reinforcement rates between the two components. Nevertheless, responding in the acquisition condition was still more sensitive to the disruptive effects of ethanol than responding in the performance component. This differential sensitivity to ethanol was most evident in measures of accuracy of responding (e.g., percentage of errors). These findings suggest that differences in overall rates of responding and reinforcement in the repeated acquisition and performance conditions contribute little, if anything, to the differential drug effects observed on those baselines.