Abortive responding during punishment of bar holding

A continuous chain of homogeneous responding was established in rats by training animals to hold a lever down for 10 sec or longer before releasing it for food reinforcement. When criterion releases were subsequently punished, completed holding chains were greatly suppressed, aborted chains increased markedly, while the rate of chain initiations remained unchanged.     

Interreinforcement time, work time, and the postreinforcement pause

Six rats were trained with food deliveries contingent upon their pressing a lever and holding it down for fixed, cumulative durations. Hold requirements were varied from 7.5 seconds to 120 seconds. Lever holding was maintained reliably at hold requirements as long as 30 seconds to 105 seconds for different rats. At longer hold requirements, lever holding was erratic and tended to occur only early in sessions. At shorter and intermediate requirements, the patterns of lever holding resembled those of responding under fixed-ratio schedules for discrete responses, with breaks in responding immediately after reinforcement alternating with relatively continuous lever holding until the next reinforcement. At longer hold requirements, postpause lever holding frequently was interrupted with additional pauses. The duration of postreinforcement pauses increased linearly with the scheduled hold requirement. However, for five of six rats, the hold requirement, which represents the actual time spent lever holding per reinforcer, accounted for somewhat less variance in pause duration than did interreinforcement time.     

Motivational control of instrumental performance: The role of prior experience of the reinforcer

In four experiments we investigated the conditions that are necessary for instrumental performance to adjust appropriately to a change in drive state. Rats were trained to press a lever and pull a chain concurrently, with one action being reinforced by sucrose solution and the other by food pellets. In Experiment 1 for animals that were hungry throughout training the rate of lever pressing in an extinction test under thirst was unaffected by the type of reinforcer produced by this action during training, even though the sucrose solution would maintain a higher rate than the food pellets during training under thirst. In contrast, animals that experienced the instrumental contingencies arranged by the concurrent schedule while thirsty at some point during training pressed the lever more during the extinction test under thirst when this action had been reinforced with the sucrose solution rather than the food pellets. The remaining three experiments demonstrated that for this incentive effect to occur it is sufficient that the sucrose solution be delivered non-contingently under thirst at some stage of training. Thus, it would appear that performance mediated by an instrumental contingency adjusts appropriately to reinforcer revaluation brought about by a drive shift only if the animals have had prior experience with the incentive under the test drive state. This observation was interpreted in terms of Tolman's “cathexis” theory of motivation.     

Control of responding by stimulus duration

Pigeons were trained on a procedure in which the key was white for 30 sec, alternating with periods of darkness, or timeout. In a nondifferential training procedure, timeout duration was held constant at either 9 or 21 sec for different animals, and pecks on the white key were reinforced on a variable-interval 36-sec schedule. After 30 sessions an extinction generalization test was conducted where the duration of the timeout was varied from 3 to 27 sec. This test showed no differences in responding following timeouts of different durations. In a differential training procedure, timeout durations of either 9 or 21 sec were randomly scheduled for each animal. The variable-internal schedule was in effect following the same timeout duration as in the prior nondifferential procedure. No pecks were reinforced after the other timeout duration. In 40 sessions, differences in response rates following the two durations gradually developed. A maintained generalization procedure was then imposed in which timeout durations were varied from 3 to 27 sec, with the variable-interval schedule in effect following only the same duration as in the previous procedures. The first maintained generalization session showed that the prior differential training had established control of the animals' behavior by the timeout duration. In continued training on the maintained generalization procedure, control by the timeout duration decreased.     

Response duration is sensitive to both immediate and delayed reinforcement

We investigated the duration of lever pressing by rats when the delivery of appetitive reinforcers was contingent upon response duration. In the first experiment, response durations increased when duration requirements were imposed, and they decreased when duration requirements were removed. This effect occurred whether reinforcers were immediate or delayed by 8 s. In order to maintain the integrity of the delay intervals, reinforcer delivery was dependent upon both lever depression and release. In a second experiment, lever depression only and a response duration of at least 4 s were required for reinforcer delivery. Compared to immediate reinforcement conditions, delayed reinforcers increased both variability and the length of the maximum response durations. In a third experiment, immediate reinforcers were delivered contingent upon lever depression and release under a variety of duration requirements. Median lever‐press durations tracked the contingencies rapidly. Across all three experiments, rats emitted numerous response durations that were too short to satisfy the reinforcer requirements, and bimodal distributions similar to those produced by differential reinforcement of low rate schedules were evident for most rats. In many aspects, response duration responds to reinforcement parameters in a fashion similar to rate of discrete responding, but an examination of this continuous dimension of behavior may provide additional information about environment–behavior relationships.     

Transfer of control of the pigeon's key peck from food reinforcement to avoidance of shock

Eight pigeons were initially trained to peck a white key for food under a variable-interval 1-min schedule of reinforcement. Then, a shock-avoidance schedule was initiated and food was no longer available in the experimental situation. Under the avoidance schedule, each peck on the key postponed shock for 40 sec. A warning signal, consisting of tone and red houselights, was presented after 30 sec without a response. If no response occurred, a shock was delivered 10 sec after warning-signal onset. Shocks were delivered every 10 sec in the presence of the warning signal until a response was made. The warning signal was terminated only by a response. Key pecking of all eight pigeons came under control of the avoidance schedule and responding continued throughout the 20-day avoidance training period.     

Fixed-ratio performance under conditions of delayed reinforcement

Four rats were trained on a schedule in which completion of a fixed number of lever presses initiated a signalled delay period, at the end of which food was delivered. Lever presses made during the delay had no scheduled consequences. Delays of 12, 3, and 0.75 sec were used, and it was found that the latency of the first response after food (the post-reinforcement pause) increased with length of delay. There was, on the other hand, no consistent effect of delay upon rates of responding after the post-reinforcement pause.     

The role of the instrumental contingency in the motivational control of performance

In four experiments we investigated an irrelevant incentive effect based upon a transition from hunger to thirst. Hungry rats were trained to lever press either for sucrose solution or for food pellets before performance was tested in extinction while they were thirsty. Reinforcer-specific motivational control was found in the first experiment in that the animals pressed the lever more on tests following training with the sucrose solution rather than with food pellets. Moreover, this effect was seen only when testing was conducted following water, but not following food deprivation. The outcome of the remaining experiments suggests that this motivational control is not mediated by the instrumental contingency between lever pressing and the sucrose reinforcer during training. In these studies lever pressing and chain pulling were reinforced concurrently, one with sucrose and the other with food pellets, in order to equate the noninstrumental functions of the incentives. Following this training, lever pressing in extinction under thirst was unaffected by the type of incentive used as its reinforcer during training.     

Conditioned suppression and conditioned enhancement with the same positive UCS: an effect of CS duration

Previous experiments have shown that positively reinforced operant responding is suppressed during a conditioned stimulus terminated with an electric shock (conditioned suppression). In the present experiment, the conditioned stimulus was terminated with a positive unconditioned stimulus, and it was found that the duration of the conditioned stimulus was a key factor in determining whether response suppression or response enhancement was observed during the stimulus. The lever-pressing responses of rats were maintained by a variable-interval schedule of food reinforcement. While the rats were pressing the lever, a light was occasionally turned on, its offset coincident with a brief period of access to a sucrose solution. In consecutive blocks of sessions, the light duration was 40 sec, 12 sec, or 120 sec. Results showed that the rate of lever pressing was substantially suppressed during the 12-sec stimulus, slightly suppressed during the 40-sec stimulus, and enhanced during the 120-sec stimulus.     

Reinforcement of mediating behavior on a spaced-responding schedule

This paper describes a procedure for gaining experimental control over mediating behavior on a spaced-responding schedule of food reinforcement. Three rats, food-deprived, were trained on a DRL 16 sec schedule of food reinforcement. Then, a concurrent schedule of food reinforcement was introduced on a second (mediating) lever, such that the first response to occur on the mediating lever, after the DRL interval had timed out, was reinforced with food, as was the next response to occur on the DRL lever. Reinforcement via the mediating lever became a discriminative stimulus for a food-reinforcement opportunity on the DRL lever. Next, food reinforcement for the mediating behavior was replaced by a conditioned reinforcer consisting of onset of a buzzer signaling timing-out of the DRL interval. Under these conditions, chaining of behavior on the two levers was strong, and timing on the DRL lever was more accurate than under ordinary DRL conditions. As the DRL requirement was lengthened from 16 sec to 24 sec to 60 sec, mediating behavior weakened slightly. When the inter-response requirement for food reinforcement on the DRL lever was made shorter than the inter-response requirement for conditioned reinforcement on the mediating lever, the mediating behavior extinguished. Performance in the experiment was analyzed into a four-component chain, and the factors contributing to the maintenance, and later extinction, of mediating behavior are discussed.     

Stimulus- and response-reinforcer contingencies in autoshaping, operant, classical, and omission training procedures in rats

Separate groups of rats received 500 trials of lever-press training under autoshaping (food delivery followed 10-second lever presentations, or occurred immediately following a response); operant conditioning (responding was necessary for food delivery); and classical conditioning (food followed lever presentations regardless of responding). Each group then received 500 trials on an omission procedure in which food was omitted on trials with a response. Another group received 1000 trials on the omission procedure, and a fifth group, random control, received 1000 uncorrelated presentations of lever and food. The autoshaping, operant, and classical groups reached high response levels by the end of initial training. Acquisition was fastest in the autoshaping group. Responding remained consistently low in the control group. The omission group responded at a level between the control group and the other three groups. During omission training, responding in these three groups declined to the omission-group level. During omission training, the rats continued contacting the lever frequently after lever pressing had declined. Response maintenance under omission training seems not to require topographic similarity between the response and reinforcer-elicited consummatory behaviors.     

Automaintenance in guinea pigs: effects of feeding regimen and omission training

Behavior maintained by stimulus-reinforcer pairings was examined. Guinea pigs maintained at 85 per cent of free-feeding weights reliably contacted a retractable lever presented before delivery of a single piece of guinea-pig chow or a 45-milligram guinea-pig pellet. When animals were given free access to one food and received the second food preceded by the lever, contact responses persisted. Such responses seldom occurred when a single food was freely available and was also delivered after lever presentation. Introduction of an omission training (negative automaintenance) procedure, in which lever contacts resulted in lever retraction and prevented food delivery, strongly reduced lever contacts. Observation indicated that mouthing the food cup, instead of the lever, became the prominent behavior during the prefood stimulus under the omission training procedure.     

Behavioral regulation of gravity: schedule effects under escape-avoidance procedures

Squirrel monkeys were restrained in a centrifuge capsule and trained to escape and avoid increases in artificial gravity. During escape-avoidance, lever responses reduced centrifugally simulated gravity or postponed scheduled increases. The effect of variation in the interval of postponement (equal to the duration of decrease produced by escape responses) was studied under a multiple schedule of four components. Three components were gravity escape-avoidance with postponement times of 20, 40, and 60 sec. The fourth component was extinction. Each component was associated with a different auditory stimulus. Rate of responding decreased with increasing postponement time and higher mean g-levels occurred at shorter intervals of postponement. Effects of the schedule parameter on response rate and mean g-level were similar to effects of the schedule on free-operant avoidance and on titration behavior maintained by shock.     

Operant hoarding: a new paradigm for the study of self-control.

In the first of four experiments, rats were exposed to a modified multiple continuous reinforcement-extinction schedule during 15-min daily sessions. In one condition (saves condition) with the cuelight on, a single lever press produced a food pellet, briefly extinguished the cuelight, and started a clock. Saves (additional lever presses with interresponse times less than 1 s) produced an additional food pellet, briefly extinguished the cuelight, and restarted the interresponse time clock. The cuelight was extinguished 1 s after the last lever press and remained off during a 10-s period of extinction, during which no food pellets were delivered. In the other condition (savings account condition), the contingencies were the same except that the cuelight was extinguished and was not reilluminated after the initial lever press, and the delivery of all food pellets in the reinforcement component was delayed until the onset of extinction. In both conditions, rats made saves, but mean saves (total saves divided by the number of reinforcement components) were slightly reduced in the savings account condition. In Experiment 2, using six equally spaced 15-min sessions per day on alternate days, saves were either followed immediately with food and brief cuelight offset (saves condition) or were not reinforced at all. Mean saves were much greater when saves were reinforced. In Experiment 3, during 5-min daily sessions, saves earned a single pellet (savings account condition) or a number of pellets equal to the ordinal number of the lever press (interest condition). Rats made fewer mean saves, with little change in the food rate, when saves earned interest. In Experiment 4, the rats earned all their food in the operant situation during 24 daily 5-min sessions, these separated by 55-min intersession intervals during which no food was available; otherwise, the conditions were the same as in Experiment 3. In Experiment 4, the shift to interest for saves led to an increase in mean daily mean saves (total daily mean saves divided by the number of daily sessions) as well as to an increase in the number of food pellets delivered in each session. The results are discussed in terms of self-control and behavioral economics.     

ASSESSING THE ROLE OF EFFORT REDUCTION IN THE REINFORCING EFFICACY OF TIMEOUT FROM AVOIDANCE

Rats responded on concurrent schedules of shock‐postponement or deletion (avoidance) and timeout from avoidance. In Experiment 1, 3 rats' responses on one lever postponed shocks for 20 s and responses on a second lever produced a 1‐min timeout according to a variable‐interval 45‐s schedule. Across conditions, a warning signal (white noise) was presented 19.5 s, 16 s, 12 s, 8 s, or 4 s before an impending shock. Raising the duration of the warning signal increased both avoidance and timeout response rates. Timeout responding, although positively correlated with avoidance responding, was not correlated with the prevailing shock rate. In Experiment 2, 3 rats' responses on one lever deleted scheduled shocks according to a variable‐cycle 30‐s schedule and responses on a second lever produced a 2‐min timeout as described above. After this baseline condition, the avoidance lever was removed and noncontingent shocks were delivered at intervals yoked to the receipt of shocks in the baseline sessions. Timeout responding decreased when the avoidance lever was removed, even though the shock‐frequency reduction afforded by the timeout remained constant. These results suggest that a key factor in the reinforcing efficacy of timeout is suspension of the requirement to work to avoid shock, rather than the reduction in shock frequency associated with timeout.     

Extinction of Sidman avoidance behavior

Extinction of Sidman avoidance behavior by eliminating the noxious stimulus was studied in Sprague-Dawley rats with bar-pressing as the response. Each of three subjects was trained and extinguished on each of the following schedules in a different order: nondiscriminated, response-shock interval = 20 sec, shock-shock interval = 5 sec; nondiscriminated, response-shock interval = 40 sec, shock-shock interval = 5 sec; discriminated, response-white noise interval = 15 sec, noise-shock interval = 5 sec, shock-shock interval = 5 sec. Less than one 4-hr session was required for extinction for all procedures. When a warning stimulus was present, resistance to extinction increased. Subjects did not, however, respond to avoid the signal. Only small differences in extinction were found after training on different schedules with no warning signal.     

Operant acceleration during a pre-reward stimulus

Stimuli of 20, 40, and 80 sec duration terminated with five non-response-contingent food pellets were superimposed upon lever pressing reinforced with single pellets on a DRL 30-sec schedule. Two rhesus monkeys served as subjects. No change in response frequency was observed during the 20- and 40-sec stimuli. During the 80-sec pre-food stimulus, overall response frequency increased to approximately 150% and 220% of pre-stimulus levels, and the temporal distributions of interresponse times shifted toward the shorter intervals. When the 80-sec stimulus was no longer terminated with food, the response frequency decreased and the temporal distributions of interresponse times gradually approached pre-stimulus levels. An increased frequency of short interresponse times and an increase in response rate was again observed when the pellet termination procedure was reinstituted with the 80-sec stimulus. No change in response frequency or interresponse times was observed in the absence of the conditioning stimulus, and performance efficiency, as reflected in the ratio of responses to reinforcements during non-stimulus periods, remained stable throughout the experiment.     

Two temporal parameters of food postponement

Rats were trained to press a lever under schedules of food postponement. In the absence of lever presses, food was delivered periodically (food-food interval). Responses initiated a second interval (response-food interval) that was reset by each additional response. Performance was first studied at different response-food intervals with the food-food interval fixed at 30 or 60 sec, or 10 min. Response-food intervals were examined in ascending order and then recovery was studied at shorter intervals. Finally, the food-food interval was manipulated with response-food interval fixed at 30 sec. At food-food intervals of 30 and 60 sec, responding first increased and then decreased as the response-food interval increased. At the 10-min food-food interval, responding decreased with increasing response-food interval. In general, very low rates of responding occurred when the response-food interval was 60 sec or more and when it equalled or exceeded the food-food interval. However, responding was maintained in one animal when the food-food interval was decreased from 120 to 15 sec with the response-food interval at 30 sec. Results, in terms of several dependent variables, are compared with data on shock avoidance. Effects of response-independent and response-produced food and shock are discussed.     

Maintaining responding during stimulus generalization testing in extinction

Resistance to extinction and generalization gradients were studied following training with a long-adjusting-interval schedule. One large reinforcer occurred at the end of each daily training session. Sessions varied in length from 20 sec to 42.66 min, but were usually the latter. Repeated generalization tests were subsequently conducted for these subjects and subjects trained with a more conventional short-random-interval schedule. The long-adjusting-interval schedule produced generalization gradients that were not qualitatively different from those produced by the conventional procedure. However, the advantages of the long-adjusting-interval schedule are: (1) greater resistance to extinction both within and across generalization tests and (2) more stable gradient slopes within and across tests.     

Rats' lever-press durations as psychophysical judgments of time

Hungry rats received food following lever-press durations exceeding a minimum value, which ranged from 0 to 6.4 sec. When no intertrial intervals separated successive presses, modal press durations remained at very short values as the minimum value required for food was increased. This was particularly true immediately after a food presentation. When an 8-sec intertrial interval followed each lever release, modal press durations were always at or beyond the minimum value required for food, and outcome of the preceding press had no effect on press duration. Possible reasons for the effects of intertrial intervals included punishment of short presses, increased delay of reinforcement of short presses, and reduced density of reinforcement. In addition, functions relating discrete-trials lever-press duration to minimum duration required for food were found to be qualitatively and quantitatively similar to the power functions recently proposed by Catania (1970) for interresponse time and response latency. This similarity was taken as support for a general psychophysical law of temporal judgments.