Concurrent performances: a baseline for the study of conditioned anxiety

Three rats were trained to lever press on concurrent random interval 2-min random interval 2-min schedules of milk reinforcement. With a 5-sec changeover delay, relative response rate matched the relative reinforcement duration associated with each lever. A stimulus, during which unavoidable shocks occurred at random intervals, was superimposed on this concurrent baseline, and shifts in preference were found. However, data from this procedure were ambiguous, apparently confounded by shock-elicited response bursts. Termination of the shocks during the stimulus resulted in a rapid recovery of matching, which was preceded by a brief facilitation of responding on the less-preferred lever. The procedure was then changed to a conventional conditioned anxiety paradigm with a variable duration pre-shock stimulus. A marked shift in relative response rate towards the preferred lever was found in all three rats; that is, responding on the preferred lever was far less suppressed during the pre-shock stimulus than responding on the less-preferred lever.     

Conditioned suppression of counting behavior in rats

Three rats were trained on a schedule in which a response on lever B was reinforced only if it was preceded by a minimum number of consecutive responses on lever A. The minimum requirement was 27 A responses for Rat 1, and 20 A responses for Rats 2 and 3. The schedule maintained high rates of responding on lever A, and a slow, spaced pattern of responding on lever B. The mean number of consecutive responses on lever A was slightly greater than the minimum required. The effect of superimposing on this behavior a stimulus that ended with an unavoidable shock was the suppression of responding on both levers during the pre-shock stimulus. Responses on lever A were more suppressed, and the proportion of relatively short response runs on lever A during the pre-shock stimulus increased. With all three rats, the mean number of consecutive responses on lever A during the pre-shock stimulus decreased to a value below the minimum requirement for reinforcement of the subsequent B response.     

Timing behavior and conditioned fear

Rats were trained on a two-response timing procedure which required that response B follow response A by at least a minimum specified interval in order to be reinforced with food. Repeated presentation (5 min on, 5 min off) of an auditory warning stimulus terminated by a brief electric shock to the feet (conditioned fear) produced a marked suppression in the frequency of A-to-B response sequences during the warning stimulus. The distribution of A-to-B interresponse times (timing behavior), however, did not change during the warning stimulus.     

Free-operant acquisition with delayed reinforcement

The acquisition of free-operant lever pressing by hungry rats was investigated under a schedule in which the first lever press in each second programmed a reinforcer delivery after a fixed delay. In two studies acquisition was observed under programmed delays ranging between 0 and 32 sec, although animals trained with a delay of 64 sec pressed no more than yoked, non-contingent controls. In the final study an attempt was made to enhance sensitivity to the instrumental contingency under a 64-sec delay by exposing the animals to the operant chamber in the absence of the lever prior to each training session. Only animals receiving such exposure pressed significantly more than their non-contingent controls.     

A note on chaining and temporal discrimination

Four pigeons were exposed to a two-key DRL procedure. At the start of a trial, key A was illuminated. A response to the lighted key turned it off and simultaneously illuminated key B. Reinforcement was available for responses on key B which followed the initial key A response by more than 2 sec. In the course of exposure to these conditions, all birds acquired superstitious response chains on key A. The distribution of the number of responses on key A preceding a key B response and the distribution of intervals elapsing from the initial key A response to the key B response were of the same form. The suggestion is made that the superstitious responding on key A served to mediate the required delay interval. However, when intervals between successive key A responses were recorded for one subject, they were found to be regularly spaced in time. Thus, the problem remains of how this behavior is itself timed.     

Conditioned suppression and conditioned enhancement with the same positive UCS: an effect of CS duration

Previous experiments have shown that positively reinforced operant responding is suppressed during a conditioned stimulus terminated with an electric shock (conditioned suppression). In the present experiment, the conditioned stimulus was terminated with a positive unconditioned stimulus, and it was found that the duration of the conditioned stimulus was a key factor in determining whether response suppression or response enhancement was observed during the stimulus. The lever-pressing responses of rats were maintained by a variable-interval schedule of food reinforcement. While the rats were pressing the lever, a light was occasionally turned on, its offset coincident with a brief period of access to a sucrose solution. In consecutive blocks of sessions, the light duration was 40 sec, 12 sec, or 120 sec. Results showed that the rate of lever pressing was substantially suppressed during the 12-sec stimulus, slightly suppressed during the 40-sec stimulus, and enhanced during the 120-sec stimulus.     

Exteroceptive control of response under delayed reinforcement

Three white rats, after 50 continuous reinforcements, were exposed successively, under dim illumination, to reinforcement delays of 1, 3, 5, 7.5, 10, 15, and 20 sec, with prolonged training at the 20-sec level. Behavior was maintained at each level, and an increase in interval was accompanied by an increase in post-reinforcement pause. Subsequently, under both 20- and 30-sec delays, the animals were tested during half of each daily session to determine the effect of introducing darkness during each delay interval. The result of this stimulus “support” was to regularize and increase response rate for each animal at both interval values.     

Delayed escape from light by the albino rat

Two albino rats were trained to terminate an aversive light for 1 min by pressing a bar. After 19 hr of conditioning they were exposed to successive delays of 1, 2, 5, and 10 sec imposed between occurrence of the escape response and light termination. No stimulus change accompanied the delay interval, and any additional responses made at this time reset the delay timer. For both rats the relative frequency of escape responses with very long latencies increased as the delay interval increased. The modal escape latency, however, remained essentially unchanged for all delay values of greater than 1 sec. “Superstitious” responding was observed during the delay interval.     

Rate changes after unscheduled omission and presentation of reinforcement

Changes in response rate similar to frustration effects were studied in a two-lever situation. Responding on one lever on a fixed-interval schedule produced access to water for 5 sec and an exteroceptive stimulus. In the presence of this stimulus, responding on another lever on a fixed-interval schedule produced access to water for 5 sec and terminated the stimulus. Occasional omission of a previously scheduled reinforcer after responding on the first lever resulted consistently in increases in rate on the second lever during the immediately succeeding interval. In another procedure, occasional presentation of a previously unscheduled reinforcer after responding on the first lever resulted consistently in decreases in rate on the second lever during the immediately succeeding interval. Changes occurred after the first omissions or presentations and were about the same in magnitude as the procedure continued over several sessions. Typically, an increase or decrease in rate was maintained throughout an entire 100-sec interval. Changes in rate on the second lever of approximately the same magnitude also occurred when rate on the first lever was near-zero under a schedule that differentially reinforced behavior other than lever pressing.     

Briefly delayed reinforcement: An interresponse time analysis

Key-peck responding of pigeons was compared under VI or DRL schedules arranging immediate reinforcement and briefly (.5 sec) delayed reinforcement. Delays were either signaled by a blackout in the chamber, unsignaled, or unsignaled with an additional requirement that responding not occur during the .5 sec interval immediately preceding reinforcement (response delay). Relative to the immediate reinforcement condition, response rates increased during the unsignaled delay, decreased during the signaled delay, and were inconsistent during the response delay condition. An analysis of interresponse times (IRTs) under the different conditions revealed a substantial increase in the frequency of short (0 to .5 sec) IRTs during the unsignaled condition and generally during the response delay conditions compared to that during the immediate reinforcement baseline. Signaled delays decreased the frequency of short (0 to .5 sec) IRTs relative to the immediate reinforcement condition. The results suggest that brief unsignaled delays and, in many instances, response delays increase the frequency of short IRTs by eliminating constraints on responding.     

Operant acceleration during a pre-reward stimulus

Stimuli of 20, 40, and 80 sec duration terminated with five non-response-contingent food pellets were superimposed upon lever pressing reinforced with single pellets on a DRL 30-sec schedule. Two rhesus monkeys served as subjects. No change in response frequency was observed during the 20- and 40-sec stimuli. During the 80-sec pre-food stimulus, overall response frequency increased to approximately 150% and 220% of pre-stimulus levels, and the temporal distributions of interresponse times shifted toward the shorter intervals. When the 80-sec stimulus was no longer terminated with food, the response frequency decreased and the temporal distributions of interresponse times gradually approached pre-stimulus levels. An increased frequency of short interresponse times and an increase in response rate was again observed when the pellet termination procedure was reinstituted with the 80-sec stimulus. No change in response frequency or interresponse times was observed in the absence of the conditioning stimulus, and performance efficiency, as reflected in the ratio of responses to reinforcements during non-stimulus periods, remained stable throughout the experiment.     

Fixed-ratio performance under conditions of delayed reinforcement

Four rats were trained on a schedule in which completion of a fixed number of lever presses initiated a signalled delay period, at the end of which food was delivered. Lever presses made during the delay had no scheduled consequences. Delays of 12, 3, and 0.75 sec were used, and it was found that the latency of the first response after food (the post-reinforcement pause) increased with length of delay. There was, on the other hand, no consistent effect of delay upon rates of responding after the post-reinforcement pause.     

The effect of informational stimuli on instrumental response rate: Signalling reward versus signalling the availability of reward

Rats were trained to press a lever for food on an interval schedule and were given a brief cue (0.5 sec) between the operative response and the reward (C condition). Some control subjects in Experiments 1, 2 and 4 were given their cue either following the end of the temporal interval during which reward had been unavailable (SD condition), or randomly with respect to food (R condition). Other control subjects in Experiments 2 and 4 received both the food-correlated cue and the temporal-interval stimulus (B condition). In all experiments, rate of responding was lowest for the C subjects and for B animals when the two cues were from different modalities. Food-correlated and temporal-interval cues did not interact, suggesting that a reward-correlated signal does not affect response rate simply by enhancing the salience of the temporal interval offset.     

Temporal control of conditioned responding in goldfish

The peak procedure was used to characterize response timing during acquisition and maintenance of conditioned responding in goldfish. Subjects received light-shock pairings with a 5- or 15-s interstimulus interval. On interspersed peak trials, the conditioned stimulus light was presented for 45 s and no shock was delivered. Peaks in the conditioned response, general activity, occurred at about the time of the expected unconditioned stimulus, and variability in the activity distribution was scalar. Modeling of the changes in the activity distributions over sessions revealed that the temporal features of the conditioned response changed very little during acquisition. The data suggest that times are learned early in training, and, contrary to I. P. Pavlov's (1927/1960) concept of "inhibition of delay," that timing is learning when to respond rather than learning when not to respond.     

Response acquisition with delayed reinforcement: a comparison of two-lever procedures.

Groups of 8 experimentally naive rats were exposed during 8-hr sessions to resetting delay procedures in which responses on one lever (the reinforcement lever) produced water after a delay of 8, 16, 32, or 64 s. For rats in one condition, responses on a second (no-consequences) lever had no programmed consequences. For rats in another condition, responses on a second (cancellation) lever during a delay initiated by a response on the reinforcement lever prevented delivery of the scheduled reinforcer; responses on the cancellation lever at other times had no programmed consequences. Under both conditions and at all delays, most subjects emitted more responses on the reinforcement lever than did control rats that never received water emitted on either lever. At 8-s delays, both conditions engendered substantially more responding on the reinforcement lever than on the other lever, and performance closely resembled that of immediate-reinforcement controls. At delays of 16 and 32 s, however, there was clear differential responding on the two levers under the cancellation condition but not under the other condition. When the delay was 64 s, differential responding on the two levers did not occur consistently under either condition. These findings provide strong evidence that the behavior of rats is sensitive to consequences delayed by 8, 16, and 32 s, but only equivocal evidence of such sensitivity to consequences delayed 64 s. They also indicate that acquisition depends, in part, on the measure of performance used to index it.     

Response-shock delay as a reinforcer in avoidance behavior

After rats received preliminary training to avoid shock on a discrete-trial retractable-bar avoidance procedure, the procedure was changed such that responses retracted the lever but did not affect the rate of shock. Responses only delayed the onset of shock. About half of the animals under these procedures responded consistently on almost 100% of the discrete-trial cycles over days. When short latencies maximized the response-shock delay, animals tended to make short-latency responses. When long latencies maximized the response-shock delay, animals tended to make long-latency responses. When all response latencies produced the same response-shock delay, animals made differing average-latency responses. And, when responses did not delay shock, most of the animals primarily engaged in shock-elicited responding while the other animals engaged in preshock responding.     

Signal functions in discriminated avoidance behavior

Four rats were trained to lever press under a discriminated avoidance/escape schedule in which separately signalled safe and warning periods were 100 sec and 32 sec respectively. The auditory and not the visual component of the compound warning signal became associated with the discriminative control of lever pressing. Avoidance behavior also came under temporal control, in that the probability of lever pressing increased as the warning period progressed. Timing began with the onset of the warning signal rather than the offset of the safe signal. However, after the warning signal had been progressively eliminated, timing began with the offset of the safe signal. When neither signal was normally available, the temporal distribution of avoidance behavior changed markedly. Drifts in the temporal distribution of lever pressing occurred throughout the study; these were manipulated for two animals.     

Second-order schedules: discrimination of components

Pigeons were exposed to a series of second-order schedules in which the completion of a fixed number of fixed-interval components produced food. In Experiment 1, brief (2 sec) stimulus presentations occurred as each fixed-interval component was completed. During the brief-stimulus presentation terminating the last fixed-interval component, a response was required on a second key, the brief-stimulus key, to produce food. Responses on the brief-stimulus key before the last brief-stimulus presentation had no scheduled consequences, but served as a measure of the extent to which the final component was discriminated from preceding components. Whether there were one, two, four, or eight fixed-interval components, responses on the brief-stimulus key occurred during virtually every brief-stimulus presentation. In Experiment 2, an attempt was made to punish unnecessary responses on the brief-stimulus key, i.e., responses on the brief-stimulus key that occurred before the last component. None of the pigeons learned to withhold these responses, even though they produced a 15-sec timeout and loss of primary reinforcement. In Experiment 3, different key colors were associated with each component of a second-order schedule (a chain schedule). In contrast to Experiment 1, brief-stimulus key responses were confined to the last component. It was concluded that pigeons do not discriminate well between components of second-order schedules unless a unique exteroceptive cue is provided for each component. The relative discriminability of the components may account for the observed differences in initial-component response rates between comparable brief-stimulus, tandem, and chain schedules.     

Thresholds for conditioned suppression using X-rays as the pre-aversive stimulus

Four male, 12-week-old Sprague-Dawley rats were used to determine the threshold for behavioral response to X-irradiation using the conditioned suppression technique. They were maintained at 80 per cent body weight and initially trained to stable performance on a VI 1 min schedule with 16 per cent sucrose solution as reinforcement. After a stable baseline was obtained, animals were placed in the instrumental conditioning box beneath the X-ray machine for a half-hour session each day. While subjects were actively pressing the lever for reinforcement, 15-sec X-ray exposures of 0.5 R/sec were administered, followed immediately by electric shock. After all animals had exhibited conditioned suppression, the dose-rate was decreased in subsequent sessions in an effort to establish threshold. The results indicate that X-rays at a dose-rate as low as 0.004 R/sec can be an effective pre-aversive stimulus for the rat.     

Reinforcement of mediating behavior on a spaced-responding schedule

This paper describes a procedure for gaining experimental control over mediating behavior on a spaced-responding schedule of food reinforcement. Three rats, food-deprived, were trained on a DRL 16 sec schedule of food reinforcement. Then, a concurrent schedule of food reinforcement was introduced on a second (mediating) lever, such that the first response to occur on the mediating lever, after the DRL interval had timed out, was reinforced with food, as was the next response to occur on the DRL lever. Reinforcement via the mediating lever became a discriminative stimulus for a food-reinforcement opportunity on the DRL lever. Next, food reinforcement for the mediating behavior was replaced by a conditioned reinforcer consisting of onset of a buzzer signaling timing-out of the DRL interval. Under these conditions, chaining of behavior on the two levers was strong, and timing on the DRL lever was more accurate than under ordinary DRL conditions. As the DRL requirement was lengthened from 16 sec to 24 sec to 60 sec, mediating behavior weakened slightly. When the inter-response requirement for food reinforcement on the DRL lever was made shorter than the inter-response requirement for conditioned reinforcement on the mediating lever, the mediating behavior extinguished. Performance in the experiment was analyzed into a four-component chain, and the factors contributing to the maintenance, and later extinction, of mediating behavior are discussed.