Complex response patterns during temporally spaced responding

A series of experiments is reported in which two monkeys emitted complex response patterns, not specified by the experimental program, during the DRL component of a multiple schedule. Administration of sodium pentobarbital and dl-amphetamine, drugs which disrupted the DRL performance, were also observed to suppress these collateral responses. Sequences of these collateral responses appear to mediate, at least in part, the timing process required for reinforced performance on the DRL schedule.     

Electroencephalographic correlations of temporally spaced responding and avoidance behavior

This study was undertaken to determine whether any EEG changes occurred during a multiple schedule of reinforcement consisting of repeated periods of avoidance behavior, a time-out (TO) period, and a period of differential reinforcement of low rates (DRL). The two monkeys were permanently implanted with bipolar electrodes in various subcortical areas. The EEG was recorded from these electrodes for several weeks before any behavioral training was undertaken. Training was then begun in the multiple schedule, and EEG recordings were taken frequently throughout the training period. When a stable performance level was achieved, drugs (dl-amphetamine and sodium pentobarbital) were introduced which disrupted the DRL performance but had little or no effect on avoidance behavior. The EEG recordings continued throughout these drug studies.

The data indicate that the marked behavioral differences noted in each component of the schedule were not reflected in the EEG. However, the EEG did correlate with the general level of alertness of the animals during the various components. The drugs which differentially affected the behavior in the components of the schedule did not differentially affect the EEG, but they did change the brain-wave activity equally in all components. Finally, EEG changes correlated with the complex motor response of lever pressing were not observed.

     

Stimulus control of temporally spaced responding in second-order schedules

Second-order schedules consisting of sequences of component differential-reinforcement-of-low-rate schedules were used to investigate two different methods of introducing exteroceptive stimuli. When different exteroceptive stimuli were associated with each component (chained schedule), periods of pausing and lower response rates developed in the early components compared to the early components of the same sequence in the presence of a single exteroceptive stimulus (tandem schedule). When a brief stimulus change occurred at the completion of each component, response rates were higher in the early components compared to the tandem schedule. Changes in response rates in the early components were directly related to changes in reinforcement frequency in the terminal components produced by the two methods of presenting the exteroceptive stimuli.     

Overt “mediating” behavior during temporally spaced responding

A rat was trained on a schedule that programmed reinforcements only when a minimum waiting time between successive responses was exceeded (DRL schedule). It was observed to fill much of the pause between lever presses with a stereotyped behavioral chain: it would take its tail in its mouth and nibble it. This behavior was shown to be functionally related to the efficiency with which the subject spaced its responses. It is thought to have served as mediating behavior, providing discriminating stimuli for appropriate lever presses.     

Suppression of random-ratio and acceleration of temporally spaced responding by the same prereward stimulus in monkeys

A 1-min tone and light signal that preceded two free pellets of food suppressed the random-ratio responding of four rhesus monkeys, but accelerated the same subjects' responding on a differential-reinforcement-of-low-rate schedule in separate sessions. Both schedule-specific interactions occurred during the first presentations of the signal that previously had been paired with food outside the operant sessions. Thus, neither effect was adventitiously produced. In two subjects, both the direction and magnitude of the prereward change in differential-reinforcement-of-low-rate responding appeared related to baseline response rates: the more rapid the baseline responding, the less was the acceleration during the signal. Suppression and acceleration did not appear as dichotomous effects with separate parameters, but as related effects at least partly determined by the characteristics of the baseline operant performance.     

Fixed-ratio reinforcement of spaced responding

Responses by rats were reinforced with food under a second-order schedule involving fixed-ratio reinforcement of temporally spaced responses. Requirements of 20, 8, and 3 responses were examined. The typical characteristic of spaced responding was maintained under the ratio schedules: interresponse time distributions were similar to those typically seen, and were not noticeably affected by the ratio value. Comparison of total response rate, correct response rate, and accuracy showed correct response rate to be the most consistently affected by changes in the ratio value. Substantial evidence of schedule control was seen only for correct responses. Incorrect response records were erratic, but rates generally declined as reinforcement was approached. Correct response records were characterized by increasing rate as reinforcement was approached. It was suggested that the pattern of fixed-ratio performance revealed may be affected by the behavioral unit examined.     

Accurate and rapid reconditioning of spaced responding

The performance maintained by reinforcing responses only when they terminated interresponse times (IRTs) of 20 sec or greater (DRL schedule) was almost the same during the first session of reconditioning as before extinction. As few as two reinforcements accurately reinstated both the pre-extinction rate of responding and the function relating the duration of an IRT to its conditional probability of occurrence (IRTs/op).     

Some properties of spaced responding in pigeons

Pigeons exposed to a schedule which reinforces interresponse times (IRTs) longer than a given value (DRL schedule) eventually reach a stable pattern of responding which is shown to be a function both of the DRL value and of previous experience with other DRL values. On any given DRL schedule, the stable performance of most pigeons which have been previously exposed to a variety of such schedules, shows an IRT distribution with median equal to the DRL value. For DRL values longer than about 30 sec, however, the median IRT falls short of the DRL value; this failure of adjustment to longer values appears to be a species characteristic of pigeons. The function relating reinforcement rate to 1/DRL value is also shown to be approximately linear over the same range, with variable slope (less than 45°) and a downturn in the vicinity of DRL 30.     

Reinforcement of spaced responding in a simultaneous discrimination

Pigeons were exposed to three stimuli simultaneously with responses reinforced according to differential-reinforcement-of-low-rate schedules. Responses to one stimulus (the positive stimulus) that were spaced appropriately resulted in food presentation. The variables manipulated were the time parameter of the schedule (5, 10, 20, 30 sec) and the consequences of responding to the other two stimuli (the negative stimuli). The percentage of the total responses that occurred to each stimulus was independent of the schedule value but was dependent on the consequences of responding to the negative stimuli. If responses to both reset the schedule timer, responding was confined largely to the positive stimulus. If responses to neither had scheduled effects, the birds were more likely to respond to those stimuli. Responding to one negative stimulus could be selectively attenuated by having responses to that stimulus alone reset the timer. With the schedule time value held constant, the absolute rate of responding to the positive stimulus was either stable or decreased with maintained exposure; it did not change as a function of increases or decreases in responding to the negative stimuli. Rather than interacting and affecting each other, responses to the three stimuli were controlled independently by their relation to reinforcement. There was no evidence that responses to the negative stimuli mediated the spacing of responses to the positive stimulus.     

Effects of timeout on spaced responding in pigeons

Three pigeons were trained under a differential-reinforcement-of-low-rate schedule of 20 sec, and then exposed to a schedule under which responses terminating interresponse times less than 20 sec produced timeout and responses terminating interresponse times greater than 20 sec produced reinforcement. Response-produced timeouts selectively decreased the probability of short interresponse times and thereby produced a higher frequency of reinforcement. The suppressive effect of timeout was independent of timeout duration, with timeouts of 5, 10, or 20 sec. Similar effects were found when the minimum interresponse time that could be terminated by response-produced reinforcement was increased to 30 sec. The suppressive effects of timeout on responding maintained by these schedules were similar to previous reports in which responding was punished with electric shock.     

A computer analysis of serial interactions in spaced responding

Serial dependencies in interresponse times were studied by means of a digital computer. In monkeys exposed to a DRL 20-sec schedule of reinforcement, serial interactions appeared at all stages of training. Early in training the serial effects consisted of trains of relatively long interresponse times interspersed among trains of relatively short ones. Later on, the serial effects appeared to be characterized by a tendency to drift up and down in long wavelength periods around the minimum interval required for reinforcement. After training to a point at which most interresponse times produced reinforcement, serial effects of a still more subtle nature appeared. These effects were made apparent by autocorrelation and power spectrum methods and consisted of both long-term and extremely short-term fluctuations in interresponse times.     

Effects of d-amphetamine and chlordiazepoxide on spaced responding in pigeons

The effects of d-amphetamine and chlordiazepoxide were studied in pigeons on performance (1) under a schedule that reinforced responses on a key (food key) if they were more than 20 sec apart, (2) under the same schedule when responses also were required on a collateral key during the interresponse time on the food key, and (3) under the same schedule when responses were required on a collateral key during the interresponse time on the food key and collateral-key responses could produce a stimulus correlated with the availability of food. Under all three spaced-responding schedules, d-amphetamine and chlordiazepoxide at low dose levels slightly increased the frequency of short interresponse times on the food key for about half the birds, and either did not affect the interresponse time patterns of the other birds, or lengthened the durations slightly. At higher dose levels, d-amphetamine and chlordiazepoxide increased the frequency of long interresponse times or abolished responding in all birds. Changes in the pattern of interresponse times on the food key did not seem to depend on changes in the rate or pattern of collateral-key responses.     

Facilitation and suppression of responding under temporally defined schedules of negative reinforcement

Two parameters for scheduling aversive stimulus presentations were studied systematically by specifying concurrent and independent probabilities of electric shock delivery for the occurrence and for the non-occurrence of a lever-press response. After preliminary training on a free-operant shock-avoidance schedule, 16 rhesus monkeys were divided into four groups, each group being assigned one shock distribution on a continuum from fixed interval to a widely ranging variable interval. Within groups, each subject was successively exposed to three values of response-dependence of shock delivery on a continuum from response-independent shock to complete dependence of shock on response occurrence (“punishment”). Introduction of shock following avoidance training produced initial response facilitation followed by suppression. Responding during both the facilitation and suppression periods was maximal when the shock schedule was periodic and response independent. Responding decreased as the inter-shock intervals were made more variable across groups, and as shock delivery was made increasingly response dependent within individual subjects.     

Temporally spaced responding by pigeons: development and effects of deprivation and extinction

In the first five or six sessions on a DRL 20-sec schedule of reinforcement there developed a stable performance characterized by a relatively constant conditional probability of occurrence (IRTs/op) of interresponse times (IRTs) of durations greater than 5 or 6 sec. Extinction and the level of deprivation changed both the overall rate of responding and the form of the function relating the duration of an IRT to its value of IRTs/op. The value of IRTs/op decreased more rapidly for short than for longer IRTs, resulting in the emergence of a finer discrimination of IRT duration.     

The allocation of time to temporally defined behaviors: responding during stimulus generalization.

In one stimulus condition, reinforcement depended on rats holding a lever for a duration having both minimum and maximum boundaries. During a second light intensity, reinforcement was not available for some rats; for others, reinforcement depended on a second response duration requirement. Generalization test stimuli controlled the same response durations found during training, and the amount of time allocated to a given response duration depended on the proximity of the test stimulus to the training stimulus which controlled that particular duration. The results indicated that a gradient of stimulus control does not reflect an underlying continuous change in responding, but is a result of the mixing of responses previously controlled by stimuli present during conditioning.     

Punishment of responding under schedules of stimulus-shock termination: effects of d-amphetamine and pentobarbital.

Responding maintained in squirrel monkeys under 5-min fixes-interval schedules of either food presentation or termination of a visual stimulus associated with electric-shock delivery was suppressed by presenting an electric shock for every thirtieth response (punishment). In monkeys responding under the schedule of food presentation, d-amphetamine sulfate only further decreased punished responding, and pentobarbital sodium markedly increased punished responding, as expected from previous reports. In monkeys responding under the schedule of stimulus-shock termination, however, the effects of the two drugs were opposite: d-amphetamine markedly increased punished responding, whereas pentobarbital only decreased responding. Thus, the effects of these drugs on punished responding were different depending on the type of event maintaining responding. These and previous results indicate that it may be misleading and inaccurate to speak of the effects of drugs on "punished responding" as though punishment were a unitary phenomenon. As with any behavior, the effects of drugs and other interventions on punished responding cannot be accurately characterized independently of the precise conditions under which the behavior occurs.