Effects of a DRL contingency added to a fixed-interval reinforcement schedule

Following 30 days of reinforcement for the bar press response of two white rats on 30-sec fixed-interval (FI), a DRL component was added so that a minimal interresponse time (IRT) for the reinforced response, in addition to the FI variable, was necessary for reinforcement. Marked control over response rate by the superimposed DRL requirement was demonstrated by an inverse hyperbolic function as the DRL component was increased from 1 to 24 sec within the constant 30-sec FI interval. Interresponse time and post-reinforcement (post-S^R) “break” distributions taken at one experimental point (DRL = 24 sec) suggested that a more precise temporal discrimination was initiated by an S^R than by a response, since the relative frequency of a sequence of two reinforced responses appeared greater than that of a sequence of a non-reinforced response followed by a reinforced one. This latter finding was confirmed with new animals in a follow-up experiment employing a conventional 24-sec DRL schedule.     

Observing behavior in monkeys (Macaca arctoides): Support for the information hypothesis

A single lever, discrete-trials observing procedure was used with stumptailed monkeys (Macaca arctoides). Lever-presses during a trial produced colored key lights (IS⁺ and IS^?) which signaled whether the trial would end with response-independent food or without food. During the baseline period, both IS⁺ and IS^? were produced on a variable-interval (VI) 15-sec schedule which began operating at the onset of the trial. The two experimental conditions involved a combination of this VI schedule and a DRL schedule. In one of these conditions, only a response that both met the VI requirement and was preceded by at least 6 sec of nonresponding could produce IS^? on nonfood trials, while the schedule for IS⁺ on food trials remained VI 15 sec. In the other experimental condition, the schedules for producing the two stimuli were the reverse. All subjects eventually learned to produce either IS⁺ or IS^? on the combined VI-DRL schedule. These data support an information hypothesis of observing in monkeys and contrast with data from pigeons which support a conditioned reinforcement hypothesis.     

Marc Richelle (1930–2021) and the Study of Temporal Regulation of Behaviour in Animals

This article discusses the contribution of Marc Richelle to the study of temporal regulation of behaviour in animals. Richelle was a pioneer of behavioural pharmacology in Europe in the 1960s, and some of his early pharmacological experiments, particular those involving chlordiazepoxide, are discussed. Richelle frequently tested drug effects on performance on fixed-interval (FI) and differential reinforcement of low rate (DRL) schedules. Much of his later work, conducted with Helga Lejeune, involved cross-species comparisons of performance on FI and DRL, and often focused on potential differences between “timing competence” and “timing performance”. His work provided an unrivalled body of research on operant behaviour in different species, involving research on animals as different as cats and fish. Much of the work was reviewed in Richelle and Lejeune’s 1980 book Time in Animal Behaviour, which contained particularly influential accounts of collateral behaviour and inter-species comparisons.     

Some properties of spaced responding in pigeons

Pigeons exposed to a schedule which reinforces interresponse times (IRTs) longer than a given value (DRL schedule) eventually reach a stable pattern of responding which is shown to be a function both of the DRL value and of previous experience with other DRL values. On any given DRL schedule, the stable performance of most pigeons which have been previously exposed to a variety of such schedules, shows an IRT distribution with median equal to the DRL value. For DRL values longer than about 30 sec, however, the median IRT falls short of the DRL value; this failure of adjustment to longer values appears to be a species characteristic of pigeons. The function relating reinforcement rate to 1/DRL value is also shown to be approximately linear over the same range, with variable slope (less than 45°) and a downturn in the vicinity of DRL 30.     

Competitive fixed-interval performance in humans

Two persons responded in the same session in separate cubicles, but under a single schedule of reinforcement. Each time reinforcement was programmed, only the first response to occur, that is, the response of only one of the subjects, was reinforced. “Competitive” behavior that developed under these conditions was examined in three experiments. In Experiment 1 subjects responded under fixed-interval (FI) 30-s, 60-s, and 90-s schedules of reinforcement. Under the competition condition, relative to baseline conditions, the response rates were higher and the pattern was “break-and-run.” In Experiment 2, subjects were exposed first to a conventional FI schedule and then to an FI competition schedule. Next, they were trained to respond under either a differential-reinforcement-of-low-rate (DRL) or fixed-ratio (FR) schedule, and finally, the initial FI competition condition was reinstated. In this second exposure to the FI competition procedure, DRL subjects responded at lower rates than were emitted during the initial exposure to that condition and FR subjects responded at higher rates. For all subjects, however, responding gradually returned to the break-and-run pattern that had occurred during the first FI competition condition. Experiment 3 assessed potential variables contributing to the effects of the competitive FI contingencies during Experiments 1 and 2. Subjects were exposed to FI schedules where (a) probability of reinforcement at completion of each fixed interval was varied, or (b) a limited hold was in effect for reinforcement. Only under the limited hold was responding similar to that observed in previous experiments.     

Reinforcement of mediating behavior on a spaced-responding schedule

This paper describes a procedure for gaining experimental control over mediating behavior on a spaced-responding schedule of food reinforcement. Three rats, food-deprived, were trained on a DRL 16 sec schedule of food reinforcement. Then, a concurrent schedule of food reinforcement was introduced on a second (mediating) lever, such that the first response to occur on the mediating lever, after the DRL interval had timed out, was reinforced with food, as was the next response to occur on the DRL lever. Reinforcement via the mediating lever became a discriminative stimulus for a food-reinforcement opportunity on the DRL lever. Next, food reinforcement for the mediating behavior was replaced by a conditioned reinforcer consisting of onset of a buzzer signaling timing-out of the DRL interval. Under these conditions, chaining of behavior on the two levers was strong, and timing on the DRL lever was more accurate than under ordinary DRL conditions. As the DRL requirement was lengthened from 16 sec to 24 sec to 60 sec, mediating behavior weakened slightly. When the inter-response requirement for food reinforcement on the DRL lever was made shorter than the inter-response requirement for conditioned reinforcement on the mediating lever, the mediating behavior extinguished. Performance in the experiment was analyzed into a four-component chain, and the factors contributing to the maintenance, and later extinction, of mediating behavior are discussed.     

Punishment: the interactive effects of delay and intensity of shock

A discrete-trial punishment procedure, with rats, was used to examine how delay-of-shock intervals of 0 to 28 sec and shock intensity interact to decrease the frequency and increase the latency of a positively reinforced response. For delay-of-shock intervals of 0, 7, 14, and 28 sec, there was a range of shock intensities, for some subjects, over which the punishing effect of shock was an increasing, monotonic function of shock intensity. For other subjects this transition was abrupt. Functions relating response frequency and latency measures to shock intensity were displaced toward higher values on the shock intensity axis with an increase in delay-of-shock interval. The effects of “gradual” and “abrupt” introduction to “severe” shock, as well as re-exposure to previously used shock intensities, were examined under both the immediate and delay-of-shock conditions. With delay-of-shock intervals of 7, 14, or 28 sec, shock intensities of approximately 0.50 milliamperes or greater were necessary to decrease substantially the number and increase the latency of the lever-pressing response. For the immediate punishment group this intensity was approximately 0.20 ma. These facts were related to Annau and Kamin's (1961) conditioned emotional response experiment in which a shock intensity of 0.49 ma or greater was required to suppress the rate of a positively reinforced response.     

Complex learning and information processing by pigeons: a critical analysis

Three models of conditional discrimination learning by pigeons are described: stimulus configuration learning, the multiple-rule model, and concept learning. A review of the literature reveals that true concept learning is not characteristic of the behavior of pigeons in matching-to-sample, oddity-from-sample, or symbolic matching studies. Instead, pigeons learn a set of sample-specific S^D rules. Transfer of the discrimination to novel stimuli, at least along the hue dimension, is predicted by a “coding hypothesis”, which holds that pigeons make a unique, but usually unobserved response, R₁, to each sample, and that the comparison stimulus chosen depends on which R₁ was emitted in the presence of the sample. Convincing evidence is found that pigeons do code sample hues, but there is little evidence that allows one to infer that the “coding event” must have behavioral properties. Parameters of the conditional discrimination paradigm are identified, and it is shown that by appropriate parametric manipulation, a variety of analogous tasks may be generated for both human and animal subjects. The tasks make possible the comparative study of complex learning, attention, memory, and information processing, with the added advantage that behavior processes may be compared systematically across tasks.     

Rapid acquisition of discrete-trial lever-press avoidance: effects of signal-shock interval

Acquisition of discrete-trial lever-press avoidance learning was studied in three experiments. Experiment I compared a new training procedure, which produces rates of lever-press avoidance learning comparable to those obtained in shuttle boxes, with a “conventional”, less efficient training procedure. A factorial design was used to compare continuous versus intermittent shock and a long-variable versus a short-fixed signal-shock interval. Learning was best in the groups trained with the long and variable interval and poorest in those trained with the short and fixed interval. Type of shock had no effect. Experiment II separated the effects of duration from those of variability of the signal-shock interval. Fixed and variable intervals of 10 and 60 sec were tested and duration was the only significant factor. Experiment III addressed the effect of the differential opportunity to avoid provided by long signal-shock intervals by varying this interval from 10 to 60 sec in 10-sec steps. Only the 10-sec group showed slow acquisition relative to the others. Analysis of avoidance response latencies showed that the distributions for all groups were positively skewed and that skewness increased with increasing duration of the signal-shock interval. At intervals longer than 20 sec, the animals made progressively less use of their increased opportunity to respond. The data do not support the opportunity-to-respond interpretation of the effects of duration of signal-shock interval and suggest that some type of inhibitory process may block lever-press avoidance learning at intervals as short as 10 sec. The significance of these findings for species-specific defense reaction and preparedness theories was emphasized.     

Discrimination learning as a function of stimulus location along an auditory intensity continuum

Eight groups of rats were trained on an auditory intensity discrimination in which the discriminative stimuli were separated by 10 decibels (db). Four pairs of stimuli were selected from different regions along a 60–100 db (SPL) intensity continuum. Counterpart groups were trained on each stimulus pair, with the relative intensity positions of the reinforced stimulus (S^D) and the non-reinforced stimulus (S^Δ) reversed for the two groups. Discrimination acquisition curves were compared to determine whether stimuli separated by equal logarithmic units were of comparable “difficulty”, and to determine the relative effectiveness of an S^D serving as the more versus less intense member of a stimulus pair. It was concluded that: (1) When S^D is the more intense, auditory intensities of constant logarithmic separation are graded in “difficulty” along the intensity continuum; high intensity discriminative stimuli are most readily discriminated. When S^Δ is the more intense, this graded effect is not evident. (2) For a given continuum location, discrimination is inferior when S^Δ is the more intense. This effect is most pronounced at the high intensity end of the continuum and is chiefly attributable to differences in the rate of S^Δ responding.     

Pigeons' performance under differential reinforcement of low rate schedules depends upon the operant

Pigeons were reinforced with grain for pecking a key or depressing a foot treadle according to differential reinforcement of low rate (DRL) schedules. Birds which depressed a treadle performed efficiently on DRL schedules as high as DRL 35-sec; while birds reinforced for keypecking showed low efficiency under DRL 14-sec. While treadle pressing and keypecking differ along a number of dimensions (including force requirement of the operant and differences in temporal distributions of responses), the present results are consistent with an interpretation based on differences in the degree to which these two responses are elicited by periodic presentations of food.     

Differential reinforcement of low rates performance by impulsive and reflective children

Third-grade boys classified as either cognitively impulsive or reflective were reinforced for key pressing according to a DRL (differential reinforcement of low rates) 6-sec schedule of reinforcement. Half of each group received instructions about the behavioral requirements for obtaining reinforcements. Prior to DRL training, impulsive Ss showed a low probability of key press responding at long interresponse time (IRT) intervals while reflective Ss exhibited an equal probability of terminating either short or long IRTs. During training and in the absence of instructions, impulsives exhibited a less precise temporal discrimination, characterized by a greater predominance of response bursts (0–2 sec IRTs) following reinforcements, than reflective Ss. While impulsive and reflective Ss displayed similar frequencies of collateral behavior between successively reinforced responses, impulsives engaged in the reinforced response more frequently and tended (p < .08) to obtain fewer reinforcements. Instructions served to enhance the DRL performance.     

Effects of dl-amphetamine under concurrent VI DRL reinforcement

Three adult, food-deprived rats were given IP injections of dl-amphetamine sulfate under DRL and concurrent VI DRL reinforcement schedules. The drug results were as follows.(1) The IRT distributions of DRL responses shifted to the left, but some temporal discrimination remained. (2) The IRT distributions of VI responses shifted slightly to the left. (3) The distinguishing characteristics of VI and DRL IRT distributions were preserved. (4) The frequency distribution of number of VI responses between two consecutive DRL responses was relatively unaffected. (5) Over-all response rates on the two components of the concurrent schedules increased more or less proportionately.These findings imply that the primary behavioral effect of dl-amphetamine was a motor excitatory one. The drug's disruption of timing behavior was not due to a derangement of internal timing mechanisms, nor to interference with the topography or pattern of behavior. Rather, it might be a secondary result of the accelerated emission of overt behavior patterns mediating the temporal spacing of DRL bar presses.     

Delayed alternation in the pigeon

Pigeons were studied in a delayed-response task requiring alternation of key pecks on two response keys. Blackouts of from 1 to 10 sec intervened between successive choices on the two keys.

The following results were obtained: (1) Birds performed at well above chance accuracy on all the delays tested. Accuracy was generally lowest at 1- and 10-sec delays. (2) Overt postural orientations during the delay interval appeared to mediate accurate key-pecking behavior. (3) The shape of the delay vs. accuracy function was discussed in terms of the possibly confounding influences of (a) stimulus “trace” variables, and (b) aversive effects of the time outs produced by incorrect responding.

     

Spaced responding and choice: a preliminary analysis

Pigeons were exposed to reinforcement both for short (2 < IRT < 3 sec) and long (10 < IRT < 11 sec) interresponse times. They developed bimodal interresponse-time distributions, which were decomposable into two independent component distributions under the control of the short and long contingencies respectively. The birds' allocation of responses between these two distributions was determined by a simple power-law relationship between reinforcement ratios, and response ratios derived from the component distributions. Comparison between this situation and concurrent choice situations raises the possibility that the power-law relation between ratios may be a more general law of choice than the matching of relative frequencies (probabilities).     

The effect of acceleration on food-reinforced DRL and FR

Performance on DRL 10 sec and FR 5 was studied after exposure to acceleration. After four rats, two on each of the above schedules, had stabilized they were exposed to 5 hr of acceleration at 5 G immediately before daily experimental sessions. Food intake was also studied in rats given access to food daily in their home cages and exposed to acceleration immediately before the free-feeding session. Weight gain of free-feeding animals and reinforcement intake of experimental animals dropped after acceleration. Over-all response rate on the FR was depressed markedly by acceleration but local response rates did not appear to be affected. IRT distributions of DRL sessions after acceleration were markedly shifted toward the long intervals. A sequential plot of IRTs on acceleration days showed an altered, but relatively stable, temporal patterning of responses followed by an abrupt return to the normal baseline toward the end of the session.     

The role of instrumental responding and contiguity of stimuli in the development of infant secondary reinforcement

Ten-month-old infants received contingent pairings of a tone (T+) and food reinforcer. Groups S^r and S^D received the food on an FI 23-sec schedule for target touching, the former group receiving T+ immediately after the response and 1.5 sec prior to food and the latter group receiving T+ at the end of the intertrial interval. Group SC received food reinforcers 1.5 sec after T+ with no response required. A second tone (Tⁿ) was heard by all groups once during each intertrial interval, at randomly determined points. All groups subsequently were given a spatial discrimination task, receiving T+ for one alternative and Tⁿ for the other. Group S^r gave significantly more responses for T+ than for Tⁿ, but neither of the other two groups produced a superiority for T+. Thus, both contiguity with a primary reinforcer and the presence of an operant during training appear to be necessary for a neutral signal to acquire the ability to enhance responding.     

Differential sample response schedules in the acquisition of conditional discriminations by pigeons

Pigeons were trained on four matching-to-sample tasks with various schedule requirements in effect on the sample key. Differential sample-schedule requirements (a differential-reinforcement-of-low-rates of 3 sec in the presence of one sample and a fixed-ratio 16 in the presence of the other) produced rapid rates of acquisition that did not differ across tasks. Nondifferential sample-schedule requirements (fixed-ratio 1, fixed-ratio 16 or a differential-reinforcement-of-low-rates of 3 sec in the presence of both samples) produced slower rates of acquisition, which depended on the difficulty of the discriminations between samples and between comparisons. Patterns of stimulus and position preferences were influenced both by the comparison stimuli in each task and by the sample-schedule requirements. Detailed analyses of acquisition revealed frequent instances of complete differential sample control of comparison responding at intermediate levels of overall “accuracy”.     

Effects of hunger and VI value on VI pacing

In two experiments, each involving four rats, responses preceded by an inter-response time between 8 and 10 sec in duration were intermittently reinforced. In Experiment I, final performance was compared under two hunger levels, while the frequency of reinforcement was held constant by a VI 5 schedule. In Experiment II, hunger was held constant and VI 3 was compared with VI 8. Both hunger and frequency of reinforcement increased the over-all rate of response, but the exact effects of these operations on temporal discrimination were different for different rats. Usually, a peak “response probability” (IRTs/Op ratio) was obtained 8 to 10 sec after the preceding response, indicating adaptation to the reinforcement contingency, but in some cases this peak was about 2 sec earlier. One rat exhibited unusually pronounced bursting which seemed to alternate with adaptive temporally spaced responding. Prolonged pauses, observable in the cumulative records, particularly following reinforcement, were attributed to the fact that inter-response times greater than 10 sec were not reinforced, so that as the interval of time since the preceding response became discriminably greater than 10 sec, the probability of a response became small.