Differential reinforcement of interresponse times with controlled probability of reinforcement per response

Pigeons were presented food after interresponse times (IRTs) longer or shorter than a fixed percentage of their most recent IRTs. This procedure controlled probability of reinforcement per response while still allowing different classes of IRTs to be reinforced differentially. Support was found for IRT-reinforcement theory in that response rates were determined by the degree and direction of differential reinforcement of IRTs, but were relatively independent of probability of reinforcement per response and of the length of the control system's IRT memory. Stimulus control of these differential response rates was also demonstrated.     

Temporal regulation of children with autism spectrum disorder exposed to a differential-reinforcement-of low-rates schedule

This study investigated temporal adjustment of children with autism spectrum disorder under a differential-reinforcement-of-low-rates (DRL) schedule. Sixteen participants, aged 3.2 to 7 years, were exposed to two conditions, DRL 5 s and DRL 20 s. Children participated in 7 sessions in each condition, except for 1 participant who attained the adjustment criteria in the DRL 5-s schedule. Temporal adjustment was measured with the proportion of reinforced interresponse times (IRTs) and the mean IRT. The operant response was a press on a touch screen and the reinforcers were cartoons. IQ and receptive language were measured prior to the DRL sessions. Results showed that the mean proportion of reinforced IRTs was slightly higher in the DRL 5-s schedule. The mean IRT was above the IRT requirement in both conditions. However, substantial individual variability was observed. Children with higher IQ and receptive language scores presented a greater proportion of reinforced IRTs in both conditions. Moreover, participants who adjusted their responses to the DRL 5-s schedule were more likely to adjust responding to the DRL 20-s schedule. This suggests that some children might be more sensitive to reinforcement contingencies than others. This study points at future research in the field of timing in children.     

Fixed-ratio performance with and without a postreinforcement timeout

Pigeons pecked a key, producing food reinforcement on fixed-ratio (FR) schedules requiring 50, 100, or 150 responses. In each session, 30-second timeouts were inserted before a random half of the FR trials, whereas the other trials began immediately after reinforcement. In general, preratio pauses were shorter on trials preceded by timeouts. On these trials, the probability of a first response tended to be highest in the first 20 seconds of the trials, suggesting that the shorter pauses were the result of transient behavioral contrast. Direct observations and analyses of interresponse times (IRTs) after the preratio pause indicated that IRTs could be grouped into three categories: (1) IRTs of about .1 second, which were produced by small head movements in the vicinity of the key; (2) IRTs of about .3 second, which were produced by distinct pecking motions; and (3) IRTs greater than .5 second, which were accompanied by pausing or movements away from the key. At all ratio sizes, as a subject progressed through a trial, the probability of a long IRT decreased, whereas the probability of an intermediate IRT usually increased at first and then decreased. The probability of a short IRT increased monotonically across a trial. The results show that responding changes systematically as a subject progresses through a ratio on an FR schedule. Some characteristics of performance varied as functions of the absolute size of the response requirement, whereas others appeared to depend on the relative location within a ratio (i.e., the proportion of the ratio completed at a given moment).     

AGGRESSION AS POSITIVE REINFORCEMENT IN MICE UNDER VARIOUS RATIO‐ AND TIME‐BASED REINFORCEMENT SCHEDULES

There is evidence suggesting aggression may be a positive reinforcer in many species. However, only a few studies have examined the characteristics of aggression as a positive reinforcer in mice. Four types of reinforcement schedules were examined in the current experiment using male Swiss CFW albino mice in a resident—intruder model of aggression as a positive reinforcer. A nose poke response on an operant conditioning panel was reinforced under fixed‐ratio (FR 8), fixed‐interval (FI 5‐min), progressive ratio (PR 2), or differential reinforcement of low rate behavior reinforcement schedules (DRL 40‐s and DRL 80‐s). In the FR conditions, nose pokes were maintained by aggression and extinguished when the aggression contingency was removed. There were long postreinforcement pauses followed by bursts of responses with short interresponse times (IRTs). In the FI conditions, nose pokes were maintained by aggression, occurred more frequently as the interval elapsed, and extinguished when the contingency was removed. In the PR conditions, nose pokes were maintained by aggression, postreinforcement pauses increased as the ratio requirement increased, and responding was extinguished when the aggression contingency was removed. In the DRL conditions, the nose poke rate decreased, while the proportional distributions of IRTs and postreinforcement pauses shifted toward longer durations as the DRL interval increased. However, most responses occurred before the minimum IRT interval elapsed, suggesting weak temporal control of behavior. Overall, the findings suggest aggression can be a positive reinforcer for nose poke responses in mice on ratio‐ and time‐based reinforcement schedules.     

A comparison of procedures for programming noncontingent reinforcement schedules

We compared two methods for programming and thinning noncontingent reinforcement (NCR) schedules during the treatment of self-injurious behavior (SIB). The participants were 3 individuals who had been diagnosed with mental retardation. Results of functional analyses indicated that all participants' SIB was maintained by positive reinforcement (i.e., access to attention or food). Following baseline, the effects of two NCR schedule-thinning procedures were compared in multielement designs. One schedule (fixed increment) was initially set at fixed-time 10-s reinforcer deliveries and was also thinned according to fixed-time intervals. The other schedule (adjusting IRT) was initially determined by participants' baseline interresponse times (IRTs) for SIB and was thinned based on IRTs observed during subsequent treatment sessions. Results indicated that both schedules were effective in initially reducing SIB and in maintaining response suppression as the schedules were thinned.     

Temporally spaced responding by pigeons: development and effects of deprivation and extinction

In the first five or six sessions on a DRL 20-sec schedule of reinforcement there developed a stable performance characterized by a relatively constant conditional probability of occurrence (IRTs/op) of interresponse times (IRTs) of durations greater than 5 or 6 sec. Extinction and the level of deprivation changed both the overall rate of responding and the form of the function relating the duration of an IRT to its value of IRTs/op. The value of IRTs/op decreased more rapidly for short than for longer IRTs, resulting in the emergence of a finer discrimination of IRT duration.     

IRT-stimulus contingencies in chained schedules: implications for the concept of conditioned reinforcement

Two experiments with pigeons investigated the effects of contingencies between interresponse times (IRTs) and the transitions between the components of 2- and 4-component chained schedules (Experiments 1 and 2, respectively). The probability of component transitions varied directly with the most recent (Lag 0) IRT in some experimental conditions and with the 4th (Lag 4) IRT preceding the most recent one in others. Mean component durations were constant across conditions, so the reinforcing effect of stimulus change was dissociated from that of delay to food. IRTs were longer in the Lag-0 than in the Lag-4 conditions of both experiments, thus demonstrating that stimulus change functioned as a reinforcer. In the Lag-0 conditions of Experiment 2, the Component-1 IRTs increased more than the Component-2 IRTs, which in turn increased more than the Component-3 IRTs. This finding runs counter to the conditioned-positive-reinforcement account of chained-schedule responding, which holds that the reinforcing effect of stimulus change should vary in strength as an inverse function of the delay to the unconditioned reinforcer at the end of the chain because conditioned reinforcement is due to first- or higher-order classical conditioning. Therefore, we present other possible explanations for this effect.     

Operant behavior and the galvanic skin potential under DRL schedule

Motor and galvanic skin potential (GSP) activity were investigated during the conditioning, extinction, and reconditioning of motor responses under a differential reinforcement of low rate (DRL) schedule of reinforcement. Interresponse time (IRT) distributions for motor responses during conditioning and reconditioning gradually stabilized at a peak just beyond the minimal IRT required for reinforcement. Few unreinforced motor responses and "bursts" of motor responses were observed during conditioning and reconditioning. Relative to conditioning and reconditioning, extinction effected larger IRTs and smaller GSP amplitudes. GSP amplitudes were greater for unreinforced than for reinforced motor responses during conditioning and reconditioning. However, GSP amplitudes associated with the unreinforced extinction responses were smaller than either the reinforced or unreinforced responses during conditioning and reconditioning.     

Effects of a delay-reinforcement procedure on performance under IRT>t schedules

Water-deprived rats were studied under a compound schedule that prescribed that responses terminating interresponse times (IRTs) greater than a fixed value t₁ (IRT > t₁ component schedule) initiated a delay of reinforcement interval t₂, at the end of which water was presented if the subject did not respond ( > t₂ component schedule). If the subject responded before the t₂ interval elapsed, the IRT > t₁ component schedule was re-initiated and water was not presented. The IRT > t₁ and > t₂ component schedules were not differentially correlated with distinctive stimuli. Rate of responding during the IRT > t₁ component decreased as a function of the value of t₂. The magnitude of the decreases in response rate appeared to be proportional to the subject's rate under the IRT > t schedule with no delay of reinforcement (t₂ = 0 sec). The effects were independent of the parameter value of the IRT > t₁ component schedule and of the rate of reinforcement. The results suggested that “efficiency” of performance under IRT > t schedules can be increased by appropriately arranging brief delays of reinforcement.     

Response rate viewed as engagement bouts: effects of relative reinforcement and schedule type.

The rate of a reinforced response is conceptualized as a composite of engagement bouts (visits) and responding during visits. Part I of this paper describes a method for estimating the rate of visit initiations and the average number of responses per visit from log survivor plots: the proportion) of interresponse times (IRTs) longer than some elapsed time (log scale) plotted as a function of elapsed time. In Part 2 the method is applied to IRT distributions from rats that obtained food pellets by nose poking a lighted key under various multiple schedules of reinforcement. As expected, total response rate increased as a function of (a) increasing the rate of reinforcement (i.e., variable-interval [VI] 4 min vs. VI 1 mi), (b) increasing the amount of the reinforcer (one food pellet vs. four pellets), (c) increasing the percentage of reinforcers that were contingent on nose poking (25% vs. 100%), and (d) requiring additional responses after the end of the VI schedule (i.e., adding a tandem variable-ratio [VR] 9 requirement). The first three of these variables (relative reinforcement) increased the visit-initiation rate. The tandem VR, in contrast, increased the number of responses per visit. Thus, variables that have similar effects on total response rate can be differentiated based on their effects on the componemts of response rate.     

Differential reinforcement of low rates performance by impulsive and reflective children

Third-grade boys classified as either cognitively impulsive or reflective were reinforced for key pressing according to a DRL (differential reinforcement of low rates) 6-sec schedule of reinforcement. Half of each group received instructions about the behavioral requirements for obtaining reinforcements. Prior to DRL training, impulsive Ss showed a low probability of key press responding at long interresponse time (IRT) intervals while reflective Ss exhibited an equal probability of terminating either short or long IRTs. During training and in the absence of instructions, impulsives exhibited a less precise temporal discrimination, characterized by a greater predominance of response bursts (0–2 sec IRTs) following reinforcements, than reflective Ss. While impulsive and reflective Ss displayed similar frequencies of collateral behavior between successively reinforced responses, impulsives engaged in the reinforced response more frequently and tended (p < .08) to obtain fewer reinforcements. Instructions served to enhance the DRL performance.     

The microanalysis of fixed-interval responding

The fixed-interval schedule of reinforcement is one of the more widely studied schedules in the experimental analysis of behavior and is also a common baseline for behavior pharmacology. Despite many intensive studies, the controlling variables and the pattern of behavior engendered are not well understood. The present study examined the microstructure and superstructure of the behavior engendered by a fixed-interval 5- and a fixed-interval 15-minute schedule of food reinforcement in the pigeon. Analysis of performance typical of fixed-interval responding indicated that the scalloped pattern does not result from smooth acceleration in responding, but, rather, from renewed pausing early in the interval. Individual interresponse-time (IRT) analyses provided no evidence of acceleration. There was a strong indication of alternation in shorter-longer IRTs, but these shorter-longer IRTs did not occur at random, reflecting instead a sequential dependency in successive IRTs. Furthermore, early in the interval there was a high relative frequency of short IRTs. Such a pattern of early pauses and short IRTs does not suggest behavior typical of reinforced responding as exemplified by the pattern found near the end of the interval. Thus, behavior from clearly scalloped performance can be classified into three states: postreinforcement pause, interim behavior, and terminal behavior.     

Performance on a fixed-ratio schedule with correlated amount of reward

Four rats were trained to bar press on FR 9 TO 30 sec. They were reinforced with a large or small amount of water according to whether their final IRT was long or short respectively. Four control rats always received the small amount of reinforcement. The control animals produced the high rates of responding typical of fixed-ratio performance. The experimental animals, with one exception, developed superstitious behavior and maintained slow responding throughout the ratio. However, some features of the results pointed to a persistent influence of the factors which favor short IRTs.     

Performance on ratio and interval schedules with matched reinforcement rates

In the first study, rats were trained to pull a chain on a schedule (RPI) that regulates the probability of reinforcement to maintain a constant average reinforcement rate without differentially reinforcing long inter-response times (IRTs). Although the response rate was sensitive to the overall rate of reinforcement, performance was unaffected by variations between 1 and 50 in the IRT memory size used in programming the schedule. In the second study, two groups of animals performed on either a random-interval (RI) schedule or a RPI schedule, with reinforcement rates determined by those generated by a third group performing on a random ratio (RR) 20 schedule. The RI group responded at a lower rate than the RPI group, which, in turn, responded at a lower rate than the RR group, even though the three groups experienced comparable rates of reinforcement. The fact that the RPI group responded at a lower rate than the RR group suggests that the standard response rate difference observed between ratio and interval schedules, which have been matched for reinforcement rate, cannot be attributed solely to the fact that conventional interval schedules differentially reinforce long IRTs.     

Discrimination and emission of temporal intervals by pigeons

Because the frequency distribution of IRTs showed little or no control by a DRL schedule, the schedule was modified so that the pigeon's behavior after each IRT would indicate whether or not it had discriminated the duration of the IRT. After every two pecks on a red key, the key changed to blue for 30 sec. Then it automatically became red again. Pecks on the blue key were reinforced with food on a VI schedule only when the preceding IRT on the red key had been longer than 18 sec. The birds did not selectively emit longer IRTs on the red key: the value of IRTs/op did not increase with IRT duration. However, they did discriminate the duration of the IRT emitted on the red key: the rate of pecking on the blue key was an increasing function of the duration of the preceding IRT on the red key.     

On the primacy of molecular processes in determining response rates under variable-ratio and variable-interval schedules

This study focused on variables that may account for response-rate differences under variable-ratio (VR) and variable-interval (VI) schedules of reinforcement. Four rats were exposed to VR, VI, tandem VI differential-reinforcement-of-high-rate, regulated-probability-interval, and negative-feedback schedules of reinforcement that provided the same rate of reinforcement. Response rates were higher under the VR schedule than the VI schedule, and the rates on all other schedules approximated those under the VR schedule. The median reinforced interresponse time (IRT) under the VI schedule was longer than for the other schedules. Thus, differences in reinforced IRTs correlated with differences in response rate, an outcome suggestive of the molecular control of response rate. This conclusion was complemented by the additional finding that the differences in molar reinforcement-feedback functions had little discernible impact on responding.     

Multiple and concurrent schedule performance: independence from concurrent and successive schedule contexts

Six pigeons were trained on multiple variable-interval schedules and performance was measured in the presence or absence of another variable-interval schedule (the common schedule) arranged concurrently with both components. Manipulations included varying the rate of reinforcement on the common schedule, leaving the common schedule unchanged while the components of the multiple schedule were varied, varying the multiple schedule components in the absence of the common schedule, and varying one component of the multiple schedule while the other component and the common schedule were unchanged. The normal rate-increasing and rate-decreasing effects of reinforcement rate increase were found, except that changing one multiple schedule component did not affect the response rate in the successively available common schedule component. Both concurrent and multiple schedule performance undermatched obtained reinforcement-rate ratios, but the degree of undermatching in multiple schedules was reliably greater. Allocation of responses between multiple schedule components was unaffected by the concurrent availability of reinforcement, and allocation of responses between concurrent schedules was unaffected by the successive availability of different reinforcement rates.     

The Isolation of Motivational, Motoric, and Schedule Effects on Operant Performance: A Modeling Approach

Dissociating motoric and motivational effects of pharmacological manipulations on operant behavior is a substantial challenge. To address this problem, we applied a response‐bout analysis to data from rats trained to lever press for sucrose on variable‐interval (VI) schedules of reinforcement. Motoric, motivational, and schedule factors (effort requirement, deprivation level, and schedule requirements, respectively) were manipulated. Bout analysis found that interresponse times (IRTs) were described by a mixture of two exponential distributions, one characterizing IRTs within response bouts, another characterizing intervals between bouts. Increasing effort requirement lengthened the shortest IRT (the refractory period between responses). Adding a ratio requirement increased the length and density of response bouts. Both manipulations also decreased the bout‐initiation rate. In contrast, food deprivation only increased the bout‐initiation rate. Changes in the distribution of IRTs over time showed that responses during extinction were also emitted in bouts, and that the decrease in response rate was primarily due to progressively longer intervals between bouts. Taken together, these results suggest that changes in the refractory period indicate motoric effects, whereas selective alterations in bout initiation rate indicate incentive‐motivational effects. These findings support the use of response‐bout analyses to identify the influence of pharmacological manipulations on processes underlying operant performance.     

The effects of morphine on the production and discrimination of interresponse times

Recent experiments suggest that the effects of drugs of abuse on the discrimination of the passage of time may differ for experimenter-imposed and subject-produced events. The current experiment examined this suggestion by determining the effects of morphine on the discrimination of interresponse times (IRTs). Pigeons pecked a center key on a random-interval 20-s schedule of matching-to-sample trials. Once the interval had timed out, a choice trial randomly followed either a short (2- to 3-s) or long (6- to 9-s) IRT on the center key. Pecking the side key lit one color produced food after a short IRT, and pecking the side key lit the other color produced food after a long IRT. Two experimental phases differed in the functional role of the different key colors. Under control conditions, the IRT distributions had two modes, one at the lower bound of the short category and a smaller one at the lower bound of the long category. Pigeons accurately categorized the duration of the IRTs: One key color was pecked following short IRTs and the other key color was pecked following long IRTs. Morphine flattened the IRT distribution and reduced the accuracy of categorizing IRTs. Categorization of long IRTs was particularly disrupted. Morphine did not produce overestimation of time as assessed by the production or categorization of IRTs. These results are similar to those obtained previously for the effects of morphine on the discrimination of the duration of experimenter-imposed events.     

Preference and resistance to change with constant- and variable-duration terminal links: independence of reinforcement rate and magnitude

Pigeons responded in a three-component multiple concurrent-chains procedure in which the variable-interval reinforcement schedules were the same across components but magnitudes differed across components. The terminal links were arranged either as a variable delay followed by presentation of a reinforcer ("variable duration") or as a fixed period of access to the schedule during which a variable number of reinforcers could be earned ("constant duration"). Relative reinforcement rate was varied parametrically across both types of conditions. After baseline training in each condition, resistance to change of terminal-link responding was assessed by delivering food during the initial links according to a variable-time schedule. Both preference and resistance to change were more sensitive to reinforcement-rate differences in the constant-duration conditions. Sensitivities of preference and resistance to change to relative reinforcement rate did not change depending on relative reinforcement magnitude. Taken together, these results confirm and extend those of prior studies, and suggest that reinforcement rate and magnitude combine additively to determine preference and resistance to change. A single structural relation linking preference and resistance to change describes all the data from this and several related studies.