Effects of a delay-reinforcement procedure on performance under IRT>t schedules

Water-deprived rats were studied under a compound schedule that prescribed that responses terminating interresponse times (IRTs) greater than a fixed value t₁ (IRT > t₁ component schedule) initiated a delay of reinforcement interval t₂, at the end of which water was presented if the subject did not respond ( > t₂ component schedule). If the subject responded before the t₂ interval elapsed, the IRT > t₁ component schedule was re-initiated and water was not presented. The IRT > t₁ and > t₂ component schedules were not differentially correlated with distinctive stimuli. Rate of responding during the IRT > t₁ component decreased as a function of the value of t₂. The magnitude of the decreases in response rate appeared to be proportional to the subject's rate under the IRT > t schedule with no delay of reinforcement (t₂ = 0 sec). The effects were independent of the parameter value of the IRT > t₁ component schedule and of the rate of reinforcement. The results suggested that “efficiency” of performance under IRT > t schedules can be increased by appropriately arranging brief delays of reinforcement.     

Inter-reinforcement times for the bar-pressing response of white rats on two DRL schedules,

A knowledge of inter-reinforcement times (IS^RT) seems of interest because, among other reasons, they bear on the variable of “reinforcement density” in behavior theory. IS^RT data were here recorded for bar-pressing responses by two white rats working on DRL 10 sec and DRL 40 sec. Sample inter-response time (IRT) distributions and cumulative records were taken simultaneously for comparison purposes. IS^RT sequences and distributions have certain characteristics that urge further investigation.     

Effects of a DRL contingency added to a fixed-interval reinforcement schedule

Following 30 days of reinforcement for the bar press response of two white rats on 30-sec fixed-interval (FI), a DRL component was added so that a minimal interresponse time (IRT) for the reinforced response, in addition to the FI variable, was necessary for reinforcement. Marked control over response rate by the superimposed DRL requirement was demonstrated by an inverse hyperbolic function as the DRL component was increased from 1 to 24 sec within the constant 30-sec FI interval. Interresponse time and post-reinforcement (post-S^R) “break” distributions taken at one experimental point (DRL = 24 sec) suggested that a more precise temporal discrimination was initiated by an S^R than by a response, since the relative frequency of a sequence of two reinforced responses appeared greater than that of a sequence of a non-reinforced response followed by a reinforced one. This latter finding was confirmed with new animals in a follow-up experiment employing a conventional 24-sec DRL schedule.     

Temporal proximity and reinforcement sensitivity in multiple schedules

Distributions of reinforcers between two components of multiple variable-interval schedules were varied over a number of conditions. Sensitivity to reinforcement, measured by the exponent of the power function relating ratios of responses in the two components to ratios of reinforcers obtained in the components, did not differ between conditions with 15-s or 60-s component durations. The failure to demonstrate the “short-component effect,” where sensitivity is high for short components, was consistent with reanalysis of previous data. With 60-s components, sensitivity to reinforcement decreased systematically with time since component alternation, and was higher in the first 15-s subinterval of the 60-s component than for the component whose total duration was 15 s. Varying component duration and sampling behavior at different times since component transition may not be equivalent ways of examining the effects of average temporal distance between components.     

Towards an empirical calculus of reinforcement value

Only one of two keys reinforces the subject with food. This key can assume one of two colors, each associated with a different fixed-ratio schedule for obtaining reinforcement. The function of the second key is to permit the animal to switch from the long schedule to the short schedule. If the difference between the ratio schedules is large enough, a preference for the shorter schedule is demonstrable. A quantitative index of preference is obtained as follows: each time the animal switches to the shorter schedule, the number of pecks required to produce the next switch is increased. As the “ante” on the switching key increases, the effective difference between the two ratio schedules decreases. After each food reinforcement, when the bird is exposed to the choice-situation, it takes longer before the bird switches again. This is used to “titrate” the bird's preference. If it does not switch within x sec, the progressively increasing ratio schedule of the switching key is decreased. A specific value, in terms of a rather specific number of responses the bird settles at on the choice key, is obtained. This equilibrium is employed as a dependent variable. Several variables of which it is a function are explored.     

Matching to relative reinforcement frequency in multiple schedules with a short component duration

Three pigeons performed on two-component multiple variable-interval variable-interval schedules of reinforcement. There were two independent variables: component duration and the relative frequency of reinforcement in a component. The component duration, which was always the same in both components, was varied over experimental conditions from 2 to 180 sec. Over these conditions, the relative frequency of reinforcement in a component was either 0.2 or 0.8 (±0.03). As the component duration was shortened, the relative frequency of responding in a component approached a value equal to the relative frequency of reinforcement in that component. When the relative frequency of reinforcement was varied over conditions in which the component duration was fixed at 5 sec, the relative frequency of responding in a component closely approximated the relative frequency of reinforcement in that component. That is, the familiar matching relationship, obtained previously only with concurrent schedules, was obtained in multiple schedules with a short component duration.     

Choice for aperiodic versus periodic ratio schedules: A comparison of concurrent and concurrent-chain procedures

Choice between mixed-ratio schedules, consisting of equiprobable ratios of 1 and 99 responses per reinforcement, and fixed-ratio schedules of food reinforcement was assessed by two commonly used procedures: concurrent schedules and concurrent-chains schedules. Rats were trained under concurrent fixed-ratio mixed-ratio schedules, in which both ratio schedules were simultaneously available, and under a concurrent-chains schedule, in which access to one of the mutually exclusive ratio schedules comprising the terminal links was contingent on a single “choice” response. The distribution of responses between the two ratio schedules was taken as the choice proportion under the concurrent procedure, and the distribution of “choice” responses was taken as the choice proportion under the concurrent-chains procedure. Seven of eight rats displayed systematic choice; of those, each displayed nearly exclusive choice for fixed-ratio 35 to the mixed-ratio schedule under the concurrent procedure, but each displayed nearly exclusive choice for the mixed-ratio schedule to fixed-ratio 35 under the concurrent-chains procedure. Thus, preference for a fixed or a mixed schedule of reinforcement depended on the procedure used to assess preference.     

Signal duration and suppression of operant responding by free reinforcement

Pigeons were trained on a VI (variable interval) schedule of food presentation with a superimposed schedule of response-independent food. Substantial suppression of the operant response rate occurred when the free food was presented without a signal. When the free food was preceded by a short (4 sec) signal, the degree of suppression was similar to that with unsignaled free food. But when the signal was lengthened to 12 sec, the degree of suppression was substantially reduced. Experiment 3 assessed the effect of signal duration using a baseline schedule of delayed reinforcement, in which contingent reinforcers were themselves preceded by a signal. The signal preceding the free reinforcers was then either the same as or different from this contingent signal. Signal duration effects occurred only when the two types of signals were different. These differences as a function of signal duration have implications for both “context-blocking” and “comparator” interpretations of the effects of noncontingent reinforcement in both Pavlovian and operant procedures.     

Choice and number of reinforcers

Pigeons were exposed to the concurrent-chains procedure in two experiments designed to investigate the effects of unequal numbers of reinforcers on choice. In Experiment 1, the pigeons were indifferent between long and short durations of access to variable-interval schedules of equal reinforcement density, but preferred a short high-density terminal link over a longer, lower density terminal link, even though in both sets of comparisons there were many more reinforcers per cycle in the longer terminal link. In Experiment 2, the pigeons preferred five reinforcers, the first of which was available after 30 sec, over a single reinforcer available at 30 sec, but only when the local interval between successive reinforcers was short. The pigeons were indifferent when this local interval was sufficiently long. The pigeons' behavior appeared to be under the control of local terminal-link variables, such as the intervals to the first reinforcer and between successive reinforcers, and was not well described in terms of transformed delays of reinforcement or reductions in average delay to reinforcement.     

Differential reinforcement of low rates performance by impulsive and reflective children

Third-grade boys classified as either cognitively impulsive or reflective were reinforced for key pressing according to a DRL (differential reinforcement of low rates) 6-sec schedule of reinforcement. Half of each group received instructions about the behavioral requirements for obtaining reinforcements. Prior to DRL training, impulsive Ss showed a low probability of key press responding at long interresponse time (IRT) intervals while reflective Ss exhibited an equal probability of terminating either short or long IRTs. During training and in the absence of instructions, impulsives exhibited a less precise temporal discrimination, characterized by a greater predominance of response bursts (0–2 sec IRTs) following reinforcements, than reflective Ss. While impulsive and reflective Ss displayed similar frequencies of collateral behavior between successively reinforced responses, impulsives engaged in the reinforced response more frequently and tended (p < .08) to obtain fewer reinforcements. Instructions served to enhance the DRL performance.     

Some properties of spaced responding in pigeons

Pigeons exposed to a schedule which reinforces interresponse times (IRTs) longer than a given value (DRL schedule) eventually reach a stable pattern of responding which is shown to be a function both of the DRL value and of previous experience with other DRL values. On any given DRL schedule, the stable performance of most pigeons which have been previously exposed to a variety of such schedules, shows an IRT distribution with median equal to the DRL value. For DRL values longer than about 30 sec, however, the median IRT falls short of the DRL value; this failure of adjustment to longer values appears to be a species characteristic of pigeons. The function relating reinforcement rate to 1/DRL value is also shown to be approximately linear over the same range, with variable slope (less than 45°) and a downturn in the vicinity of DRL 30.     

On the exponent in the "generalized" matching equation

A power function equation between ratios of behavior and ratios of reinforcement rates has been called a generalized form of Herrnstein's (1961) matching law, even without a formal relationship having been shown between the two equations. The present work uses a functional relationship to prove that when ratios of reinforcement are not equivalent to ratios of behavior, and the transform leading to this inequality is consistent for every pair of reinforcement rates, the result is a power function relationship between response and reinforcement ratios. The label “generalized matching equation” for the power function equation is thus validated formally.     

The effects of morphine on the production and discrimination of interresponse times

Recent experiments suggest that the effects of drugs of abuse on the discrimination of the passage of time may differ for experimenter-imposed and subject-produced events. The current experiment examined this suggestion by determining the effects of morphine on the discrimination of interresponse times (IRTs). Pigeons pecked a center key on a random-interval 20-s schedule of matching-to-sample trials. Once the interval had timed out, a choice trial randomly followed either a short (2- to 3-s) or long (6- to 9-s) IRT on the center key. Pecking the side key lit one color produced food after a short IRT, and pecking the side key lit the other color produced food after a long IRT. Two experimental phases differed in the functional role of the different key colors. Under control conditions, the IRT distributions had two modes, one at the lower bound of the short category and a smaller one at the lower bound of the long category. Pigeons accurately categorized the duration of the IRTs: One key color was pecked following short IRTs and the other key color was pecked following long IRTs. Morphine flattened the IRT distribution and reduced the accuracy of categorizing IRTs. Categorization of long IRTs was particularly disrupted. Morphine did not produce overestimation of time as assessed by the production or categorization of IRTs. These results are similar to those obtained previously for the effects of morphine on the discrimination of the duration of experimenter-imposed events.     

Performance on a fixed-ratio schedule with correlated amount of reward

Four rats were trained to bar press on FR 9 TO 30 sec. They were reinforced with a large or small amount of water according to whether their final IRT was long or short respectively. Four control rats always received the small amount of reinforcement. The control animals produced the high rates of responding typical of fixed-ratio performance. The experimental animals, with one exception, developed superstitious behavior and maintained slow responding throughout the ratio. However, some features of the results pointed to a persistent influence of the factors which favor short IRTs.     

Effects of hunger and VI value on VI pacing

In two experiments, each involving four rats, responses preceded by an inter-response time between 8 and 10 sec in duration were intermittently reinforced. In Experiment I, final performance was compared under two hunger levels, while the frequency of reinforcement was held constant by a VI 5 schedule. In Experiment II, hunger was held constant and VI 3 was compared with VI 8. Both hunger and frequency of reinforcement increased the over-all rate of response, but the exact effects of these operations on temporal discrimination were different for different rats. Usually, a peak “response probability” (IRTs/Op ratio) was obtained 8 to 10 sec after the preceding response, indicating adaptation to the reinforcement contingency, but in some cases this peak was about 2 sec earlier. One rat exhibited unusually pronounced bursting which seemed to alternate with adaptive temporally spaced responding. Prolonged pauses, observable in the cumulative records, particularly following reinforcement, were attributed to the fact that inter-response times greater than 10 sec were not reinforced, so that as the interval of time since the preceding response became discriminably greater than 10 sec, the probability of a response became small.     

Response bias and the discrimination of stimulus duration

Pigeons discriminated stimulus duration in a psychophysical choice situation. Following presentation of any duration from a set of short duration (11 to 15 sec), responses on a red key were reinforced intermittently. Following presentation of any duration from a set of long durations (16 to 22 sec), responses on a green key were reinforced intermittently. Relative reinforcement rates were manipulated for choice responses across conditions. As relative reinforcement rates were varied, psychometric functions showed shifts in green-key responses at all durations. A signal-detection analysis showed that sensitivity remained roughly constant across conditions while response bias changed as a function of changes in relative reinforcement rate. Relative error rates tended to match relative reinforcement rates.     

Choice behavior on discrete trials: a demonstration of the occurrence of a response strategy

Three pairs of pigeons were trained to peck at two keys presented simultaneoulsy in discrete trials with intertrial intervals of 1, 22, or 120 sec. Left-key responses incremented the probability of reinforcement for the first right-key response and, conversely, right-key responses incremented the probability of reinforcement for the first left-key response. In terms of relative response rates, it was found that all birds' choices were described by a momentary maximizing strategy, but this fact was not reflected in the detailed sequential statistics for birds with the longer (22 or 120 sec) intertrial intervals. It was hypothesized that choice behavior, in general, may be accurately described by a momentary maximizing sequence, but that prior failures to demonstrate this were due to “errors” in executing the momentary maximizing sequence. These misappropriated responses, which are hypothesized to be randomly distributed among the responses defining the momentary maximizing sequence, caused successive choices to appear to be statistically independent when, in fact, they were not.     

Magnitude and frequency of reinforcement and frequencies of interresponse times

The relative magnitude and relative frequency of reinforcement for two concurrent interresponse times (1.5 to 2.5 sec and 3.5 to 4.5 sec) were simultaneously varied in an experiment in which pigeons obtained grain by pecking on a single key. Visual discriminative stimuli accompanied the two time intervals in which reinforcements were arranged by a one-minute variable-interval schedule. The resulting interresponse times of each of three pigeons fell into two groups; "short" (1.0 to 2.5 sec) and "long" (3.0 to 4.5 sec). Steady-state relative frequencies of these interresponse times were orderly functions of both reinforcement variables. The combined effects of both independent variables were well summarized by a linear function of one variable, relative access to food. Unlike corresponding two-key concurrent variable-interval schedules, the present schedule did not produce an equality between the relative frequency of an operant and either the relative magnitude or the relative frequency of reinforcement of that operant. A tentative account is provided for this difference between one-key and two-key functions.     

Stimulus control and delayed reinforcement

Pigeons were tested for generalization along the line-orientation dimension, after being trained on various two-component multiple schedules. The first component contained either a variable-interval 1-min schedule of immediate reinforcement or an extinction schedule and was associated with a plain white key (S1). The second component contained a variable-interval 1-min schedule of delayed reinforcement and was associated with a black line on a white background (S2). The major results showed that (a) decremental gradients were obtained around the stimulus associated with the delayed reinforcement component when S1 was associated with extinction, but that incremental gradients were obtained when S1 was associated with immediate reinforcement, (b) the subjects' pretraining did not affect the generalization gradients if sufficient training on the terminal multiple schedule was provided, and (c) changing the S1 schedule from immediate reinforcement to extinction produced behavioral contrast if reinforcement was delayed for 10 sec during S2, but not if it was delayed for 20 sec.     

Molecular maximizing characterizes choice on Vaughan's (1981) procedure

Pigeons keypecked on a two-key procedure in which their choice ratios during one time period determined the reinforcement rates assigned to each key during the next period (Vaughan, 1981). During each of four phases, which differed in the reinforcement rates they provided for different choice ratios, the duration of these periods was four minutes, duplicating one condition from Vaughan's study. During the other four phases, these periods lasted six seconds. When these periods were long, the results were similar to Vaughan's and appeared compatible with melioration theory. But when these periods were short, the data were consistent with molecular maximizing (see Silberberg & Ziriax, 1982) and were incompatible with melioration, molar maximizing, and matching. In a simulation, stat birds following a molecular-maximizing algorithm responded on the short- and long-period conditions of this experiment. When the time periods lasted four minutes, the results were similar to Vaughan's and to the results of the four-minute conditions of this study; when the time periods lasted six seconds, the choice data were similar to the data from real subjects for the six-second conditions. Thus, a molecular-maximizing response rule generated choice data comparable to those from the short- and long-period conditions of this experiment. These data show that, among extant accounts, choice on the Vaughan procedure is most compatible with molecular maximizing.