Effect of reinforcement duration on fixed-interval responding

Five different reinforcement durations occurred randomly within each session on fixed interval 60-sec. Postreinforcement pause was directly related (and “running” rate inversely related) to the duration of reinforcement initiating each fixed interval.     

Differential reinforcement of vocal duration

The effects of differential reinforcement of vocal duration were examined in a series of experiments in which each of 28 subjects (Ss) emitted a vowel whenever a light was flashed. In the first phase of each experiment, a penny was dispensed after each of 20 responses. In the second and subsequent phases, only those responses whose durations exceeded a criterion were reinforced; when 10 successive reinforcements were presented, one phase was terminated and the next begun. The criterion for reinforcement in each phase was determined by a different schedule in each of six experiments; it ranged from 80 to 120 per cent of the mean duration of the 10 terminal responses in the prior phase. Differential reinforcement effected a large and systematic change in the duration of vocal responses as long as the responses selected for reinforcement had a sufficiently high probability of occurrence. This requirement was formulated as the difference between the criterion duration and the mean duration of the terminal responses in the prior phase, divided by their standard deviation. This statistic, named the shaping index, was correlated with the number of responses emitted before each phase was terminated. It was found to be large whenever the shaping process failed. Many Ss failed to tact the reinforcement contingency despite marked changes in their vocal behavior and extensive probing by a questionnaire, administered at the end of each session.     

Effects of response rate, reinforcement frequency, and the duration of a stimulus preceding response-independent food

Food-reinforced key pecking in the pigeon was maintained under a four-component multiple schedule. In two components, responding was maintained at high rates under a random-ratio schedule. In the other two components, responding was maintained at low rates under a schedule that specified a minimum interresponse time. For both high and low response rates, one of the schedule components was associated with a high reinforcement frequency and the other components with a lower reinforcement frequency. During performance under these schedules, a stimulus terminated by access to response-independent food was periodically presented. The duration of this pre-food stimulus was 5, 30, 60, or 120 sec. Changes in rate of key pecking during the pre-food stimulus were systematically related to baseline response rate and the duration of the stimulus. Both high and low response rates were increased during the 5-sec stimulus. At longer stimulus durations, low response rates were unaffected and high response rates were decreased during the stimulus. For two of three pigeons, high response rates maintained under a lower frequency of reinforcement tended to be decreased more than high response rates maintained under a higher reinforcement frequency. In general, the magnitude of decrease in high response rates was inversely related to the duration of the pre-food stimulus.     

Conditioned reinforcement as a function of duration of stimulus

Pigeons were provided with three keys. Pecking the center key produced grain on a schedule that alternated at unpredictable times between a variable-interval component and extinction. On concurrent variable-interval schedules, pecking either side key produced a stimulus associated with the variable-interval component on the center key provided that said schedule was currently in effect. The independent variable was the length of time this stimulus remained on the keys. Pecking one side key produced the stimulus for 27 seconds, whereas the duration produced by pecking the other key varied for successive blocks of sessions. For the first four birds, the values tested were 3, 9, 27, and 81 seconds. For the second group, numbering three birds, the values tested were 1, 3, 9, and 27 seconds. The dependent variable was the proportion of total side key pecks that occurred on the variable key. For all birds, the function was positive in slope and negative in acceleration. This finding supports a formulation that ascribes the maintenance of observing responses in a normal setting to the fact that the subject exposes itself to the positive discriminative stimulus for a longer mean duration than it does to the negative stimulus.     

Short-component multiple schedules: effects of relative reinforcement duration

Pigeons were exposed to multiple variable-interval 2-min variable-interval 2-min schedules of food presentation in which relative duration of food presentation was manipulated. When components alternated every 5 sec and were scheduled on separate response keys, relative response rates closely matched relative reinforcement duration in three of four pigeons. On the other hand, relative response rates were insensitive to relative reinforcement duration when components scheduled on a single response key alternated every 5 sec, and when components scheduled on separate response keys alternated every 2 min. Thus, both rapid alternation and spatial separation of components were necessary to produce approximate matching of relative responding to relative reinforcement duration. This finding contrasts with previous findings that only rapid component alternation is necessary for matching when relative rate of reinforcement is manipulated.     

The discrimination of differences in duration of a thermal reinforcement

The sides of a contour square appear successively when context lines are presented one before and one after the square. As the interstimulus interval (ISI) between lines and square is increased to 140 msec, reports of succession increase. If the sides of the square are presented successively and interpart interval (IPI) varied by a single staircase method in order to determine a succession threshold, the average magnitude of this threshold varies as a function of ISI, decreasing up to 140 msec lSI. Each S, however, shows a maximum decrease in threshold between 80 and 120 msec. Reports of this illusion are not dependent on beta movement of either context lines or square.     

Reduction of shock duration as negative reinforcement in free-operant avoidance

Rats were trained on a free-operant procedure in which shock duration was controlled by responses within a limited range of interresponse times. Shocks of 1.6-mA intensity occurred randomly with average density of 10 shocks per minute. As long as interresponse times were 15 seconds or less, any shocks received were at the briefer of two durations (.3 second). Whenever interresponse times exceeded 15 seconds, any shocks received were at the longer duration (1.0 second). For six of eight animals, avoidance responding developed quickly and reached levels of better than 90%. Four yoked animals stopped responding within the first few sessions. Shock duration reduction without change in shock probability or intensity was sufficient for the acquisition and maintenance of avoidance responding.     

Signal duration and suppression of operant responding by free reinforcement

Pigeons were trained on a VI (variable interval) schedule of food presentation with a superimposed schedule of response-independent food. Substantial suppression of the operant response rate occurred when the free food was presented without a signal. When the free food was preceded by a short (4 sec) signal, the degree of suppression was similar to that with unsignaled free food. But when the signal was lengthened to 12 sec, the degree of suppression was substantially reduced. Experiment 3 assessed the effect of signal duration using a baseline schedule of delayed reinforcement, in which contingent reinforcers were themselves preceded by a signal. The signal preceding the free reinforcers was then either the same as or different from this contingent signal. Signal duration effects occurred only when the two types of signals were different. These differences as a function of signal duration have implications for both “context-blocking” and “comparator” interpretations of the effects of noncontingent reinforcement in both Pavlovian and operant procedures.     

Intermittent reinforcement of a continuous response

Konorski showed that when a go/no-go procedure was used, sound quality discriminations were rapidly acquired and sound location discriminations were slowly acquired. These findings have been interpreted as a general constraint on the acquisition of auditory discriminations (quality-location effect). However, experiments carried out within an evolutionary framework (Harrison, 1984) have shown that the rate of acquisition of sound location discriminations varies widely as a function of the inclusion or exclusion of naturalistic features. These data suggest that Konorski's findings were a function of the special conditions of the experiments. The first purpose of the present experiments was to assess whether rats showed the effects noted by Konorski when studied under similar conditions. The second purpose was to study the effect of manipulating two natural features (novelty and stimulus-response adjacency) to assess whether the acquisition rates of quality and location discriminations could be greatly modified or made approximately equal, or both. When a go/no-go procedure was used and the other conditions were similar to those of Konorski, rats acquired a quality discrimination but did not acquire a location discrimination. However, when the S+ or S- were presented through a closely adjacent speaker, the sound location discrimination was acquired as rapidly as the quality discrimination. Finally, preexposing the animal to either S+ or S- retarded the rate of or prevented the acquisition of the quality discrimination. The experiments showed that the quality-location effect was determined primarily by the conditions used in Konorski's experiments, and that the effect is not a general constraint on learning.     

Effects on preference of reinforcement delay, number of reinforcers, and terminal-link duration

Three experiments used concurrent-chains procedures to examine the effects of reinforcement delay, number of reinforcers, and terminal-link duration on preference. In Condition 30 of Experiment 1, food was delivered after 30 seconds in each 150-second terminal link, with four additional food deliveries occurring at 30-second intervals in one of the links. In Condition 5, food was delivered after 5 seconds in each 25-second terminal link, and the four additional reinforcers were delivered at 5-second intervals. Preferences for the multiple-food chain were greater in Condition 30. In Experiment 2, the terminal link(s) providing only one reinforcer terminated immediately after delivery of the reinforcer. Preferences for the multiple-food chain were smaller than in Experiment 1. In Condition 5 of Experiment 3, food was delivered after 5, 75, 100, 125, and 150 seconds in one 150-second link and after 5 seconds in the other. Condition 50 differed only in that the first (or only) reinforcer in each link was delivered after 50 seconds instead of after 5 seconds. Preferences for the multiple-food chain were greater in Condition 50. Results of Experiments 1 and 2 do not correspond to results obtained by Moore (1979).     

Concurrent schedules of response-independent reinforcement: duration of a reinforcing stimulus

Presentations of grain to three pigeons were determined by two response-independent schedules. Interpresentation intervals varied with a mean interval of 1.5 min for each schedule. Both were concurrently operative, but grain was presented by one only when the chamber was illuminated with blue light and by the other only during amber illumination. A response on a white key, the only key in the chamber, alternated the stimulus conditions and the effective schedule. Grain presentation durations associated with the illumination conditions were varied from 1.5 to 4.5 sec. The proportion of the total session time spent in an illumination condition closely approximated the relative grain presentation duration provided in that illumination. For two of the birds, the proportion of the total number of grain presentations obtained in an illumination condition was an increasng function of the presentation duration in that illumination.     

Interresponse time duration in fixed-interval schedules of reinforcement: control by ordinal position and time since reinforcement

The times between each of the first thirteen responses after reinforcement (the first twelve interresponse times) were determined for two pigeons whose pecking was reinforced on fixed-interval schedules of food reinforcement ranging from 0.5 min to 5 min. These interresponse times were classified with respect to their ordinal position in the sequence of responses and with respect to the time since the preceding reinforcement at which the initiating response occurred. The median interresponse time durations were essentially constant after the sixth response after reinforcement regardless of the time at which the interresponse time was initiated. The durations of the first few interresponse times after reinforcement decreased as the number of preceding responses increased and as the time since the preceding reinforcement increased.     

Differential reinforcement of interresponse times with controlled probability of reinforcement per response

Pigeons were presented food after interresponse times (IRTs) longer or shorter than a fixed percentage of their most recent IRTs. This procedure controlled probability of reinforcement per response while still allowing different classes of IRTs to be reinforced differentially. Support was found for IRT-reinforcement theory in that response rates were determined by the degree and direction of differential reinforcement of IRTs, but were relatively independent of probability of reinforcement per response and of the length of the control system's IRT memory. Stimulus control of these differential response rates was also demonstrated.     

Contrast, component duration, and the following schedule of reinforcement

Experiment 1 investigated component duration in a four-component multiple schedule designed to separate the effects of the preceding schedule of reinforcement from those of the following schedule of reinforcement. The preceding schedule in the preceding component had no consistent effect regardless of component duration. The following schedule was a powerful determinant of behavior, however, with higher response rates resulting from a following period of extinction. Moreover, the effect of the following schedule was greater with short component durations, which strongly suggests that component-duration effects previously found with multiple schedules are due generally to variation in the degree of control by the following schedule. Experiments 2--4 investigated the basis of the effects of the following schedule. In some situations differential responding in the following component was the controlling variable, but in others it was differential reinforcement in the following component. The results are consistent with the view that response rate during a stimulus is inversely related to the "value" of the following component, where the calculation of value must include both primary and conditioned reinforcement.     

Effects of magnitude of food reinforcement on free-operant response rates

In Experiment 1 rats were trained to press a lever on a variable-ratio schedule of food presentation and were then exposed to progressively increasing magnitudes of food reinforcement. Response running rates (rates exclusive of the postreinforcement pause) were found to increase as a function of increasing reinforcement magnitudes. The effect of reinforcement magnitude on response rates inclusive of the postreinforcement pause, however, was less pronounced. Increases in the magnitude of reinforcement were also found to increase the length of the postreinforcement pause. Rats in Experiment 2 were trained to respond on a chained differential-reinforcement-of-low-rate variable-ratio schedule, and were exposed to increasing magnitudes of reinforcement as in Experiment 1. Response running rates increased in the variable-ratio component but decreased in the other component of the schedule. The results are discussed with reference to incentive accounts of reinforcement and the action of reinforcement on the response units generated by the operative contingencies.     

Behavioral contrast and response independent reinforcement

Four pigeons received pre-training that included presentation of the reinforcer independently of behavior and then baseline training on a variable-interval schedule of reinforcement. With the introduction of a multiple schedule, in which the first stimulus was associated with a response contingent and a second stimulus with a response independent, 1-min variable-interval schedule, a reduction in response rate was obtained in the second component, which was not accompanied by a behavioral contrast effect in the first component. A further three pigeons were given the same pre-training and baseline training before the introduction of an otherwise identical multiple schedule, in which no reinforcement occured in the second component. Behavioral contrast was obtained from all three subjects. The results indicated that under conditions of constant reinforcement density a reduction in responding is not a sufficient condition for the occurrence of behavioral contrast.