Effect of reinforcement duration on fixed-interval responding

Five different reinforcement durations occurred randomly within each session on fixed interval 60-sec. Postreinforcement pause was directly related (and “running” rate inversely related) to the duration of reinforcement initiating each fixed interval.     

Differential reinforcement of vocal duration

The effects of differential reinforcement of vocal duration were examined in a series of experiments in which each of 28 subjects (Ss) emitted a vowel whenever a light was flashed. In the first phase of each experiment, a penny was dispensed after each of 20 responses. In the second and subsequent phases, only those responses whose durations exceeded a criterion were reinforced; when 10 successive reinforcements were presented, one phase was terminated and the next begun. The criterion for reinforcement in each phase was determined by a different schedule in each of six experiments; it ranged from 80 to 120 per cent of the mean duration of the 10 terminal responses in the prior phase. Differential reinforcement effected a large and systematic change in the duration of vocal responses as long as the responses selected for reinforcement had a sufficiently high probability of occurrence. This requirement was formulated as the difference between the criterion duration and the mean duration of the terminal responses in the prior phase, divided by their standard deviation. This statistic, named the shaping index, was correlated with the number of responses emitted before each phase was terminated. It was found to be large whenever the shaping process failed. Many Ss failed to tact the reinforcement contingency despite marked changes in their vocal behavior and extensive probing by a questionnaire, administered at the end of each session.     

Effects of reinforcement history on response rate and response pattern in periodic reinforcement

Several researchers have suggested that conditioning history may have long-term effects on fixed-interval performances of rats. To test this idea and to identify possible factors involved in temporal control development, groups of rats initially were exposed to different reinforcement schedules: continuous, fixed-time, and random-interval. Afterwards, half of the rats in each group were studied on a fixed-interval 30-s schedule of reinforcement and the other half on a fixed-interval 90-s schedule of reinforcement. No evidence of long-term effects attributable to conditioning history on either response output or response patterning was found; history effects were transitory. Different tendencies in trajectory across sessions were observed for measures of early and late responding within the interreinforcer interval, suggesting that temporal control is the result of two separate processes: one involved in response output and the other in time allocation of responding and not responding.     

Effects of response rate, reinforcement frequency, and the duration of a stimulus preceding response-independent food

Food-reinforced key pecking in the pigeon was maintained under a four-component multiple schedule. In two components, responding was maintained at high rates under a random-ratio schedule. In the other two components, responding was maintained at low rates under a schedule that specified a minimum interresponse time. For both high and low response rates, one of the schedule components was associated with a high reinforcement frequency and the other components with a lower reinforcement frequency. During performance under these schedules, a stimulus terminated by access to response-independent food was periodically presented. The duration of this pre-food stimulus was 5, 30, 60, or 120 sec. Changes in rate of key pecking during the pre-food stimulus were systematically related to baseline response rate and the duration of the stimulus. Both high and low response rates were increased during the 5-sec stimulus. At longer stimulus durations, low response rates were unaffected and high response rates were decreased during the stimulus. For two of three pigeons, high response rates maintained under a lower frequency of reinforcement tended to be decreased more than high response rates maintained under a higher reinforcement frequency. In general, the magnitude of decrease in high response rates was inversely related to the duration of the pre-food stimulus.     

Effect of signaled reinforcement on response variability

Three experiments examined the effect of signaling reinforcement on rats' lever pressing on contingencies that reinforced variable responding to extend the exploration of signaled reinforcement to a schedule that has previously not been examined in this respect. In Experiment 1, rats responding on a lag-8 variability schedule with signaled reinforcement displayed greater levels of variability (U values) than rats on the same schedule lacking a reinforcement signal. In Experiment 2, rats responding on a differential reinforcement of least frequent responses schedule also displayed greater operant variability with a signal for reinforcement compared with rats without a reinforcement signal. In Experiment 3, a reinforcement signal decreased the variability of a response sequence when there was no variability requirement. These results offer empirical corroboration that operant variability responds to manipulations in the same manner as do other forms of operant response and that a reinforcement signal facilitates the emission of the required operant.     

Conditioned reinforcement as a function of duration of stimulus

Pigeons were provided with three keys. Pecking the center key produced grain on a schedule that alternated at unpredictable times between a variable-interval component and extinction. On concurrent variable-interval schedules, pecking either side key produced a stimulus associated with the variable-interval component on the center key provided that said schedule was currently in effect. The independent variable was the length of time this stimulus remained on the keys. Pecking one side key produced the stimulus for 27 seconds, whereas the duration produced by pecking the other key varied for successive blocks of sessions. For the first four birds, the values tested were 3, 9, 27, and 81 seconds. For the second group, numbering three birds, the values tested were 1, 3, 9, and 27 seconds. The dependent variable was the proportion of total side key pecks that occurred on the variable key. For all birds, the function was positive in slope and negative in acceleration. This finding supports a formulation that ascribes the maintenance of observing responses in a normal setting to the fact that the subject exposes itself to the positive discriminative stimulus for a longer mean duration than it does to the negative stimulus.     

Short-component multiple schedules: effects of relative reinforcement duration

Pigeons were exposed to multiple variable-interval 2-min variable-interval 2-min schedules of food presentation in which relative duration of food presentation was manipulated. When components alternated every 5 sec and were scheduled on separate response keys, relative response rates closely matched relative reinforcement duration in three of four pigeons. On the other hand, relative response rates were insensitive to relative reinforcement duration when components scheduled on a single response key alternated every 5 sec, and when components scheduled on separate response keys alternated every 2 min. Thus, both rapid alternation and spatial separation of components were necessary to produce approximate matching of relative responding to relative reinforcement duration. This finding contrasts with previous findings that only rapid component alternation is necessary for matching when relative rate of reinforcement is manipulated.     

Effects of stimulus duration on observing behavior maintained by differential reinforcement magnitude

Pigeons made observing responses for stimuli signalling the availability of either 10-sec or 2-sec access to grain on fixed-interval 1-min schedules. If observing responses did not occur, food-producing responses occurred to a stimulus common to both reinforcement magnitudes. When the stimuli remained on for the duration of the components and signalled differential reinforcement magnitudes, observing responses were maintained; however, when the stimuli remained on for 10 sec, observing responses decreased markedly. In addition, it was shown that the occasional presentation of the stimulus signalling 10-sec access to grain was necessary for the maintenance of observing behavior. A control condition demonstrated that when all the available stimuli signalled 6-sec access to grain, observing responses declined. Taken together, the results demonstrated that the occasional presentation of the stimulus that remained on for the duration of the component and signalled the larger reinforcement magnitude was necessary for the maintenance of observing behavior.     

The discrimination of differences in duration of a thermal reinforcement

The sides of a contour square appear successively when context lines are presented one before and one after the square. As the interstimulus interval (ISI) between lines and square is increased to 140 msec, reports of succession increase. If the sides of the square are presented successively and interpart interval (IPI) varied by a single staircase method in order to determine a succession threshold, the average magnitude of this threshold varies as a function of ISI, decreasing up to 140 msec lSI. Each S, however, shows a maximum decrease in threshold between 80 and 120 msec. Reports of this illusion are not dependent on beta movement of either context lines or square.     

Signal duration and suppression of operant responding by free reinforcement

Pigeons were trained on a VI (variable interval) schedule of food presentation with a superimposed schedule of response-independent food. Substantial suppression of the operant response rate occurred when the free food was presented without a signal. When the free food was preceded by a short (4 sec) signal, the degree of suppression was similar to that with unsignaled free food. But when the signal was lengthened to 12 sec, the degree of suppression was substantially reduced. Experiment 3 assessed the effect of signal duration using a baseline schedule of delayed reinforcement, in which contingent reinforcers were themselves preceded by a signal. The signal preceding the free reinforcers was then either the same as or different from this contingent signal. Signal duration effects occurred only when the two types of signals were different. These differences as a function of signal duration have implications for both “context-blocking” and “comparator” interpretations of the effects of noncontingent reinforcement in both Pavlovian and operant procedures.     

Reduction of shock duration as negative reinforcement in free-operant avoidance

Rats were trained on a free-operant procedure in which shock duration was controlled by responses within a limited range of interresponse times. Shocks of 1.6-mA intensity occurred randomly with average density of 10 shocks per minute. As long as interresponse times were 15 seconds or less, any shocks received were at the briefer of two durations (.3 second). Whenever interresponse times exceeded 15 seconds, any shocks received were at the longer duration (1.0 second). For six of eight animals, avoidance responding developed quickly and reached levels of better than 90%. Four yoked animals stopped responding within the first few sessions. Shock duration reduction without change in shock probability or intensity was sufficient for the acquisition and maintenance of avoidance responding.     

Conditioned reinforcement and response strength

Stimuli associated with primary reinforcers appear themselves to acquire the capacity to strengthen behavior. This paper reviews research on the strengthening effects of conditioned reinforcers within the context of contemporary quantitative choice theories and behavioral momentum theory. Based partially on the finding that variations in parameters of conditioned reinforcement appear not to affect response strength as measured by resistance to change, long-standing assertions that conditioned reinforcers do not strengthen behavior in a reinforcement-like fashion are considered. A signposts or means-to-an-end account is explored and appears to provide a plausible alternative interpretation of the effects of stimuli associated with primary reinforcers. Related suggestions that primary reinforcers also might not have their effects via a strengthening process are explored and found to be worthy of serious consideration.     

Intermittent reinforcement of a continuous response

Konorski showed that when a go/no-go procedure was used, sound quality discriminations were rapidly acquired and sound location discriminations were slowly acquired. These findings have been interpreted as a general constraint on the acquisition of auditory discriminations (quality-location effect). However, experiments carried out within an evolutionary framework (Harrison, 1984) have shown that the rate of acquisition of sound location discriminations varies widely as a function of the inclusion or exclusion of naturalistic features. These data suggest that Konorski's findings were a function of the special conditions of the experiments. The first purpose of the present experiments was to assess whether rats showed the effects noted by Konorski when studied under similar conditions. The second purpose was to study the effect of manipulating two natural features (novelty and stimulus-response adjacency) to assess whether the acquisition rates of quality and location discriminations could be greatly modified or made approximately equal, or both. When a go/no-go procedure was used and the other conditions were similar to those of Konorski, rats acquired a quality discrimination but did not acquire a location discrimination. However, when the S+ or S- were presented through a closely adjacent speaker, the sound location discrimination was acquired as rapidly as the quality discrimination. Finally, preexposing the animal to either S+ or S- retarded the rate of or prevented the acquisition of the quality discrimination. The experiments showed that the quality-location effect was determined primarily by the conditions used in Konorski's experiments, and that the effect is not a general constraint on learning.     

Concurrent schedules of response-independent reinforcement: duration of a reinforcing stimulus

Presentations of grain to three pigeons were determined by two response-independent schedules. Interpresentation intervals varied with a mean interval of 1.5 min for each schedule. Both were concurrently operative, but grain was presented by one only when the chamber was illuminated with blue light and by the other only during amber illumination. A response on a white key, the only key in the chamber, alternated the stimulus conditions and the effective schedule. Grain presentation durations associated with the illumination conditions were varied from 1.5 to 4.5 sec. The proportion of the total session time spent in an illumination condition closely approximated the relative grain presentation duration provided in that illumination. For two of the birds, the proportion of the total number of grain presentations obtained in an illumination condition was an increasng function of the presentation duration in that illumination.     

Response duration is sensitive to both immediate and delayed reinforcement

We investigated the duration of lever pressing by rats when the delivery of appetitive reinforcers was contingent upon response duration. In the first experiment, response durations increased when duration requirements were imposed, and they decreased when duration requirements were removed. This effect occurred whether reinforcers were immediate or delayed by 8 s. In order to maintain the integrity of the delay intervals, reinforcer delivery was dependent upon both lever depression and release. In a second experiment, lever depression only and a response duration of at least 4 s were required for reinforcer delivery. Compared to immediate reinforcement conditions, delayed reinforcers increased both variability and the length of the maximum response durations. In a third experiment, immediate reinforcers were delivered contingent upon lever depression and release under a variety of duration requirements. Median lever‐press durations tracked the contingencies rapidly. Across all three experiments, rats emitted numerous response durations that were too short to satisfy the reinforcer requirements, and bimodal distributions similar to those produced by differential reinforcement of low rate schedules were evident for most rats. In many aspects, response duration responds to reinforcement parameters in a fashion similar to rate of discrete responding, but an examination of this continuous dimension of behavior may provide additional information about environment–behavior relationships.     

Effects on preference of reinforcement delay, number of reinforcers, and terminal-link duration

Three experiments used concurrent-chains procedures to examine the effects of reinforcement delay, number of reinforcers, and terminal-link duration on preference. In Condition 30 of Experiment 1, food was delivered after 30 seconds in each 150-second terminal link, with four additional food deliveries occurring at 30-second intervals in one of the links. In Condition 5, food was delivered after 5 seconds in each 25-second terminal link, and the four additional reinforcers were delivered at 5-second intervals. Preferences for the multiple-food chain were greater in Condition 30. In Experiment 2, the terminal link(s) providing only one reinforcer terminated immediately after delivery of the reinforcer. Preferences for the multiple-food chain were smaller than in Experiment 1. In Condition 5 of Experiment 3, food was delivered after 5, 75, 100, 125, and 150 seconds in one 150-second link and after 5 seconds in the other. Condition 50 differed only in that the first (or only) reinforcer in each link was delivered after 50 seconds instead of after 5 seconds. Preferences for the multiple-food chain were greater in Condition 50. Results of Experiments 1 and 2 do not correspond to results obtained by Moore (1979).