Delayed reinforcement of fixed-ratio performance without mediating exteroceptive conditioned reinforcement

The performance of pigeons was studied under conditions in which the completion of a fixed-ratio requirement was not contiguous with the presentation of a reinforcer. Timein and timeout periods alternated throughout the experimental sessions. Responses made by an experimental bird during the timein period were accumulated, and when a fixed-ratio requirement had been met, grain was presented to the experimental bird and a yoked control following their first response in the next timein period. Across most manipulations of the fixed-ratio requirement and of the duration of the timeout period, the response rates of the experimental birds were considerably higher than those of their controls, suggesting that the response-reinforcer dependency controlled the behavior of the experimental bird in the absence of a close temporal association between responding on the ratio schedule and reinforcer presentations.     

Enhancement of conditioned reinforcement by uncertainty

Pigeons were trained in three conditions. In the baseline condition, the birds responded on a fixed-interval schedule with the response key white. When the interval was completed, the key turned either red or green for a delay interval that was terminated by a grain presentation dependent on no key pecks during the final 2 sec. In the uncertainty condition, no grain was presented at the end of the delay periods when the key was red. In the certainty condition, the white light appeared only on occasions when pecking would turn the key green and produce food. Otherwise, the key was illuminated red throughout the total time period. The highest response rate in white occurred in the uncertainty condition, the next highest in the certainty condition, and the lowest in baseline. The results suggest that uncertainty facilitated responding, although uncertainty is not a necessary condition for conditioned reinforcement.     

Effects of ratio reinforcement schedules on discrimination performance by Japanese monkeys

In Experiment 1, Japanese monkeys were trained on three conditional position-discrimination problems with colors as the conditional cues. Within each session, each problem was presented for two blocks of ten reinforcements; correct responses were reinforced under continuous-reinforcement, fixed-ratio 5, and variable-ratio 5 schedules, each assigned to one of the three problems. The assignment of schedules to problems was rotated a total of three times (15 sessions per assignment) after 30 sessions of acquisition training. Accuracy of discrimination increased to a moderate level with fewer trials under CRF than under ratio schedules. In contrast, the two ratio schedules, fixed and variable, were more effective in maintaining accurate discrimination than was CRF. With further training, as asymptotes were reached, accuracy was less affected by the schedule differences. These results demonstrated an interaction between the effects of reinforcement schedules and the level of acquisition. In Experiment 2, ratio sizes were gradually increased to 30. Discrimination accuracy was maintained until the ratio reached 20; ratio 30 strained the performance. Under FR conditions, accuracy increased as correct choice responses cumulated after reinforcement.     

Fixed-interval behavior maintained by conditioned reinforcement

The key-pecking of a pigeon was reinforced with grain on an 18-min second-order schedule. During the 18 min, a key peck which completed a 3-min fixed interval produced a stimulus of 0.5-sec duration. The first 3-min fixed interval completed after 18 min resulted in primary reinforcement. Behavior characteristic of fixed-interval schedules was produced on both the 3-min components and the 18-min schedule. This performance was shown to be enhanced whenever the 0.5-sec stimulus was also presented before the presentation of grain.     

Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus

Three experiments examined appetitive Pavlovian-instrumental interactions by presenting separately trained conditioned stimuli (CSs) during reinforced instrumental responding in rabbits. Intra-oral reinforcement was used to minimize interference from peripheral responses such as magazine approach. In experiment 1, the rabbits were first trained to perform an instrumental head-raising response for sucrose reward. A conditioned jaw movement response was then established to a 2-sec CS by pairing it with sucrose; a control stimulus was unpaired with sucrose. Instrumental responding maintained by a variable-interval 40-sec schedule was enhanced during 10-sec presentations of the paired, but not the unpaired, CS. Responding on a variable-ratio 15 schedule was unaffected except on trials on which the pre-CS baseline response rate was low; in such cases the paired CS caused a long-lasting acceleration of responding. Noncontingent presentation of the sucrose reinforcer itself briefly suppressed responding but had no long-term effect. In Experiment 2, a CS that had been conditioned at a 10-sec duration produced the same pattern of effects as in the first study, indicating that facilitation resulted from CS presentation rather than from the frustrative effects of non-reinforcement of the CS. In Experiment 3 an inhibitory CS blocked facilitation by the excitatory CS but did not itself affect instrumental responding. These results support the view that Pavlovian processes play a positive role in instrumental performance and suggest that previous findings of suppression by a short-duration CS reflect peripheral interference. The dependence of facilitation on the baseline level of responding is discussed in terms of associative and motivational theories of Pavlovian mediation.     

Schedules of response-independent conditioned reinforcement

Rates and patterns of responding of pigeons under response-independent and response-dependent schedules of brief-stimulus presentation were compared by superimposing 3-min brief-stimulus schedules on a 15-min fixed-interval schedule of food presentation. The brief-stimulus schedules were fixed time, fixed interval, variable time, and variable interval. When the brief stimulus was paired with food presentation, its effects depended upon the schedule and ongoing rates. Fixed- and variable-interval brief-stimulus schedules enhanced the low rates normally occurring early in the 15-min interval, whereas fixed- and variable-time schedules suppressed these rates. Although the overall rates later in the interval were not affected to any great extent, the fixed brief-stimulus schedules generated patterns of positively accelerated responding between stimulus presentations. These patterns appeared less frequently under the variable brief-stimulus schedules. Initially, when not paired with food delivery, presentations of the brief stimulus produced relatively little effect on either response rate or patterning. However, once the stimulus had accompanied food presentation, the original performance under the nonpaired condition was not recovered. The effects were more like those occurring when the stimulus was paired with food.     

Control of preference in children by conditioned positive reinforcement

A preference measure was employed with children to evaluate the conditioned positive reinforcing function of a stimulus that preceded reinforcement. A match-to-sample procedure was arranged in which subjects could respond to either the form or color dimension of a compound sample stimulus. Intermittent token reinforcement was provided equally for color and form matches. Two stimuli were employed (Stimulus A and Stimulus B), each consisting of a distinctive tone and colored light. One of these stimuli (the paired stimulus) preceded each token delivery, and the other did not (nonpaired stimulus). The paired stimulus was dependent upon each response to one match dimension, and the nonpaired stimulus followed each response to the other dimension. Three of the five subjects responded primarily to the dimension that was followed by the paired stimulus. This effect was obtained regardless of which stimulus (A or B) was paired and on which match dimension (color or form) the paired stimulus was dependent. These results were unaltered by discontinuing the nonpaired stimulus. The other two subjects demonstrated consistent preferences for the form dimension and Stimulus A, respectively.     

Effects on choice of reinforcement delay and conditioned reinforcement

Pigeons chose between fixed-interval schedules of different durations presented in the terminal links of concurrent-chains schedules. The pair of schedules was always in the ratio of 2:1, but the absolute duration of the fixed intervals varied. In one set of conditions, the different terminal-link schedules were associated with different keylight stimuli (cued conditions). In a second set of conditions, the different terminal-link schedules were associated with the same stimulus (uncued conditions). Results from the cued conditions replicated previous findings that preference for the shorter fixed-interval schedule increased with fixed-interval duration. Preferences in the uncued conditions were lower than in the corresponding cued conditions but also increased with fixed-interval length. In addition, the degree of control under the uncued conditions was correlated with the extent to which the schedule during the terminal link was discriminated immediately upon entry into the terminal link. The pattern of results in both conditions was inconsistent with the notion that choice behavior matches relative immediacy of reinforcement. Reanalysis of previous evidence for matching (Chung and Herrnstein, 1967) showed that matching in fact did not occur, as the preferences of their subjects for the shorter of two delays also increased with the absolute size of the delays.     

Positive conditioned suppression: Transfer of performance between contingent and noncontingent reinforcement situations

Five homing pigeons were trained on concurrent variable-interval schedules. A fixed-duration stimulus was occasionally presented on one key; and, in various conditions, this stimulus terminated (a) without reinforcement, (b) in noncontingent reinforcement, (c) with reinforcement contingent on a response on the key on which the stimulus was presented, and (d) with reinforcement contingent on a response on the key on which the stimulus was not presented. Initially, a stimulus terminating in noncontingent reinforcement generally produced decreased response rates on both keys during the stimulus. Contingencies, however, reliably produced increased rates during the stimulus on the key on which the contingency was arranged, relative to the rate on the concurrently available key. Contingency conditions were followed by noncontingency conditions in which the separation of rates caused by contingencies was maintained. When rates during the stimulus were compared with response rates on the same keys in the absence of the stimulus, contingency-caused rate increases and decreases were again found, but only the rate decreases were maintained in subsequent noncontingency conditions. Further data suggested that the contingency-caused rate changes were not maintained when the stimulus terminated without reinforcement, and that they were unaffected by a threefold decrease in the reinforcement rate provided by the baseline schedules. The results support the suggestion that performance in the positive conditioned suppression procedure results from concurrent and multiple schedule interactions. They further suggest that the production of either acceleration or suppression is dependent on adventitious and historical contingencies.     

Choice and conditioned reinforcement.

A potential weakness of one formulation of delay-reduction theory is its failure to include a term for rate of conditioned reinforcement, that is, the rate at which the terminal-link stimuli occur in concurrent-chains schedules. The present studies assessed whether or not rate of conditioned reinforcement has an independent effect upon choice. Pigeons responded on either modified concurrent-chains schedules or on comparable concurrent-tandem schedules. The initial link was shortened on only one of two concurrent-chains schedules and on only one of two corresponding concurrent-tandem schedules. This manipulation increased rate of conditioned reinforcement sharply in the chain but not in the tandem schedule. According to a formulation of delay-reduction theory, when the outcomes chosen (the terminal links) are equal, as in Experiment 1, choice should depend only on rate of primary reinforcement; thus, choice should be equivalent for the tandem and chain schedules despite a large difference in rate of conditioned reinforcement. When the outcomes chosen are unequal, however, as in Experiment 2, choice should depend upon both rate of primary reinforcement and relative signaled delay reduction; thus, larger preferences should occur in the chain than in the tandem schedules. These predictions were confirmed, suggesting that increasing the rate of conditioned reinforcement on concurrent-chains schedules may have no independent effect on choice.     

Preference for conditioned reinforcement.

Pigeons were presented with a concurrent-chains schedule in which both choice alternatives led to the same terminal-link stimulus, which was followed by food. Superimposed on the food-reinforced presentations of the terminal-link stimulus was a second schedule of presentations of the same stimulus that were followed by no food. The absolute number of these no-food stimulus presentations was held constant while their relative frequency assigned to one or the other choice alternative was systematically varied. Preference for a given choice alternative tracked the relative frequency of these stimulus presentations, thus demonstrating that they served as reinforcers. These results resolve conflicts in the literature regarding the effect of conditioned reinforcement on choice.     

Effects of reinforcement magnitude and ratio values on behaviour maintained by a cyclic ratio schedule of reinforcement

In Experiments 1 and 2, lever pressing by rats was reinforced on a cyclic ratio schedule of food reinforcement, comprising a repeated sequence of fixed-ratio component schedules. Reinforcement magnitude was varied, on occasional sessions in Experiment 1 and across blocks of sessions in Experiment 2, from one to two or three 45-mg food pellets. In the one-pellet condition, post-reinforcement pauses increased with component schedule value. At higher magnitudes, post-reinforcement pauses increased, and overall response rates declined. Response rate on component schedules was a decreasing linear function of the obtained rate of reinforcement in all conditions. Plotted against component schedule value, response rate increased exponentially to an asymptote that decreased when reinforcement magnitude increased. These findings are consistent with regulatory accounts of food reinforced behaviour. In Experiment 3, rats were trained under a cyclic ratio schedule comprising fixed-ratio components including higher values, and some inverted U-shaped response functions were obtained. Those rats that did not showthis relationship were trained on cyclic ratios with even higher values, and all showed inverted U-shaped response functions. This suggests that behaviour on cyclic ratio schedules can reflect activating of reinforcement as well as the satiating effects seen in Experiments 1 and 2.     

The conditioned reinforcement of repeated acquisition

Three monkeys were trained to emit a chain of three responses on three separate levers in a set of six levers to obtain food. The chain producing food (correct chain) was changed each day. During a trial, a press on any lever produced a feedback stimulus; a press on a correct lever produced an additional distinctive stimulus; the third correct press produced a food pellet. Test sessions in which either the food or the distinctive stimuli were removed were interspersed with baseline sessions. In tests without food presentations, the subjects acquired the correct chain rapidly, with a level of accuracy comparable to baseline. Removing the distintive stimuli for either the first or second member of the correct chain greatly retarded acquisition of that member of the chain. Removing all distinctive stimuli often reduced accuracy throughout the chain to chance level, even though food was presented following each correct chain. These results were interpreted as evidence that the distinctive stimuli presented after correct responses functioned as conditioned reinforcers. Reductions in accuracy following an omitted distinctive stimulus indicated that they were also discriminative stimuli for correct responding in their presence.     

Temporal context and conditioned reinforcement value

The effectiveness of a stimulus as a conditioned reinforcer depends on the temporal context of reinforcement, that is, the overall rate of reinforcement in the situation. The dominant view has been that context determines the learned value of a stimulus directly, according to delay-reduction theory. By contrast, the contextual choice model (CCM) maintains that value is independent of context and incorporates the effects of context on choice in the framework of the matching law. The authors report 2 experiments with pigeons as subjects that use transfer tests to assess the value of stimuli in the concurrent-chains procedure. Results strongly support the assumption of CCM that pigeons learn the temporal relations between events independently of context but that context modulates the expression of that learning as choice.     

The emergence of conditioned reinforcement from observation

We report an experiment in which observations of peers by six 3-5-year-old participants under specific conditions functioned to convert a small plastic disc or, for one participant, a small piece of string, from a nonreinforcer to a reinforcer. Prior to the observational procedure, we compared each participant's responding on (a) previously acquired performance tasks in which the child received either a preferred food item or the disc (string) for correct responses, and (b) the acquisition of new repertoires in which the disc (string) was the consequence for correct responses. Verbal corrections followed incorrect responses in the latter tasks. The results showed that discs and strings did not reinforce correct responses in the performance tasks, but the food items did; nor did the discs and strings reinforce correct responses in learning new repertoires. We then introduced the peer observation condition in which participants engaged in a different performance task in the presence of a peer who also performed the task. A partition blocked the participants from seeing the peers' performance. However, participants could observe peers receiving discs or strings. Participants did not receive discs or strings regardless of their performance. Peer observation continued until the participants either requested discs or strings repeatedly, or attempted to take discs or strings from the peers. Following the peer observation condition, the same performance and acquisition tasks in which participants had engaged prior to observation were repeated. The results showed that the discs and strings now reinforced correct responding for both performance and acquisition for all participants. We discuss the results with reference to research involving nonhuman subjects that demonstrated the observational conditioning of reinforcers.