An invariant relation between changing over and reinforcement

Although concurrent schedules may arrange reinforcers irregularly, relatively large numbers of reinforcers are obtained when an animal changes from one schedule to the other. This paper proposes a quantitative relation that predicts the proportion of reinforcers obtained when an animal is working on a schedule and the proportion when the animal changes over to a schedule. Basically the relation states that the number of reinforcers obtained while an animal works on a schedule varies directly with the relative amount of time spent working on that schedule; and the number of reinforcers obtained when an animal changes to a schedule varies directly with the relative amount of time spent on the alternate schedule. An important aspect of this relation is that when relative reinforcement rates are less than .50, more reinforcers are obtained just after an animal changes to a schedule than at all other times when this schedule is engaged. Data obtained both from a stat-bird and a live pigeon were in close agreement with the quantitative predictions. The relation between changing over and reinforcement held across several procedural changes that included changes in relative reinforcement rate, changes from independent to interdependent scheduling procedures, and changes in the variable-interval reinforcement distributions. The results are discussed in terms of the effects of the local distribution of reinforcement on responding. The local reinforcement distribution can affect local response rates and affects the resulting matching relation. This arrangement has implications for explanations of choice.     

Partial reinforcement effects and type of reward

In two experiments 68 rats were trained to bar press or run down a straight runway for food or for water under conditions of either continuous reinforcement or partial reinforcement. In both experiments, there was greater persistence of behavior which had been reinforced with food than with water. In Expt 2, the partial reinforcement extinction effect (PREE) was observed with food reward, but not with water.Within the context of the experimental procedures used, it can be concluded that the rat has mechanisms for developing persistence which are dependent on the specific motivational system involved. This conclusion is related to theories of partial reinforcement effects and to possible biological origins of the mechanisms.     

The relation between response rates and reinforcement rates in a multiple schedule

In a multiple schedule, exteroceptive stimuli change when the reinforcement schedule is changed. Each performance in a multiple schedule may be considered concurrent with other behavior. Accordingly, two variable-interval schedules of reinforcement were arranged in a multiple schedule, and a third, common variable-interval schedule was programmed concurrently with each of the first two. A quantitative statement was derived that relates as a ratio the response rates for the first two (multiple) variable-interval schedules. The value of the ratio depends on the rates of reinforcement provided by those schedules and the reinforcement rate provided by the common variable-interval schedule. The following implications of the expression were evaluated in an experiment with pigeons: (a) if the reinforcement rates for the multiple variable-interval schedules are equal, then the ratio of response rates is unity at all reinforcement rates of the common schedule; (b) if the reinforcement rates for the multiple schedules are unequal, then the ratio of response rates increases as the reinforcement rate provided by the common schedule increases; (c) the limit of the ratio is equal to the ratio of the reinforcement rates. Satisfactory confirmation was obtained for the first two implications, but the third was left in doubt.     

The relation between memory and expectancy as revealed by percentage and sequence of reward investigations

There is growing agreement that to explain instrumental learning properly, one should emphasize memory as well as expectancy. I call this approachmemory-expectancy theory. Amsel’s (1992) frustration theory is one variety of memory-expectancy theory. Capaldi’s (1994) sequential theory is another. In this report, I examine in considerable detail the effects of percentage and sequence of reward on extinction following different levels of acquisition training. These extinction findings, taken together with certain serial learning acquisition findings, seem to support a novel version of memory-expectancy theory, one that in some respects is similar to and in some respects is different from that suggested by Amsel. First, on the basis of this analysis, we may reject two ideas: that animals remember only the prior reward event and that animals anticipate only the reward event contingent upon the current response. Second, the analysis supports three salient propositions of the present memory-expectancy approach. Memories of reward events may serve as conditioned stimuli for expectancies of reward events. On any current trial, the animal may remember each of the reward events associated with one or more prior trials. On any current trial, the animal may anticipate not only the current reward event, but also reward events contingent upon subsequent trials. Essentially, according to this model, the stimuli that elicit expectancies, as well as the expectancies themselves, may change progressively over a series of learning trials.     

Early developments in the ability to understand the relation between stimulus and reward

Delayed nonmatching to sample (DNMS) is used to test the recognition memory function dependent on the medial temporal lobe. Children cannot succeed on this task until about 21 months. Because robust recognition is present well before then, the late emergence of another ability must account for the late success on DNMS. Evidence is presented here that the critical late-maturing competence is the ability to grasp the relation between stimulus and reward--that is, to understand that the stimulus is a symbol or marker for the reward. Infants of 9 and 12 months were tested on 3 conditions of DNMS. A sample object was presented. After a delay, the sample and a novel object appeared; choice of the novel object was rewarded. In the standard task, the reward was in a well beneath the stimulus. In the verbal-reward condition the reward was not a separate object but was praise and applause. In the Velcro condition, the reward, although a separate and separable object, was attached to the base of the stimulus. Most infants at both ages succeeded in the verbal-reward and Velcro conditions but not in the standard condition.     

Immediate reinforcement in delayed reward learning in pigeons

Pigeons were trained on simultaneous red-green discrimination procedures with delayed reward and sequences of stimuli during the delay. In Experiment 1, three stimuli appeared during the 60-second intervals between the correct responses and reward, and the incorrect responses and nonreward. The stimulus that immediately followed a correct response also preceded nonreward, and the stimulus that followed an incorrect response preceded reward. These stimuli were 10 or .33 second in duration for different groups. Stimuli during the remainder of the delay interval differed following correct and incorrect responses. Group 10 initially persisted in the nonrewarded choice, but shifted to a preponderance of rewarded responses after further training. Group .33 rapidly acquired the correct response. Similar results were obtained in Experiment 2 where delay intervals consisted of opposite sequences of two stimuli of equal duration and total delays were 6, 20, or 60 seconds. Early in training, generalization of differential conditioned-reinforcing properties from the conditions preceding reward and nonreward to postchoice conditions had a greater effect relative to backchaining than it did later. It was concluded that delayed-reward learning is best analyzed in terms of the conditioned-reinforcing value of the patterns of cues that follow immediately after rewarded and nonrewarded responses.     

Transfer between downshift in reward magnitude and continuous delay of reward

Two separate experiments were conducted to investigate transfer of persistence between delay and a downshift in reward magnitude. In the first experiment, experimental rats were initially downshifted in reward magnitude and later tested for persistence to continuous delay of large reward. It was found that these rats were more persistent to the effects of delay than control rats which did not receive prior experience with a downshift in reward magnitude. In the second experiment, experimental rats were first trained to receive large reward under delayed conditions and then tested for persistence to a downshift in reward magnitude. Compared to control rats which received no prior experience with delay, the experimental rats showed a significantly smaller negative contrast effect. The results were interpreted as supporting Amsel's theory of persistence as well as Capaldi's recent interpretation of contrast effects.