Effects of D-amphetamine on response acquisition with immediate and delayed reinforcement.

The present study examined in 8-hour sessions the effects of d-amphetamine (1.0, 5.6, and 10 mg/kg) on the acquisition of lever-press responding in rats that were exposed to procedures in which water delivery was delayed by 0, 8, or 16 seconds relative to the response that produced it. Both nonresetting- and resetting-delay conditions were studied. Although neither shaping nor autoshaping occurred, substantial levels of operative-lever responding developed under all conditions in which responses produced water. The lowest dose (1.0 mg/kg) of d-amphetamine either had no effect on or increased operative-lever pressing, whereas higher doses typically produced an initial reduction in lever pressing. Nonetheless, overall rates of operative-lever pressing at these doses were as high as, or higher than, those observed with vehicle. Thus, response acquisition was observed under all reinforcement procedures at all drug doses. In the absence of the drug, most responding occurred on the operative lever when reinforcement was immediate. Such differential responding also developed under both nonresetting- and resetting-delay procedures when the delay was 8 seconds, but not when it was 16 seconds. d-Amphetamine did not affect the development of differential responding under any procedure. Thus, consistent with d-amphetamine's effects under repeated acquisition procedures, the drug had no detrimental effect on learning until doses that produced general behavioral disruption were administered.     

Assessment and choice: an operant simulation of foraging in patches

Pigeons were presented with an operant simulation of two prey patches using concurrent random-ratio schedules of reinforcement. An unstable patch offered a higher initial reinforcement probability, which then declined unpredictably to a zero reinforcement probability in each session. A stable patch offered a low but unvarying reinforcement probability. When the reinforcement probability declined to zero in a single step, the birds displayed shorter giving-up times in the unstable patch when the ratio between the initial reinforcement probabilities in the unstable and stable patches was greater and when the combined magnitude of the reinforcement probabilities in the two patches was greater. When the unstable patch declined in two steps, the birds behaved as if their giving-up times were influenced heavily by events encountered during the most recent step of the double-step change. This effect was observed, however, only when the reinforcement probability in that step was .04, not when it was .06. All of these data agree with the predictions of a capture-probability model based on a comparison of the estimated probability of receiving a reinforcer in the current patch with that in alternative patches.     

Stimulus Effects On Behavior Allocation In Three-alternative Choice

Six pigeons were trained on three-alternative concurrent variable-interval schedules that were available through a switching response and were signaled by colored stimuli. The discriminative stimuli for two of the schedules were always 560 nm and 630 nm, but the stimulus signaling the third alternative was varied across conditions over seven levels between these colors. For each third-alternative stimulus condition, the relative frequency of reinforcers was varied over three conditions with 4:1 and 16:1 reinforcer ratios between each pair of alternatives. The distribution of responses between the alternatives was dependent jointly on the third-alternative reinforcer rate and on the disparity between the stimulus signaling the third alternative and those signaling the other alternatives. A generalized matching approach was unable to provide invariant measures of the discriminability between constant stimuli, but a contingency-discriminability approach provided excellent fits and sensible and invariant stimulus discriminability measures.     

Procrastination by pigeons: preference for larger, more delayed work requirements.

In three experiments, pigeons chose between alternatives that required the completion of a small ratio schedule early in the trial or a larger ratio schedule later in the trial. Completion of the ratio requirement did not lead to an immediate reinforcer, but simply allowed the events of the trial to continue. In Experiment 1, the ratio requirements interrupted periods in which food was delivered on a variable-time schedule. In Experiments 2 and 3, each ratio requirement was preceded and followed by a delay, and only one reinforcer was delivered, at the end of each trial. Two of the experiments used an adjusting-ratio procedure in which the ratio requirement was increased and decreased over trials so as to estimate an indifference point--a ratio size at which the two alternatives were chosen about equally often. These experiments found clear evidence for "procrastination"--the choice of a larger but more delayed response requirement. In some cases, subjects chose the more delayed ratio schedule even when it was larger than the more immediate alternative by a factor of four or more. The results suggest that as the delay to the start of a ratio requirement is increased, it has progressively less effect on choice behavior, in much the same way that delaying a positive reinforcer reduces it effect on choice.     

Effects Of Histories Of Differential Reinforcement Of Response Rate On Variable-interval Responding

Three pigeons were exposed first to multiple differential-reinforcement-of-high-rate and differential-reinforcement-of-low-rate schedules that were correlated with green and red keys, respectively, and then were shifted to a variable-interval schedule arranged on a white key. In subsequent test sessions, the variable-interval schedule continued to operate, but green and red keys replaced the white key in alternate sessions. In Part 1 of the experiment, the variable-interval schedule correlated with the white key was introduced immediately after the multiple-schedule condition, and the test condition began 15 days later. This sequence was repeated twice, with a reversal of the correlation of the key colors with the components of the multiple schedule at the start of each new cycle. Part 2 added a 6-month break between the multiple-schedule history and the white-key variable-interval schedule followed by test sessions. The procedure was then repeated with a reversal of the correlation between key colors and multiple-schedule components. In the test sessions of Part 1, all pigeons consistently responded faster in the presence of the key color most recently correlated with the differential-reinforcement-of-high-rate contingency than during the color most recently correlated with the differential-reinforcement-of-low-rate contingency. Similar but smaller effects were observed in Part 2. The effects of the reversals in these two parts of the experiment showed that only the most recent contingency exerted an influence on subsequent responding. The data suggest that this effect of the most recent history continues to operate on behavior under current contingencies even after a long lapse of time.     

Learning about food: starlings, skinner boxes, and earthworms

Despite its importance as a tool for understanding a wide range of animal behavior, the study of reinforcement schedules in the laboratory has suffered from difficulties in the biological interpretation of its findings. This study is an operant-laboratory investigation of the ability of European starlings, Sturnus vulgaris, to learn to respond adaptively to the problem of foraging on patchily distributed prey that are uncertainly located in space. In order to maximize the biological relevance of the laboratory study, variation in the aggregation of earthworms, Lumbricus terrestris (a prey species), was rigorously quantified from the field, and the experimental birds were presented with reinforcement schedules designed to represent the extremes of the observed variation. The results demonstrate that, even for a single prey species, the degree to which individuals are aggregated can vary markedly over a range of spatial scales, and that starlings can rapidly learn to respond, in an adaptive manner, to these variations. These findings suggest that starlings are capable of adjusting their behavior to facilitate the efficient exploitation of prey that occurs in patches of an uncertain nature, and thus illustrate the heuristic value of an ecologically informed operant-laboratory approach to studying foraging behavior.     

Pigeons' preference for free choice: number of keys versus key area.

In concurrent-chains schedules, pigeons prefer terminal links that provide two keys correlated with reinforcers (free choice) over those that provide only one key (forced choice), terminal-link reinforcement rates being equal. With same-size keys, free choice provides a larger area available for pecking. Preferences were examined using terminal links that differed in key number only (one or two) or key size only (small and medium or medium and large), or that equated the area of the two free-choice keys with that of the forced-choice key. Medium (standard) keys were typically preferred to small keys, but indifference was typically obtained between medium and large keys. The size preference usually overrode free-choice preference with one medium key pitted against two small keys, but free-choice preference was reliably observed with one large key pitted against two medium keys. In other words, preferences were a joint function of key number and key area, implying that free-choice preference is not reducible to preference for larger key areas. Free-choice preference requires separate keys rather than larger areas; the relevant behavioral units are the discriminated operants correlated with each terminal-link key rather than classes defined by topographical features such as area or perimeter.     

Conjoint schedules of timeout deletion in pigeons.

This experiment attempted to bring behavior under joint control of two distinct contingencies, one that provided food and a second that extended the periods during which that food was available. Pigeons' responses on each of two keys were reinforced according to a single random-interval schedule of food presentation except during signaled timeout periods during which the schedule was temporarily disabled. By means of a conjoint schedule, responses on the initially less preferred key not only produced food but also canceled impending timeouts. When behavior came to predominate on this conjoint alternative, the consequences of responding on the two keys were reversed. Responding in 3 of 4 pigeons proved sensitive to the conjoint scheduled consequences, as evidenced by systematic shifts in response rates favoring the conjoint key. In 2 of these 3 pigeons, sensitivity to the conjoint contingency was evident under time-in:timeout ratios of 2:1 (time-in = 120 s, timeout = 60 s) and 1:5 (time-in = 30 s, timeout = 150 s), whereas for the other pigeon preference for the conjoint key was observed only under the latter sequence of conditions. There was only weak evidence of control by the conjoint scheduled consequences in the 4th subject, despite extended training and forced exposure to the conjoint alternative. The overall pattern of results is consistent with studies of timeout avoidance but also shares features in common with positively reinforced behavior.     

A COLLATERAL EFFECT OF REWARD PREDICTED BY MATCHING THEORY

Matching theory describes a process by which organisms distribute their behavior between two or more concurrent schedules of reinforcement (Herrnstein, 1961). In an attempt to determine the generality of matching theory to applied settings, 2 students receiving special education were provided with academic response alternatives. Using a combined simultaneous treatments design and reversal design, unequal ratio schedules of reinforcement were varied across two academic responses. Findings indicated that both subjects allocated higher rates of responses to the richer schedule of reinforcement, although only one responded exclusively to the richer schedule. The present results lend support to a postulation that positive reinforcement may have undesirable collateral effects that are predicted by matching theory (Balsam & Bondy, 1983).     

Choice, experience, and the generalized matching law

Five pigeons were exposed to different pairs of concurrent variable-interval, variable-interval schedules on nine experimental conditions of 30 sessions each. For every session, the parameters of the generalized matching equation were computed for the first five, six, seven, eight, and nine experimental conditions. The exponent a, both for response and time distribution, tended to decrease with increases in number of experimental conditions and to increase with number of sessions per condition, but values of k (bias) varied unsystematically. When the subjects were exposed to five new pairs of schedules, with 55 sessions per condition, the findings were confirmed. Data from the literature on the generalized matching law suggest that the variability of exponent values may be explained in part by the use of naive or experienced subjects in different investigations and by the variability in number of experimental conditions and in number of sessions per condition.