Response acquisition under targeted percentile schedules: a continuing quandary for molar models of operant behavior.

The number of responses rats made in a "run" of consecutive left-lever presses, prior to a trial-ending right-lever press, was differentiated using a targeted percentile procedure. Under the nondifferential baseline, reinforcement was provided with a probability of .33 at the end of a trial, irrespective of the run on that trial. Most of the 30 subjects made short runs under these conditions, with the mean for the group around three. A targeted percentile schedule was next used to differentiate run length around the target value of 12. The current run was reinforced if it was nearer the target than 67% of those runs in the last 24 trials that were on the same side of the target as the current run. Programming reinforcement in this way held overall reinforcement probability per trial constant at .33 while providing reinforcement differentially with respect to runs more closely approximating the target of 12. The mean run for the group under this procedure increased to approximately 10. Runs approaching the target length were acquired even though differentiated responding produced the same probability of reinforcement per trial, decreased the probability of reinforcement per response, did not increase overall reinforcement rate, and generally substantially reduced it (i.e., in only a few instances did response rate increase sufficiently to compensate for the increase in the number of responses per trial). Models of behavior predicated solely on molar reinforcement contingencies all predict that runs should remain short throughout this experiment, because such runs promote both the most frequent reinforcement and the greatest reinforcement per press. To the contrary, 29 of 30 subjects emitted runs in the vicinity of the target, driving down reinforcement rate while greatly increasing the number of presses per pellet. These results illustrate the powerful effects of local reinforcement contingencies in changing behavior, and in doing so underscore a need for more dynamic quantitative formulations of operant behavior to supplement or supplant the currently prevalent static ones.     

Control over response number by a targeted percentile schedule: reinforcement loss and the acute effects of d-amphetamine.

Two fixed-consecutive-number-like procedures were used to examine effects of acute d-amphetamine administration on control over response number. In both procedures, rats were required to press the left lever at least once and then press the right lever to complete a trial. The consecutive left-lever presses on each trial comprised a "run." Under the targeted percentile schedule, reinforcement was provided if the current run length was closer to the target length (16) than half of the most recent 24 runs. This differentially reinforced run length while holding reinforcement probability constant at .5. A second group acquired the differentiation under the targeted percentile schedule, but were then shifted to a procedure that yoked reinforcement probability by subject and run length to that obtained under the targeted percentile schedule. The two procedures generated practically identical control run lengths, response rates, reinforcement probabilities, and reinforcement rates. Administration of d-amphetamine disrupted percentile responding to a greater degree than yoked control responding. This disruption decreased reinforcement frequency less in the former than the latter procedure. The similar baseline responding under these two procedures suggests that this difference in sensitivity was due to behavioral adjustments to drug prompted by reduction of reinforcement density in the yoked control but not the percentile schedule. These adjustments attenuate the drug's effects under the former, but not the latter, procedure.     

Food-paired stimuli as conditioned reinforcers: effects of d-amphetamine.

Seven pigeons were studied in two experiments in which key pecks were reinforced under a second-order schedule wherein satisfaction of variable-interval schedule requirements produced food or a brief stimulus. In the second part of each session, responses produced only the brief stimulus according to a variable-interval schedule (food extinction). For the 4 pigeons in Experiment 1, the response key was red throughout the session. In separate phases, the brief stimulus was either paired with food, not paired with food, or not presented during extinction. d-Amphetamine (0.3 to 10.0 mg/kg) dose-dependently reduced food-maintained responding during the first part of the session and, at intermediate dosages, increased responding during the extinction portion of the session. The magnitude of these increases, however, did not consistently depend on whether the brief stimulus was paired, not paired, or not presented. It was also true that under nondrug conditions, response rates during extinction did not differ reliably depending on pairing operations for the brief stimulus. In Experiment 2, 3 different pigeons responded under a procedure wherein the key was red in the component with food presentations and blue in the extinction component (i.e., multiple schedule). Again, d-amphetamine produced dose-related decreases in responding during the first part of a session and increases in responding in the second part of the session. These increases, however, were related to the pairing operations; larger increases were observed when the brief stimulus was paired with food than when it was not or when it was not presented at all. Under nondrug conditions, the paired brief stimulus controlled higher response rates during extinction than did a nonpaired stimulus or no stimulus. These findings suggest that d-amphetamine can enhance the efficacy of conditioned reinforcers, and that this effect may be more robust if conditioned reinforcers occur in the context of a signaled period of extinction.     

Cocaine and food as reinforcers: effects of reinforcer magnitude and response requirement under second-order fixed-ratio and progressive-ratio schedules.

Reinforcer magnitude and fixed-ratio requirement were varied under two second-order schedules. Under one, the first sequence of a fixed number of responses completed after the lapse of a 10-min fixed interval produced reinforcement. Under the second, a second-order progressive-ratio schedule, the fixed number of responses increased after each reinforcement. Either cocaine (0 to 300 micrograms/kg/inj) or food (0 to 5,700 mg/delivery) reinforcers were delivered. Under some conditions, a 2-s illumination of stimulus lights occurred on completion of each ratio sequence. Under the second-order schedule, as cocaine dose or amount of food increased, rates of responding increased; at the highest values, rates of responding decreased. Increases in the ratio requirement from 10 to 170 responses minimally decreased overall response rates. Under the second-order progressive-ratio schedule, increases in dose of cocaine or amount of food increased rates of responding; at the highest amounts of food, rates of responding decreased but response rates at the highest dose of cocaine remained relatively high. The highest ratio requirement that was completed (breaking point) depended on the dose of cocaine but was less dependent on the amount of food. Removing brief-stimulus presentations had a greater effect on completion of ratio requirements with cocaine compared to food.     

Molar And Molecular Control In Variable-interval And Variable-ratio Schedules

Response rates are typically higher under variable-ratio than under variable-interval schedules of reinforcement, perhaps because of differences in the dependence of reinforcement rate on response rate or because of differences in the reinforcement of long interresponse times. A variable-interval-with-added-linear-feedback schedule is a variable-interval schedule that provides a response rate/reinforcement rate correlation by permitting the minimum interfood interval to decrease with rapid responding. Four rats were exposed to variable-ratio 15, 30, and 60 food reinforcement schedules, variable-interval 15-, 30-, and 60-s food reinforcement schedules, and two versions of variable-interval-with-added-linear-feedback 15-, 30-, and 60-s food reinforcement schedules. Response rates on the variable-interval-with-added-linear-feedback schedule were similar to those on the variable-interval schedule; all three schedules led to lower response rates than those on the variable-ratio schedules, especially when the schedule values were 30. Also, reinforced interresponse times on the variable-interval-with-added-linear-feedback schedule were similar to those on variable interval and much longer than those produced by variable ratio. The results were interpreted as supporting the hypothesis that response rates on variable-interval schedules in rats are lower than those on comparable variable-ratio schedules, primarily because the former schedules reinforce long interresponse times.     

Further evaluation of reinforcer magnitude effects in noncontingent schedules

We closely replicated the procedures of a previous study that showed a positive relationship between reinforcer magnitude and the response-rate-reducing effects of noncontingent schedules (NCS). NCS reduced response rates, as expected, but the NCS-magnitude effect was not reproduced, illuminating possible weaknesses of current arbitrary-response procedures and suggesting avenues for future research.     

Optimality And Concurrent Variable-interval Variable-ratio Schedules

Despite claims to the contrary, all leading theories about operant choice may be seen as models of optimality. Although melioration is often contrasted with global maximization, both make the same core assumptions as other versions of optimality theory, including momentary maximizing, hill climbing, and the various versions of optimal foraging theory. The present experiment aimed to test melioration against more global optimality and to apply the visit-by-visit analysis suggested by foraging theory. Rats were exposed to concurrent schedules in which one alternative was always variable-ratio 10 and the other alternative was a variable-interval schedule. Although choice relations varied from rat to rat, the overall results roughly confirmed the matching law, a result often taken to support melioration. Pooling the data across sessions and across rats, however, resulted in no increment in unsystematic variance, lending support to the contention by Ziriax and Silberberg (1984) that the choice relation is partly constrained. When the data were analyzed at the level of visits, the results either disconfirmed predictions of melioration or showed regularities about which melioration is silent. Instead, performance tended toward a rough optimization, in which responding favored the variable ratio, but with relatively brief visits to the variable interval. There were no asymmetries in travel or variability that would indicate that different processes were involved in generating visits at the two different schedules. The findings point toward a more global optimality model than melioration and demonstrate the value of per-visit analysis in the study of concurrent performances.     

Reversed effects in closed and open economies

Pigeons received food according to either fixed-interval, fixed-ratio, or random-interval schedules in both closed and open feeding economies. In the closed economy, they were not food deprived, they controlled the amount of food received at each meal, and they had no other source of food. In the open economy, each feeding bout consisted of one feeder cycle, and the pigeons received supplemental feeding as needed to maintain them at 80% of their free-feeding weights. Response rate always increased with larger schedule requirements in the closed economy, but it either decreased steadily or increased and then decreased in the open economy. Initial pauses lengthened with longer fixed intervals or fixed ratios (FR) in the open economy but less so in the closed economy. Responding continued under FR 10,000 schedules in the closed economy, but never survived FR 400 in the open economy. In the open economy, fixed-interval schedules could maintain far more behavior than could either fixed ratios or random intervals. Familiar concepts such as matching and arousal can describe at least some of the behavior in the open economy, but current theory does not apply well to behavior in the closed economy. An explanation of economy-dependent effects might begin with the possibility that the two economies invoke different evolved survival strategies. These strategies influence behavior by means of different mechanisms and laws. The strategy for the closed economy may relate to weight conservation, but that for the open economy may be based on energy conservation.     

Nonstable concurrent choice in pigeons

Six pigeons were trained on concurrent variable-interval schedules in which the arranged reinforcer ratios changed from session to session according to a 31-step pseudorandom binary sequence. This procedure allows a quantitative analysis of the degree to which performance in an experimental session is affected by conditions in previous sessions. Two experiments were carried out. In each, the size of the reinforcer ratios arranged between the two concurrent schedules was varied between 31-step conditions. In Experiment 1, the concurrent schedules were arranged independently, and in Experiment 2 they were arranged nonindependently. An extended form of the generalized matching law described the relative contribution of past and present events to present-session behavior. Total performance in sessions was mostly determined by the reinforcer ratio in that session and partially by reinforcers that had been obtained in previous sessions. However, the initial exposure to the random sequence produced a lower sensitivity to current-session reinforcers but no difference in overall sensitivity to reinforcement. There was no evidence that the size of the reinforcer ratios available on the concurrent schedules affected either overall sensitivity to reinforcement or the sensitivity to reinforcement in the current session. There was also no evidence of any different performance between independent and nonindependent scheduling. Because of these invariances, this experiment validates the use of the pseudorandom sequence for the fast determination of sensitivity to reinforcement.     

Conditioned reinforcement dynamics in three-link chained schedules.

In two experiments rats were trained on three-link concurrent-chains schedules of reinforcement. In Experiment 1, additional entries to one terminal link were added during one of the middle links to a baseline schedule that was otherwise equal for the two chains, and, depending on the condition, these additional terminal-link presentations ended either in food or in no food. When food occurred, preference was always in favor of the chain with the additional terminal-link presentations (which also entailed a higher rate of reinforcement). When no food occurred at the end of the additional terminal links, the outcome depended on the nature of the stimuli associated with these additional terminal links. When stimuli different from the reinforced baseline terminal links were used for the no-food terminal links, preference was against the choice alternative that led to the extra periods of extinction. When the same stimulus was used for the two kinds of terminal links, preference was near indifference, that is, significantly greater than when different stimuli were used. In Experiment 2, rats learned repeated reversals of a simultaneous discrimination under a three-link concurrent-chains schedule, in which the food or no-food choice outcomes were delayed until the end of the chain. Different conditions were defined by the point in the chain at which differential stimuli occurred. When the middle and terminal links provided no differential stimuli, discrimination was acquired more slowly than when differential stimuli occurred in both links. When differential stimuli occurred in the middle but not the terminal links, acquisition rates were intermediate. Both experiments together show that the effects of stimuli in a chain schedule are due partly to the time to food correlated with the stimuli and partly to the time to the next conditioned reinforcer in the sequence.