DYNAMICS OF CHOICE: RELATIVE RATE AND AMOUNT AFFECT LOCAL PREFERENCE AT THREE DIFFERENT TIME SCALES

To examine extended control over local choice, the present study investigated preference in transition as food‐rate ratio provided by two levers changed across seven components within daily sessions, and food‐amount ratio changed across phases. Phase 1 arranged a food‐amount ratio of 4:1 (i.e., the left lever delivered four pellets and the right lever one pellet); Phase 2 reversed the food‐amount ratio to 1:4, and in Phase 3 the food‐amount ratio was 3:2. At a relatively extended time scale, preference was described well by a linear relation between log response ratio and log rate ratio (the generalized matching law). A small amount of carryover occurred from one rate ratio to the next but disappeared after four food deliveries. Estimates of sensitivity to food‐amount ratio were around 1.0 and were independent of rate ratio. Analysis across food deliveries within rate‐ratio components showed that the effect of a small amount was diminished by the presence of a large amount—that is, when a larger amount was present in the situation (three or four pellets), the value of a small amount (one or two pellets) became paltry. More local analysis of visits to the levers between food deliveries showed that postfood visits following a large amount were disproportionately longer than following a small amount. Continuing food deliveries from the same source tended to make visits less dependent on relative amount, but a discontinuation (i.e., food from the other lever) reinstated dependence on relative amount. Analysis at a still smaller time scale revealed preference pulses following food deliveries that confirmed the tendency toward dependence on absolute amount with continuing deliveries, and toward dependence on relative amount following discontinuations. A mathematical model based on a linear‐operator equation accounts for many of the results. The larger and longer preference following a switch to a larger amount is consistent with the idea that local preference depends on relatively extended variables even on short time scales.     

Development and maintenance of choice in a dynamic environment

Four pigeons were exposed to a concurrent procedure similar to that used by Davison, Baum, and colleagues (e.g., Davison & Baum, 2000, 2006) in which seven components were arranged in a mixed schedule, and each programmed a different left∶right reinforcer ratio (1∶27, 1∶9, 1∶3, 1∶1, 3∶1, 9∶1, 27∶1). Components within each session were presented randomly, lasted for 10 reinforcers each, and were separated by 10-s blackouts. These conditions were in effect for 100 sessions. When data were aggregated over Sessions 16-50, the present results were similar to those reported by Davison, Baum, and colleagues: (a) preference adjusted rapidly (i.e., sensitivity to reinforcement increased) within components; (b) preference for a given alternative increased with successive reinforcers delivered via that alternative (continuations), but was substantially attenuated following a reinforcer on the other alternative (a discontinuation); and (c) food deliveries produced preference pulses (immediate, local, increases in preference for the just-reinforced alternative). The same analyses were conducted across 10-session blocks for Sessions 1-100. In general, the basic structure of choice revealed by analyses of data from Sessions 16-50 was preserved at a smaller level of aggregation (10 sessions), and it developed rapidly (within the first 10 sessions). Some characteristics of choice, however, changed systematically across sessions. For example, effects of successive reinforcers within a component tended to increase across sessions, as did the magnitude and length of the preference pulses. Thus, models of choice under these conditions may need to take into account variations in behavior allocation that are not captured completely when data are aggregated over large numbers of sessions.     

Relative sensitivity to reinforcer amount and delay in a self-control choice situation.

Rats were exposed to concurrent-chains schedules in which a single variable-interval schedule arranged entry into one of two terminal-link delay periods (fixed-interval schedules). The shorter delay ended with the delivery of a single food pellet; the longer day ended with a larger number of food pellets (two under some conditions and six under others). In Experiment 1, the terminal-link delays were selected so that under all conditions the ratio of delays would exactly equal the ratio of the number of pellets. But the absolute duration of the delays differed across conditions. In one condition, for example, rats chose between one pellet delayed 5 s and six pellets delayed 30 s; in another condition rats chose between one pellet delayed 10 s and six pellets delayed 60 s. The generalized matching law predicts indifference between the two alternatives, assuming that the sensitivity parameters for amount and delay of reinforcement are equal. The rats' choices were, in fact, close to indifference except when the choice was between one pellet delayed 5 s and six pellets delayed 30 s. That deviation from indifference suggests that the sensitivities to amount and delay differ from each other depending on the durations of the delays. In Experiment 2, rats chose between one pellet following a 5-s delay and six pellets following a delay that was systematically increased over sessions to find a point of indifference. Indifference was achieved when the delay to the six pellets was approximately 55 s. These results are consistent with the possibility that the relative sensitivities to amount and delay differ as a function of the delays.     

Choice and amount of reinforcement in rats

Rats were exposed to concurrent-chains schedules in which the terminal links were equal, fixed-interval (FI) schedules terminating in one or a varying number of food pellets. In most rats, choice proportions for the larger reinforcer increased with increases in reinforcer amount (e.g., from one to five food pellets). When log response ratios were plotted against log reinforcer amount ratios, the results indicated that the effects of reinforcer amount depended on the length of fixed-interval terminal links, by showing that rats undermatched their response ratios to reinforcer amount ratios with the shorter terminal links (FI 5 s, Experiment 1), whereas they overmatched with the longer terminal links (FI 20 s, Experiment 2). These results demonstrated that the manipulation of FI terminal-link schedules affected the sensitivity of choice to reinforcer amount, and are consistent with the previous findings that choice proportions for the larger of two reinforcers (one vs three food pellets) increased with increases in the length of FI terminal-link schedules.     

ISOLATING BEHAVIORAL MECHANISMS OF INTERTEMPORAL CHOICE: NICOTINE EFFECT ON DELAY DISCOUNTING AND AMOUNT SENSITIVITY

Many drugs of abuse produce changes in impulsive choice, that is, choice for a smaller—sooner reinforcer over a larger—later reinforcer. Because the alternatives differ in both delay and amount, it is not clear whether these drug effects are due to the differences in reinforcer delay or amount. To isolate the effects of delay, we used a titrating delay procedure. In phase 1, 9 rats made discrete choices between variable delays (1 or 19 s, equal probability of each) and a delay to a single food pellet. The computer titrated the delay to a single food pellet until the rats were indifferent between the two options. This indifference delay was used as the starting value for the titrating delay for all future sessions. We next evaluated the acute effects of nicotine (subcutaneous 1.0, 0.3, 0.1, and 0.03 mg/kg) on choice. If nicotine increases delay discounting, it should have increased preference for the variable delay. Instead, nicotine had very little effect on choice. In a second phase, the titrated delay alternative produced three food pellets instead of one, which was again produced by the variable delay (1 s or 19 s) alternative. Under this procedure, nicotine increased preference for the one pellet alternative. Nicotine‐induced changes in impulsive choice are therefore likely due to differences in reinforcer amount rather than differences in reinforcer delay. In addition, it may be necessary to include an amount sensitivity parameter in any mathematical model of choice when the alternatives differ in reinforcer amount.     

Choice between sequences of fixed-ratio schedules: effects of ratio values and probability of food delivery.

Pigeons were exposed to schedules of food delivery that consisted of two sequential fixed ratios. When alternative sequences provided two food deliveries per 50 responses, the schedule with the shorter initial fixed-ratio value was consistently preferred. Progressively reducing from 1.0 to .25 the probability of food delivery following completion of the second fixed ratio of the sequence with the shorter initial fixed ratio did not reduce preference for this sequence. Moreover, the sequence with the shorter initial fixed ratio also was preferred when the probability of food delivery following completion of the initial ratio in that sequence was progressively reduced from 1.0 to .5, although preference shifted to the alternative when the probability was reduced to 0. These findings suggest that the length of the initial fixed ratio was a primary determinant of choice. Subsequent manipulations demonstrated, however, that when the initial fixed ratios of the two alternatives were equal, changes in the ratio value and probability of food delivery following completion of the second fixed ratio lawfully affected choice.     

Local effects of delayed food

Five pigeons were trained on a procedure in which seven concurrent variable-interval schedules arranged seven different food-rate ratios in random sequence in each session. Each of these components lasted for 10 response-produced food deliveries, and components were separated by 10-s blackouts. We varied delays to food (signaled by blackout) between the two response alternatives in an experiment with three phases: In Phase 1, the delay on one alternative was 0 s, and the other was varied between 0 and 8 s; in Phase 2, both delays were equal and were varied from 0 to 4 s; in Phase 3, the two delays summed to 8 s, and each was varied from 1 to 7 s. The results showed that increasing delay affected local choice, measured by a pulse in preference, in the same way as decreasing magnitude, but we found also that increasing the delay at the other alternative increased local preference. This result casts doubt on the traditional view that a reinforcer strengthens a response depending only on the reinforcer's value discounted by any response-reinforcer delay. The results suggest that food guides, rather than strengthens, behavior.     

Reinforcement: Food Signals the Time and Location of Future Food

It has long been understood that food deliveries may act as signals of future food location, and not only as strengtheners of prefood responding as the law of effect suggests. Recent research has taken this idea further—the main effect of food deliveries, or other “reinforcers”, may be signaling rather than strengthening. The present experiment investigated the ability of food deliveries to signal food contingencies across time after food. In Phase 1, the next food delivery was always equally likely to be arranged for a left‐ or a right‐key response. Conditions were arranged such that the next food delivery was likely to occur either sooner on the left (or right) key, or sooner on the just‐productive (or not‐just‐productive) key. In Phase 2, similar contingencies were arranged, but the last‐food location was signaled by a red keylight. Preference, measured in 2‐s bins across interfood intervals, was jointly controlled by the likely time and location of the next food delivery. In Phase 1, when any food delivery signaled a likely sooner next food delivery on a particular key, postfood preference was strongly toward that key, and moved toward the other key across the interreinforcer interval. In other conditions in which food delivery on the two keys signaled different subsequent contingencies, postfood preference was less extreme, and quickly moved toward indifference. In Phase 2, in all three conditions, initial preference was strongly toward the likely‐sooner food key, and moved to the other key across the interfood interval. In both phases, at a more extended level of analysis, sequences of same‐key food deliveries caused a small increase in preference for the just‐productive key, suggesting the presence of a “reinforcement effect”, albeit one that was very small.     

RISKY CHOICE IN PIGEONS: PREFERENCE FOR AMOUNT VARIABILITY USING A TOKEN-REINFORCEMENT SYSTEM

Pigeons were given repeated choices between variable and fixed numbers of token reinforcers (stimulus lamps arrayed above the response keys), with each earned token exchangeable for food. The number of tokens provided by the fixed‐amount option remained constant within blocks of sessions, but varied parametrically across phases, assuming values of 2, 4, 6, or 8 tokens per choice. The number of tokens provided by the variable‐amount option varied between 0 and 12 tokens per choice, arranged according to an exponential or rectangular distribution. In general, the pigeons strongly preferred the variable option when the fixed option provided equal or greater numbers of tokens than the variable amount. Preference for the variable amount decreased only when the alternatives provided widely disparate amounts favoring the fixed amount. When tokens were removed from the experimental context, preference for the variable option was reduced or eliminated, suggesting that the token presentation played a key role in maintaining risk‐prone choice patterns. Choice latencies varied inversely with preferences, suggesting that local analyses may provide useful ancillary measures of reinforcer value. Overall, the results indicate that systematic risk sensitivity can be attained with respect to reinforcer amount, and that tokens may be critical in the development of such preferences.     

STIMULUS EFFECTS ON LOCAL PREFERENCE: STIMULUS—RESPONSE CONTINGENCIES,STIMULUS—FOOD PAIRING,AND STIMULUS—FOOD CORRELATION

Four pigeons were trained in a procedure in which concurrent‐schedule food ratios changed unpredictably across seven unsignaled components after 10 food deliveries. Additional green‐key stimulus presentations also occurred on the two alternatives, sometimes in the same ratio as the component food ratio, and sometimes in the inverse ratio. In eight experimental conditions, we varied the contingencies surrounding these additional stimuli: In two conditions, stimulus onset and offset were noncontingent; in another two, stimulus onset was noncontingent, and offset was response contingent. In four conditions, both stimulus onset and offset were contingent, and in two of these conditions the stimulus was simultaneously paired with food delivery. Sensitivity to component food ratios was significantly higher when stimulus onset was response contingent compared to when it was noncontingent. Choice changes following food delivery were similar in all eight conditions. Choice changes following stimuli were smaller than those following food, and directionally were completely determined by the food‐ratio:stimulus‐ratio correlation, not by the stimulus contingency nor by whether the stimulus was paired with food or not. These results support the idea that conditional reinforcers may best be viewed as signals for next‐food location rather than as stimuli that have acquired hedonic value, at least when the signals are differential with respect to future conditions.     

Rapid acquisition of choice and timing and the provenance of the terminal-link effect

Eight pigeons responded in a concurrent-chains procedure in which terminal-link schedules changed pseudorandomly across sessions. Pairs of terminal-link delays either summed to 15 s or to 45 s. Across sessions, the location of the shorter terminal link changed according to a pseudorandom binary sequence. On some terminal links, food was withheld to obtain start and stop times, measures of temporal control. Log initial-link response ratios stabilized within the first half of each session. Log response ratio was a monotonically-increasing but nonlinear function of programmed log terminal-link immediacy ratio. There was an effect of absolute terminal-link duration on log response ratio: For most subjects, preference for the relatively shorter terminal-link delay was stronger when absolute delays were long than when absolute delays were short. Polynomial regressions and model comparison showed that differences in degree of nonlinearity, not in sensitivity to log immediacy ratio, produced this effect. Temporal control of stop times was timescale invariant with scalar variability, but temporal control of start times was not consistent across subjects or terminal-link durations.     

Every reinforcer counts: reinforcer magnitude and local preference

Six pigeons were trained on concurrent variable-interval schedules. Sessions consisted of seven components, each lasting 10 reinforcers, with the conditions of reinforcement differing between components. The component sequence was randomly selected without replacement. In Experiment 1, the concurrent-schedule reinforcer ratios in components were all equal to 1.0, but across components reinforcer-magnitude ratios varied from 1:7 through 7:1. Three different overall reinforcer rates were arranged across conditions. In Experiment 2, the reinforcer-rate ratios varied across components from 27:1 to 1:27, and the reinforcer-magnitude ratios for each alternative were changed across conditions from 1:7 to 7:1. The results of Experiment 1 replicated the results for changing reinforcer-rate ratios across components reported by Davison and Baum (2000, 2002): Sensitivity to reinforcer-magnitude ratios increased with increasing numbers of reinforcers in components. Sensitivity to magnitude ratio, however, fell short of sensitivity to reinforcer-rate ratio. The degree of carryover from component to component depended on the reinforcer rate. Larger reinforcers produced larger and longer postreinforcer preference pulses than did smaller reinforcers. Similar results were found in Experiment 2, except that sensitivity to reinforcer magnitude was considerably higher and was greater for magnitudes that differed more from one another. Visit durations following reinforcers measured either as number of responses emitted or time spent responding before a changeover were longer following larger than following smaller reinforcers, and were longer following sequences of same reinforcers than following other sequences. The results add to the growing body of research that informs model building at local levels.     

DO CONDITIONAL REINFORCERS COUNT?

Six pigeons were trained on a procedure in which seven components arranged different food-delivery ratios on concurrent variable-interval schedules each session. The components were unsignaled, lasted for 10 food deliveries, and occurred in random order with a 60-s blackout between components. The schedules were arranged using a switching-key procedure in which two responses on a center key changed the schedules and associated stimuli on two side keys. In Experiment 1, over five conditions, an increasing proportion of food deliveries accompanied by a magazine light was replaced with the presentation of the magazine light only. Local analyses of preference showed preference pulses toward the alternative that had just produced either a food-plus-magazine-light or magazine-light-only presentation, but pulses after food deliveries were always greater than those after magazine lights. Increasing proportions of magazine lights did not change the size of preference pulses after food or magazine-light presentations. Experiment 2 investigated the effects of correlations between food ratios and magazine-light ratios: In Condition 6, magazine-light ratios in components were inversely correlated (-1.0) with food ratios, and in Condition 7, magazine-light ratios were uncorrelated with food ratios. In Conditions 8 and 9, pecks also produced occasional 2.5-s flashes of a green keylight. In Condition 8, food and magazine-light ratios were correlated 1.0 whereas food and green-key ratios were correlated -1.0. In Condition 9, food and green-key ratios were correlated 1.0 whereas food and magazine-light ratios were correlated -1.0. Preference pulses toward alternatives after magazine lights and green keys depended on the correlation between these event ratios and the food ratios: If the ratios were correlated +1.0, positive preference pulses resulted; if the correlation was -1.0, preference pulses were negative. These results suggest that the Law of Effect has more to do with events signaling consequences than with strengthening responses.     

Choice between rewards differing in amount and delay: Toward a choice model of self control

A concurrent-chain procedure was used to study pigeons' choices between rewards differing in both amount and delay. The shorter delay terminated with a 2-second access to grain whereas the longer delay terminated with a 6-second access to grain. The ratio of the delays was constant within a given condition while their absolute values were varied. Over conditions, ratios of 6:1, 3:1, and 3:2 were studied. As the absolute values of the delays to reinforcement increased, preference for the longer-delayed but larger reward decreased under both the 6:1 and 3:1 ratios, but increased under the 3:2 ratio. These results are inconsistent with choice models predicting no change in preference when the ratios of delays and amounts are held constant. In addition, the change in preference under the 3:1 ratio is inconsistent with a simple multiplicative interaction of the trade off between reinforcer amount and delay, and suggests that delay is a more potent determinant of choice than is amount. These results have implications for models that view choice between small immediate rewards and large but delayed rewards as underlying the behavior commonly called self control.     

Determinants of pigeons' choices in token-based self-control procedures

Four pigeons were exposed to a token-based self-control procedure with stimulus lights serving as token reinforcers. Smaller-reinforcer choices produced one token immediately; larger-reinforcer choices produced three tokens following a delay. Each token could be exchanged for 2-s access to food during a signaled exchange period each trial. The main variables of interest were the exchange delays (delays from the choice to the exchange stimulus) and the food delays (also timed from the choice), which were varied separately and together across blocks of sessions. When exchange delays and food delays were shorter following smaller-reinforcer choices, strong preference for the smaller reinforcer was observed. When exchange delays and food delays were equal for both options, strong preference for the larger reinforcer was observed. When food delays were equal for both options but exchange delays were shorter for smaller-reinforcer choices, preference for the larger reinforcer generally was less extreme than under conditions in which both exchange and food delays were equal. When exchange delays were equal for both options but food delays were shorter for smaller-reinforcer choices, preference for the smaller reinforcer generally was less extreme than under conditions in which both exchange and food delays favored smaller-reinforcer choices. On the whole, the results were consistent with prior research on token-based self-control procedures in showing that choices are governed by reinforcer immediacy when exchange and food delays are unequal and by reinforcer amount when exchange and food delays are equal. Further, by decoupling the exchange delays from food delays, the results tentatively support a role for the exchange stimulus as a conditioned reinforcer.     

Matching and conditioned reinforcement rate

Attempts to examine the effects of variations in relative conditioned reinforcement rate on choice have been confounded by changes in rates of primary reinforcement or changes in the value of the conditioned reinforcer. To avoid these problems, this experiment used concurrent observing responses to examine sensitivity of choice to relative conditioned reinforcement rate. In the absence of observing responses, unsignaled periods of food delivery on a variable-interval 90-s schedule alternated with extinction on a center key (i.e., a mixed schedule was in effect). Two concurrently available observing responses produced 15-s access to a stimulus differentially associated with the schedule of food delivery (S+). The relative rate of S+ deliveries arranged by independent variable-interval schedules for the two observing responses varied across conditions. The relation between the ratio of observing responses and the ratio of S+ deliveries was well described by the generalized matching law, despite the absence of changes in the rate of food delivery. In addition, the value of the S+ deliveries likely remained constant across conditions because the ratio of S+ to mixed schedule food deliveries remained constant. Assuming that S+ deliveries serve as conditioned reinforcers, these findings are consistent with the functional similarity between primary and conditioned reinforcers suggested by general choice theories based on the concatenated matching law (e.g., contextual choice and hyperbolic value-added models). These findings are inconsistent with delay reduction theory, which has no terms for the effects of rate of conditioned reinforcement in the absence of changes in rate of primary reinforcement.     

Effects of reinforcement magnitude and ratio values on behaviour maintained by a cyclic ratio schedule of reinforcement

In Experiments 1 and 2, lever pressing by rats was reinforced on a cyclic ratio schedule of food reinforcement, comprising a repeated sequence of fixed-ratio component schedules. Reinforcement magnitude was varied, on occasional sessions in Experiment 1 and across blocks of sessions in Experiment 2, from one to two or three 45-mg food pellets. In the one-pellet condition, post-reinforcement pauses increased with component schedule value. At higher magnitudes, post-reinforcement pauses increased, and overall response rates declined. Response rate on component schedules was a decreasing linear function of the obtained rate of reinforcement in all conditions. Plotted against component schedule value, response rate increased exponentially to an asymptote that decreased when reinforcement magnitude increased. These findings are consistent with regulatory accounts of food reinforced behaviour. In Experiment 3, rats were trained under a cyclic ratio schedule comprising fixed-ratio components including higher values, and some inverted U-shaped response functions were obtained. Those rats that did not showthis relationship were trained on cyclic ratios with even higher values, and all showed inverted U-shaped response functions. This suggests that behaviour on cyclic ratio schedules can reflect activating of reinforcement as well as the satiating effects seen in Experiments 1 and 2.     

Response acquisition and fixed-ratio escalation based on interresponse times in rats

The effectiveness of a fixed‐ratio (FR) escalation procedure, developed by Pinkston and Branch (2004) and based on interresponse times (IRTs), was assessed during lever‐press acquisition. Forty‐nine experimentally naïve adult male Long Evans rats were deprived of food for 24 hr prior to an extended acquisition session. Before the start of the session, three food pellets were placed in the magazine. Otherwise, no magazine training, shaping, nor autoshaping procedure was employed. The first 20 presses each resulted in the delivery of a 45‐mg food pellet. Then, the FR increased (2, 4, 8, 11, 16, 20, 25, 30) when each IRT in the ratio was less than 2 s during three consecutive ratios. Sessions lasted 13 hr or until 500 pellets were earned. On average, rats reached a terminal ratio of 11 (mean) or 16 (median) during the first session. Seven rats reached the maximum value of FR 30 and only one rat did not acquire the response. In most rats, a break‐and‐run pattern of responding characteristic of FR schedules began to develop in this acquisition session. Subsequently, the ratio‐escalation procedure continued during daily 2‐hr sessions. In these sessions, the starting ratio requirement was set at the terminal ratio reached in the previous session. Using this procedure, over half (26) of the rats reached the FR 30 requirement by the fourth session. These data demonstrate that a ratio‐escalation procedure based on IRTs provides a time‐efficient way of establishing ratio responding.     

Concurrent-schedule performance in transition: changeover delays and signaled reinforcer ratios

Six pigeons were trained in experimental sessions that arranged six or seven components with various concurrent-schedule reinforcer ratios associated with each. The order of the components was determined randomly without replacement. Components lasted until the pigeons had received 10 reinforcers, and were separated by 10-s blackout periods. The component reinforcer ratios arranged in most conditions were 27:1, 9:1, 3:1, 1:1, 1:3, 1:9 and 1:27; in others, there were only six components, three of 27:1 and three of 1:27. In some conditions, each reinforcement ratio was signaled by a different red-yellow flash frequency, with the frequency perfectly correlated with the reinforcer ratio. Additionally, a changeover delay was arranged in some conditions, and no changeover delay in others. When component reinforcer ratios were signaled, sensitivity to reinforcement values increased from around 0.40 before the first reinforcer in a component to around 0.80 before the 10th reinforcer. When reinforcer ratios were not signaled, sensitivities typically increased from zero to around 0.40. Sensitivity to reinforcement was around 0.20 lower in no-changeover-delay conditions than in changeover-delay conditions, but increased in the former after exposure to changeover delays. Local analyses showed that preference was extreme towards the reinforced alternative for the first 25 s after reinforcement in changeover-delay conditions regardless of whether components were signaled or not. In no-changeover-delay conditions, preference following reinforcers was either absent, or, following exposure to changeover delays, small. Reinforcers have both local and long-term effects on preference. The former, but not the latter, is strongly affected by the presence of a changeover delay. Stimulus control may be more closely associated with longer-term, more molar, reinforcer effects.     

Effects of free food deliveries and temporal contiguity on choice under concurrent-chain schedules

Eight pigeons responded in a concurrent chain with variable-interval (VI) 10-sec and VI 20-sec terminal links. Free food deliveries were then added to the initial links according to a variable-time (VT) 20-sec schedule in two conditions that differed in terms of whether a differential reinforcement of other (DRO) contingency was also arranged, which ensured that those deliveries could not occur within 2-sec of a response. Preference for the VI 10-sec terminal link increased when VT food was added, but not when the DRO contingency was operative, showing that free food deliveries affected preference only when those deliveries could be temporally contiguous with choice responding. This finding suggests that Mazur’s (2003) report of increased preference with added VT food, replicated here, was due to adventitious reinforcement. Current models for behavioral choice are limited because they are based entirely on temporal relations between stimuli and reinforcers and fail to take into account response-reinforcer contiguity.