The role of the instrumental contingency in the motivational control of performance

In four experiments we investigated an irrelevant incentive effect based upon a transition from hunger to thirst. Hungry rats were trained to lever press either for sucrose solution or for food pellets before performance was tested in extinction while they were thirsty. Reinforcer-specific motivational control was found in the first experiment in that the animals pressed the lever more on tests following training with the sucrose solution rather than with food pellets. Moreover, this effect was seen only when testing was conducted following water, but not following food deprivation. The outcome of the remaining experiments suggests that this motivational control is not mediated by the instrumental contingency between lever pressing and the sucrose reinforcer during training. In these studies lever pressing and chain pulling were reinforced concurrently, one with sucrose and the other with food pellets, in order to equate the noninstrumental functions of the incentives. Following this training, lever pressing in extinction under thirst was unaffected by the type of incentive used as its reinforcer during training.     

Reinforcement: Food Signals the Time and Location of Future Food

It has long been understood that food deliveries may act as signals of future food location, and not only as strengtheners of prefood responding as the law of effect suggests. Recent research has taken this idea further—the main effect of food deliveries, or other “reinforcers”, may be signaling rather than strengthening. The present experiment investigated the ability of food deliveries to signal food contingencies across time after food. In Phase 1, the next food delivery was always equally likely to be arranged for a left‐ or a right‐key response. Conditions were arranged such that the next food delivery was likely to occur either sooner on the left (or right) key, or sooner on the just‐productive (or not‐just‐productive) key. In Phase 2, similar contingencies were arranged, but the last‐food location was signaled by a red keylight. Preference, measured in 2‐s bins across interfood intervals, was jointly controlled by the likely time and location of the next food delivery. In Phase 1, when any food delivery signaled a likely sooner next food delivery on a particular key, postfood preference was strongly toward that key, and moved toward the other key across the interreinforcer interval. In other conditions in which food delivery on the two keys signaled different subsequent contingencies, postfood preference was less extreme, and quickly moved toward indifference. In Phase 2, in all three conditions, initial preference was strongly toward the likely‐sooner food key, and moved to the other key across the interfood interval. In both phases, at a more extended level of analysis, sequences of same‐key food deliveries caused a small increase in preference for the just‐productive key, suggesting the presence of a “reinforcement effect”, albeit one that was very small.     

Acquisition of lever-press responding in rats with delayed reinforcement: A comparison of three procedures

The present study examined the acquisition of lever pressing in rats under three procedures in which food delivery was delayed by 4, 8, and 16 seconds relative to the response. Under the nonresetting delay procedure, food followed the response selected for reinforcement after a specified interval elapsed; responses during this interval had no programmed effect. Under the resetting procedure, the response selected for reinforcement initiated an interval to food delivery that was reset by each subsequent response. Under the stacked delay procedure, every response programmed delivery of food t seconds after its occurrence. Two control groups were studied, one that received food immediately after each lever press and another that never received food. With the exception of the group that did not receive food, responding was established with every procedure at every delay value without autoshaping or shaping. Although responding was established under the resetting delay procedure, response rates were generally not as high as under the other two procedures. These findings support the results of other recent investigations in demonstrating that a response not previously reinforced can be brought to strength by delayed reinforcement in the absence of explicit training.     

On the motivational effects of positive verbal reinforcement on performance: Toward an inverted-U relationship

The purpose of this study was to ascertain the motivational effects of positive verbal reinforcement on the performance of a qualitative task. Male subjects performed on a task consisting of 24 slides that allowed them to test their decision-making abilities in simulated situations. Subjects performed under one of five noncontingent verbal reinforcement ratios (.00, .25, .50, .75, or 1.00). Subjects' accuracy and reaction times to each of the 24 slide stimuli were recorded and transformed into a total performance index. In line with an inverted-U hypothesis, it was hypothesized that if motivational effects of positive verbal reinforcement exist, they should induce a curvilinear relationship between reinforcement ratio and performance. This hypothesis was supported by the results of a trend analysis that showed that only the quadratic curvilinear term was significant. Inspection of means revealed the presence of an inverted-U curve such that performance gradually increased from ratio 0 and reached a maximum at ratio .50, following which a decrease in performance was obtained for ratios .75 and 1.00. Results support a motivational interpretation of the effects of positive verbal reinforcement on performance. Findings are discussed in light of the interaction between motivational and attentional processes.     

Contingency tracking during unsignaled delayed reinforcement

Three experiments were conducted with rats in which responses on one lever (labeled the functional lever) produced reinforcers after an unsignaled delay period that reset with each response during the delay. Responses on a second, nonfunctional, lever did not initiate delays, but, in the first and third experiments, such responses during the last 10 s of a delay did postpone food delivery another 10 s. In the first experiment, the location of the two levers was reversed several times. Responding generally was higher on the functional lever, though the magnitude of the difference diminished with successive reversals. In the second experiment, once a delay was initiated by a response on the functional lever, in different conditions responses on the nonfunctional lever either had no effect or postponed food delivery by 30 s. The latter contingency typically lowered response rates on the nonfunctional lever. In the first two experiments, both the functional and nonfunctional levers were identical except for their location; in the third experiment, initially, a vertically mounted, pole-push lever defined the functional response and a horizontally mounted lever defined the nonfunctional response. Higher response rates occurred on the functional lever. These results taken together suggest that responding generally tracked the response-reinforcer contingency. The results further show how nonfunctional-operanda responses are controlled by a prior history of direct reinforcement of such responses, by the temporal delay between such responses and food delivery, and as simple generalization between the two operanda.     

Prepared to eat: how immediate affective and motivational responses to food cues are influenced by food deprivation

Three studies examined how food deprivation influences the immediate valence of food stimuli as well as spontaneous motivational tendencies toward them. We assumed that immediate reactions towards food stimuli should be tuned to the basic needs of the organism. In Study 1, the immediate valence of food names as a function of need state was assessed using an Implicit Association Test (IAT) in a quasi‐experimental design. Food deprivation led to a more positive immediate valence of food items. In Study 2, these results were replicated using the Extrinsic Affective Simon Task. In Study 3, immediate motivational reactions toward pictorial food stimuli were assessed. As hypothesized, approach reactions were facilitated for participants tested before as compared to after lunch, even in a sample with eating disorders. We thus conclude that the immediate valence of edible objects partially reflects regulation in the service of need fulfillment. Copyright © 2006 John Wiley & Sons, Ltd.     

Choice with delayed and probabilistic reinforcers: effects of variability, time between trials, and conditioned reinforcers.

In a discrete-trials procedure with pigeons, a response on a green key led to a 4-s delay (during which green houselights were lit) and then a reinforcer might or might not be delivered. A response on a red key led to a delay of adjustable duration (during which red houselights were lit) and then a certain reinforcer. The delay was adjusted so as to estimate an indifference point--a duration for which the two alternatives were equally preferred. Once the green key was chosen, a subject had to continue to respond on the green key until a reinforcer was delivered. Each response on the green key, plus the 4-s delay that followed every response, was called one "link" of the green-key schedule. Subjects showed much greater preference for the green key when the number of links before reinforcement was variable (averaging four) than when it was fixed (always exactly four). These findings are consistent with the view that probabilistic reinforcers are analogous to reinforcers delivered after variable delays. When successive links were separated by 4-s or 8-s "interlink intervals" with white houselights, preference for the probabilistic alternative decreased somewhat for 2 subjects but was unaffected for the other 2 subjects. When the interlink intervals had the same green houselights that were present during the 4-s delays, preference for the green key decreased substantially for all subjects. These results provided mixed support for the view that preference for a probabilistic reinforcer is inversely related to the duration of conditioned reinforcers that precede the delivery of food.     

The integration of habits maintained by food and water reinforcement through stimulus compounding.

In Experiment 1, a light and a tone were correlated independently with water reinforcement of bar pressing by rats. With different naive subjects in Experiment 2, one of these stimuli was correlated with food and the other with water reinforcement (counterbalanced). In both experiments the absence of tone and light signaled extinction. Tests of stimulus-reinforcer independence in Experiment 2 indicated that tone and light controlled behavior whose rate was specifically affected by deprivation state. In the stimulus-compounding tests of both experiments, response rates were higher to tone-plus-light than to tone or light presented alone (additive summation). This is the first report of additive summation produced through compounding stimuli paired with different reinforcers. The results are discussed in the context of the effects of incentive motivation on operant performance.     

Self-control in mentally retarded adolescents: choice as a function of amount and delay of reinforcement.

Three severely mentally retarded adolescents were studied under discrete-trial procedures in which a choice was arranged between edible reinforcers that differed in magnitude and, in some conditions, delay. In the absence of delays the larger reinforcer was consistently chosen. Under conditions in which the smaller reinforcer was not delayed, increasing the delay to delivery of the larger reinforcer decreased the percentage of trials in which that reinforcer was chosen. All subjects directed the majority of choice responses to the smaller reinforcer when the larger reinforcer was sufficiently delayed, although the value at which this occurred differed across subjects. Under conditions in which the larger reinforcer initially was sufficiently delayed to result in preference for the smaller one, progressively increasing in 5-s increments the delay to both reinforcers increased percentage of trials with the larger reinforcer chosen. At sufficiently long delays, 2 of the subjects consistently chose the larger, but more delayed, reinforcer, and the 3rd subject chose that reinforcer on half of the trials. These results are consistent with the findings of prior studies in which adult humans responded to terminate noise and pigeons responded to produce food.     

STIMULUS EFFECTS ON LOCAL PREFERENCE: STIMULUS—RESPONSE CONTINGENCIES,STIMULUS—FOOD PAIRING,AND STIMULUS—FOOD CORRELATION

Four pigeons were trained in a procedure in which concurrent‐schedule food ratios changed unpredictably across seven unsignaled components after 10 food deliveries. Additional green‐key stimulus presentations also occurred on the two alternatives, sometimes in the same ratio as the component food ratio, and sometimes in the inverse ratio. In eight experimental conditions, we varied the contingencies surrounding these additional stimuli: In two conditions, stimulus onset and offset were noncontingent; in another two, stimulus onset was noncontingent, and offset was response contingent. In four conditions, both stimulus onset and offset were contingent, and in two of these conditions the stimulus was simultaneously paired with food delivery. Sensitivity to component food ratios was significantly higher when stimulus onset was response contingent compared to when it was noncontingent. Choice changes following food delivery were similar in all eight conditions. Choice changes following stimuli were smaller than those following food, and directionally were completely determined by the food‐ratio:stimulus‐ratio correlation, not by the stimulus contingency nor by whether the stimulus was paired with food or not. These results support the idea that conditional reinforcers may best be viewed as signals for next‐food location rather than as stimuli that have acquired hedonic value, at least when the signals are differential with respect to future conditions.     

Control of behavior by an establishing stimulus

Seventeen pigeons were exposed to a three-key discrete-trial procedure in which a peck on the lit center key produced food if, and only if, the left keylight was lit. The center key was illuminated by a peck on the lit right key. Of interest was whether subjects pecked the right key before or after the response-independent onset of the left keylight. Pecks on the right key after left-keylight onset suggest control of behavior by the left keylight—an establishing stimulus. In three experiments, the strength of center-keylight onset as conditioned reinforcer for a response on the right key was manipulated by altering the size of the reduction in time to food delivery correlated with its onset. Control of pigeons' key pecks by onset of the left keylight occurred on more trials per session when the center keylight was a relatively weak conditioned reinforcer and on fewer trials per session when the center keylight was a relatively strong condtioned reinforcer. Differences across conditions in the degree of control by onset of the establishing stimulus were greatest when changes in conditioned reinforcer strength occurred relatively frequently and were signaled. The results provide evidence of the function of an establishing stimulus.     

Matching and conditioned reinforcement rate

Attempts to examine the effects of variations in relative conditioned reinforcement rate on choice have been confounded by changes in rates of primary reinforcement or changes in the value of the conditioned reinforcer. To avoid these problems, this experiment used concurrent observing responses to examine sensitivity of choice to relative conditioned reinforcement rate. In the absence of observing responses, unsignaled periods of food delivery on a variable-interval 90-s schedule alternated with extinction on a center key (i.e., a mixed schedule was in effect). Two concurrently available observing responses produced 15-s access to a stimulus differentially associated with the schedule of food delivery (S+). The relative rate of S+ deliveries arranged by independent variable-interval schedules for the two observing responses varied across conditions. The relation between the ratio of observing responses and the ratio of S+ deliveries was well described by the generalized matching law, despite the absence of changes in the rate of food delivery. In addition, the value of the S+ deliveries likely remained constant across conditions because the ratio of S+ to mixed schedule food deliveries remained constant. Assuming that S+ deliveries serve as conditioned reinforcers, these findings are consistent with the functional similarity between primary and conditioned reinforcers suggested by general choice theories based on the concatenated matching law (e.g., contextual choice and hyperbolic value-added models). These findings are inconsistent with delay reduction theory, which has no terms for the effects of rate of conditioned reinforcement in the absence of changes in rate of primary reinforcement.     

Self-serving bias in the attribution of responsibility: Cognitive versus motivational explanations

The purpose of this study is to compare the cognitive and motivational explanations of differences in the responsibility assigned to the victim of a crime. It has generally been found that the victim of a crime is assigned more responsibility for that crime by similar others if its consequences are severe. The motivational explanation for this finding is that subjects are threatened by the idea that a serious crime can happen by chance. The cognitive explanation is that serious crimes are rarer and subjects attribute more of the responsibility for rarer events to those who experience them. This study tested these two explanations by independently varying the severity and the likelihood of a crime and examining attributions of responsibility to its victim. The results of the study support the cognitive explanation. Similar others were found to attribute more responsibility for a crime to the victim when that crime was presented as rare, irrespective of whether its consequences were mild or severe. In addition, when likelihood was controlled experimentally no effect of severity upon attributions of responsibility was found. These results suggest that, within the context of attributions of responsibility to victims, patterns of attribution consistent with those predicted by theories suggesting motivational influences upon attribution may actually represent the operation of cognitive processes.     

Delay or rate of food delivery as determiners of response rate

Pigeons were confronted with two keys: a green food key and a white changeover key. Food became available for a peck to the green key after variable intervals of time (mean = 113 seconds). A single peck on the changeover key changed the color of the food key to red for a fixed period of time during which the timing of the variable-interval schedule in green was suspended and the switching option eliminated and after which the conditions associated with green were reinstated. In Experiment 1 a single food presentation was obtainable during each red-key period after a minimum delay timed from the switch. This delay and the duration of the red-key period were held constant during a condition but varied between conditions (delay = 2.5, 7.5, 15, or 30 seconds; red-period duration = 30, 60, 120, 240, or 480 seconds). In Experiment 2 additional food presentations were scheduled during a 240-second red-key period with the delay to the first food delivery held constant at 30 seconds, and the delays to later food deliveries varied over conditions. Considering the data from both experiments, the rate of switching to red was a decreasing function of the delay to the first food, the delay to the second food, and perhaps the delay to the third food after a switch. There was no clear evidence that the rate of food in the red-key period made an independent contribution. The ordering of response rates among conditions was consistent with the view that each food presentation after a response adds an incremental effect to the rate of the response and that each food presentation's contribution is a decreasing function of its delay timed from the response.     

Substitution effects in a generalized token economy with pigeons

Pigeons made repeated choices between earning and exchanging reinforcer‐specific tokens (green tokens exchangeable for food, red tokens exchangeable for water) and reinforcer‐general tokens (white tokens exchangeable for food or water) in a closed token economy. Food and green food tokens could be earned on one panel; water and red water tokens could be earned on a second panel; white generalized tokens could be earned on either panel. Responses on one key produced tokens according to a fixed‐ratio schedule, whereas responses on a second key produced exchange periods, during which all previously earned tokens could be exchanged for the appropriate commodity. Most conditions were conducted in a closed economy, and pigeons distributed their token allocation in ways that permitted food and water consumption. When the price of all tokens was equal and low, most pigeons preferred the generalized tokens. When token‐production prices were manipulated, pigeons reduced production of the tokens that increased in price while increasing production of the generalized tokens that remained at a fixed price. The latter is consistent with a substitution effect: Generalized tokens increased and were exchanged for the more expensive reinforcer. When food and water were made freely available outside the session, token production and exchange was sharply reduced but was not eliminated, even in conditions when it no longer produced tokens. The results join with other recent data in showing sustained generalized functions of token reinforcers, and demonstrate the utility of token‐economic methods for assessing demand for and substitution among multiple commodities in a laboratory context.     

The blocking of reinforcement control

Two experiments were conducted to extend the blocking effect to the reinforcement of a response. A delayed reinforcement contingency was presented to subjects with or without a previously pretrained response available during the delay interval. The interpolated response had no scheduled effect on delivery of the reinforcer, but its availability reduced strengthening of the initial response, which completely extinguished for some subjects. The results were interpreted as support for blocking as a fundamental principle of behavior, and as evidence against the principle of reinforcement being stated solely in terms of temporal proximity between response and reinforcer.     

Operant hoarding: a new paradigm for the study of self-control.

In the first of four experiments, rats were exposed to a modified multiple continuous reinforcement-extinction schedule during 15-min daily sessions. In one condition (saves condition) with the cuelight on, a single lever press produced a food pellet, briefly extinguished the cuelight, and started a clock. Saves (additional lever presses with interresponse times less than 1 s) produced an additional food pellet, briefly extinguished the cuelight, and restarted the interresponse time clock. The cuelight was extinguished 1 s after the last lever press and remained off during a 10-s period of extinction, during which no food pellets were delivered. In the other condition (savings account condition), the contingencies were the same except that the cuelight was extinguished and was not reilluminated after the initial lever press, and the delivery of all food pellets in the reinforcement component was delayed until the onset of extinction. In both conditions, rats made saves, but mean saves (total saves divided by the number of reinforcement components) were slightly reduced in the savings account condition. In Experiment 2, using six equally spaced 15-min sessions per day on alternate days, saves were either followed immediately with food and brief cuelight offset (saves condition) or were not reinforced at all. Mean saves were much greater when saves were reinforced. In Experiment 3, during 5-min daily sessions, saves earned a single pellet (savings account condition) or a number of pellets equal to the ordinal number of the lever press (interest condition). Rats made fewer mean saves, with little change in the food rate, when saves earned interest. In Experiment 4, the rats earned all their food in the operant situation during 24 daily 5-min sessions, these separated by 55-min intersession intervals during which no food was available; otherwise, the conditions were the same as in Experiment 3. In Experiment 4, the shift to interest for saves led to an increase in mean daily mean saves (total daily mean saves divided by the number of daily sessions) as well as to an increase in the number of food pellets delivered in each session. The results are discussed in terms of self-control and behavioral economics.     

Partial reinforcement effects and type of reward

In two experiments 68 rats were trained to bar press or run down a straight runway for food or for water under conditions of either continuous reinforcement or partial reinforcement. In both experiments, there was greater persistence of behavior which had been reinforced with food than with water. In Expt 2, the partial reinforcement extinction effect (PREE) was observed with food reward, but not with water.Within the context of the experimental procedures used, it can be concluded that the rat has mechanisms for developing persistence which are dependent on the specific motivational system involved. This conclusion is related to theories of partial reinforcement effects and to possible biological origins of the mechanisms.     

Topography of signal-centered behavior in the rat: Effects of deprivation state and reinforcer type

In a series of three experiments, groups of food-deprived and water-deprived rats were given pairings of a retractable lever (CS⁺) with response-independent deliveries of either solid or liquid reinforcers. In Experiment 1 food-deprived rats given a solid-pellet reinforcer differentially tended to sniff, paw, mouth, and bite the CS⁺ lever more often than a lever that was not paired with food (CS⁻), whereas food-deprived rats given a liquid reinforcer tended to differentially sniff, paw, and lick the CS⁺ lever. 23½-hour water-deprived rats given liquid reinforcers showed very little CS⁺ contact. In Experiment 2 increasing the severity of water deprivation from 23½ to 47½ hours significantly increased CS⁺ contact. In Experiment 3, subjects that were simultaneously food and water deprived and given a water reinforcer failed to exhibit differential CS⁺ contact, but subjects that were simultaneously food and water deprived and given a food reinforcer did acquire differential CS⁺-contact behavior. These results suggest that (a) even under a single motivational state the nature of signal-centered behavior can be determined by type of reinforcer, (b) although water reinforcement produces less signal contact than food reinforcement, this can be facilitated with more severe water-deprivation levels, and (c) high CS-contact rates using food reinforcement are not simply a product of reductions in body weight with food deprivation.     

Renewal of operant performance formerly eliminated by omission or noncontingency training upon return to the acquisition context

Renewal of operant performance formerly eliminated by omission or noncontingency training was explored in two experiments with rats. When pressing a lever was trained with food reinforcement in one context (A) and then eliminated in a second context (B), responding was renewed by returning the rats to the original context (A). This ABA renewal effect was demonstrated in Experiment 1 when the elimination training was an omission procedure (delivery of food for withholding responding) and in Experiment 2 when it was a noncontingency procedure (delivery of food irrespective of responding). Because omission training (differential reinforcement of other behavior) and noncontingency training have been used in applied settings as effective procedures to reduce undesired human behaviors, the clinical implications of our findings for the relapse of undesirable behavior were discussed.