Fix and sample with rats in the dynamics of choice

The generality of the molar view of behavior was extended to the study of choice with rats, showing the usefulness of studying order at various levels of extendedness. Rats' presses on two levers produced food according to concurrent variable-interval variable-interval schedules. Seven different reinforcer ratios were arranged within each session, without cues identifying them, and separated by blackouts. To alternate between levers, rats pressed on a third changeover lever. Choice changed rapidly with changes in component reinforcer ratio, and more presses occurred on the lever with the higher reinforcer rate. With continuing reinforcers, choice shifted progressively in the direction of the reinforced lever, but shifted more slowly with each new reinforcer. Sensitivity to reinforcer ratio, as estimated by the generalized matching law, reached an average of 0.9 and exceeded that documented in previous studies with pigeons. Visits to the more-reinforced lever preceded by a reinforcer from that lever increased in duration, while all visits to the less-reinforced lever decreased in duration. Thus, the rats' performances moved faster toward fix and sample than did pigeons' performances in previous studies. Analysis of the effects of sequences of reinforcer sources indicated that sequences of five to seven reinforcers might have sufficed for studying local effects of reinforcers with rats. This study supports the idea that reinforcer sequences control choice between reinforcers, pulses in preference, and visits following reinforcers.     

Punishment and the potential for negative reinforcement with histamine injection

The present study examined punishment of responding with histamine injection, and its potential to generate avoidance of punishment. Sprague–Dawley rats were trained under concurrent schedules in which responses on one lever (the punishment lever) produced food under a variable‐interval schedule, and under some conditions intermittent injections of histamine, which suppressed behavior. Responses on a second (avoidance) lever prevented histamine injections scheduled on the punishment lever. After stabilization of punished responding, a variable‐interval 15‐s schedule of cancellation of histamine (avoidance) was added for responding on the second/avoidance lever, without subsequent acquisition of responding on that lever. Progressive decreases in the length of the punishment variable‐interval schedule increased suppression on the punishment lever without increases in response rates on the avoidance lever. Exchanging contingencies on the levers ensured that response rates on the avoidance lever were sufficiently high to decrease the histamine injection frequency; nonetheless response rates on the avoidance lever decreased over subsequent sessions. Under no condition was responding maintained on the avoidance lever despite continued punishing effectiveness of histamine throughout. The present results suggest that avoidance conditioning is not a necessary condition for effective punishment, and confirm the importance of empirical rather than presumed categorization of behavioral effects of stimulus events.     

Concurrent performances: reinforcement by different doses of intravenous cocaine in rhesus monkeys

Different doses of intravenous cocaine reinforced the lever pressing of rhesus monkeys under two-lever concurrent or concurrent-chain schedules. Under the concurrent procedure, responding produced drug reinforcers arranged according to independent variable-interval 1-min schedules. Under the concurrent-chain procedure, responding in the variable-interval link led to one of two mutually exclusive, equal-valued, fixed-ratio links; completion of the ratio produced a drug reinforcer. Under both procedures, responding on one lever produced a constant dose of 0.05 or 0.1 mg/kg/injection, while on the other lever, dose was systematically varied within a range of 0.013 to 0.8 mg/kg/injection. Preference, indicated by relative response frequency on the variable-dose lever during the variable-interval link, was always for the larger of the doses. Relative response frequencies on the variable-dose lever roughly matched relative drug intake (mg/kg of drug obtained on variable lever divided by mg/kg of drug obtained on both levers). For many dose comparisons, responding occurred and reinforcers were obtained almost exclusively on the preferred lever. Overall variable-interval rates generally were lower than with other reinforcers, and these low rates, under the experimental conditions, may have occasioned the exclusive preferences.     

Fixed-interval schedule of cocaine reinforcement: effect of dose and infusion duration

Rhesus monkeys were trained on a fixed-interval 9-min limited-hold 3-min schedule of intravenous cocaine reinforcement. A 15-min timeout followed each reinforcement or limited-hold expiration. An identical schedule of food reinforcement was interspersed in the session to assess rate-modifying effects of the drug infusions not specific to drug reinforcement. In one experiment, response rate for cocaine reinforcement was shown to be a positive function of reinforcement magnitude for a dose range from 0 to 800 ug/kg/inj. At these doses, there was little effect on food reinforced responding except at the highest dose, where responding decreased. Results of the second experiment indicated that increasing the duration of the cocaine infusion produced a change in response rate similar to decreasing unit dose. The response rate change for a given increase in infustion duration was less at a unit dose of 400 ug/kg than at 200 ug/kg.     

Risky choice as a function of amount and variance in food supply.

In Experiment 1, 4 rats earned their daily food ration by choosing on a trials basis between a "risky" and a "riskless" lever. The risky lever produced either 15 45-mg food pellets or no pellets, and on average provided five pellets per choice. The riskless lever always produced three pellets. Across conditions, the number of trials per session was varied. Body weight and choice of the risky lever decreased as the number of trials per session decreased, even though body weight could only be defended by increased choice of the risky lever. In Experiment 2, trials per session were fixed, but the number of pellets delivered by the risky and riskless levers was either at the same level as in Experiment 1 or tripled from those levels. Now choice of the risky lever was inversely related to the size of reinforcement and to body weight. The results of these experiments show that risk aversion covaries with the amount of food available in a session and the daily variance in the amount of food earned.     

Behavioral and pharmacological variables affecting risky choice in rats.

The effects of manipulations of response requirement, intertrial interval (ITI), and psychoactive drugs (ethanol, phencyclidine, and d-amphetamine) on lever choice under concurrent fixed-ratio schedules were investigated in rats. Responding on the "certain' lever produced three 45-mg pellets, whereas responding on the "risky" lever produced either 15 pellets (p = .33) or no pellets (p .67). Rats earned all food during the session, which ended after 12 forced trials and 93 choice trials or 90 min, whichever occurred first. When the response requirement was increased from 1 to 16 and the ITI was 20 s, percentage of risky choice was inversely related to fixed-ratio value. When only a single response was required but the ITI was manipulated between 20 and 120 s (with maximum session duration held constant), percentage of risky choice was directly related to length of the ITI. The effects of the drugs were investigated first at an ITI of 20 s, when risky choice was low for most rats, and then at an ITI of 80 s, when risky choice was higher for most rats. Ethanol usually decreased risky choice. Phencyclidine did not usually affect risky choice when the ITI was 20 s but decreased it in half the rats when the ITI was 80 s. For d-amphetamine, the effects appeared to he related to baseline probability of risky choice; that is, low probabilities were increased and high probabilities were decreased. Although increase in risky choice as a function of the ITI is at variance with previous ITI data, it is consistent with foraging data showing that risk aversion decreases as food availability decreases. The pharmacological manipulations showed that drug effects on risky choice may be influenced by the baseline probability of risky choice, just as drug effects can be a function of baseline response rate.     

A method for determining the aversive threshold in the rat using repeated measures: Tests with morphine sulfate

The procedure described here provides an objective automated technique for the psychophysical assessment of the aversive threshold and allows the animal Ss to serve as their own controls. Sensitivity to drug effects, as well as comparisons between drugs, using the same animals are possible also. Using a rectangular tilt cage, the aversive threshold to grid shock was defined as that intensity avoided 75% of the time. Each shock intensity (30, 60, 90, 120, and 150 microA) was presented for 5 min on one side of the cage and then switched to the other side for 5 min, thereby forcing the animal to sample each shock intensity. Aversive thresholds were determined daily for each rat. The analgesic effects of morphine sulfate were compared to saline. Animals were tested for 3 days at each morphine dose level (2, 4, 8, and 16 mg/kg). Each 3-day morphine series alternated with 3 days of testing under saline. Significant differences were detected between saline and morphine at 4, 8, and 16 mg/kg.     

Response induction during the acquisition and maintenance of lever pressing with delayed reinforcement

The acquisition of lever pressing by rats and the occurrence of unreinforced presses at a location different from that of the reinforced response were studied using different delays of reinforcement. An experimental chamber containing seven identical adjoining levers was used. Only presses on the central (operative) lever produced food pellets. Groups of 3 rats were exposed to one of seven different tandem random-interval (RI) fixed-time (FT) schedules. The average RI duration was the complement of the FT duration such that their sum yielded a nominal 32-s interreinforcement interval on average. Response rate on the operative lever decreased as the FT value was lengthened. The spatial distribution of responses on the seven levers converged on the operative lever when the FT was 0 or 2 s and spread across the seven levers as the FT value was lengthened to 16 or 32 s. Presses on the seven levers were infrequent during the FT schedule. Both operative- and inoperative-lever pressing intertwined in repetitive patterns that were consistent within subjects but differed between subjects. These findings suggest that reinforcer delay determined the response-induction gradient.     

Discrimination between reinforced action patterns in the rat

Laboratory rats were rewarded for face-washing, rearing, or scratching by being given the opportunity to press retractable levers for food reward. Yoked control animals received the same number of lever presentations and food rewards, but did not have to face-wash, rear, or scratch to obtain the levers. The experimental animals showed increases in number of bouts of reinforced target behavior above control levels, and the total amount of time spent face-washing increased when a 1.5-sec criterion for reinforced bout length was introduced. The activities in this experiment were made to serve a discriminative as well as an instrumental function, since the cue to tell the rat which lever to press for reward when the levers were presented was the activity that the rat had engaged in to obtain lever presentation. In two separate experiments high levels of discrimination between behaviors were obtained. Discrimination was worse following scratching than after other actions, and scratching also showed relatively poor instrumental conditioning. The relationship between Pavlovian and instrumental conditioning processes in this situation is discussed.     

Effects of cocaine on fixed-interval responding reinforced by the opportunity to run

Rate-dependent drug effects have been observed for operant responding maintained by food, water, heat, light onset, electrical brain stimulation, shock-stimulus termination, and shock presentation. The present study sought to determine if the effects of cocaine on lever pressing maintained by the opportunity to run could also be described as rate dependent. Seven male Wistar rats were trained to respond on levers for the opportunity to run in a wheel. The schedule of reinforcement was fixed-interval 60 s, and the reinforcing consequence was the opportunity to run for 60 s. On this schedule, overall rates of responding were low, usually below six presses per minute, and pauses frequently exceeded the 60-s interval. Despite these differences, an overall scalloped pattern of lever pressing was evident for each rat. Doses of 1, 2, 4, 8, and 16 mg/kg cocaine were administered 10 min prior to a session. Only at the 16 mg/kg dose did the responding of the majority of rats change in a manner suggestive of a rate-dependent drug effect. Specifically, lower response rates at the beginning of the intervals increased and higher rates at the end of the intervals decreased, as indicated by the fact that slopes from the regression of drug rates on control rates decreased. These data provide tentative support for the generalization of rate-dependent effects to operant responding maintained by wheel running. Differences in the baseline performance maintained by wheel running compared to those for food and water point to the need for further experimentation before this effect can be firmly established.     

The conditioned reinforcement of repeated acquisition

Three monkeys were trained to emit a chain of three responses on three separate levers in a set of six levers to obtain food. The chain producing food (correct chain) was changed each day. During a trial, a press on any lever produced a feedback stimulus; a press on a correct lever produced an additional distinctive stimulus; the third correct press produced a food pellet. Test sessions in which either the food or the distinctive stimuli were removed were interspersed with baseline sessions. In tests without food presentations, the subjects acquired the correct chain rapidly, with a level of accuracy comparable to baseline. Removing the distintive stimuli for either the first or second member of the correct chain greatly retarded acquisition of that member of the chain. Removing all distinctive stimuli often reduced accuracy throughout the chain to chance level, even though food was presented following each correct chain. These results were interpreted as evidence that the distinctive stimuli presented after correct responses functioned as conditioned reinforcers. Reductions in accuracy following an omitted distinctive stimulus indicated that they were also discriminative stimuli for correct responding in their presence.     

Economic substitutability of electrical brain stimulation, food, and water.

Concurrent variable-ratio schedules of electrical brain stimulation, food, and water were paired in various combinations as reinforcement of rats' lever presses. Relative prices of the concurrent reinforcers were varied by changing the ratio of the response requirements on the two levers. Economic substitutability, measured by the sensitivity of response ratio to changes in relative price, was highest with brain stimulation reinforcement of presses on both levers and lowest with food reinforcement of presses on one lever and water reinforcement of presses on the other. Substitutability with brain stimulation reinforcement of presses on one lever and either food or water reinforcement for presses on the other was about as high as with brain stimulation for presses on both levers. Electrical brain stimulation for rats may thus serve as an economic substitute for two reinforcers, neither of which is substitutable for the other.     

Effects of chlordiazepoxide and cocaine on concurrent food and avoidance-of-timeout schedules.

Five rats were trained on a concurrent schedule in which responses on one lever produced a food pellet on a random-interval 30-s schedule during 10 s of food availability associated with distinctive exteroceptive stimuli. Responses on another lever postponed for 20 s the presentation of a 50-s timeout, during which all stimuli were extinguished and the schedule contingencies on the food lever were suspended. The response rates maintained by the random-interval schedule exceeded those maintained by the avoidance contingency, but both provided a stable baseline to assess the behavioral effects of different drugs. Low doses of cocaine hydrochloride (1 and 3 mg/kg) did not affect food-reinforced responding or avoidance response rates. Intermediate doses (5.6, 10, and 13 mg/kg) produced a dose-dependent decrease in food-maintained and avoidance response rates, and both types of responding were virtually eliminated after administration of the highest doses (17 and 30 mg/kg) of cocaine. Low doses of chlordiazepoxide (1 and 3 mg/kg) increased food-maintained and avoidance response rates, and both rates decreased systematically after 10 and 30 mg/kg of this drug. The effects of cocaine and chlordiazepoxide on response rates maintained by avoidance of timeout from food presentation were unlike those reported when subjects responded to avoid shock presentation. The results of this experiment thus provide evidence to suggest that the effects of drug administration on avoidance behavior may be a function of the nature of the consequent event to be avoided.     

Drug discrimination in rats under concurrent variable-interval variable-interval schedules

Eight rats were trained to discriminate pentobarbital from saline under a concurrent variable-interval (VI) VI schedule, on which responses on the pentobarbital-biased lever after pentobarbital were reinforced under VI 20 s and responses on the saline-biased lever were reinforced under VI 80 s. After saline, the reinforcement contingencies programmed on the two levers were reversed. The rats made 62.3% of their responses on the pentobarbital-biased lever after pentobarbital and 72.2% on the saline-biased lever after saline, both of which are lower than predicted by the matching law. When the schedule was changed to concurrent VI 50 s VI 50 s for test sessions with saline and the training dose of pentobarbital, responding on the pentobarbital-biased lever after the training dose of pentobarbital and on the saline-biased lever after saline became nearly equal, even during the first 2 min of the session, suggesting that the presence or absence of the training drug was exerting minimal control over responding and making the determination of dose-effect relations of drugs difficult to interpret. When the pentobarbital dose-response curve was determined under the concurrent VI 50-s VI 50-s schedule, responding was fairly evenly distributed on both levers for most rats. Therefore, 6 additional rats were trained to respond under a concurrent VI 60-s VI 240-s schedule. Under this schedule, the rats made 62.6% of their responses on the pentobarbital-biased lever after pentobarbital and 73.5% of their responses on the saline-biased lever after saline, which also is lower than the percentages predicted by perfect matching. When the schedule was changed to a concurrent VI 150-s VI 150-s schedule for 5-min test sessions with additional drugs, the presence or absence of pentobarbital continued to control responding in most rats, and it was possible to generate graded dose-response curves for pentobarbital and other drugs using the data from these 5-min sessions. The dose-response curves generated under these conditions were similar to the dose-response curves generated using other reinforcement schedules and other species.     

Overmatching in rats: the barrier choice paradigm

The barrier choice paradigm was used to impose a cost on rats' behavior of traveling between two levers: Pressing on two levers was reinforced with food on concurrent random-interval schedules, but rats had to climb over a barrier to move from one lever to another. The height of the barrier separating the levers was increased from 30.5 to 45.7 cm across two phases that involved various pairs of random-interval schedules. With the 30.5-cm barrier, the generalized matching law showed slopes equal to or slightly above 1.0 for response and time allocation. With the 45.7-cm barrier, the generalized matching law showed slopes above 1.2 for responses, indicating that sensitivity to reinforcement increased with increasing barrier height. For time allocation the slopes remained close to 1.0; sensitivity to reinforcement did not seem to increase with increasing barrier height. The role of locomotion effort in choice situations is discussed.     

Effects of methamphetamine and scopolamine on variability of response location.

Methamphetamine and scopolamine were studied in monkeys responding under a multiple fixed-ratio fixed-interval schedule of reinforcement. A response on any one of six levers could satisfy the schedule requirements. Variability of response location was evaluated in terms of switches, where a switch was defined as a response on one lever followed by a response on a different lever. Under baseline conditions the fixed-ratio schedule generated a high rate of responding and a low level of variability, while the fixed-interval schedule generated a low rate of responding and a high level of variability. Both methamphetamine (0.1 to 0.5 mg/kg) and scopolamine (2.4 to 240 microgram/kg) decreased overall response rate and increased variability of response location in each component of the multiple schedule with increasing doses of drug. At lower doses both drugs were found to decrease rate without affecting response variability.     

Conditioned reinforcement and choice

In a series of three experiments, rats were exposed to successive schedule components arranged on two levers, in which lever pressing produced a light, and nose-key pressing produced water in 50% of the light periods. When one auditory signal was presented only during those light periods correlated with water on one lever, and a different signal was presented only during those light periods correlated with nonreinforcement on the other lever, the former lever was preferred in choice trials, and higher rates of responding were maintained on the former lever in nonchoice (forced) trials. Thus, the rats preferred a schedule component that included a conditioned reinforcer over one that did not, with the schedules of primary reinforcement and the information value of the signals equated. Preferences were maintained when one or the other of the auditory signals was deleted, but were not established in naive subjects when training began with either the positive or negative signal only. Discriminative control of nose-key pressing by the auditory signals was highly variable across subjects and was not correlated with choice.     

DYNAMICS OF CHOICE: RELATIVE RATE AND AMOUNT AFFECT LOCAL PREFERENCE AT THREE DIFFERENT TIME SCALES

To examine extended control over local choice, the present study investigated preference in transition as food‐rate ratio provided by two levers changed across seven components within daily sessions, and food‐amount ratio changed across phases. Phase 1 arranged a food‐amount ratio of 4:1 (i.e., the left lever delivered four pellets and the right lever one pellet); Phase 2 reversed the food‐amount ratio to 1:4, and in Phase 3 the food‐amount ratio was 3:2. At a relatively extended time scale, preference was described well by a linear relation between log response ratio and log rate ratio (the generalized matching law). A small amount of carryover occurred from one rate ratio to the next but disappeared after four food deliveries. Estimates of sensitivity to food‐amount ratio were around 1.0 and were independent of rate ratio. Analysis across food deliveries within rate‐ratio components showed that the effect of a small amount was diminished by the presence of a large amount—that is, when a larger amount was present in the situation (three or four pellets), the value of a small amount (one or two pellets) became paltry. More local analysis of visits to the levers between food deliveries showed that postfood visits following a large amount were disproportionately longer than following a small amount. Continuing food deliveries from the same source tended to make visits less dependent on relative amount, but a discontinuation (i.e., food from the other lever) reinstated dependence on relative amount. Analysis at a still smaller time scale revealed preference pulses following food deliveries that confirmed the tendency toward dependence on absolute amount with continuing deliveries, and toward dependence on relative amount following discontinuations. A mathematical model based on a linear‐operator equation accounts for many of the results. The larger and longer preference following a switch to a larger amount is consistent with the idea that local preference depends on relatively extended variables even on short time scales.     

Effects of reinforcer delays on choice as a function of income level

Three rats earned their daily food ration by responding during individual trials either on a lever that delivered one food pellet immediately or on a second lever that delivered three pellets after a delay that was continuously adjusted to ensure substantial responding to both alternatives. Choice of the delayed reinforcer increased when the number of trials per session was reduced. This result suggests that models seeking closure on choice effects must include a parameter reflecting how preference changes with sessionwide income. Moreover, models positing that reinforcer probability and immediacy (1/delay) function equivalently in choice are called into question by the finding that probability and immediacy produce opposing effects when income level is changed.     

Reversibility of single-incentive selective associations.

Rats were trained to press a lever in the presence of a tone-light compound stimulus and not to press in its absence. In each of two experiments, schedules were designed to make the compound a conditioned punisher for one group and a conditioned reinforcer for the other. In Experiment 1, one group's responding produced food in the presence of the compound but not in its absence. The other group's responding terminated the compound stimulus, and food was presented only in its absence. When tone and light were later presented separately, light controlled more responding than did tone in the former group, but tone gained substantial control in the latter. The same effects were also observed within subjects when the training schedules were switched over groups. In Experiment 2, two groups avoided shock in the presence of the compound stimulus. In the absence of the compound, one group was not shocked, and the other received both response-independent and response-produced shock. When tone and light were presented separately, the former group's responding was mainly controlled by tone, but the latter group's responding was almost exclusively controlled by light. These effects were also observed within subjects when the training schedules were switched over groups. Thus, these single-incentive selective association effects (appetitive in Experiment 1 and aversive in Experiment 2) were completely reversible. The schedules in which the compound should have been a conditioned reinforcer consistently produced visual control, and auditory control increased when the compound should have become a conditioned punisher. Currently accepted accounts of selective associations based on affinities between shock and auditory stimuli and between food and visual stimuli (i.e., stimulus-reinforcer interactions) do not adequately address these results. The contingencies of reinforcement most recently associated with the compound and with its absence, rather than the nature of the reinforcer, determined whether auditory or visual stimulus control developed.