Resurgence as Choice in Context: Treatment duration and on/off alternative reinforcement

Resurgence as Choice (RaC) is a quantitative theory suggesting that an increase in an extinguished target behavior with subsequent extinction of an alternative behavior (i.e., resurgence) is governed by the same processes as choice more generally. We present data from an experiment with rats examining a range of treatment durations with alternative reinforcement plus extinction and demonstrate that increases in treatment duration produce small but reliable decreases in resurgence. Although RaC predicted the relation between target responding and treatment duration, the model failed in other respects. First, contrary to predictions, the present experiment also replicated previous findings that exposure to cycling on/off alternative reinforcement reduces resurgence. Second, RaC did a poor job simultaneously accounting for target and alternative behaviors across conditions. We present a revised model incorporating a role for more local signaling effects of reinforcer deliveries or their absence on response allocation. Such signaling effects are suggested to impact response allocation above and beyond the values of the target and alternative behaviors as longer-term repositories of experience. The new model provides an excellent account of the data and can be viewed as an integration of RaC and a quantitative approximation of some aspects of Context Theory.     

Resurgence and downshifts in alternative reinforcement rate

Resurgence refers to an increase in a previously suppressed target behavior with a relative worsening of conditions for a more recently reinforced alternative behavior. This experiment examined the relation between resurgence and the magnitude of a reduction in the rate of reinforcement for the alternative behavior. Groups of both male and female rats initially pressed a target lever for food on a variable-interval (VI) 30-s schedule. In a second phase, responding to the target lever was extinguished for all groups and pressing an alternative lever was reinforced on a VI 10-s schedule. Next, the rate of reinforcement for alternative behavior was reduced differentially across groups by arranging extinction, VI 80-s, VI 40-s, VI 20-s, or continued VI 10-s reinforcement. Target responding increased as an exponential function of the magnitude of the reduction in alternative reinforcement rates. With the exception that males appeared to show higher rates of target responding in baseline and higher rates of alternative responding in other phases, the overall pattern of responding across phases was not meaningfully different between sexes. The pattern of both target and alternative response rates across sessions and phases was well described quantitatively by the Resurgence as Choice in Context model.     

The matching law in and within groups of rats

In each of the two experiments, a group of five rats lived in a complex maze containing four small single-lever operant chambers. In two of these chambers, food was available on variable-interval schedules of reinforcement. In Experiment I, nine combinations of variable intervals were used, and the aggregate lever-pressing rates (by the five rats together) were studied. The log ratio of the rates in the two chambers was linearly related to the log ratio of the reinforcement rates in them; this is an instance of Herrnstein's matching law, as generalized by Baum. Summing over the two food chambers, food consumption decreased, and response output increased, as the time required to earn each pellet increased. In Experiment II, the behavior of individual rats was observed by time-sampling on selected days, while different variable-interval schedules were arranged in the two chambers where food was available. Individual lever-pressing rates for the rats were obtained, and their median bore the same “matching” relationship to the reinforcement rates as the group aggregate in Experiment I. There were differences between the rats in their distribution of time and responses between the two food chambers; these differences were correlated with differences in the proportions of reinforcements the rats obtained from each chamber.     

Time horizons in rats: the effect of operant control of access to future food.

The primary goal of this experiment was to determine whether the addition of an operant requirement for access to a less costly (continuous reinforcement) patch of future food increased the time horizon over which that future patch decreased intake in a currently available depleting (progressive-ratio) patch. Three groups of 4 rats were tested. Each member of the earned-time group was required to cumulate a fixed-time outside the progressive-ratio patch to obtain access to food in the less costly patch; the fixed-time requirement ranged from 2 to 64 min. Rats in the matched-time group received response-independent access to less costly food at the average delay shown by the earned-time group. Rats in the matched-time no-food group were removed from the chamber at the same average delay without receiving access to less costly food. Two of the earned-time rats showed an increased time horizon relative to that shown by the matched-time rats (approaching 40 min for 1 rat). The other 2 earned-time rats markedly increased instrumental responding but showed suppression of intake only when food was less than 20 min away. The matched-time group showed less suppression of intake over a similar range of delay intervals. Surprisingly, the matched-time no-food animals also showed suppression of intake concentrated at the end of the session, possibly reflecting the receipt of their entire daily ration 30 min after the session. The potential importance of time horizons to the foraging process is clear, but experimenters are still working out paradigms for investigation of these horizons.     

Social reinforcement of operant behavior in rats: a methodological note.

An apparatus was developed to study social reinforcement in the rat. Four Long-Evans female rats were trained to press a lever via shaping, with the reinforcer being access to a castrated male rat. Responding under a fixed-ratio schedule and in extinction was also observed. Social access was found to be an effective reinforcer. When social reinforcement was compared with food reinforcement under similar conditions of deprivation and reinforcer duration, no significant differences were observed.     

Overmatching in rats: the barrier choice paradigm

The barrier choice paradigm was used to impose a cost on rats' behavior of traveling between two levers: Pressing on two levers was reinforced with food on concurrent random-interval schedules, but rats had to climb over a barrier to move from one lever to another. The height of the barrier separating the levers was increased from 30.5 to 45.7 cm across two phases that involved various pairs of random-interval schedules. With the 30.5-cm barrier, the generalized matching law showed slopes equal to or slightly above 1.0 for response and time allocation. With the 45.7-cm barrier, the generalized matching law showed slopes above 1.2 for responses, indicating that sensitivity to reinforcement increased with increasing barrier height. For time allocation the slopes remained close to 1.0; sensitivity to reinforcement did not seem to increase with increasing barrier height. The role of locomotion effort in choice situations is discussed.     

The effects of 100 dB 1‐kHz and 22‐kHz tones as punishers on lever pressing in rats

Aversive control is an important yet understudied process of learning. One reason aversive control may be relatively understudied is ethical concerns about painful stimuli (e.g., electric shock). High decibel broad‐band noise and 22‐kHz vocalizations both demonstrably affect rodent behavior while not necessarily being painful. The goal of this study was to determine if 100‐dB 22‐kHz‐pure tones were differentially more effective in reducing operant response rates in rats. We examined whether 22‐kHz pure tones would function as aversive stimuli, specifically as positive punishers. The effects of response‐dependent as well as continuously presented 22‐kHz and 1‐kHz tones on rate of response maintained by variable interval 30‐s food deliveries were assessed across several conditions. We found that response rates were lower when tones were presented response dependently than when tones were presented continuously throughout a session. We also found that the lower response rates obtained with response‐dependent 22‐kHz tones were not significantly different from response rates obtained with response‐dependent 1‐kHz tones. The primary conclusion of this experiment is that both 1‐kHz and 22‐kHz tones functioned as punishers, but that the 22‐kHz tones were not differentially more effective in reducing response rate.     

Fix and sample with rats in the dynamics of choice

The generality of the molar view of behavior was extended to the study of choice with rats, showing the usefulness of studying order at various levels of extendedness. Rats' presses on two levers produced food according to concurrent variable-interval variable-interval schedules. Seven different reinforcer ratios were arranged within each session, without cues identifying them, and separated by blackouts. To alternate between levers, rats pressed on a third changeover lever. Choice changed rapidly with changes in component reinforcer ratio, and more presses occurred on the lever with the higher reinforcer rate. With continuing reinforcers, choice shifted progressively in the direction of the reinforced lever, but shifted more slowly with each new reinforcer. Sensitivity to reinforcer ratio, as estimated by the generalized matching law, reached an average of 0.9 and exceeded that documented in previous studies with pigeons. Visits to the more-reinforced lever preceded by a reinforcer from that lever increased in duration, while all visits to the less-reinforced lever decreased in duration. Thus, the rats' performances moved faster toward fix and sample than did pigeons' performances in previous studies. Analysis of the effects of sequences of reinforcer sources indicated that sequences of five to seven reinforcers might have sufficed for studying local effects of reinforcers with rats. This study supports the idea that reinforcer sequences control choice between reinforcers, pulses in preference, and visits following reinforcers.     

The effect of ethanol treatment on social behavior in male rats

The effect of 0.50 g/kg of EtOH in male rats interacting with a stimulus male juvenile in a newly developed test of social interaction was examined. The adult male rats were treated with EtOH (8.0 to 12.0 g/kg/day) or equicaloric dextrin maltose for 2 weeks (studies 1 and 2) or 8 weeks (study 3) and social interaction was assessed both before and after chronic drug treatment was ended in study 1 and after chronic drug treatment was ended in studies 2 and 3. It was found that prior to chronic drug treatment, in study 1, 0.50 g/kg of EtOH increased both aggressive behavior and time spent interacting with the stimulus juvenile male from the first presentation of the juvenile to the second presentation (20 min apart) while saline injection decreased it. After chronic drug treatment was ended, in study 1 animals treated chronically with EtOH were more aggressive when they were not intoxicated than when they had been treated with 0.50 g/kg of EtOH. In studies 2 and 3, after chronic drug treatment was ended, aggressive behavior and time spent interacting with the juvenile were greater in the animals treated chronically with EtOH, regardless of whether they were injected with saline or 0.50 g/kg of EtOH. The results of these studies showed that chronic EtOH treatment can produce long-lasting changes in social behavior after drug treatment is over and can alter the animal's normal response to EtOH in a social setting.     

Intertrial-interval effects on sensitivity (A') and response bias (B") in a temporal discrimination by rats.

Killeen and Fetterman's (1988) behavioral theory of animal timing predicts that decreases in the rate of reinforcement should produce decreases in the sensitivity (A') of temporal discriminations and a decrease in miss and correct rejection rates (decrease in bias toward "long" responses). Eight rats were trained on a 10- versus 0.1-s temporal discrimination with an intertrial interval of 5 s and were subsequently tested on probe days on the same discrimination with intertrial intervals of 1, 2.5, 5, 10, or 20 s. The rate of reinforcement declined for all animals as intertrial interval increased. Although sensitivity (A') decreased with increasing intertrial interval, all rats showed an increase in bias to make long responses.     

Preference reversals with food and water reinforcers in rats

Rats were given a choice between a smaller, immediately available reward and a larger reward available after a delay. In one phase, the reward was food and in another phase, the reward was water. Constant delays were added between the choice presentation and the delivery of the reward alternatives. As the time between choice and reward delivery increased from 0 to 25 s, all rats (except one in the water phase) reversed their preference from the smaller, sooner alternative to the larger, later alternative. These findings extend the generality of the preference-reversal animal model to qualitatively different reinforcers. Furthermore, the presence of both impulsive and self-control choices within the same animal is consistent with the view that self-control may be understood as choice behavior, and that species differences in self-control may be differences in degree, not kind.     

The long-term effect of high- and low-rate responding histories on fixed-interval responding in rats

Tell rats were given extended lever-press training on a fixed-interval (FI) 30-s food reinforcement schedule from the outset or following exposure to one or two previous reinforcement schedules. For 4 rats the previots schedule was either fixed-ratio 20, which generated high response rates, or differential-reinforcement-of-low-rate 20 s, which produced low response rates. For 4 additional rats the extended training on FI 30 s was preceded by experience with two schedules: fixed-ratio 20 followed by differential-reinforcement-of-low-rate 20 s; or the same two schedules in the reverse order. Fixed-interval response rates were initially affected by the immediately preceding schedule, but after 80 to 100 sessions, all traces of prior schedule history had disappeared. The results also showed no long-term effect of schedule history on the interfood-interval patterns of responding on the FI 30-s schedule. These results support one of the most central tenets of the experimental analysis of behavior: control by the immediate consequences of behavior.     

Delay discounting in Lewis and Fischer 344 rats: steady-state and rapid-determination adjusting-amount procedures

Lewis rats have been shown to make more impulsive choices than Fischer 344 rats in discrete-trial choice procedures that arrange fixed (i.e., nontitrating) reinforcement parameters. However, nontitrating procedures yield only gross estimates of preference, as choice measures in animal subjects are rarely graded at the level of the individual subject. The present study was designed to examine potential strain differences in delay discounting using an adjusting-amount procedure, in which distributed (rather than exclusive) choice is observed due to dynamic titration of reinforcer magnitude across trials. Using a steady-state version of the adjusting-amount procedure in which delay was manipulated between experimental conditions, steeper delay discounting was observed in Lewis rats compared to Fischer 344 rats; further, delay discounting in both strains was well described by the traditional hyperbolic discounting model. However, upon partial completion of the present study, a study published elsewhere (Wilhelm & Mitchell, 2009) demonstrated no difference in delay discounting between these strains with the use of a more rapid version of the adjusting-amount procedure (i.e., in which delay is manipulated daily). Thus, following completion of the steady-state assessment in the present study, all surviving Lewis and Fischer 344 rats completed an approximation of this rapid-determination procedure in which no strain difference in delay discounting was observed.     

Stimulus- and response-reinforcer contingencies in autoshaping, operant, classical, and omission training procedures in rats

Separate groups of rats received 500 trials of lever-press training under autoshaping (food delivery followed 10-second lever presentations, or occurred immediately following a response); operant conditioning (responding was necessary for food delivery); and classical conditioning (food followed lever presentations regardless of responding). Each group then received 500 trials on an omission procedure in which food was omitted on trials with a response. Another group received 1000 trials on the omission procedure, and a fifth group, random control, received 1000 uncorrelated presentations of lever and food. The autoshaping, operant, and classical groups reached high response levels by the end of initial training. Acquisition was fastest in the autoshaping group. Responding remained consistently low in the control group. The omission group responded at a level between the control group and the other three groups. During omission training, responding in these three groups declined to the omission-group level. During omission training, the rats continued contacting the lever frequently after lever pressing had declined. Response maintenance under omission training seems not to require topographic similarity between the response and reinforcer-elicited consummatory behaviors.     

Individual differences in impulsive choice and timing in rats

Individual differences in impulsive choice behavior have been linked to a variety of behavioral problems including substance abuse, smoking, gambling, and poor financial decision-making. Given the potential importance of individual differences in impulsive choice as a predictor of behavioral problems, the present study sought to measure the extent of individual differences in a normal sample of hooded Lister rats. Three experiments utilized variations of a delay discounting task to measure the degree of variation in impulsive choice behavior across individual rats. The individual differences accounted for 22-55% of the variance in choice behavior across the three experiments. In Experiments 2 and 3, the individual differences were still apparent when behavior was measured across multiple choice points. Large individual differences in the rate of responding, and modest individual differences in timing of responding were also observed during occasional peak trials. The individual differences in timing and rate, however, did not correlate consistently with individual differences in choice behavior. This suggests that a variety of factors may affect choice behavior, response rate, and response timing.     

Preference for mixed versus constant delays of reinforcement: Effect of probability of the short, mixed delay

Preference for mixed versus constant delays of reinforcement was studied with a concurrent-chain procedure. Lever pressing by rats in concurrently available variable-interval 60-second initial links occasionally produced mutually exclusive terminal-link reinforcement delays. A constant delay of reinforcement (either 15 seconds or 30 seconds) composed one terminal link and mixed delays (.2 second and twice the value of the constant delay) were arranged in the other terminal link. The proportion of .2-second delays in the mixed-delay terminal link took on values of 0, .1, .25, .5, .75, .9, and 1.0 over experimental conditions. Based on relative rates of responding in the initial links, preference for the mixed delays was a negatively accelerated function of the proportion of short, mixed delays. Three of five rats preferred the mixed delays to the constant delays when the proportion of short, mixed delays was .1 or higher, and all five rats preferred the mixed delays when the proportion of short, mixed delays was .25 or higher. Neither Squires and Fantino's (1971) delay-reduction model of choice nor a model based on the harmonic mean reinforcement delay provided a close estimate of choice proportions over the range of short-delay proportions studied. The delay-reduction model underestimated choice for the mixed delays at low and intermediate proportions of short delays, and the harmonic-mean-delay model overestimated choice for the mixed delays at intermediate and high proportions of short delays.     

The effects of d-amphetamine on the temporal control of operant responding in rats during a preshock stimulus.

The operant behavior of six rats was maintained by a random-interval schedule of reinforcement. Three-minute periods of noise were superimposed on this behavior, each period ending with the delivery of an unavoidable shock. Overall rates of responding were generally lower during the periods of noise than in its absence (conditioned suppression). These suppressed response rates also exhibited temporal patterning, with responding becoming less frequent as each noise period progressed. The effects of d-amphetamine on this behavioral baseline were then assessed. In four animals the relative response rates during the noise and in its absence suggested that the drug produced a dose-related decrease in the amount of conditioned suppression. However, this effect was often due to a decrease in the rates of responding in the absence of the preshock stimulus, rather than to an increase in response rates during the stimulus. Temporal patterning in response rates during the preshock stimulus was abolished, an effect that was interpreted in terms of rate-dependent effect of d-amphetamine. This study thus extends rate-dependent analyses of the effects of amphetamines to the patterns of operant behavior that occur during a preshock stimulus, and which have been discussed in terms of the disrupting effects of anxiety on operant behavior.     

Polydipsia induced in rats by second-order schedules of reinforcement.

Three rats were exposed to a second-order schedule in which fixed-interval components ended either with food or with a brief stimulus that was never paired with food. Food and the brief stimulus occurred in a random sequence (variable-ratio 2 overall schedule). Another three rats were exposed to a similar second-order schedule, the only difference being that the food or the stimulus was presented independently of operant behavior (fixed-time components). The three rats exposed to the fixed-interval components licked at a water spout after each food presentation. These rats also licked in the intervals after the brief stimulus. Although the discriminative properties of food and of the brief stimulus were identical in relation to subsequent reinforcement, licking after the stimulus was less than after food. The three rats exposed to the second-order schedules with fixed-time components also licked at the water spout after food, but these rats did not lick consistently after brief stimulus presentations.     

Quinine pellets as an inferior good and a Giffen good in rats.

In Experiment 1, 4 rats earned their daily food ration by choosing between two levers. One lever delivered two regular and one quinine-adulterated food pellets, and the other delivered two regular and four quinine pellets. A 20-s intertrial interval separated successive choices. Sessions began with 10 forced trials during which only one lever, selected with p = .5 and cued by a light above it, could deliver its reinforcer. Forced trials were followed by 30 or 150 trials, depending on the condition, during which choices to either lever could be reinforced. Over this range, absolute choice of the four-quinine, two-regular-pellet lever was inversely related to the number of free-choice trials, establishing this reinforcer as an inferior good. In Condition 1 of Experiment 2, the prior design was altered in two ways: (a) one lever delivered four quinine pellets, and the other lever delivered one standard pellet; and (b) sessions ended after 140 free-choice trials. When the number of free-choice trials was reduced to 100 (Condition 2), all 3 rats increased their preference for quinine pellets, confirming their status as an inferior good. In the next several conditions, the number of quinine pellets provided for selecting its associated lever was varied between three and four. Preference for the quinine-pellet alternative was inversely related to the number of pellets it provided, a result defining it as a Giffen good. These findings are not accommodated readily by extant choice models and complicate the search for a unitary model of choice.