Discriminative properties of shock-correlated reinforcement in an operant context

A two-choice discrete operant procedure was devised for the study of shock-correlated reinforcement effects in rats. In the presence of one auditory stimulus, responding on one response lever was reinforced with food; with another auditory stimulus, responding on a second lever was reinforced. It was found that discrimination performance of one group, relative to appropriate control groups, was facilitated when electric shock was correlated with reinforcement on one lever and not on the other. Further, relative discrimination levels were found to be higher on the lever correlated with the shock than on the alternate lever. The significance of the results for operant within-S studies and for a mediational theory of shock-correlated reinforcement was discussed.     

Transfer of persistence to the acquisition of a new behaviour

Two experiments tested whether the degree of effort required for the reinforcement of one behaviour would affect the acquisition of a second behaviour. In the first experiment, rats were placed in a conditioning chamber and: (a) were required to press a lever for food pellets on a fixed ratio schedule, (b) received free presentation of the pellets, or (c) did not receive pellets. Next, all rats were rewarded for a new behaviour, round trips across the length of a runway. As predicted, the fixed-ratio group had the greatest shuttle rate. In the second experiment, two groups were required to press a lever, and the number of presses per pellet was varied. For two other groups not required to press the lever, the amount of food presented per approach to the feeder was varied. The greater required number of lever presses and the lesser number of pellets per approach to the feeder produced the higher subsequent shuttle rates. Two alternative explanations were compared: the degree of accustomed effort per reinforcer becomes a learned component of behaviour, or high effort increases the habituation of frustration-produced disruptive responses.     

Resurgence and alternative‐reinforcer magnitude

Resurgence is defined as an increase in the frequency of a previously reinforced target response when an alternative source of reinforcement is suspended. Despite an extensive body of research examining factors that affect resurgence, the effects of alternative‐reinforcer magnitude have not been examined. Thus, the present experiments aimed to fill this gap in the literature. In Experiment 1, rats pressed levers for single‐pellet reinforcers during Phase 1. In Phase 2, target‐lever pressing was extinguished, and alternative‐lever pressing produced either five‐pellet, one‐pellet, or no alternative reinforcement. In Phase 3, alternative reinforcement was suspended to test for resurgence. Five‐pellet alternative reinforcement produced faster elimination and greater resurgence of target‐lever pressing than one‐pellet alternative reinforcement. In Experiment 2, effects of decreasing alternative‐reinforcer magnitude on resurgence were examined. Rats pressed levers and pulled chains for six‐pellet reinforcers during Phases 1 and 2, respectively. In Phase 3, alternative reinforcement was decreased to three pellets for one group, one pellet for a second group, and suspended altogether for a third group. Shifting from six‐pellet to one‐pellet alternative reinforcement produced as much resurgence as suspending alternative reinforcement altogether, while shifting from six pellets to three pellets did not produce resurgence. These results suggest that alternative‐reinforcer magnitude has effects on elimination and resurgence of target behavior that are similar to those of alternative‐reinforcer rate. Thus, both suppression of target behavior during alternative reinforcement and resurgence when conditions of alternative reinforcement are altered may be related to variables that affect the value of the alternative‐reinforcement source.     

Operant hoarding: a new paradigm for the study of self-control.

In the first of four experiments, rats were exposed to a modified multiple continuous reinforcement-extinction schedule during 15-min daily sessions. In one condition (saves condition) with the cuelight on, a single lever press produced a food pellet, briefly extinguished the cuelight, and started a clock. Saves (additional lever presses with interresponse times less than 1 s) produced an additional food pellet, briefly extinguished the cuelight, and restarted the interresponse time clock. The cuelight was extinguished 1 s after the last lever press and remained off during a 10-s period of extinction, during which no food pellets were delivered. In the other condition (savings account condition), the contingencies were the same except that the cuelight was extinguished and was not reilluminated after the initial lever press, and the delivery of all food pellets in the reinforcement component was delayed until the onset of extinction. In both conditions, rats made saves, but mean saves (total saves divided by the number of reinforcement components) were slightly reduced in the savings account condition. In Experiment 2, using six equally spaced 15-min sessions per day on alternate days, saves were either followed immediately with food and brief cuelight offset (saves condition) or were not reinforced at all. Mean saves were much greater when saves were reinforced. In Experiment 3, during 5-min daily sessions, saves earned a single pellet (savings account condition) or a number of pellets equal to the ordinal number of the lever press (interest condition). Rats made fewer mean saves, with little change in the food rate, when saves earned interest. In Experiment 4, the rats earned all their food in the operant situation during 24 daily 5-min sessions, these separated by 55-min intersession intervals during which no food was available; otherwise, the conditions were the same as in Experiment 3. In Experiment 4, the shift to interest for saves led to an increase in mean daily mean saves (total daily mean saves divided by the number of daily sessions) as well as to an increase in the number of food pellets delivered in each session. The results are discussed in terms of self-control and behavioral economics.     

The role of the instrumental contingency in the motivational control of performance

In four experiments we investigated an irrelevant incentive effect based upon a transition from hunger to thirst. Hungry rats were trained to lever press either for sucrose solution or for food pellets before performance was tested in extinction while they were thirsty. Reinforcer-specific motivational control was found in the first experiment in that the animals pressed the lever more on tests following training with the sucrose solution rather than with food pellets. Moreover, this effect was seen only when testing was conducted following water, but not following food deprivation. The outcome of the remaining experiments suggests that this motivational control is not mediated by the instrumental contingency between lever pressing and the sucrose reinforcer during training. In these studies lever pressing and chain pulling were reinforced concurrently, one with sucrose and the other with food pellets, in order to equate the noninstrumental functions of the incentives. Following this training, lever pressing in extinction under thirst was unaffected by the type of incentive used as its reinforcer during training.     

Anticipation of reward omission as a cue for choice behavior

A within-subject partial reinforcement extinction effect was obtained in a conditional discriminative choice task. In choice acquisition, one group had learned a task in which one choice alternative was continuously reinforced (CRF) and the other alternative was partially reinforced (PRF). Most errors in the acquisition phase were the choosing of the CRF lever when the conditional stimulus signaled the PRF lever; in extinction this pattern of choice errors reversed. In extinction, most errors were choosing of the PRF lever when the conditional stimulus signaled the CRF lever. The predictions of frustration theory and sequential theory were compared with the choice data. The results were interpreted as consistent with the anticipatory frustration construct within an associative mediational theory.     

Suppression of operant behavior and schedule-induced licking in rats

The first experiment studied the effects of punishment on rats' lever pressing maintained by a fixed-interval schedule of food reinforcement and on the associated schedule-induced licking. When licking was followed by shock, licking was suppressed but lever pressing was largely unaffected. When lever pressing was followed by shock, lever pressing was suppressed but licking was unaffected. In both cases, the punished behavior recovered its previous unpunished level when the shocks were discontinued. In a second experiment, the rats' lever pressing was maintained by a variable-interval schedule of food reinforcement under which polydipsic licking also developed. Both lever pressing and licking were partially suppressed during a stimulus correlated with occasional unavoidable electric shocks. With a higher shock intensity, both behaviors were suppressed further. Both lever pressing and licking recovered their previous levels when shocks were discontinued. These results show that schedule-induced licking, which has been described as adjunctive behavior, can be suppressed by procedures that suppress reinforced lever pressing, an operant behavior.     

The roles of stimulus control and reinforcement frequency in modulating the behavioral effects of d-amphetamine in the rat.

The behavioral effects of d-amphetamine have been shown to be modulated by stimulus control, with less impairment of performance occurring when control is great. When the fixed-consecutive-number schedule is used (on which at least a specified consecutive number of responses must be made on one operandum before a single response on another will produce a reinforcer), response rate tends to be invariant but reinforcement frequency is not. This study asks whether the differences in reinforcement frequency that usually accompany changes in stimulus control could themselves be responsible for the performance differences. Two versions of the fixed-consecutive-number schedule of reinforcement were combined into a multiple schedule within which stimulus control was varied but differences in reinforcement frequency were minimized by omitting some reinforcer deliveries during the component that usually had the higher reinforcement frequency. In one component, a compound discriminative stimulus was added with the eighth consecutive response on the first lever; a single response on the second lever was then reinforced. In the other component, no such stimulus was presented. With no added stimulus, large decreases occurred in the number of runs satisfying the minimum requirement for reinforcement at doses of drug that produced only minimal changes when an added stimulus controlled behavior. Thus, increased stimulus control diminishes the behavioral changes produced by d-amphetamine even when the possible contribution by baseline reinforcement rate is minimized.     

Effects of reinforcement magnitude and ratio values on behaviour maintained by a cyclic ratio schedule of reinforcement

In Experiments 1 and 2, lever pressing by rats was reinforced on a cyclic ratio schedule of food reinforcement, comprising a repeated sequence of fixed-ratio component schedules. Reinforcement magnitude was varied, on occasional sessions in Experiment 1 and across blocks of sessions in Experiment 2, from one to two or three 45-mg food pellets. In the one-pellet condition, post-reinforcement pauses increased with component schedule value. At higher magnitudes, post-reinforcement pauses increased, and overall response rates declined. Response rate on component schedules was a decreasing linear function of the obtained rate of reinforcement in all conditions. Plotted against component schedule value, response rate increased exponentially to an asymptote that decreased when reinforcement magnitude increased. These findings are consistent with regulatory accounts of food reinforced behaviour. In Experiment 3, rats were trained under a cyclic ratio schedule comprising fixed-ratio components including higher values, and some inverted U-shaped response functions were obtained. Those rats that did not showthis relationship were trained on cyclic ratios with even higher values, and all showed inverted U-shaped response functions. This suggests that behaviour on cyclic ratio schedules can reflect activating of reinforcement as well as the satiating effects seen in Experiments 1 and 2.     

Duration discrimination: effects of probability of stimulus presentation

Monkeys initiated a stimulus by pressing on the center of three levers and the stimulus terminated independently of behavior 60, 80, 90, or 100 sec later. Presses on the right lever were reinforced with food following the three briefer durations, and presses on the left lever, following the 100-sec duration. Incorrect responses produced a 10-sec timeout. Probability of presenting the 100-sec duration was manipulated in the range from 0.25 to 0.75, with the probabilities of the briefer durations remaining equal and summing to one minus the probability of the 100-sec duration. Percentage of responses on either side lever was functionally related to both the probability of presenting the 100-sec stimulus and to stimulus duration. An analysis of the data based on the theory of signal detection resulted in operating characteristics that were linear when plotted on normal-normal coordinates. The percentage of responses on either lever approximated the optimal values for maximizing reinforcement probability in each condition of the experiment.     

Discrete and continuous measures of dimensional stimulus control

In two sets of experiments, we examined dimensional stimulus control of pigeons' responses to a visual flicker-rate continuum. In the first experiment, responses to a single key were reinforced periodically during stimuli from one half of the stimulus continuum, and responses during other stimuli were extinguished. In the second experiment, two response keys were simultaneously available, with reinforcement for each response alternative associated with different halves of the stimulus continuum. Conditions of the second experiment involved either free-operant or discrete-trial stimulus presentations. Results from these experiments show that positive dimensional contrast appeared in discrimination tasks with one or two response alternatives, but only with free-operant procedures. In addition, discrimination between stimulus classes established by differential reinforcement was assessed as accurately by continuous rate measures as by discrete response choice in the two-alternative situation. The general implication of these experiments is that response rate measures, when properly applied, may reveal sources of variation within stimulus classes, such as dimensional contrast, that are not evident with discrete measures.     

Temporal discrimination learning of operant feeding in goldfish (Carassius auratus)

Operant temporal discrimination learning was investigated in goldfish. In the first experiment, there was a fixed daily change in illumination. Eight subjects were trained to operate a lever that reinforced each press with food. The period during which responses were reinforced was then progressively reduced until it was 1 hr in every 24. The final 1-hr feeding schedule was maintained over 4 weeks. The feeding period commenced at the same time each day throughout. The food dispensers were then made inactive, and a period of extinction ensued for 6 days. The pattern of responding suggested that the fish were able to exhibit temporal discrimination in anticipation of feeding time. This pattern of responding persisted for a limited number of days during the extinction procedure. The second experiment produced evidence that operant temporal discrimination could develop under continuous illumination.     

Reinforcement of mediating behavior on a spaced-responding schedule

This paper describes a procedure for gaining experimental control over mediating behavior on a spaced-responding schedule of food reinforcement. Three rats, food-deprived, were trained on a DRL 16 sec schedule of food reinforcement. Then, a concurrent schedule of food reinforcement was introduced on a second (mediating) lever, such that the first response to occur on the mediating lever, after the DRL interval had timed out, was reinforced with food, as was the next response to occur on the DRL lever. Reinforcement via the mediating lever became a discriminative stimulus for a food-reinforcement opportunity on the DRL lever. Next, food reinforcement for the mediating behavior was replaced by a conditioned reinforcer consisting of onset of a buzzer signaling timing-out of the DRL interval. Under these conditions, chaining of behavior on the two levers was strong, and timing on the DRL lever was more accurate than under ordinary DRL conditions. As the DRL requirement was lengthened from 16 sec to 24 sec to 60 sec, mediating behavior weakened slightly. When the inter-response requirement for food reinforcement on the DRL lever was made shorter than the inter-response requirement for conditioned reinforcement on the mediating lever, the mediating behavior extinguished. Performance in the experiment was analyzed into a four-component chain, and the factors contributing to the maintenance, and later extinction, of mediating behavior are discussed.     

Operant acceleration during a pre-reward stimulus

Stimuli of 20, 40, and 80 sec duration terminated with five non-response-contingent food pellets were superimposed upon lever pressing reinforced with single pellets on a DRL 30-sec schedule. Two rhesus monkeys served as subjects. No change in response frequency was observed during the 20- and 40-sec stimuli. During the 80-sec pre-food stimulus, overall response frequency increased to approximately 150% and 220% of pre-stimulus levels, and the temporal distributions of interresponse times shifted toward the shorter intervals. When the 80-sec stimulus was no longer terminated with food, the response frequency decreased and the temporal distributions of interresponse times gradually approached pre-stimulus levels. An increased frequency of short interresponse times and an increase in response rate was again observed when the pellet termination procedure was reinstituted with the 80-sec stimulus. No change in response frequency or interresponse times was observed in the absence of the conditioning stimulus, and performance efficiency, as reflected in the ratio of responses to reinforcements during non-stimulus periods, remained stable throughout the experiment.     

The ambiguous-cue problem: The effects of reduced similarity between trials and the schedule of reinforcement

In a series of seven experiments pigeons were trained on the two-choice ambiguous-cue problem that contains PA and NA trials. On PA trials choice of P (the positive stimulus) was reinforced and choice of A (the so-called ambiguous stimulus) was not reinforced; on NA trials choice of A was reinforced and choice of N (the negative stimulus) was not reinforced. When the two types of trials were distinguished by only the presence of P or N and each trial was terminated by one response, subjects quickly attained high levels of performance on NA trials but showed only chance levels of performance on PA trials. PA performance was markedly improved when a variety of additional stimuli distinguished the two types of trials and when a variety of schedules of intermittent reinforcement were employed; these manipulations had little effect on NA performance.     

Effects upon rats' responses on a running wheel of single alternation of large and small rewards and external cues

Thirty-two rats were trained to run on a running wheel for one or six pellets of food, in either single alternation (SA) or quasi-random sequences of reward magnitude and with external cues either informing or not informing the rats of the reward magnitude. Reward-magnitude discrimination was clearly shown when the informative cues were given, but without such cues the SA patterning of responding appeared only late in training. The reward-magnitude discrimination developed most quickly when internal (SA) cues and external informative cues were both present; the effects of these two classes of cues on discrimination were interactive rather than additive. Possible explanations are given for the superior discrimination under the condition where both the internal and external cues were present over the condition when only the external cues were present.     

Loss of control by a less valid or by a less salient stimulus compounded with a better predictor of reinforcement

An unsalient stimulus, or one imperfectly correlated with reinforcement, may acquire significant control over responding, provided that it is the only available signal for reinforcement, but may fail to acquire control if it is reinforced only in conjunction with a second, more salient or more valid stimulus. A stimulus imperfectly correlated with reinforcement may also lose control over responding if having initially been reinforced in isolation, it is subsequently reinforced only in conjunction with another, more valid stimulus. If the effects of relative salience are to be explained in exactly the same way as those of relative validity, we should expect a similar loss of control by an unsalient stimulus, A, if, after initial consistently reinforced trials to A alone, subjects subsequently receive reinforcement only in the presence of a compound stimulus, A + B. Two experiments on discrete-trial discrimination learning in pigeons and one on conditioned suppression in rats confirm this expectation. The results have implications for theories of selective association in conditioning and discrimination learning.     

Control of responding by sounds of different quality: an evolutionary analysis.

Two experiments investigated the acquisition of discriminations between two acoustic stimuli of different quality (noise bursts vs. a 2-kHz pulsed signal) when features of the everyday environment were incorporated into the experiments. In Experiment 1, rats were trained, using food, to press a lever. Throughout all sessions, 5-s trials of noise bursts (the random stimulus) were presented, after variable intertrial intervals, through a remote speaker mounted outside the experimental enclosure. The noise burst occurred randomly with respect to reinforcement of lever pressing and had no programmed relationship to the animal's behavior. When lever pressing was established, the 2-kHz signal was presented through a speaker adjacent to the response lever according to a different set of variable intertrial intervals. A response in the presence of the 2-kHz signal terminated the trial and was reinforced. The 2-kHz signal acquired control of responding within the first few trials, whereas the random stimulus exerted no control of responding. In Experiment 2, rats were trained to press the lever in the presence of the 2-kHz signal presented through the adjacent speaker on a variable intertrial interval. After 14 sessions, 5-s trials of noise bursts (random stimulus) were presented through the remote speaker on the second variable intertrial interval. The random stimulus initially elicited exploratory behavior, which then rapidly declined. Subsequently, the random stimulus exerted no or weak control of responding. The introduction of the random stimulus had no effect on responding in the presence of the adjacent stimulus. In Experiments 3 and 4 the random stimulus was presented through the adjacent speaker, and the stimulus correlated with reinforcement was presented through the remote speaker. In both experiments, there was persistent control of responding by the random stimulus and slow development of control by the stimulus correlated with reinforcement. In Experiment 5, both stimuli were presented through the adjacent speaker. There was persistent control of responding by the random stimulus.     

Behavior of rats under fixed consecutive number schedules: effects of drugs of abuse.

Four rats responded under a simple fixed consecutive number schedule in which eight or more consecutive responses on the run lever, followed by a single response on the reinforcement lever, produced the food reinforcer. Under this simple schedule, dose-response curves were determined for diazepam, morphine, pentobarbital, and phencyclidine. The rats were then trained to respond under a multiple fixed consecutive number schedule in which a discriminative stimulus signaled when the response requirement on the run lever had been completed in one of the two fixed consecutive number component schedules. Under control conditions, the percentage of reinforced runs under the multiple-schedule component with the discriminative stimulus added was much higher than the percentage of reinforced runs under the multiple-schedule component without the discriminative stimulus. All of the drugs decreased the percentage of reinforced runs under each of the fixed consecutive number schedules by increasing the conditional probability of short run lengths. This effect was most consistently produced by morphine. The drugs produced few differences in responding between the multiple fixed consecutive number components. Responding under the simple fixed consecutive number schedule, however, was affected at lower doses of the drugs than was responding under the same fixed consecutive number schedule when it was a component of the multiple schedule. This result may be due to the difference in schedule context or, perhaps, to the order of the experiments.     

The repeated acquisition of behavioral chains

Monkeys were trained with food reinforcement in a chamber containing four groups of three levers. For each session the monkey's task was to learn a new four-response chain by pressing the correct lever in each group. A stable pattern of learning resulted, and the number of errors reached a steady state from session to session. The technique was then used to determine how various durations of timeouts, following errors, affected the acquisition of new chains. With no timeout, the monkeys made a great many errors, due in large part to superstitious responses within the reinforced chain. Timeout durations ranging from 1 sec to 4 min reduced the number of errors substantially. A second experiment investigated the effects upon acquisition errors of presenting a single light (an “instruction” stimulus) over the correct lever. When this light did not influence the monkeys' responses to the three alternatives, the chains were learned as without it. When the light did control responding, the monkey pressed the appropriate sequence of levers but did not learn the sequence. Thus, when the light was removed, the monkey performed as if learning that sequence for the first time.