The effects of a signal for free or for earned reward: Implications for the role of response-reinforcer associations in instrumental performance

Pearce and Hall (1978) investigated the effects of making a brief flash of light contingent upon response in rats lever-pressing for food on a variable-interval (VI) schedule. When this signal occurred in conjunction only with reinforced responses the response rate was lowered with respect to a condition in which an equal number of light flashes occurred uncorrelated with reinforcement. The experiments reported here compared these effects with those produced by signalling “free” food deliveries in a similar way. Experiments I and II compared the effects of presenting correlated and uncorrelated schedules of light and food to rats given no opportunity to lever-press. The different schedules did not produce differences in response rate when the levers were made available. In Experiment III, free food was delivered to rats responding on a VI schedule. Signalling the delivery of earned food pellets produced a low response rate in comparison with a condition in which the free pellets were signalled. It is concluded that signalling food delivery is effective only when the rat must respond to earn the food and it is argued that the signal has its effect by overshadowing a response-reinforcer association.     

Instrumental performance on negative schedules

Three experiments examined the performance of rats pressing a lever for food reinforcement on a schedule in which high rates of response resulted in lowered rates of reinforcement (i.e. a schedule with a negative component). In Experiment 1, rats responded on a variable interval (VI) schedule with a conjoint component such that every 30 responses a reinforcement programmed by the VI schedule was cancelled. These subjects generally emitted a lower response rate than rats responding on a VI schedule yoked to the former subjects with respect to the delivery of reinforcement, although response rate differences were sometimes not large. Similar response-rate effects were obtained in Experiment 2 using a within-subject yoking procedure. In Experiment 3, reinforced interresponse times were matched on negative and VI schedules yoked in terms of reinforcement rate, and the response rate emitted in these conditions were similar. These results give support to theories of instrumental conditioning that stress the strengthening and shaping properties of reinforcement.     

The schedule dependency of the signaled reinforcement effect

In Experiment 1, rats leverpressed for food reinforcement on either a variable ratio (VR) 30 schedule or a variable interval (VI) 15-s schedule. One group in each condition received a signal filling a 500-ms delay of reinforcement. This treatment enhanced rates on the VR schedule, and attenuated rates on the VI schedule, relative to the rate seen in an unsignaled control condition. In Experiment 2 there was no delay of reinforcement and the signal and food were presented simultaneously. Attenuated rates of responding were observed on VI schedules with a range of mean interval values (15 to 300 s). Experiment 3 used a range of VR schedules (10 to 150) with simultaneous presentations of signal and food. A signal-induced enhancement of response rate was found at all VR values. In Experiment 4, a signal elevated response rates on a tandem VI VR schedule, but depressed rates on a tandem VR VI schedule, compared to control conditions receiving unsignaled delayed reinforcement. These results are taken to show that the effect of a signal accompanying reinforcement depends upon the nature of the behavior that is reinforced during exposure to a given schedule.     

ALTERNATIVE REINFORCEMENT INCREASES RESISTANCE TO CHANGE: PAVLOVIAN OR OPERANT CONTINGENCIES?

Two multiple-schedule experiments with pigeons examined the effect of adding food reinforcement from an alternative source on the resistance of the reinforced response (target response) to the decremental effects of satiation and extinction. In Experiment 1, key pecks were reinforced by food in two components according to variable-interval schedules and, in some conditions, food was delivered according to variable-time schedules in one of the components. The rate of key pecking in a component was negatively related to the proportion of reinforcers from the alternative (variable-time) source. Resistance to satiation and extinction, in contrast, was positively related to the overall rate of reinforcement in the component. Experiment 2 was conceptually similar except that the alternative reinforcers were contingent on a specific concurrent response. Again, the rate of the target response varied as a function of its relative reinforcement, but its resistance to satiation and extinction varied directly with the overall rate of reinforcement in the component stimulus regardless of its relative reinforcement. Together the results of the two experiments suggest that the relative reinforcement of a response (the operant contingency) determines its rate, whereas the stimulus-reinforcement contingency (a Pavlovian contingency) determines its resistance to change.     

Effects of reinforcement magnitude and ratio values on behaviour maintained by a cyclic ratio schedule of reinforcement

In Experiments 1 and 2, lever pressing by rats was reinforced on a cyclic ratio schedule of food reinforcement, comprising a repeated sequence of fixed-ratio component schedules. Reinforcement magnitude was varied, on occasional sessions in Experiment 1 and across blocks of sessions in Experiment 2, from one to two or three 45-mg food pellets. In the one-pellet condition, post-reinforcement pauses increased with component schedule value. At higher magnitudes, post-reinforcement pauses increased, and overall response rates declined. Response rate on component schedules was a decreasing linear function of the obtained rate of reinforcement in all conditions. Plotted against component schedule value, response rate increased exponentially to an asymptote that decreased when reinforcement magnitude increased. These findings are consistent with regulatory accounts of food reinforced behaviour. In Experiment 3, rats were trained under a cyclic ratio schedule comprising fixed-ratio components including higher values, and some inverted U-shaped response functions were obtained. Those rats that did not showthis relationship were trained on cyclic ratios with even higher values, and all showed inverted U-shaped response functions. This suggests that behaviour on cyclic ratio schedules can reflect activating of reinforcement as well as the satiating effects seen in Experiments 1 and 2.     

Resurgence is greater following a return to the training context than remaining in the extinction context

The present study examined whether resurgence of a previously reinforced target response upon removing alternative reinforcement would be greater when (1) returning to the original training context (ABA context changes) versus (2) remaining in the analogue treatment context in which the alternative response was differentially reinforced (ABB context changes). Experiment 1 arranged reinforcement of button pressing with points exchangeable for money in university students. Experiment 2 arranged reinforcement of lever pressing with food for rats. Experiment 3 arranged reinforcement of responses to a touchscreen with small bites of food with children diagnosed with ASD. Overall, resurgence of target responding tended to be greater when returning to the original training context (A) than when remaining in the analogue treatment context (B). These findings suggest context changes with differential reinforcement treatments could exacerbate the recurrence of problem behavior resulting from reductions in treatment integrity through failure to reinforce appropriate behavior.     

Positive and negative conditioned suppression in the pigeon: Effects of the locus and modality of the CS

In Experiment I, four pigeons were exposed to trials in which a 12-sec key light illumination was followed by free food. These trials were superimposed upon a baseline of key pecking for food reinforcement on a variable-interval schedule. When the signal for food was on the operant key, response rate was substantially higher during the signal than during the baseline procedure. When the signal was on a second, signal key, operant responding was suppressed during the signal and substantial pecking of the signal key occurred. The sum of signal key and operant key pecks far exceeded the operant baseline rate of responding. An explanation of opposite results obtained with rats and pigeons as subjects in experiments of this type was suggested in terms of the spatial relation between the signal for free food and the operant target which usually characterizes these experiments. Experiment II assessed the importance of signal location when shock rather than food was the US. Suppression of operant key pecking was unaffected by signal location. Experiment III assessed the relative effectiveness of visual and auditory stimuli (clicks) as signals for food and shock, and found that all combinations of signal and US were equally effective in suppressing operant key responding. The three experiments together suggested that the identification of important effects of species—typical behavior in one experimental situation does not imply that there will be like effects in similar situations.     

Visual Reinforcement Signals Interfere with the Effects of Reinforcer Magnitude Manipulations

Three experiments examined the effect of reinforcement magnitude on free-operant response rates. In Experiment 1, rats that received four food pellets responded faster than rats that received one pellet on a variable ratio 30 schedule. However, when the food hopper was illuminated during reinforcer delivery, there was no difference between the rates of response produced by the two magnitudes of reward. In Experiment 2, there was no difference in response rates emitted by rats receiving either one or four pellets of food as reward on a random interval (RI) 60-s schedule. In Experiment 3, rats responding on an RI 30-s schedule did so at a lower rate with four pellets as reinforcement than with one pellet. This effect was abolished by the illumination of the food hopper during reinforcement delivery. These results indicate that the influence of magnitude is obscured by manipulations which signal the delivery of reinforcement.     

Behavioral contrast produced by a signaled decrease in local rate of reinforcement

Pigeons' key pecking in the presence of one stimulus (S1) was reinforced according to a response-dependent variable-interval schedule. Pecking rate during S1 increased (behavioral contrast) when a second stimulus (S2) [associated with either a response-dependent fixed-interval schedule (Experiment I) or a response-independent reinforcement schedule in which reinforcement availability was signaled by visual (Experiment II) or temporal (Experiment III) stimuli] alternated with S1. These experiments suggest that a discriminable, signaled decrease in local reinforcement rate during S2 is an antecedent of the behavioral contrast response rate increases during S1.     

The effect of signaled reinforcement on rats' fixed-interval responding

Four experiments examined the effect on rats' response rate of presenting a brief (500 ms) stimulus simultaneously with the delivery of food on fixed-interval (FI) schedules. In Experiment 1, reinforcement signals that were spatially diffuse (both tones and lights) elevated rates of responding, but responding was attenuated by localized visual stimuli. The remaining experiments examined the signal-induced potentiation of responding. In Experiment 2, a tone reinforcement signal potentiated response rates on an FI schedule, but attenuated response rates on a variable-interval (VI) schedule. This difference was obtained even though the overall rate of responding was equated on the two schedules before the introduction of the signal. Signal-induced potentiation of responding occurred over a range of FI values employed in Experiment 3. In Experiment 4, presenting a reinforcement signal when high local rates of response had occurred immediately before reinforcement resulted in potentiated rates of responding on an FI schedule. The opposite effect on response rate occurred when the reinforcement signal followed only low local rates of response. These results indicate that a variety of factors influence the effects of a reinforcement signal. They imply, however, that the local rate of response at the time of reinforcement is a key factor in establishing the nature of the signaling effect.     

Timeout postponement without increased reinforcement frequency

Three experiments were conducted to examine pigeons' postponement of signaled extinction periods (timeouts) from a schedule of food reinforcement when such responding neither decreased overall timeout frequency nor increased the overall frequency of food reinforcement. A discrete-trial procedure was used in which a response during the first 5 s of a trial postponed an otherwise immediate 60-s timeout to a later part of that same trial but had no effect on whether the timeout occurred. During time-in periods, responses on a second key produced food according to a random-interval 20-s schedule. In Experiment 1, the response-timeout interval was 45 s under postponement conditions and 0 s under extinction conditions (responses were ineffective in postponing timeouts). The percentage of trials with a response was consistently high when the timeout-postponement contingency was in effect and decreased to low levels when it was discontinued under extinction conditions. In Experiment 2, the response-timeout interval was also 45 s but postponement responses increased the duration of the timeout, which varied from 60 s to 105 s across conditions. Postponement responding was maintained, generally at high levels, at all timeout durations, despite sometimes large decreases in the overall frequency of food reinforcement. In Experiment 3, timeout duration was held constant at 60 s while the response-timeout interval was varied systematically across conditions from 0 s to 45 s. Postponement responding was maintained under all conditions in which the response-timeout interval exceeded 0 s (the timeout interval in the absence of a response). In some conditions of Experiment 3, which were designed to control for the immediacy of food reinforcement and food-correlated (time-in) stimuli, responding postponed timeout but the timeout was delayed whether a response occurred or not. Responding was maintained for 2 of 3 subjects, suggesting that behavior was negatively reinforced by timeout postponement rather than positively reinforced by the more immediate presentation of food or food-correlated (time-in) stimuli.     

Effects of a conditioned reinforcer upon accuracy of match-to-sample behavior in pigeons

Three pigeons were trained to perform a two-key sequential match-to-sample task. During baseline conditions, food reinforcement was contingent upon the first match response to occur following 8-min periods, and orange illumination of both keys preceded the delivery of food by 0.5 sec. The baseline schedule of food reinforcement was in effect throughout the study. In some conditions, a 0.5-sec flash of orange keylight alone was presented contingent upon mismatch responses that followed variable time periods averaging 1 min. Rate of mismatch responses increased and accuracy of matching performances decreased as compared with baseline conditions. The ability of the 0.5-sec orange flash to reinforce mismatch responses was markedly reduced when it no longer immediately preceded the delivery of food.     

Persistence and relapse of reinforced behavioral variability

The present study examined persistence and relapse of reinforced behavioral variability in pigeons. Pigeons emitted four‐response sequences across two keys. Sequences produced food according to a lag schedule, in which a response sequence was followed by food if it differed from a certain number of previous sequences. In Experiment 1, food was delivered for sequences that satisfied a lag schedule in both components of a multiple schedule. When reinforcement was removed for one component (i.e., extinction), levels of behavioral variability decreased for only that component. In Experiment 2, food was delivered for sequences satisfying a lag schedule in one component of a multiple schedule. In the other component, food was delivered at the same rate, but without the lag variability requirement (i.e., yoked). Following extinction, levels of behavioral variability returned to baseline for both components after response‐independent food delivery (i.e., reinstatement). In Experiment 3, one group of pigeons responded on a lag variability schedule, and the other group responded on a lag repetition schedule. For both groups, levels of behavioral variability increased when alternative reinforcement was suspended (i.e., resurgence). In each experiment, we observed some evidence for extinction‐induced response variability and for variability as an operant dimension of behavior.     

Nest material and food as reinforcers for fixed-ratio responding in mice

Three experiments investigated the importance of operant-reinforcer distance as a factor affecting continuous reinforcement (CRF) and fixed-ratio (FR) performance in mice, with nest material and food as reinforcers. In Experiment 1, nest material (paper strips) was presented on a series of FR schedules of increasing size, with the operandum located as close as possible to the reinforcer dispenser. Subjects compensated for increases in FR size by proportionately increasing their response output, but ratio strain occurred at low FR values. In Experiment 2, response rate was found to be inversely related to operant-reinforcer distance on a CRF schedule with nest material as reinforcer. In Experiment 3, food was presented on a series of FR schedules at two levels of deprivation, and with three operant-reinforcer distances. Operant-reinforcer distance was found to affect CRF response rate, degree of compensation for increases in FR size, and occurrence of ratio strain, but only when deprivation level was low. The results support the view that nest material and food share fundamentally similar reinforcing properties, but that nest material is a weaker reinforcer under normal test conditions.     

Operant hoarding: a new paradigm for the study of self-control.

In the first of four experiments, rats were exposed to a modified multiple continuous reinforcement-extinction schedule during 15-min daily sessions. In one condition (saves condition) with the cuelight on, a single lever press produced a food pellet, briefly extinguished the cuelight, and started a clock. Saves (additional lever presses with interresponse times less than 1 s) produced an additional food pellet, briefly extinguished the cuelight, and restarted the interresponse time clock. The cuelight was extinguished 1 s after the last lever press and remained off during a 10-s period of extinction, during which no food pellets were delivered. In the other condition (savings account condition), the contingencies were the same except that the cuelight was extinguished and was not reilluminated after the initial lever press, and the delivery of all food pellets in the reinforcement component was delayed until the onset of extinction. In both conditions, rats made saves, but mean saves (total saves divided by the number of reinforcement components) were slightly reduced in the savings account condition. In Experiment 2, using six equally spaced 15-min sessions per day on alternate days, saves were either followed immediately with food and brief cuelight offset (saves condition) or were not reinforced at all. Mean saves were much greater when saves were reinforced. In Experiment 3, during 5-min daily sessions, saves earned a single pellet (savings account condition) or a number of pellets equal to the ordinal number of the lever press (interest condition). Rats made fewer mean saves, with little change in the food rate, when saves earned interest. In Experiment 4, the rats earned all their food in the operant situation during 24 daily 5-min sessions, these separated by 55-min intersession intervals during which no food was available; otherwise, the conditions were the same as in Experiment 3. In Experiment 4, the shift to interest for saves led to an increase in mean daily mean saves (total daily mean saves divided by the number of daily sessions) as well as to an increase in the number of food pellets delivered in each session. The results are discussed in terms of self-control and behavioral economics.     

Multiple determinants of the effects of reinforcement magnitude on free-operant response rates

Four experiments examined the effects of increasing the number of food pellets given to hungry rats for a lever-press response. On a simple variable-interval 60-s schedule, increased number of pellets depressed response rates (Experiment 1). In Experiment 2, the decrease in response rate as a function of increased reinforcement magnitude was demonstrated on a variable-interval 30-s schedule, but enhanced rates of response were obtained with the same increase in reinforcement magnitude on a variable-ratio 30 schedule. In Experiment 3, higher rates of responding were maintained by the component of a concurrent variable-interval 60-s variable-interval 60-s schedule associated with a higher reinforcement magnitude. In Experiment 4, higher rates of response were produced in the component of a multiple variable-interval 60-s variable-interval 60-s schedule associated with the higher reinforcement magnitude. It is suggested that on simple schedules greater reinforcer magnitudes shape the reinforced pattern of responding more effectively than do smaller reinforcement magnitudes. This effect is, however, overridden by another process, such a contrast, when two magnitudes are presented within a single session on two-component schedules.     

Signal duration and suppression of operant responding by free reinforcement

Pigeons were trained on a VI (variable interval) schedule of food presentation with a superimposed schedule of response-independent food. Substantial suppression of the operant response rate occurred when the free food was presented without a signal. When the free food was preceded by a short (4 sec) signal, the degree of suppression was similar to that with unsignaled free food. But when the signal was lengthened to 12 sec, the degree of suppression was substantially reduced. Experiment 3 assessed the effect of signal duration using a baseline schedule of delayed reinforcement, in which contingent reinforcers were themselves preceded by a signal. The signal preceding the free reinforcers was then either the same as or different from this contingent signal. Signal duration effects occurred only when the two types of signals were different. These differences as a function of signal duration have implications for both “context-blocking” and “comparator” interpretations of the effects of noncontingent reinforcement in both Pavlovian and operant procedures.     

Examining stimuli paired with alternative reinforcement to mitigate resurgence in children diagnosed with autism spectrum disorder and pigeons

In two laboratory experiments, we examined whether stimuli paired with alternative reinforcers could mitigate resurgence of a previously reinforced target response with pigeons (Experiment 1) and children diagnosed with Autism Spectrum Disorder (Experiment 2). In Phase 1, we arranged food reinforcement according to a variable-ratio schedule for engaging in a target response. In Phase 2, we arranged extinction for target responding and differentially reinforced alternative responding according to a fixed-ratio schedule, with every alternative-reinforcer delivery paired with a change in keylight color (Experiment 1) or automated verbal (praise) statement (Experiment 2). In Phase 3, we assessed resurgence during extinction of target and alternative responding in the presence versus absence of continued presentation of the paired stimulus. Despite variation across sessions, resurgence on average was lower when continuing to present the paired stimuli in all pigeons and children while maintenance of alternative responding did not differ between assessments. These findings indicate that stimuli paired with alternative reinforcement can modestly decrease resurgence, but further examination of their efficacy and a better understanding of the underlying processes are necessary before they can be recommended for clinical use in reducing resurgence of clinically relevant problem behavior.     

Effects of different delay of reinforcement procedures on variable-interval responding

Two experiments studied responding in the rat when the first bar press after a variable period of time produced a cue light that remained on for either 10, 30, or 100 sec and terminated with the delivery of food. In Experiment I, response rate decreased and time to the first response after reinforcement increased as the delay of reinforcement increased. Similar results were obtained whether the delay consisted of retracting the lever during the delay, a fixed delay with no scheduled consequence for responding, or every response during the delay restarted the delay interval. In Experiment II, fixed-delay and fixed-interval schedules of the same duration during the delay period had no differential effect on either response rate or time to the first response after reinforcement, but differentially controlled responding during the delay periods.     

The influence of positive and negative reinforcement on selective attention in the rat

In Experiments 1 and 2 rats were trained under two multiple schedules of reinforcement. In one, bar pressing during a tone-light compound stimulus was reinforced under a variable-interval food reinforcement schedule. In the other multiple schedule, bar pressing avoided grid shock on a free-operant schedule. In both multiple schedules, a discrimination was maintained by an extinction schedule that was operative during the absence of the tone-light compound. In Experiments 1 and 2 the intensity of the tone-light compound was manipulated over three levels. Subsequent extinction tests revealed that light was attended to, almost exclusively of the tone, when food reinforcement had maintained bar pressing. On the other hand, the tone gained considerable attentional control under the shock avoidance schedule. This stimulus-reinforcer interaction was maintained for all three levels of the compound intensity. In Experiment 3 it was investigated whether this interaction was associative by presenting shock during the absence of the tone-light compound when food reinforcement maintained responding, and food during the absence of the compound when shock avoidance maintained responding. Since both food and shock were presented during a single session for both schedules, nonassociative effects of the reinforcing stimuli were equivalent across the schedules. Nevertheless, the stimulus-reinforcer interaction was maintained, indicating that the interaction was an associative effect.