The effects of a signal for free or for earned reward: Implications for the role of response-reinforcer associations in instrumental performance

Pearce and Hall (1978) investigated the effects of making a brief flash of light contingent upon response in rats lever-pressing for food on a variable-interval (VI) schedule. When this signal occurred in conjunction only with reinforced responses the response rate was lowered with respect to a condition in which an equal number of light flashes occurred uncorrelated with reinforcement. The experiments reported here compared these effects with those produced by signalling “free” food deliveries in a similar way. Experiments I and II compared the effects of presenting correlated and uncorrelated schedules of light and food to rats given no opportunity to lever-press. The different schedules did not produce differences in response rate when the levers were made available. In Experiment III, free food was delivered to rats responding on a VI schedule. Signalling the delivery of earned food pellets produced a low response rate in comparison with a condition in which the free pellets were signalled. It is concluded that signalling food delivery is effective only when the rat must respond to earn the food and it is argued that the signal has its effect by overshadowing a response-reinforcer association.     

Temporal properties of responding during stimuli that precede response-independent food

Pigeons' keypecks were reinforced with grain on the average of once per minute by schedules that maintained low response rates and by schedules that maintained high response rates. During these schedules, a fixed-duration conditioned stimulus (CS) ranging from 7.5 to 120 sec in duration across conditions terminated with response-independent food. Response rates during the CS were inversely related to CS duration. The rates and the temporal patterns of responding during the shortest CS were similar whether the ongoing schedule maintained high response rates or low response rates. As CS duration increased, the rate and pattern of responding during the CS converged on the rate and pattern of responding maintained by the baseline schedule. These data indicate that changes in responding during stimuli that signal response-independent reinforcement are not homogeneous throughout the CS; that response measures, such as “suppression ratios”, which presume homogeneity may mislead us; and that conditioned suppression and conditioned enhancement may be better talked about in terms of species-specific approach and avoidance than in terms of emotional states.     

Resistance to the response-decrementing effects of response-independent reinforcement produced by delay and non-delay schedules of reinforcement

In two experiments, animals were initially exposed to response-dependent schedules of food before exposure to response-independent reinforcement matched for overall rate and temporal distribution of reinforcers to the preceding condition. In Experiment I, response decrements during the response-independent phase were smaller after delayed reinforcement training than after a comparable immediate reinforcement schedule, for both doves and rats. In Experiment II variable-interval and variable-ratio schedules, both with either immediate or delayed reinforcement, were used with rats. Both the delayed reinforcement schedules produced resistance to subsequent response-independent reinforcement, but response decrements were larger after either of the immediate reinforcement conditions. It was concluded that the critical factor in response maintenance under response-independent reinforcement was the type of response-reinforcer contiguities permitted under the response-dependent schedule rather than perception of response-reinforcer “contingencies”. If the response-dependent schedule was arranged so that behaviours other than a designated operant (key pecking or lever pressing) could be contiguous with food, responding was maintained well under response-independent schedules.     

Choice between delayed reinforcers in an adjusting-delay schedule: The effects of absolute reinforcer size and deprivation level

Choice between two reinforcers differing in magnitude and delay was investigated in rats using an adjusting-delay discrete-trials schedule in which the two reinforcers were associated with two levers (A and B). The delay to Reinforcer A (the smaller reinforcer) was always 2 sec, whereas the delay to Reinforcer B was varied in accordance with the distribution of choices in successive blocks of trials. In Experiment 1, the mean delay to the large reinforcer during the last 5 of 60 training sessions was greater when the rats were maintained at 80% than when they were maintained at 90% of their free-feeding body weights. In Experiment 2, the delay to the larger reinforcer was greater when the two reinforcers consisted of one and two 45-mg food pellets than when they consisted of three and six pellets. The results are consistent with a model of “self-control” which posits hyperbolic relations between reinforcer value and reinforcer magnitude, and between reinforcer value and delay of reinforcement.     

Locus of control and the perception of freedom in an agent of reinforcement

Internals and externals observed a staged videotape presentation that portrayed a “teacher” and a “learner” engaging in a 20-item anagrams task. For externals, any variability (10, 50, and 90% schedule) in the reinforcing behavior of the “teacher” (agent of reinforcement) resulted in significantly greater perceived freedom than when no variability (0 and 100% schedule) occurred in behavior. For internals, perceived freedom was a negative linear function of the amount of reinforcement administered. Internals indicated that they could influence the behavior of the “teacher” in all reinforcement conditions, whereas externals indicated that they could influence only the “teachers” in the variable reinforcement conditions. It was concluded that greater freedom is attributed to a person whose behavior is perceived as congruent with the person's inner disposition.     

Contingency effects with maintained instrumental reinforcement

In three experiments we investigated the effect on the performance of thirsty rats of varying the instrumental contingency between lever pressing and the delivery of a saccharin reinforcer. In Experiment 1, the subjects performed more slowly in a non-contingent condition, in which the momentary probability of reinforcement was unaffected by whether or not the animals pressed, than in a contingent condition in which the reinforcer was never presented except following a lever press. This was true of performance under both random ratio and interval schedules in which the function determining the probability of reinforcement following a lever press remained the same across the contingent and non-contingent conditions. Experiment 2 demonstrated that instrumental performance was less affected when the contingency was degraded by the introduction of free reinforcers if these reinforcers were signalled. In Experiment 3, lever pressing was reinstated to some degree after non-contingent training by giving non-reinforced exposure to the operant chamber in the absence of the lever. These results suggest that free reinforcers depress instrumental behaviour through a performance mechanism engaged by their ability to support conditioning of the contextual cues.     

Habituation of conditioned suppression

Habituation of a conditioned emotional response was investigated using a procedure which eliminated contaminating temporal discriminations. Three rats were trained to bar press on a random interval 60 sec schedule of milk reinforcement and variable duration tone-shock pairings were superimposed upon this baseline. Very little recovery from conditioned suppression was found over 60 sessions of testing and no systematic differences were found after a month's “vacation” from the procedure. Analysis of responding within the CS period showed uniform suppression. The data are discussed in terms of stimulus predictability.     

Alternative reinforcement effects on fixed-interval performance

Pigeons' key pecks were reinforced with food on a fixed-interval schedule. Food also was available at variable time periods either independently of responding or for not key pecking (a differential-reinforcement-of-other-behavior schedule). The latter condition arranged reinforcement following the first pause of t seconds after it became available according to a variable-time schedule. This schedule allowed separation of the effects of pause requirements ≤ five-seconds and reinforcement frequency. The time spent pausing increased as the duration of the pause required for reinforcement increased from 0 to 30 seconds and as the frequency of reinforcement for pausing increased from 0 to 2 reinforcers per minute. Key pecking was more evenly distributed within each fixed interval with shorter required pauses and with more frequent reinforcement for pausing. The results complement those obtained with other concurrent schedules in which the same operant response was reinforced in both components.     

Effects of a conditioned reinforcer upon accuracy of match-to-sample behavior in pigeons

Three pigeons were trained to perform a two-key sequential match-to-sample task. During baseline conditions, food reinforcement was contingent upon the first match response to occur following 8-min periods, and orange illumination of both keys preceded the delivery of food by 0.5 sec. The baseline schedule of food reinforcement was in effect throughout the study. In some conditions, a 0.5-sec flash of orange keylight alone was presented contingent upon mismatch responses that followed variable time periods averaging 1 min. Rate of mismatch responses increased and accuracy of matching performances decreased as compared with baseline conditions. The ability of the 0.5-sec orange flash to reinforce mismatch responses was markedly reduced when it no longer immediately preceded the delivery of food.     

Sensitization and habituation regulate reinforcer effectiveness

We argue that sensitization and habituation occur to the sensory properties of reinforcers when those reinforcers are presented repeatedly or for a prolonged time. Sensitization increases, and habituation decreases, the ability of a reinforcer to control behavior. Supporting this argument, the rate of operant responding changes systematically within experimental sessions even when the programmed rate of reinforcement is held constant across the session. These within-session changes in operant responding are produced by repeated delivery of the reinforcer, and their empirical characteristics correspond to the characteristics of behavior undergoing sensitization and habituation. Two characteristics of habituation (dishabituation, stimulus specificity) are particularly useful in separating habituation from alternative explanations. Arguing that habituation occurs to reinforcers expands the domain of habituation. The argument implies that habituation occurs to biologically important, not just to neutral, stimuli. The argument also implies that habituation may be observed in “voluntary” (operant), not just in reflexive, behavior. Expanding the domain of habituation has important implications for understanding operant and classical conditioning. Habituation may also contribute to the regulation of motivated behaviors. Habituation provides a more accurate and a less cumbersome explanation for motivated behaviors than homeostasis. Habituation also has some surprising, and easily testable, implications for the control of motivated behaviors.     

Stimulus-reinforcer contingencies and local behavioral contrast

Four pigeons were exposed to a series of multiple schedules of variable-interval reinforcement in which pecks were required on one key (operant key) and components were signalled on a second key (signal key). Four additional pigeons experienced identical conditions, except that a yoking procedure delivered food on variable-time schedules, with no key pecks required. One of the components of the multiple schedule was constant throughout the experiment as a variable-interval (or variable-time) 30-second schedule. Operant-key responding during the constant component was uniform throughout the component, uninfluenced by changes in the duration of the variable component, and only slightly influenced by changes in reinforcement frequency correlated with the variable component. By comparison, signal-key response rate during the constant component was highest at the onset of the component, was higher when the variable component was 60-sec long than when it was 1-sec long, and was higher when no reinforcement occurred in the variable component than when reinforcement was scheduled in the variable component. These characteristics of signal-key pecking matched characteristics of local positive behavioral contrast. These data are taken to support the “additivity theory” of behavioral contrast and to suggest that Pavlovian stimulus-reinforcer relations contribute primarily to the phenomenon of local positive contrast.     

Enhancement of food-rewarded instrumental responding by an appetitive conditioned stimulus

Four experiments are reported in which a stimulus (with a minimum duration of 60 s) signalling the delivery of “free” food was presented to rats lever-pressing for food available on a variable interval schedule. It was found that responding was enhanced in the presence of the stimulus when the baseline schedule of reinforcement was lean (Experiment I) and that the enhancement was dependent upon the pairing of the stimulus with free food (Experiments II and III). Experiment IV showed that an enhancement could be found after initial training in which stimulus-food pairings were given to subjects that were not concurrently lever pressing for food. It is argued that these results are consistent with the suggestion that an appetitive conditioned stimulus can energise appetitive instrumental behaviour.     

Operant conditioning of autogrooming in vervet monkeys: Cercopithecus aethiops

Vervet monkeys received food reinforcement contingent on autogrooming. Experiment 1 reinforced grooming on a schedule of increasing intermittency and grooming increased in frequency and duration; with only pauses reinforced, grooming decreased in frequency and duration. Experiment 2 demonstrated differentiation of operant autogrooming; in each session a different single form of grooming was reinforced (for example, grooming the tail only), and that form increased in frequency while other forms became less frequent. In Experiment 3 scratching was succesfully conditioned with a method that selectively reinforced variety in behavior; reinforcement was contingent on a shift in scratching form. In Experiment 4, with no contingencies on grooming, a prefood stimulus did not increase autogrooming whether or not grooming had previously resulted in contingent reinforcement. The form of conditioned autogrooming resembled the form of unconditioned autogrooming. The discussion suggests how reinforcement principles can account for changes in the topography of operant behavior.     

Operant and nonoperant vocal responding in the mynah: Complex schedule control and deprivation-induced responding

Several recent studies have been concerned with operant responses that are also affected by nonoperant factors, (e.g., biological constraints, innate behavior patterns, respondent processes). The major reason for studying mynah vocal responding concerned the special relation of avian vocalizations to nonoperant emotional and reflexive systems. The research strategy was to evaluate operant and nonoperant control by comparing the schedule control obtained with the vocal response to that characteristic of the motor responses of other animals. We selected single, multiple, and chain schedules that ordinarily produce disparate response rates at predictable times. In multiple schedules with one component where vocal responding (“Awk”) was reinforced with food (fixed-ratio or fixed-interval schedule) and one where the absence of vocal responding was reinforced (differential reinforcement of other behavior), response rates never exceeded 15 responses per minute, but clear schedule differences developed in response rate and pause time. Nonoperant vocal responding was evident when responding endured across 50 extinction sessions at 25% to 40% of the rate during reinforcement. The “enduring extinction responding” was largely deprivation induced, because the operant-level of naive mynahs under food deprivation was comparable in magnitude, but without deprivation the operant level was much lower. Food deprivation can induce vocal responding, but the relatively precise schedule control indicated that operant contingencies predominate when they are introduced.     

The effects of punishment intensity on squirrel monkeys

Responses of squirrel monkeys were maintained by a variable-interval schedule of food reinforcement. Concurrently, punishment consisting of a brief electric shock followed each response. As has been found for pigeons and rats, punishment did not produce extreme, all-or-none reactions. By gradually increasing the punishment intensity it was possible to produce response rates intermediate to no suppression and complete suppression. Similarly, the moment-to-moment response rate was free of extreme fluctuations. A “warm-up” effect occurred in which the punished responses were especially suppressed during the initial part of a session. The pre-punishment performance was negatively accelerated within a session, and punishment reduced the degree of negative acceleration. When punishment was discontinued, responding recovered immediately except when suppression had been complete or prolonged. When the punishment intensity was decreased gradually, more suppression resulted at a given intensity than when intensity was increased gradually. This suggests a “behavioral inertia” effect wherein behavior at a new punishment intensity is biased toward the behavior at the previous value. A corollary generalization is that the larger the change in intensity, the less the behavior at the new value will be biased toward the behavior at the previous value.     

Reinforcement duration and the peak shift in post-discrimination gradients

Pigeons were trained to key-peck for food, first with single-stimulus training and then with successive discrimination (multiple schedule) training. In the multiple schedule, two different wavelengths were each correlated with equally frequent variable-interval reinforcement but different durations (6 sec vs. 2 sec) of access to grain. For some birds, the different durations of feeding cycle were cued by different intensities of the food hopper light. For some of these “cued” birds, single-stimulus training had been carried out with 6-sec feedings and when multiple-schedule training was introduced, the novel stimulus was correlated with 2-sec feedings. For the others, 2-sec feedings were originally used, and the novel stimulus was then present during the 6-sec reinforcement duration. The cueing procedure enhanced discrimination performance, and was necessary for the consistent production of a peak shift. In addition, the condition in which original training had been carried out with 6-sec feedings, and thus reinforcement duration was reduced in the presence of the novel stimulus, led to the best performance.     

The effect of negative stimulus presentations on observing-response rates

Theories of observing differ in predicting whether or not a signal for absence of reinforcement (S−) is capable of reinforcing observing responses. Experiments in which S− was first removed from and then restored to the procedure have yielded mixed results. The present experiments suggest that failure to control for the direct effect of presenting S− may have been responsible. Pigeons and operant procedures were used. Experiment 1 showed that presentations of S−, even when not contingent on observing, can raise the rate of an observing response that was reinforced only by presentations of a signal (S+) that accompanied a schedule of food delivery. Experiment 2 showed that this effect resulted from bursts of responding that followed offsets of S−. Experiment 3 showed that, when the presence of S− was held constant, lower rates occurred when S− was dependent on, rather than independent of, observing. These results support theories that characterize S− as incapable of reinforcing observing responses.     

The effects of isolation rearing on behavioural inhibition in the rat

The possibility that isolation-rearing in the rat affects the development of inhibitory mechanisms was studied in a series of experiments. It was found that socially-isolated rats were (1) slower to learn both a lever-panel alternation, and a two-lever alternation schedule of reinforcement, (2) more persistent than controls in pressing a lever for food when a supply of identical “free food” was introduced into the operant chamber, but (3) similar to control rats in their response to preloading with food, a procedure which inhibited lever pressing to the same extent in the two groups. Finally, it was shown in a separate experiment that the effects of increased food deprivation on lever pressing in the presence of free food were qualitatively different from the effects of social isolation, and therefore the social/isolate difference cannot be interpreted as motivational. The possible contributions of neophobia to the difference are discussed. It is concluded that isolates may well suffer from a disinhibitory defect, but that there are probably other effects of isolation in addition.     

Economic and biological influences on a pigeon's key peck

Pigeons were studied in a two-component multiple schedule. In the first phase of the experiment, key pecks were reinforced on a variable-interval 2-min schedule in both components and free food was delivered additionally during one component. When components alternated every 8 sec, all pigeons pecked at a much higher rate during the component with free food than during the other component. At a component duration of 16 min, the reverse was true: all pigeons pecked at a higher rate during the component without free food. In the second phase, the additional food during one component was made contingent on pecking. Responding during the component without the extra food remained essentially unchanged, as expected, since rate of reinforcement remained identical to that in the previous phase. However, rate of responding during the component with the extra food (now contingent on pecking) was elevated, compared to the rate in the first phase, and did not show the marked decline as component duration was increased.     

Conditioned suppression and conditioned enhancement with the same positive UCS: an effect of CS duration

Previous experiments have shown that positively reinforced operant responding is suppressed during a conditioned stimulus terminated with an electric shock (conditioned suppression). In the present experiment, the conditioned stimulus was terminated with a positive unconditioned stimulus, and it was found that the duration of the conditioned stimulus was a key factor in determining whether response suppression or response enhancement was observed during the stimulus. The lever-pressing responses of rats were maintained by a variable-interval schedule of food reinforcement. While the rats were pressing the lever, a light was occasionally turned on, its offset coincident with a brief period of access to a sucrose solution. In consecutive blocks of sessions, the light duration was 40 sec, 12 sec, or 120 sec. Results showed that the rate of lever pressing was substantially suppressed during the 12-sec stimulus, slightly suppressed during the 40-sec stimulus, and enhanced during the 120-sec stimulus.