Stimulus-reinforcer contingencies and local behavioral contrast

Four pigeons were exposed to a series of multiple schedules of variable-interval reinforcement in which pecks were required on one key (operant key) and components were signalled on a second key (signal key). Four additional pigeons experienced identical conditions, except that a yoking procedure delivered food on variable-time schedules, with no key pecks required. One of the components of the multiple schedule was constant throughout the experiment as a variable-interval (or variable-time) 30-second schedule. Operant-key responding during the constant component was uniform throughout the component, uninfluenced by changes in the duration of the variable component, and only slightly influenced by changes in reinforcement frequency correlated with the variable component. By comparison, signal-key response rate during the constant component was highest at the onset of the component, was higher when the variable component was 60-sec long than when it was 1-sec long, and was higher when no reinforcement occurred in the variable component than when reinforcement was scheduled in the variable component. These characteristics of signal-key pecking matched characteristics of local positive behavioral contrast. These data are taken to support the “additivity theory” of behavioral contrast and to suggest that Pavlovian stimulus-reinforcer relations contribute primarily to the phenomenon of local positive contrast.     

Determinants of contrast in the signal-key procedure: Evidence against additivity theory

Two experiments are reported that challenge the interpretation of previous results with the signal-key procedure, in which the discriminative stimuli are located on a response key different from the key associated with the operant response requirement. Experiment 1 replicated the procedure of Keller (1974), and found that contrast effects on the operant key occurred reliably for only one of four subjects. High rates to the signal key initially occurred for only one subject, but modifications of the procedure produced substantial rates to the signal key for all subjects. In all cases, however, signal-key behavior was greatly reduced by the addition of a changeover delay which prevented reinforcement within 2 seconds of the last peck to the signal key, suggesting that signal-key pecking was maintained primarily by adventitious reinforcement. Experiment 2 modified the signal-key procedure by using three response keys, so that the discriminative stimuli on the signal key controlled different responses during all phases of training. With this modification, reliable contrast effects on the operant key occurred for all subjects, suggesting that the failure to find contrast in previous studies has been due to the confounding of changes in the discrimination requirements with changes in relative rate of reinforcement. The results challenge the additivity theory of contrast, and suggest that “elicited” behavior plays a minor role, if any, in the determination of contrast effects in multiple schedules.     

Signalled reinforcement and multiple schedules

The responses of four pigeons were first reinforced in the presence of two different wave-lengths (green and red) on a two-ply multiple schedule with identical variable-interval 3-min schedules of reinforcement associated with each component. While the constant-component reinforcement schedule remained unchanged during the experiment, the schedule associated with the variable component was changed to (1) signalled variable time, (2) unsignalled variable time, or (3) signalled variable interval. The probability with which the availability of the reinforcer was signalled in the variable-interval schedules was either 0.5 or 1.0. Positive contrast occurred in both signalled variable-interval and variable-time schedules, but only when the availability of all the variable-component reinforcers was signalled. Signalling the availability of only 50% of the reinforcers in signalled variable-interval schedules resulted in negative induction. The present data suggest that positive behavioral contrast resulting from signalled reinforcer availability is due to the presence of an extinction-correlated stimulus.     

Elicited responding to signals for reinforcement: the effects of overall versus local changes in reinforcement probability

Pigeons were studied on a three-component multiple schedule where all reinforcement was independent of responding. Two components were cued by different keylights and were associated with different rates of reinforcement. The third was always a no-key period associated with extinction. After a few sessions, pecking was elicited by the keylights signalling the reinforcement and continued to be maintained indefinitely. The duration and sequence of the three components were varied to determine if the primary controlling variable was differences in the overall probability of reinforcement, or if it was the immediate change in reinforcement signalled by the onset and/or offset of the stimulus. Both variables were found to control behavior. When 30-sec components were used, the primary controlling variable was the overall probability of reinforcement, but when 3-min components were used, overall probability had little effect. Control by local changes in reinforcement also occurred, although the type of local control varied both across subjects and experimental conditions. Some behaviors were controlled more by the change in reinforcement signalled by the onset of the stimulus, while others were controlled more by the change signalled by the offset of the stimulus.     

Control of responding during stimuli that precede transitions in reinforcement frequency

Pigeons' responses were reinforced on a variant of a mixed variable-interval extinction schedule of reinforcement in which the transition to the higher reinforcement rate was signaled by a trace stimulus projected on the response key prior to the onset of the component correlated with food delivery. In the first of two experiments, the duration of the trace stimulus preceding the component correlated with food delivery was varied from 1.5 to 50.0 s and in the second experiment, the reinforcement frequency in the same component was varied from 10 to 60 reinforcers per hour. Pigeons pecked at the trace stimulus preceding the onset of the component correlated with food delivery even though responding was not reinforced in its presence and only one of the changes in reinforcement rate (i.e., from extinction to reinforcement) was signaled. The rate of pecking during the trace stimulus was a function of its duration but not of the reinforcement frequency in the following component. Higher rates generally occurred at the shorter trace-stimulus durations. Component responding following the offset of the trace stimulus was under discriminative control of the trace stimulus whether or not responding occurred in the presence of the trace stimulus.     

Behavioral contrast of time allocation

Pigeons' standing on a platform produced food reinforcement according to two-component multiple schedules in which either both components consisted of the same variable-interval schedule or one of these was replaced with a component without reinforcement (extinction). The components of the multiple schedule alternated every 30 sec, and were signalled by changes in the color of diffuse overhead illumination. Changing the schedule of one of the components to extinction increased the percentage of time spent on the platform during the unchanged component (behavioral contrast). This result casts doubt on accounts that attribute behavioral contrast to variations in the rate of noninstrumental elicited responses.     

Inverse relations between preference and contrast

Pigeons were trained on a multiple schedule in which two target components with identical reinforcement schedules were followed by either the same-valued schedule or by extinction. Response rate increased in both target components but was higher in the target component followed by extinction, replicating previous findings of positive anticipatory contrast. A similar design was used to study negative contrast, in that the two target components were followed either by the same-valued schedule or by a higher valued schedule. Negative contrast occurred equally, on average, in both target components, thus failing to demonstrate negative contrast that is specifically anticipatory in nature. When the stimuli correlated with the two target components were paired in choice tests, the pattern of preference was in the opposite direction. For the positive contrast procedure, no significant preference between the two target stimuli was evident. But for the negative contrast procedure, preference favored the stimulus followed by the higher valued schedule. The results demonstrate a functional dissociation between positive and negative contrast in relation to stimulus value. More generally, the results demonstrate an inverse relation between response rate and preference and challenge existing accounts of contrast in terms of the concept of relative value.     

Short-component multiple schedules: effects of relative reinforcement duration

Pigeons were exposed to multiple variable-interval 2-min variable-interval 2-min schedules of food presentation in which relative duration of food presentation was manipulated. When components alternated every 5 sec and were scheduled on separate response keys, relative response rates closely matched relative reinforcement duration in three of four pigeons. On the other hand, relative response rates were insensitive to relative reinforcement duration when components scheduled on a single response key alternated every 5 sec, and when components scheduled on separate response keys alternated every 2 min. Thus, both rapid alternation and spatial separation of components were necessary to produce approximate matching of relative responding to relative reinforcement duration. This finding contrasts with previous findings that only rapid component alternation is necessary for matching when relative rate of reinforcement is manipulated.     

Shortcomings of the behavioral competition theory of contrast: Reanalysis of McLean (1992)

McLean (1992) presented significant data showing that the occurrence of behavioral contrast in a multiple schedule was correlated with shifts in the frequency of reinforcers from a second source between components of the schedule, and interpreted his results as showing that contrast was due to changes in the degree of response competition within the constant component of the multiple schedule. Reanalysis of his data shows that there was an effect of reinforcement in the alternative component of the schedule independent of the shifts in reinforcers between components. Thus, the effect of relative rate of reinforcement cannot be ascribed, at least entirely, to the mechanisms proposed by the behavioral competition theory of contrast.     

Behavioral contrast: Pavlovian effects and anticipatory contrast.

Two sources of behavioral contrast have been identified previously: Pavlovian stimulus-reinforcer relations and component sequence effects (anticipatory contrast). This study sought to isolate these sources of control procedurally in a four-ply multiple schedule composed of two fixed two-component sequences. Different cues were associated with the first component of each sequence, and contrast effects were studied in these target components. In Experiment 1, differential cuing of Component 2 between sequences and availability of reinforcement during target components were varied across three groups of pigeons; the stimulus-reinforcer relation between target-component cues and schedule of reinforcement in Component 2 was varied within subjects. Control by the Pavlovian relation was demonstrated under all conditions, and anticipatory contrast was not observed. In Experiment 2, target-component duration was systematically varied in the three groups of Experiment 1. Control by the Pavlovian relation was reliably obtained only when target-component behavior was unreinforced, and diminished with increases in component duration. Anticipatory contrast emerged in the two groups for which target-component reinforcement was available. These and other data indicate that Pavlovian effects in multiple schedules may be obscured when the requisite conditions for anticipatory contrast are present.     

Key pecking in pigeons produced by pairing keylight with inaccessible grain

In Experiment I, keylight was paired with inaccessible grain delivery (under two conditions of keylight intensity) to determine if autoshaping would occur in the absence of primary reinforcement. In Experiment II, the procedure was repeated with accessible grain, for comparison. In Experiment III, the procedures were repeated with explicitly unpaired presentations of keylight and either inaccessible or accessible grain. The results indicated that key pecking occurred as quickly in the presence of keylight pairings with inaccessible grain as with accessible grain, though (except for one bird) key pecking was not maintained with inaccessible grain. Furthermore, compared to the dim keylight, the bright keylight greatly suppressed key pecking when paired with inaccessible grain, and reduced the rate of key pecking when paired with accessible grain. Little key pecking occurred in groups exposed to explicitly unpaired presentations of keylight (whether bright or dim) and grain (whether accessible or inaccessible). When the birds in Experiment III were retested with explicitly paired presentations of keylight and grain, little key pecking was observed, suggesting suppressive effects of prior explicitly unpaired presentations. It is suggested that the effects of key-brightness manipulation were produced by the association of grain with cues other than the response key, or by distraction produced by partial illumination of the grain hopper.     

Factors influencing responding under multiple schedules of conditioned and unconditioned reinforcement

Two experiments examined pigeons' responses under multiple schedules of conditioned and unconditioned reinforcement. In one component, responses produced food according to a fixed-interval schedule; in a second component, responses produced brief stimuli according to a fixed-ratio schedule. When brief-stimulus presentations were paired with food in the first component, rates in the second component were usually higher than 10 responses per minute. When pairing in the first component was eliminated, responding continued to be maintained in the second component. Elimination of food presentation from the first component substantially decreased responding in the second component, even though the brief stimulus had not been paired with food. Experiment II demonstrated that response rate was affected by the duration of both the second component and the brief stimulus. The results suggest that three conditions are important in maintaining responding with brief-stimulus presentations: (1) pairing the brief stimulus, at least initially, with food, (2) maintaining unconditioned reinforcement in one component, and (3) employing optimal brief-stimulus and component durations.     

Contrast and induction in multiple schedules of discrete-trial concurrent reinforcement

Three pigeons were exposed to two-key discrete-trial concurrent schedules of reinforcement. Red and white key colors alternated irregularly and the assignment of reinforcers depended on key color. The red-key schedules were held constant, with the scheduled relative frequency of reinforcement for left-key pecks set at 0.75, while the white-key schedules varied. When the location of white-key reinforcement was changed from one side to the other, while its overall frequency was constant, red-key choices shifted in the same direction as white-key choices, an induction effect. When the overall frequency of white-key reinforcement was changed while its location remained constant, red key choices shifted in a direction opposite to white-key choices, a contrast effect. Both induction and contrast effects were clearer when the overall frequency of red-key reinforcement was reduced. These data demonstrate that the allocation of responding may exhibit schedule interaction effects similar to those commonly reported for response rate.     

Behavioral contrast as a function of the temporal location of reinforcement

Pigeons were trained on a multiple variable-interval variable-interval schedule of reinforcement. One component was then changed to a variation of a fixed-interval schedule in which the same rate of reinforcement was obtained as previously but the location of the reinforcer was fixed within the component. The effects of different temporal locations were compared. An increase in response rate for the unchanged variable-interval component (behavioral contrast) occurred when the reinforcer was located in the middle or at the end of the FI component, but response suppression occurred when it was located at the beginning of the component. The pattern of results cannot be explained by any previous theories of contrast. The overall response rates, and the pattern of local rates within the components, were consistent with the hypothesis that the major determinant of the contrast effect was the transition to a lower reinforcement rate following the unchanged component.     

Neurochemical changes correlated with behavior maintained under fixed-interval and fixed-ratio schedules of reinforcement.

Key pecking of 4 pigeons was maintained under a multiple 3-min fixed-interval, 30-response fixed-ratio schedule of food presentation. Only one schedule was in effect during an experimental session, and each was correlated with a different keylight stimulus and location (left vs. right). The different schedule components alternated across days or weeks. Cerebrospinal fluid was collected from chronically implanted intracerebroventricular cannulae following sessions with the different schedules, as well as following sessions in which reinforcement was withheld (extinction), when response-independent food was delivered, and when the experimental chamber was dark and there were no scheduled events. Metabolites of the neurotransmitters serotonin, norepinephrine, and dopamine were assayed in cerebrospinal fluid using high-performance liquid chromatography with electrochemical detection. Compared to the fixed-ratio condition, responding maintained under the fixed-interval schedule resulted in consistently higher levels of the serotonin metabolite 5-hydroxyindoleacetic acid and of the dopamine metabolite homovanillic acid in all pigeons. Levels of 3-methoxy-4-hydroxyphenylethylene glycol, a metabolite of norepinephrine, and dihydroxyphenylacetic acid, another dopamine metabolite, were also higher in 3 of the 4 pigeons following exposure to the fixed-interval schedules when compared to levels of these metabolites after exposure to the fixed-ratio schedule. Extinction of fixed-ratio responding resulted in large increases in 5-hydroxyindoleacetic acid compared to levels of this metabolite under the fixed-ratio schedule, whereas this serotonin metabolite decreased during extinction of responding under the fixed-interval schedule. Control procedures suggested that the neurochemical changes were not related to the rate of responding but were a function of the specific experimental conditions. Distinctive neurochemical changes that accompany schedule-controlled responding show the sensitivity of the neurochemical environment to behavioral contingencies and demonstrate further the profound impact that such contingencies have on biobehavioral processes.     

Operant conditioning of behavioral variability using a percentile reinforcement schedule.

The present investigation developed and tested a new percentile reinforcement schedule suited to study pattern variability, whose main feature was the relative dissociation it provided between the variability requirement defining criterional responses and overall probability of reinforcement. In a discrete-trials procedure, pigeons produced patterns of four pecks on two response keys. If the pattern emitted on the current trial differed from the N preceding patterns, reinforcement was delivered with probability mu. The schedule continuously adjusted the criterion N such that the probability of a criterional response, estimated from the subject's recent behavior, was always constant. In these circumstances, the criterion corresponded to an invariant percentile in the distribution of recent responses. Using a between-subjects design, Experiment 1 manipulated the variability requirement--the percentile--while keeping overall reinforcement probability constant. The degree of variability varied directly with the requirement. In addition, an inverse relationship existed between the requirement and within-group variance. Experiment 2 manipulated probability of reinforcement while maintaining the variability requirement constant. No consistent relationship was found between variability and reinforcement probability. A tentative hypothesis was advanced ascribing the operant conditioning of behavioral variability to a process of probability-dependent selection.     

Stimulus control of respondent and operant key pecking: A single key procedure

Pigeons' responses to a uniformly illuminated response key were either reinforced on a variable-interval one-minute schedule of reinforcement or extinguished for one-minute periods. When 1.5 second signals were presented at the beginning of each component, so as to differentially predict reinforcement, the pigeons pecked at the signals, at rates higher than rates during the remainder of the component. When the brief signals were not differentially predictive of reinforcement, pecking in their presence decreased to near zero levels. Similar results were obtained with signals based upon colors and upon line orientations. Changes in rates of (unreinforced) pecking occurred during the signal whether pigeons responded differentially during the remainder of the component or not. Experiment II demonstrated that the presence of the signal correlated with extinction was not necessary for pecking to develop at the signal which preceded the component in which responding was intermittently reinforced. The experiments demonstrated a clear dissociation of respondent control from operant control of a response. In addition, operant behavior was shown to be relatively insensitive to differing rates of reinforcement, as compared to the sensitivity of respondent behavior to differing rates of reinforcement produced by the very same operant behavior.     

Competition between stimulus-reinforcer contingencies and anticipatory contrast.

Procedures used to study anticipatory contrast are conceptually similar to those used to study autoshaping, in that two target stimuli signal either higher or lower rates of reinforcement in the following components of the schedule. Despite this signal contingency, anticipatory contrast entails response rates that are higher to the target stimulus followed by the lower rate of reinforcement. To determine the relation between such effects and autoshaping, different variations of the procedure were used in which the signal contingency was presented in the absence of reinforcement in the target components themselves and in which the reinforcement schedules in the different following components were signaled by the same stimulus. Autoshaping effects of this signal contingency were demonstrated when no reinforcement was available during the target-component signals themselves. Intermediate patterns of behavior occurred when reinforcement was available during the target-component signals and when their different following schedules were correlated with the same stimulus. Attempts to isolate these signal and contrast effects functionally by using the signal-key procedure were unsuccessful. The results demonstrate that Pavlovian stimulus contingencies are in competition with the dynamics of anticipatory contrast, thus reducing its occurrence under some circumstances.     

An equation for behavioral contrast.

Pigeons were trained on a three-component multiple schedule in which the rates of reinforcement in the various components were systematically varied. Response rates were described by an equation that posits that the response-strengthening effects of reinforcement are inversely related to the context of reinforcement in which it occurs, and that the context is calculated as the weighted average of the various sources of reinforcement in the situation. The quality of fits was comparable to that found with previous quantitative analyses of concurrent schedules, especially for relative response rates, with over 90% of the variance accounted for in every case. As with previous research, reinforcements in the component that was to follow received greater weights in determining the context than did reinforcements in the preceding component.     

Resistance to reinforcement change in multiple and concurrent schedules assessed in transition and at steady state

Behavioral momentum theory relates resistance to change of responding in a multiple-schedule component to the total reinforcement obtained in that component, regardless of how the reinforcers are produced. Four pigeons responded in a series of multiple-schedule conditions in which a variable-interval 40-s schedule arranged reinforcers for pecking in one component and a variable-interval 360-s schedule arranged them in the other. In addition, responses on a second key were reinforced according to variable-interval schedules that were equal in the two components. In different parts of the experiment, responding was disrupted by changing the rate of reinforcement on the second key or by delivering response-independent food during a blackout separating the two components. Consistent with momentum theory, responding on the first key in Part 1 changed more in the component with the lower reinforcement total when it was disrupted by changes in the rate of reinforcement on the second key. However, responding on the second key changed more in the component with the higher reinforcement total. In Parts 2 and 3, responding was disrupted with free food presented during intercomponent blackouts, with extinction (Part 2) or variable-interval 80-s reinforcement (Part 3) arranged on the second key. Here, resistance to change was greater for the component with greater overall reinforcement. Failures of momentum theory to predict short-term differences in resistance to change occurred with disruptors that caused greater change between steady states for the richer component. Consistency of effects across disruptors may yet be found if short-term effects of disruptors are assessed relative to the extent of change observed after prolonged exposure.