Determinants of contrast in the signal-key procedure: Evidence against additivity theory

Two experiments are reported that challenge the interpretation of previous results with the signal-key procedure, in which the discriminative stimuli are located on a response key different from the key associated with the operant response requirement. Experiment 1 replicated the procedure of Keller (1974), and found that contrast effects on the operant key occurred reliably for only one of four subjects. High rates to the signal key initially occurred for only one subject, but modifications of the procedure produced substantial rates to the signal key for all subjects. In all cases, however, signal-key behavior was greatly reduced by the addition of a changeover delay which prevented reinforcement within 2 seconds of the last peck to the signal key, suggesting that signal-key pecking was maintained primarily by adventitious reinforcement. Experiment 2 modified the signal-key procedure by using three response keys, so that the discriminative stimuli on the signal key controlled different responses during all phases of training. With this modification, reliable contrast effects on the operant key occurred for all subjects, suggesting that the failure to find contrast in previous studies has been due to the confounding of changes in the discrimination requirements with changes in relative rate of reinforcement. The results challenge the additivity theory of contrast, and suggest that “elicited” behavior plays a minor role, if any, in the determination of contrast effects in multiple schedules.     

Stimulus-reinforcer contingencies and local behavioral contrast

Four pigeons were exposed to a series of multiple schedules of variable-interval reinforcement in which pecks were required on one key (operant key) and components were signalled on a second key (signal key). Four additional pigeons experienced identical conditions, except that a yoking procedure delivered food on variable-time schedules, with no key pecks required. One of the components of the multiple schedule was constant throughout the experiment as a variable-interval (or variable-time) 30-second schedule. Operant-key responding during the constant component was uniform throughout the component, uninfluenced by changes in the duration of the variable component, and only slightly influenced by changes in reinforcement frequency correlated with the variable component. By comparison, signal-key response rate during the constant component was highest at the onset of the component, was higher when the variable component was 60-sec long than when it was 1-sec long, and was higher when no reinforcement occurred in the variable component than when reinforcement was scheduled in the variable component. These characteristics of signal-key pecking matched characteristics of local positive behavioral contrast. These data are taken to support the “additivity theory” of behavioral contrast and to suggest that Pavlovian stimulus-reinforcer relations contribute primarily to the phenomenon of local positive contrast.     

Discrimination learning by macaca mulatta with option to switch between S+ and S−

Three rhesus monkeys were trained to press either of two response keys. A response on the reinforcement key during presentation of the reinforced stimulus produced a sucrose pellet followed by an intertrial interval, but during presentation of the unreinforced stimulus produced only the intertrial interval. A response on the switching key changed the discriminative stimulus from reinforced to unreinforced or from unreinforced to reinforced. The reinforced stimulus was presented automatically on half the trials, but could be produced only by a switching response on the other half. Switching tended to occur in three distinct stages during acquisition of discriminative behavior. The first stage was identified as "nondiscriminative switching"; the second as "nonswitching"; and the third as "discriminative switching".     

Responding maintained by the opportunity to attack during an interval food reinforcement schedule

Pigeons responded in a two-key situation. Responses on the right key (food key) were reinforced with food presentation on a response-initiated fixed-interval schedule, (i.e., first response after a fixed period of time was reinforced); responses on the left key (target key) were reinforced on a fixed-ratio schedule (i.e., every nth response was reinforced) with the presentation of a target bird that could be attacked. When the interval value of the food reinforcement schedule was varied from 1 min to 5 min, both the rate of attack responding on the target bird and the rate of responding on the target key were a function of the interval value. Responding on the target key was not maintained by the stimulus change associated with target availability, and was successively extinguished and reconditioned by removing and returning the target bird to the restraining box. When food was delivered independently of behavior, responding on the target key either remained unchanged or decreased, but was not eliminated. Responding on the target key was not maintained in the absence of an intermittent schedule of food presentation.     

Two-key concurrent paced variable-interval paced variable-interval schedules of reinforcement

Nine pigeons were used in two experiments in which a response was reinforced if a variable-interval schedule had assigned a reinforcement and if the response terminated an interresponse time within a certain interval, or class, of interresponse times. One such class was scheduled on one key, and a second class was scheduled on a second key. The procedure was, therefore, a two-key concurrent paced variable-interval paced variable-interval schedule. In Exp. I, the lengths of the two reinforced interresponse times were varied. The relative frequency of responding on a key approximately equalled the relative reciprocal of the length of the interresponse time reinforced on that key. In Exp. II, the relative frequency and relative magnitude of reinforcement were varied. The relative frequency of responding on the key for which the shorter interresponse time was reinforced was a monotonically increasing, negatively accelerated function of the relative frequency of reinforcement on that key. The relative frequency of responding depended on the relative magnitude of reinforcement in approximately the same way as it depended on the relative frequency of reinforcement. The relative frequency of responding on the key for which the shorter interresponse time was reinforced depended on the lengths of the two reinforced interresponse times and on the relative frequency and relative magnitude of reinforcement in the same way as the relative frequency of the shorter interresponse time depended on these variables in previous one-key concurrent schedules of reinforcement for two interresponse times.     

Reciprocal response interactions in concurrent variable-interval and discrete-trial fixed-ratio schedules

Abstract.— Pecking a red key by pigeons was reinforced with grain on a continuously accessible variable-interval schedule. Pecking a second key was reinforced on a discrete-trial fixed-ratio schedule; occasionally the second key was illuminated green and after a single run on the fixed-ratio schedule a reinforcer was presented and the green light was turned off. The experiment investigated the effects of acquisition, extinction, and re-acquisition of pecking the second key. All pigeons changed over immediately from pecking the red key to pecking the green key whenever the green light controlled a high rate of pecking this key. Pecking the red key was completely suppressed during pecking the green key. The experiment shows that a changeover from one response to a second response can come under discriminative control of a stimulus during which the second response is intermittently reinforced. All pigeons frequently emitted observing and orienting behaviors towards the dark key that was occasionally lit green.     

Frustration, preference and behavioural contrast

Pigeons' pecks on both a red (left-hand) and a striped (right-hand) response key were reinforced on a concurrent variable-interval 2-min. schedule until the proportion of responses given to each key had stabilized. In alternate sessions, the right-hand key was covered, while responding to a green stimulus on the left-hand key was reinforced on variable-interval 1-min. When responding to green was later extinguished, more responses were made to the striped key in reinforcement sessions, although the rate of responding to the other, red key increased. Replacing extinction during green by reinforcement returned the preference and the response rates to their previous levels. These results are compared with a previous experiment in which the striped key was not present, where a similar increase in response rate to red was observed after extinction on green. The shift in preference coupled with the usual contrast effect in the present experiment supports an interpretation of behavioural contrast in terms of the frustrative effects of extinction.     

Resurgence of temporal patterns of responding

The resurgence of temporal patterns of key pecking by pigeons was investigated in two experiments. In Experiment 1, positively accelerated and linear patterns of responding were established on one key under a discrete-trial multiple fixed-interval variable-interval schedule. Subsequently, only responses on a second key produced reinforcers according to a variable-interval schedule. When reinforcement on the second key was discontinued, positively accelerated and linear response patterns resurged on the first key, in the presence of the stimuli previously correlated with the fixed- and variable-interval schedules, respectively. In Experiment 2, resurgence was assessed after temporal patterns were directly reinforced. Initially, responding was reinforced if it approximated an algorithm-defined temporal pattern during trials. Subsequently, reinforcement depended on pausing during trials and, when it was discontinued, resurgence of previously reinforced patterns occurred for each pigeon and for 2 of 3 pigeons during a replication. The results of both experiments demonstrate the resurgence of temporally organized responding and replicate and extend previous findings on resurgence of discrete responses and spatial response sequences.     

Control of responding by location of auditory stimuli: adjacency of sound and response.

Four rhesus monkeys were trained to respond on one key when a one-second noise burst was presented through one speaker and to respond on a second key when the noise burst was presented through a second speaker. The acquisition of stimulus control was studied under three conditions, in each of which the relationship between the sound source and the response-key positions varied: an adjacent condition in which the noise burst was presented through the key and a response on this key was reinforced; a reversed-adjacent condition in which the noise burst was presented through one key and responding on the other key was reinforced: and a nonadiacent condition in which responding on the key nearer the sound was reinforced. Under adjacent conditions, stimulus control developed within one or two sessions. Under reversed and nonadjacent conditions, 10 sessions were required for the development of control. The asymptote of correct responding was the same under each condition in all animals.     

Note on changes in response latency following discrimination training in the monkey

Two monkeys were trained to press and hold down a telegraph key in the presence of a red light. Subsequent release of the key in response to a white cross superimposed on the red background was followed by reinforcement. Key release in response to a white circle on the red background was never reinforced. Latencies for the key release response to the reinforced stimulus (cross) were considerably shorter and less variable than those to the unreinforced stimulus (circle).     

Control of responding by location of auditory stimuli: role of differential and non-differential reinforcement

Sound was presented to monkeys through one of two loudspeakers, each adjacent to a response key. A response on the key adjacent to the sound source was reinforced (correct response). A response on the other key produced a timeout (incorrect response). Under these conditions, over 90% of responses were correct within one or two sessions. When the procedure was changed so that a response on either key was reinforced independently of which speaker was sounding, similar control by location developed within one or two sessions. When conditions were modified by moving the keys away from the immediate vicinity of the speakers, the animals required about 20 sessions to reach a stable level of greater than 90% correct responses under differential reinforcement conditions. No control by location developed under nondifferential reinforcement conditions.     

Aggression as a reinforcer: operant behavior in the mouse-killing rat

Two experiments examined mouse killing as a reinforcer of key pressing by rats that killed mice. In Experiment I, mouse-killing rats performed the key-pressing response when each press was reinforced with presentation of a mouse. Offered a choice between a key that yielded presentation of mice and one that did not, the rats preferred the key that yielded mice. When the contingency was reversed, the rats preferred the other key and continued to kill mice. In Experiment II, mouse-killing rats that did not kill rat pups performed a key-pressing response reinforced with presentation of mice on a variable-interval schedule. In tests for responding reinforced on that schedule with presentation of normal mice, anesthetized mice, dead mice, or rat pups, these rats that killed mice but not rat pups exhibited a decline in response rate when rat pups were the reinforcer. Altering the condition of the mice did not significantly affect performance.     

Choice with a fixed requirement for food, and the generality of the matching relation

Pigeons were trained on choice procedures in which responses on each of two keys were reinforced probabilistically, but only after a schedule requirement had been met. Under one arrangement, a fixed-interval choice procedure was used in which responses were not reinforced until the interval was over; then a response on one key would be reinforced, with the effective key changing irregularly from interval to interval. Under a second, fixed-ratio choice procedure, responses on either key counted towards completion of the ratio and then, once the ratio had been completed, a response on the probabilistically selected key would produce food. In one experiment, the schedule requirements were varied for both fixed-interval and fixed-ratio schedules. In the second experiment, relative reinforcement rate was varied. And in a third experiment, the duration of an intertrial interval separating choices was varied. The results for 11 pigeons across all three experiments indicate that there were often large deviations between relative response rates and relative reinforcement rates. Overall performance measures were characterized by a great deal of variability across conditions. More detailed measures of choice across the schedule requirement were also quite variable across conditions. In spite of this variability, performance was consistent across conditions in its efficiency of producing food. The absence of matching of behavior allocation to reinforcement rate indicates an important difference between the present procedures and other choice procedures; that difference raises questions about the specific conditions that lead to matching as an outcome.     

Peak shift in concurrent schedules

Pigeons were exposed to two keys, a main key and a changeover key. Initially non-differential training was given in which pecking the main key was reinforced on a variable-interval 2-min schedule when the key displayed the first stimulus, a black line on a blue background, and was reinforced on an identical but independent variable-interval 2-min schedule when the key displayed a plain blue stimulus. Later, differential training was given in which pecking the main key was reinforced on a variable-interval 2-min schedule when the first stimulus was displayed; and was reinforced on a variable-interval 10-min schedule when a second stimulus, a black line of another orientation on a blue background, was displayed. During non-differential and differential training, each peck on the changeover key changed the stimulus on the main key. Generalization tests were given before and after the differential training. These consisted of presentations on the main key of seven orientations of the black line on the blue background, including the first and second stimuli, with no reinforcements being given. Changeover-key pecks changed the stimuli on the main key. Generalization gradients were obtained using three measures: time spent, responses, and response rate in the presence of each test stimulus. Typically, maximum values on these measures occurred to stimuli away from the first in a direction opposite the second stimulus, and minimum values occurred to stimuli away from the second in a direction opposite the first.     

Pigeons presented with sequences of light flashes use behavior to count but not to time

On randomly ordered trials, pigeons were presented with either a blue or a white key that flashed red for 200 ms at a fast (2 flashes/s), medium (1 flash/s), or slow (0.5 flashes/s) rate. The blue key signaled a fixed-interval (FI) schedule in which the first response after 20 s was reinforced, and the white key signaled a fixed-number (FN) schedule in which the first response after 20 flashes was reinforced. In Experiments 1 and 2, pigeons showed depressed responding to the flash on FI-cued trials and accelerated responding to the flash on FN-cued trials. When the response key was periodically blacked out in Experiments 3 and 4, counting but not timing was eliminated.     

Concurrent performances: inhibition of one response by reinforcement of another

In an analysis of interactions between concurrent performances, variable-interval reinforcement was scheduled, in various sequences, for both keys, for only one key, or for neither key of a two-key pigeon chamber. With changeover delays of 0.5 or 1.0 sec, and with each key's reinforcements discriminated on the basis of key-correlated feeder stimuli, reinforcement of pecks on one key reduced the pecking maintained by reinforcement on the other key. The decrease in pecking early after reinforcement was discontinued on one key was not substantially affected by whether pecks on the other key were reinforced, but after reinforcement was discontinued on both keys, reinstatement of reinforcement for one key sometimes produced transient increases in pecking on the other key. Correlating the availability of right-key reinforcements with a stimulus, which maintained right-key reinforcement while reducing right-key pecking to negligible levels, demonstrated that these interactions depended on concurrent reinforcement, not concurrent responding. Thus, reinforcement of a response, but not necessarily the occurrence of the response, inhibits other reinforced responses. Compared with accounts in terms of excitatory effects of extinction, often invoked in treatments of behavioral contrast, this inhibitory account has the advantage of dealing only with observed dimensions of behavior.     

Choice between response rates

Three pigeons were required to peck a single key at a higher and a lower rate, corresponding to two classes of shorter and longer concurrently reinforced interresponse times. Food reinforcers arranged by a single variable-interval schedule were randomly allocated to the two reinforced interresponse times. The absolute durations of reinforced interresponse times were varied while the total reinforcements per hour was held constant and the relative duration, i.e., the relative reciprocal, of the shorter reinforcer class was held constant at 0.70. Preference for the higher rate of responding, as measured by the relative frequency of responses terminating interresponse times in the shorter reinforced class, depended on the absolute reinforced response rates. Preference for the higher reinforced rate increased from a level of near-indifference (0.50) at high reinforced response rates, through the matching level (0.70) at intermediate reinforced response rates, to a virtually exclusive preference (>0.90) at low reinforced response rates. These results resemble corresponding preference functions obtained with two-key concurrent-chains schedules and thereby provide another sense in which it may be said that interresponse-time distributions from interval schedules estimate preference functions for the component response rates corresponding to different classes of reinforced interresponse times.     

THE LOCAL ORGANIZATION OF BEHAVIOR: DISCRIMINATION OF AND MEMORY FOR SIMPLE BEHAVIORAL PATTERNS

A procedure was developed to enable nonverbal organisms to report what they remember of the temporal organization of their recent behavior. A baseline behavior with known temporal structure was established by a concurrent variable-interval variable-interval schedule for two temporal patterns of behavior (two different classes of reinforced interresponse times). The five pigeon subjects emitted these two temporal patterns on a center key and were occasionally given a short-term memory probe for their most-recently-emitted pattern. The probes consisted of symbolic delayed matching-to-sample tests, in which a response on a green side key was reinforced if the most recent pattern belonged to the shorter reinforced class, and a response to a red side key was reinforced if the most recent pattern belonged to the longer reinforced class. All subjects could report with over ninety percent accuracy what their most recently emitted behavioral pattern was when a retention interval separating the pattern from the memory probe was only .1 seconds. The retention interval was then manipulated, and it was found that recall for a pattern was frequently above chance after a delay of as much as eight seconds. Thus, pigeons can remember their most recent interresponse time not only right after it is emitted, but for several seconds thereafter. In other conditions, the patterns themselves were manipulated. It was found that as the patterns became more similar, discrimination became poorer. These results agree with the view that reinforcement tends to organize and integrate the local structure of behavior to the extent to which that structure is remembered.     

ALTERNATIVE REINFORCEMENT INCREASES RESISTANCE TO CHANGE: PAVLOVIAN OR OPERANT CONTINGENCIES?

Two multiple-schedule experiments with pigeons examined the effect of adding food reinforcement from an alternative source on the resistance of the reinforced response (target response) to the decremental effects of satiation and extinction. In Experiment 1, key pecks were reinforced by food in two components according to variable-interval schedules and, in some conditions, food was delivered according to variable-time schedules in one of the components. The rate of key pecking in a component was negatively related to the proportion of reinforcers from the alternative (variable-time) source. Resistance to satiation and extinction, in contrast, was positively related to the overall rate of reinforcement in the component. Experiment 2 was conceptually similar except that the alternative reinforcers were contingent on a specific concurrent response. Again, the rate of the target response varied as a function of its relative reinforcement, but its resistance to satiation and extinction varied directly with the overall rate of reinforcement in the component stimulus regardless of its relative reinforcement. Together the results of the two experiments suggest that the relative reinforcement of a response (the operant contingency) determines its rate, whereas the stimulus-reinforcement contingency (a Pavlovian contingency) determines its resistance to change.     

Uninstructed human responding: sensitivity to ratio and interval contingencies

College students' presses on a telegraph key were occasionally reinforced by light onsets in the presence of which button presses (consummatory responses) produced points later exchangeable for money. One student's key presses were reinforced according to a variable-ratio schedule; key presses of another student in a separate room were reinforced according to a variable-interval schedule yoked to the interreinforcement intervals produced by the first student. Instructions described the operation of the reinforcement button, but did not mention the telegraph key; instead, key pressing was established by shaping. Performances were comparable to those of infrahuman organisms: variable-ratio key-pressing rates were higher than yoked variable-interval rates. With some yoked pairs, schedule effects occurred so rapidly that rate reversals produced by schedule reversals were demonstrable within one session. But sensitivity to these contingencies was not reliably obtained with other pairs for whom an experimenter demonstrated key pressing or for whom the reinforcer included automatic point deliveries instead of points produced by button presses. A second experiment with uninstructed responding demonstrated sensitivity to fixed-interval contingencies. These findings clarify prior failures to demonstrate human sensitivity to schedule contingencies: human responding is maximally sensitive to these contingencies when instructions are minimized and the reinforcer requires a consummatory response.