Testing the reinforcing properties of S-: a replication of Lieberman''s procedure

A critical issue in testing theories of observing is whether the stimulus associated with extinction (the S-) reinforces observing responses. In previous experiments, subjects have been trained to make observing responses that produce both the S- and the stimulus correlated with reinforcement (the S+). Then, either the S+ or the S- has been withheld. Conflicting results have been attributed to differences among species. In the present experiments, pecking one key by master pigeons was reinforced with grain on a variable-ratio extinction schedule. Yoked pigeons received the grain on a variable-interval, extinction schedule controlled by the variable-ratio performances of the master birds. For both groups, concurrent pecking on a second key was reinforced on a variable-interval schedule with displays of discriminative stimuli. Subsequently, either the S+ or the S- was eliminated from the procedure. Omission of S+ produced a large decrease, as predicted by traditional conditioned reinforcement accounts of observing. By itself, S- did not maintain observing. A smaller and less reliable decrease, comparable to that obtained by Lieberman (1972) with rhesus monkeys, occurred when S- was eliminated. This replication with pigeons of Lieberman's results indicates that they are not species-specific, and the fact that observing was not maintained by S- alone suggests that the decrease obtained when S- was omitted is not attributable to the reinforcing power of S-.     

Required pecking alters judgments of the passage of time by pigeons

There is evidence that how humans perceive time is affected by the activity in which they are engaged when they are judging time. In humans, typically, the more demanding the task, the faster time seems to pass. We asked whether a similar effect could be found in pigeons. Pigeons were trained to discriminate between a short-(2-sec) and a long-(10-sec) duration stimulus. Depending on the color of the stimulus (white or blue), the pigeons were required to peck (at least once per second or the trial was aborted) or to refrain from pecking (pecks aborted the trial). Once these tasks had been acquired to a high degree, probe trials involving white and blue stimuli were presented at durations between 2 and 10 sec. On trials in which the pigeons were required to peck, the point of subjective equality (i.e., the point at which pigeons are equally likely to choose the stimulus associated with long stimuli as the stimulus associated with short stimuli) was almost 1 sec longer than on trials in which the pigeons were required to refrain from pecking. In other words, on trials that required pecking, more time passed before the pigeons indicated that the probe duration was at the subjective midpoint between 2 and 10 sec than on trials that did not require pecking. This result suggests that like humans, the pigeons underestimated the passage of time when they were active or when attention to time-related cues had to be shared with attention to satisfying the response rate requirement.     

Differential autoshaping to common and distinctive elements of positive and negative discriminative stimuli

The learning by hungry pigeons of a discrimination between two successively presented compound visual stimuli was investigated using a two-key autoshaping procedure. Common and distinctive stimulus elements were simultaneously presented on separate keys and either followed by food delivery, S+, or not, S−. The subjects acquired both between-trial and within-trial discriminations. On S+ trials, pigeons pecked the distinctive stimulus more than the common stimulus; before responding ceased on S− trials, they pecked the common stimulus more than the distinctive one. Mastery of the within-display discrimination during S+ trials preceded mastery of the between-trials discrimination. These findings extend the Jenkins-Sainsbury analysis of discriminations based upon a single distinguishing feature to discriminations in which common and distinctive elements are associated with both the positive and negative discriminative stimuli. The similarity of these findings to other effects found in autoshaping—approach to signals that forecast reinforcement and withdrawal from signals that forecast nonreinforcement—is also discussed.     

The role of the response-reinforcer contingency in negative automaintenance

When a response key is briefly illuminated before a grain reinforcer is presented, key pecking is reliably developed and maintained in pigeons, even if pecking prevents reinforcement (negative automaintenance). This experiment demonstrated that pigeons are sensitive to a negative response-reinforcer contingency, even though it does not eliminate responding. Within individual pigeons, two kinds of trials were compared: red key trials, in which reinforcement was negatively contingent on responding, and white key trials, in which reinforcement was unrelated to responding. Reinforcement frequency in non-contingent trials was yoked to the obtained reinforcement frequency in negatively contingent trials. All eight pigeons pecked substantially more on the non-contingent key than on the negative key, and preferred the non-contingent key to the negative key on occasional “choice” trials where both were presented together. When the stimuli correlated with the two conditions were reversed, the pigeons' behavior also shifted. These response differences are taken as evidence that pigeons are sensitive to the negative response-reinforcer contingency.     

An investigation of peak shift and behavioral contrast for autoshaped and operant behavior.

Instrumental treadle press and nonreinforced key peck responses were monitored during discrimination training and generalization testing in pigeons on positive and negative reinforcement schedules. In Experiment 1, six pigeons pressed a treadle for food on a multiple variable-interval extinction schedule. In Experiment 2, three pigeons pressed a treadle to avoid shock on a multiple free-operant avoidance extinction schedule. Different color keylights signaled S+ and S- components. Some positive behavioral contrast occurred during discrimination training, but the effect was small. Pecking occurred to the S+ keylight in Experiment 1 but not in Experiment 2. On stimulus generalization tests, all subjects displayed a positive peak shift when pressing the treadle for food or to avoid shock. However, peak shift was not found for nonreinforced "autopecks" on the stimulus key, although an area shift was observed in Experiment 1. This is the first demonstration of peak shift for pigeons pressing treadles and the only reliable demonstration of peak shift when negative reinforcement maintained responding. These results, in combination with previous demonstrations of peak shift for rats pressing levers and pigeons pecking keys, indicate that peak shift is a general by-product of operant discrimination learning, since it occurs across a variety of the organisms, responses, and reinforcers.     

Effects of time between trials on rats' and pigeons' choices with probabilistic delayed reinforcers

Parallel experiments with rats and pigeons examined reasons for previous findings that in choices with probabilistic delayed reinforcers, rats' choices were affected by the time between trials whereas pigeons' choices were not. In both experiments, the animals chose between a standard alternative and an adjusting alternative. A choice of the standard alternative led to a short delay (1 s or 3 s), and then food might or might not be delivered. If food was not delivered, there was an "interlink interval," and then the animal was forced to continue to select the standard alternative until food was delivered. A choice of the adjusting alternative always led to food after a delay that was systematically increased and decreased over trials to estimate an indifference point--a delay at which the two alternatives were chosen about equally often. Under these conditions, the indifference points for both rats and pigeons increased as the interlink interval increased from 0 s to 20 s, indicating decreased preference for the probabilistic reinforcer with longer time between trials. The indifference points from both rats and pigeons were well described by the hyperbolic-decay model. In the last phase of each experiment, the animals were not forced to continue selecting the standard alternative if food was not delivered. Under these conditions, rats' choices were affected by the time between trials whereas pigeons' choices were not, replicating results of previous studies. The differences between the behavior of rats and pigeons appears to be the result of procedural details, not a fundamental difference in how these two species make choices with probabilistic delayed reinforcers.     

On conditioned reinforcing effects of negative discriminative stimuli

Observing responses by pigeons were studied during sessions in which a food key and an observing key were available continuously. A variable-interval schedule and extinction alternated randomly on the food key. In one condition, food-key pecking during extinction decreased reinforcement frequency during the next variable-interval component, and in the other condition such pecking did not affect reinforcement frequency. Observing responses either changed both keylight colors from white to green (S+) or to red (S−) depending on the condition on the food key, or the observing responses never produced the S+ but produced the S− when extinction was in effect on the food key. Observing responses that produced only S− were maintained only when food-key pecking during extinction decreased reinforcement frequency in the subsequent variable-interval component. The red light conformed to conventional definitions of a negative discriminative stimulus, rendering results counter to previous findings that production of S− alone does not maintain observing. Rather than offering support for an informational account of conditioned reinforcement, the results are discussed in terms of a molar analysis to account for how stimuli acquire response-maintaining properties.     

Effects of differences between stimuli, responses, and reinforcer rates on conditional discrimination performance

In a discrete-trial conditional discrimination procedure, 4 pigeons obtained food reinforcers by pecking a key with a short latency on trials signaled by one stimulus and by pecking the same key with a long latency on trials signaled by a second stimulus. The physical difference between the two stimuli and the temporal separation between the latency values required for reinforcement were varied factorially over four sets of conditions, and the ratio of reinforcer rates for short and long latencies was varied within each set of conditions. Stimulus discrimination varied directly with both stimulus and response differences and was unaffected by the reinforcer ratio. Sensitivity to reinforcement, estimated by generalized-matching-law fits to the data within each set of conditions, varied directly with the response difference but inversely with the stimulus difference arranged between sets of conditions. Because variations in stimulus differences, response differences, and reinforcer differences did not have equivalent effects, these findings question the functional equivalence of the three terms of the discriminated operant: antecedent stimuli, behavior, and consequences.     

The nature of control by spoken words over visual stimulus selection.

Eight retarded adolescents were trained to select one (a trained S+) of two visual stimuli in response to a spoken word (a trained word). Two different visual stimuli alternated randomly as the S-. To determine if the spoken work was merely a temporal discriminative stimulus for when to respond, or if it also specified which visual stimulus to select, the subjects were given intermittent presentations of untrained (novel) spoken words. All subjects consistently selected the trained S+ in response to the trained spoken word and selected the previous S- in response to the untrained spoken words. It was hypothesized that the subjects were responding away from the trained S+ in response to untrained spoken words, and control by untrained spoken words would not be observed when the trained S+ was not present. The two visual S- stimuli selected on trials of untrained spoken words were presented simultaneously. The untrained spoken words presented on these trials no longer controlled stimulus selections for seven subjects. The results supported the hypothesis that previous control by spoken words was due to responding away from the trained S+ in response to untrained spoken words.     

Stimulus generalization: the ordering and spacing of test stimuli

Twenty-four pigeons learned a successive discrimination between 500 mmu (S+) and 574 mmu (S-). When tested in extinction, some birds received stimuli around S+, with no S- presentations. These birds showed a positive peak shift, with maximum responding not at 550 mmu, but displaced to 538 mmu and 544 mmu. Other birds were tested with stimuli around S-, with no S+ presentations. These birds showed a negative shift, with least responding not at 574 mmu, but at 586 mmu. Though the first group was tested around S+ and the second around S-, total responding between groups did not differ. When retested on the other half of the continuum, however, birds that had gone from the S+ half to the S- half responded fewer times than those that had gone from the S- half to the S+ half. In a second experiment, reducing stimulus spacing from 6 mmu to 2 mmu produced flatter gradients and decreased the amount of positive shift. In a third experiment, birds were tested across the whole continuum with stimuli presented in serial order. A sequence from 538 mmu to 586 mmu produced no responding after the first part of the session; a sequence from 586 mmu to 538 mmu produced responding throughout the session.     

Categorization of natural movements by pigeons: visual concept discrimination and biological motion

In three experiments, pigeons were exposed to a discriminated autoshaping procedure in which categories of moving stimuli, presented on videotape, were differentially associated with reinforcement. All stimuli depicted pigeons making defined responses. In Experiment 1, one category consisted of several different scenes of pecking and the other consisted of scenes of walking, flying, head movements, or standing still. Four of the 4 birds for which pecking scenes were positive stimuli discriminated successfully, whereas only 1 of the 4 for which pecking was the negative category did so. In the pecking-positive group, there were differences between the pecking rates in the presence of the four negative actions, and these differences were consistent across subjects. In Experiment 2, only the categories of walking and pecking were used; some but not all birds learned this discrimination, whichever category was positive, and these birds showed some transfer to new stimuli in which the same movements were represented only by a small number of point lights (Johansson's “biological motion” displays). In Experiment 3, discriminations between pecking and walking movement categories using point-light displays were trained. Four of the 8 birds discriminated successfully, but transfer to fully detailed displays could not be demonstrated. Pseudoconcept control groups, in which scenes from the same categories of motion were used in both the positive and negative stimulus sets, were used in Experiments 1 and 3. None of the 8 pigeons trained under these conditions showed discriminative responding. The results suggest that pigeons can respond differentially to moving stimuli on the basis of movement cues alone.     

A procedure for generating differential "sample" responding without different exteroceptive stimuli

Sidman (1994, 2000) suggested that responses as well as stimuli can join equivalence classes, a hypothesis difficult to test because differential responding typically requires different stimuli. The present experiments describe a procedure with pigeons that avoids this potential confounding effect. In Experiment 1, spacing two responses 3 s apart (a differential-reinforcement-of-low-rate [DRL] schedule) to a white stimulus on some trials produced food or the comparison stimuli in a matching task, whereas pecking 10 or more times with no temporal restrictions (a fixed-ratio [FR] schedule) produced the same effect on other trials. Completing the alternative (unscheduled) requirement terminated the white stimulus and repeated the trial. Following such errors, pigeons learned to switch to the alternative response pattern on the repeat trials. In addition, the correct response pattern functioned as a conditional cue for comparison choice. In Experiment 2, mixed DRL-FR training was preceded by two-sample/two-alternative matching-to-sample with DRL and FR sample-response requirements. In a subsequent transfer test in which the correct response pattern to white served as the sample, pigeons preferentially chose the comparison previously reinforced following that pattern in the baseline task. This "unsignaled response" procedure may be useful for assessing whether differential responses can be members of acquired equivalence classes.     

Acquisition of the autoshaped key peck as a function of amount of preliminary magazine training

Three experiments evaluated the effect of magazine training on acquisition of the pigeon's key peck during autoshaping. In Experiment I, pigeons were exposed to two days of extended magazine training, followed on the third day by keylight-only presentations. All pigeons pecked the keylight early in the keylight-only session. Experiment II examined the relationship between the number of magazine-training trials and trials to the first peck. Pigeons were given either 0, 3, 10, or 25 magazine-training trials followed by the standard autoshaping procedure. The number of trials to the first peck was related to the number of magazine-training trials. In Experiment III, pigeons were exposed to the standard autoshaping procedure without prior magazine training. The data from Experiment III suggested that key pecking will occur only after the response of eating from the lighted hopper has occurred. Taken together, these results suggest that initial magazine training is an important variable in autoshaping. Key pecking is discussed as a generalized consummatory response.     

The relation between stimulus function and equivalence class formation

Fifty participants were exposed to a simple discrimination-training procedure during which six S+ functions were established for six arbitrary stimuli, and S- functions were established for a further six stimuli. Following this training, each participant was exposed to one of five conditions. In the S+ condition, participants were exposed to a stimulus equivalence training and testing procedure using only the six S+ stimuli as samples and comparisons. In the S+/S- condition, participants were exposed to the same training and testing sequence as in the S+ condition, the difference being that three S+ and three S- stimuli were used as sample and comparison stimuli, with each set of three corresponding to the trained equivalence relations. In the S+/S- mixed condition, the S+ and S- stimuli were assigned to their roles as samples and comparisons in a quasi-random order. In the S- condition, all six S- stimuli were used. The no-function condition served as a control condition and employed stimuli for which no stimulus-control functions had been established. The results showed that, on average, participants required more testing trials to form equivalence relations when the stimuli involved were functionally similar rather than functionally different. Moreover, participants required more test trials to form equivalence relations when novel arbitrary stimuli, rather than functionally distinct stimuli, were used as samples and comparisons. The speed of acquisition of stimulus equivalence was also related to the number of functionally similar stimuli established before training. These findings indicate a variety of ways in which the emergence of equivalence relations is affected by the functional classes in which the relevant stimuli participate.     

Resurgence of temporal patterns of responding

The resurgence of temporal patterns of key pecking by pigeons was investigated in two experiments. In Experiment 1, positively accelerated and linear patterns of responding were established on one key under a discrete-trial multiple fixed-interval variable-interval schedule. Subsequently, only responses on a second key produced reinforcers according to a variable-interval schedule. When reinforcement on the second key was discontinued, positively accelerated and linear response patterns resurged on the first key, in the presence of the stimuli previously correlated with the fixed- and variable-interval schedules, respectively. In Experiment 2, resurgence was assessed after temporal patterns were directly reinforced. Initially, responding was reinforced if it approximated an algorithm-defined temporal pattern during trials. Subsequently, reinforcement depended on pausing during trials and, when it was discontinued, resurgence of previously reinforced patterns occurred for each pigeon and for 2 of 3 pigeons during a replication. The results of both experiments demonstrate the resurgence of temporally organized responding and replicate and extend previous findings on resurgence of discrete responses and spatial response sequences.     

Acquisition and maintenance of autoshaped key pecking as a function of food stimulus and key stimulus similarity.

The results of a number of recent studies suggest that acquisitions of autoshaped key pecking in pigeons is affected by the similarity of the grain-hopper stimulus and response-key stimulus. In Experiment 1 this hypothesis was tested by training independent groups of pigeons to key peck under six different hopper-stimulus and key-stimulus similarity conditions, and three procedures containing either immediate reinforcement, variable delay of reinforcement, or omission of reinforcement for key pecking. Number of trials to acquisition was found to be related to the similarity variable. Maintained responding was affected by the response-reinforcer contingency. This effect was found both within and between subjects. Under two of the contingencies (automaintenance and omission), maintained responding continued to be affected by the similarity of the hopper stimulus and key stimulus. In Experiment 2 pigeons were given omission training with a hopper light on or off. Both acquisition and maintenance of key pecking were facilitated by the presence of the hopper light. The present findings suggest that much of the responding reported in various automatic shaping and training procedures may reflect the effects of key stimulus/food stimulus similarity.     

Magnitude and frequency of reinforcement and frequencies of interresponse times

The relative magnitude and relative frequency of reinforcement for two concurrent interresponse times (1.5 to 2.5 sec and 3.5 to 4.5 sec) were simultaneously varied in an experiment in which pigeons obtained grain by pecking on a single key. Visual discriminative stimuli accompanied the two time intervals in which reinforcements were arranged by a one-minute variable-interval schedule. The resulting interresponse times of each of three pigeons fell into two groups; "short" (1.0 to 2.5 sec) and "long" (3.0 to 4.5 sec). Steady-state relative frequencies of these interresponse times were orderly functions of both reinforcement variables. The combined effects of both independent variables were well summarized by a linear function of one variable, relative access to food. Unlike corresponding two-key concurrent variable-interval schedules, the present schedule did not produce an equality between the relative frequency of an operant and either the relative magnitude or the relative frequency of reinforcement of that operant. A tentative account is provided for this difference between one-key and two-key functions.     

Maintenance of key pecking by response-independent food presentation: the role of the modality of the signal for food

Three pigeons were exposed to a series of procedures in which periods of response-independent food presentation, on a variable-time schedule, alternated with periods in which food was never presented. The stimuli that signalled periods of food availability or non-availability varied from one procedure to the next, and were sometimes key colors, sometimes tones, and sometimes compounds of both. Key pecking was initiated and maintained when key color was a signal for food; key pecking was not initiated when a tone was the signal for food. However, control of key pecking that was already established could be transferred from key color to tone, and subsequently, initiated by the tone. It is suggested that for pigeons, pre-experimental relationships exist among food, visual stimuli, and pecking, and that a similar relationship, which includes auditory stimuli, must be induced in the laboratory.     

Attention in the pigeon: testing for excitatory and inhibitory control by the weak elements.

In two experiments, pigeons were trained on a successive discrimination between a color and either a compound S+ or a compound S- consisting of a form superimposed on a second color. Two stimulus control tests followed discrimination training: an attention test in which the form and colors used in training were presented singly and in combination, and then a resistance-to-reinforcement test using the form element of S+ or S- and a novel form. In the attention test, the birds trained with a compound S+ responded most to the S+ compound, less to the S+ color alone, and still less to the S+ form on a dark key. Few responses were made to the negative stimulus, either alone or with the S+ form added. The birds trained with a compound S- pecked most at the S+ color and to a compound of the S+ color with the S- form added. The resistance-to-reinforcement test showed that the birds trained with a compound S+ responded more to the S+ form than to a novel form. However, the birds trained with a compound S- did not reliably respond more to a novel form than to the S- form. These findings suggested that the form element of a compound S+ gains some excitatory control, but the form element of a compound S- does not acquire inhibitory control. The possibility existed that low levels of responding to the S+ form on a dark background in the first experiment were due to use of a darkened key to separate S+ and S- periods during discrimination training. However, the essential findings were the same in a second experiment in which darkening of the chamber separated S+ and S- periods.     

Effect of delay-interval stimuli on delayed symbolic matching to sample in the pigeon

In Experiment 1, food-deprived pigeons received delayed symbolic matching to sample training in a darkened Skinner box. Trials began with the illumination of the grain feeder lamp (no food sample), or illumination of this lamp, accompanied by the raising of the feeder tray (food sample). After a delay of a few seconds, the two side response keys were illuminated, one with red and one with green light, with positions counterbalanced over trials. Pecking the red (green) comparison produced grain reinforcement if the trial had started with food (no food); pecking red after a no-food sample or green after a food sample was not reinforced. Once matching performance was stable, four stimuli were presented during the delay interval, and their effects on matching accuracy were evaluated. Both illumination of the houselight and the center key with white geometric forms decreased matching accuracy, whereas presentation of a tone and vibration of the test chamber did not. In Experiment 2, pecking the red center key was reinforced with food according to a variable interval schedule. The effects of occasional brief presentations of the four stimuli used in the first experiment on ongoing pecking were assessed. The houselight and form disturbed key pecking, but the tone and vibration did not. Thus, stimuli that interfered with delayed matching also interfered with simple operant behavior. Implications of these results for theories of remembering are discussed.