Effects of a brief stimulus accompanying reinforcement on instrumental responding in pigeons

The responding of pigeons on a variable interval schedule of reinforcement was investigated in four experiments. In some conditions in each experiment reinforced keypecks were accompanied by a brief (0.5-sec) flash of the houselight. This procedure resulted in a low rate of response in comparison with that found in conditions when response-contingent light flashes occurred uncorrelated with reinforcement (Experiments 1 and 2) or when no light flash was presented (Experiment 3). Experiment 4 allowed a comparison between the effects of a signal accompanying the reinforced response and one accompanying the delivery of “free” food. Signaling the delivery of earned food produced a lower rate of response than did signaling the delivery of free food. The role of stimulus-reinforcer and response-reinforcer associations in producing these effects is discussed.     

Reinforcement of behavioral patterns: shaping a scallop

Temporal patterns of key pecking by pigeons were shaped by a schedule in which the delivery of food was contingent upon a measure of the overall extent to which the temporal pattern of behavior within a 5-sec trial conformed to a required pattern. This pattern approximated a constant rate of change in the rate of key pecking throughout the 5-sec trial. In comparison with behavior maintained by a classical fixed-interval 5-sec schedule, the new schedule controlled a better approximation to a "scallop" within individual trials and greatly reduced intersubject variability. These results are consistent with the view that the delivery of a reinforcer after a behavioral pattern a few seconds in duration may strengthen the entire pattern as a unit, or operant. The response topography contiguous with reinforcement may be a negligible fraction of the strengthened operant. One implication of this view is that mean response rate for such brief responses as key pecks and lever presses is a byproduct of whatever patterns are strengthened, and generally will not reveal fundamental controlling relationships, whenever a reinforcer is not contiguous with all the behavior on which it is contingent.     

Aversive aspects of a schedule of positive reinforcement

Six male White Carneaux pigeons were trained to peck at one of two keys to obtain food on several fixed-ratio schedules of reinforcement. Concurrently, the first response on a second key could, I—change the conditions of visual stimulation and remove the food reinforcement contingency, II—change the conditions of stimulation and have no effect upon the reinforcement contingency, or III—do nothing. The second response on the stimulus change key always restored baseline conditions. When second-key responses produced a stimulus change, the number of such responses was a function of the ratio value on the first key. Typically, second-key responses occurred before the start of fixed-ratio runs. The duration of stimulus change periods was an exponential function of the number of responses required for reinforcement when the possibility for reinforcement was not disturbed by periods of stimulus change (Condition II).     

Punishment of an extinguishing shock-avoidance response by time-out from positive reinforcement

Two experiments were conducted to determine the effects of punishment by time-out from positive reinforcement on the extinction of discriminated shock-avoidance responding. Subjects were trained initially to bar press for food on an intermittent schedule of reinforcement and, concurrently, to avoid shock at the onset of a warning signal. Experiment I compared avoidance extinction performance under no punishment and when avoidance responding resulted in a 30-sec TO from reinforced appetitive responding. In Exp II, the contingent use of TO punishment was compared with its random, or noncontingent use. The results of both experiments showed that in the absence of punishment, avoidance extinction was characterized by short latencies and nearly 100% avoidance responding. Avoidance responding in extinction was little affected by noncontingent TO punishment. When TO was made contingent upon avoidance responding, however, avoidance latencies immediately increased and the frequency of avoidance responses subsequently decreased to zero.     

The effects of a signal for free or for earned reward: Implications for the role of response-reinforcer associations in instrumental performance

Pearce and Hall (1978) investigated the effects of making a brief flash of light contingent upon response in rats lever-pressing for food on a variable-interval (VI) schedule. When this signal occurred in conjunction only with reinforced responses the response rate was lowered with respect to a condition in which an equal number of light flashes occurred uncorrelated with reinforcement. The experiments reported here compared these effects with those produced by signalling “free” food deliveries in a similar way. Experiments I and II compared the effects of presenting correlated and uncorrelated schedules of light and food to rats given no opportunity to lever-press. The different schedules did not produce differences in response rate when the levers were made available. In Experiment III, free food was delivered to rats responding on a VI schedule. Signalling the delivery of earned food pellets produced a low response rate in comparison with a condition in which the free pellets were signalled. It is concluded that signalling food delivery is effective only when the rat must respond to earn the food and it is argued that the signal has its effect by overshadowing a response-reinforcer association.     

Concurrent schedules of primary and conditioned reinforcement in rats

Rats responded on a fixed-interval schedule during which a 3-sec stimulus preceded each water reinforcement. The stimulus was then scheduled concurrently for responses on the same lever according to either a variable ratio. Although water reinforcement continued on a fixed-interval schedule, the pattern of responding became typical of a variable-interval or variable-ratio schedule. When the 3-sec stimulus was presented on a variable-interval or variable-ratio schedule, but was omitted on the fixed-interval schedule, the response rate decreased. When the stimulus occurred after the same time periods as those of the variable-interval schedule, but at least 7-sec after the last response, the rate decreased. The rate became higher when the fixed-interval schedule was discontinued and each presentation of the 3-sec stimulus was followed by water on a variable-interval schedule. When both water and the 3-sec stimulus were discontinued for a period of time, resulting in extinction of the lever response, and the 3-sec stimulus alone then presented on a variable-interval or variable-ratio schedule after lever responses, rate increased and then gradually decreased.     

Resistance to change and relapse of observing

Four experiments examined relapse of extinguished observing behavior of pigeons using a two-component multiple schedule of observing-response procedures. In both components, unsignaled periods of variable-interval (VI) food reinforcement alternated with extinction and observing responses produced stimuli associated with the availability of the VI schedule (i.e., S+). The components differed in the rate of food arranged (Rich = VI 30 s; Lean = VI 120 s). In Experiment 1, following baseline training, extinction of observing involved removal of both food and S+ deliveries, and reinstatement was examined by presenting either response-independent food or S+ deliveries. In Experiment 2, extinction involved removal of only food deliveries while observing responses continued to produce S+. Reinstatement was examined by delivering food contingent upon the first two food-key responses occurring in the presence of the S+. Experiment 3 assessed ABA renewal of observing by extinguishing food-key and observing responses in the presence of one contextual stimulus (i.e., B) and then returning to the original training context (i.e., A) during continued extinction. Experiment 4 examined resurgence by introducing food reinforcement for an alternative response during extinction, and subsequently removing that alternative source of food. Across experiments, relative resistance to extinction and relapse of observing tended to be greater in the component previously associated with the higher rate of primary reinforcement. Relapse of observing or attending to stimuli associated with primary reinforcement appears to be impacted by frequency of primary reinforcement in a manner similar to responding maintained directly by primary reinforcement.     

Persistent behavior maintained by unavoidable shocks

Squirrel monkeys were trained on a multiple schedule in which 10-min periods on a continuous shock avoidance schedule, indicated by a yellow light, alternated with 10-min periods on a 1.5-min variable interval schedule of food reinforcement (VI 1.5). A white light indicated that VI 1.5 was in effect, except for the middle 2 min of the period on VI 1.5, in which a blue light appeared and terminated with the delivery of a 0.5-sec unavoidable shock. Stable response rates developed in the avoidance and VI 1.5 components. However, the highest response rates occurred in the blue, preshock stimulus. A series of experiments showed that responding in the blue stimulus persisted even when responding had been extinguished on both the VI schedule of food reinforcement and the shock avoidance schedule. Responding in the blue stimulus ceased when the blue stimulus terminated without shock or when it terminated with a response-contingent shock. Each time responding ceased, it was restored by terminating the blue stimulus with an unavoidable shock. When the blue stimulus was on throughout each session and unavoidable shocks were delivered at regular 10-min intervals, responding was well maintained. These results show that in monkeys that have been trained on a continuous avoidance schedule, unavoidable shocks can maintain responding even under conditions where responses have no programmed consequences.     

The effects upon food-reinforced pecking and treadle-pressing of auditory and visual signals for response-independent food

Seven pigeons whose key-pecking was maintained by food reinforcement on a differential-reinforcement-of-low-rates 12-sec limited-hold 4-sec schedule and 12 other pigeons whose treadle-pressing was maintained by the same schedule received appetitive Pavlovian conditioning trials superimposed upon the instrumental baseline. Half the birds in each group received a tone as the CS, and the other half received a stimulus change on the key. Each CS was 20 sec long, and was immediately followed by 10-sec access to grain. The visual CS markedly facilitated the rate of pecking on the key for the birds whose baseline response was pecking. The visual CS produced auto-shaping of the key-peck and tended to produce suppression of treadle-pressing for the birds whose baseline response was treadle-pressing. The auditory CS produced inconsistent effects across birds regardless of the baseline response. In all cases the conditioned effects extinguished when response-independent food was omitted.     

Signal duration and suppression of operant responding by free reinforcement

Pigeons were trained on a VI (variable interval) schedule of food presentation with a superimposed schedule of response-independent food. Substantial suppression of the operant response rate occurred when the free food was presented without a signal. When the free food was preceded by a short (4 sec) signal, the degree of suppression was similar to that with unsignaled free food. But when the signal was lengthened to 12 sec, the degree of suppression was substantially reduced. Experiment 3 assessed the effect of signal duration using a baseline schedule of delayed reinforcement, in which contingent reinforcers were themselves preceded by a signal. The signal preceding the free reinforcers was then either the same as or different from this contingent signal. Signal duration effects occurred only when the two types of signals were different. These differences as a function of signal duration have implications for both “context-blocking” and “comparator” interpretations of the effects of noncontingent reinforcement in both Pavlovian and operant procedures.     

Effects of signaled and unsignaled shock on schedule-controlled lever pressing and schedule-induced licking: Shock intensity and body weight

Schedule-controlled lever pressing and schedule-induced licking were studied in rats under a multiple fixed-interval fixed-interval schedule of food reinforcement. Following acquisition of stable rates of pressing and licking, a multiple variable-time variable-time schedule of electric-shock delivery was superimposed upon the baseline schedule. In only one component of the multiple schedule, a 5-sec stimulus preceded each shock (signaled shock). In the other component shock was unsignaled. Several shock intensities (Experiment 1) and body weights (Experiment 2) were studied. Lever pressing and licking were affected similarly by experimental manipulations, although with parametric differences. Depending upon shock intensity and body weight, rates of lever pressing and licking were hardly suppressed, suppressed primarily in the unsignaled shock component (differential suppression), or markedly suppressed in both components. Differential suppression during components with signaled and unsignaled shock and conditioned suppression of responding during the preshock stimulus appeared not to be functionally related. Differential suppression depended more on the discriminability of shock-free time, and on shock intensity, body weight, and the type of response than on the “preparatory” behavior preceding shock.     

Comparisons between variable-interval and fixed-interval schedules of electric shock delivery

Responding maintained in rats under a variable-interval 35-sec food schedule was suppressed more by 60-sec and 240-sec fixed-interval schedules of shock delivery than by 60-sec and 240-sec variable-interval schedules of shock delivery. When the delivery of shock was preceded by a 5-sec visual stimulus, little overall response suppression was found with either fixed-interval or variable-interval schedules. In a third experiment, the percentage of occasions on which a cue preceded each shock delivery was varied from 0% to 100%. For the fixed-interval shock condition, the most suppression occurred with the 0% treatment, the least with 100%, and an intermediate amount with the 50% treatment. For the variable-interval groups, the most suppression occurred in the 50% condition, the least in the 100% group, and an intermediate amount with 0%.     

COMPARISON OF EXTINCTION,DRO 0-SEC,AND DRO 6-SEC IN THE ELIMINATION OF IMITATIVE RESPONDING UNDER DISCRETE-TRIAL PARADIGMS1

Imitative response levels were observed for normal 5-yr-old children within a discrete-trial imitation paradigm. Following imitation training sessions, children were observed under conditions of contingent reinforcement for three experimental sessions. Following the contingent reinforcement phase, children were then observed under reversal conditions for four sessions. A 3 × 4 repeated measures design was employed for observing response decrements under conditions of extinction, DRO 6-sec, and DRO 0-sec. The DRO 0-sec group demonstrated a significantly greater decrement in imitative responding than did the DRO 6-sec and extinction group. The differences in effects between the two DRO conditions and results from previous studies in which DRO procedures have had little effect in reducing imitative responding, suggest an incomplete understanding of the ways in which DRO procedures may operate.     

Quasireinforcement: Control of behavior by second-order interval schedules

Pigeons were trained on a variable-interval 66-sec schedule of reinforcement that was segmented into either fixed- or variable-interval 10-sec components. Three-second access to food followed some components according to the overall VI 66-sec schedule, but 3-sec periods of nonreinforcement followed the other components. With both FI 10-sec and VI 10-sec segments, overall response rates were generally higher when the completion of unreinforced segments was signaled by a red key (never paired with food) than when it was unsignaled. Response rates during the red-key periods dropped to zero. Brief presentations of the red key engendered the distinctive (FI or VI) patterns of responding which would be expected if each segment were followed by food. These data demonstrate behavioral control by brief stimuli which are not paired with primary reinforcement and show that such control may develop even when the sequences of behavior required to produce food or brief stimuli are variable in duration.     

Reinforcement contingencies maintaining collateral responding under a DRL schedule

Two-key conjunctive schedules were studied with one key (food key) under a differential-reinforcement-of-low-rate 20-sec schedule, while the consequences of responding on another key (collateral key) were varied. When food depended not only upon a food-key interresponse time in excess of 20 sec, but also upon the occurrence of one or more collateral-key responses during the food-key interresponse time, the rate of collateral-key responding was low and food-key interresponse times rarely exceeded 20 sec. When collateral-key responses could produce a discriminative stimulus correlated with the availability of food under the DRL schedule, the discriminative stimulus functioned as a conditioned reinforcer to maintain higher rates of collateral-key responding, and the spacing of food-key responses increased. If the occurrence of the discriminative stimulus was independent of collateral-key responses, the rate of collateral-key responding was again low, but the spacing of food-key responses was still controlled by the discriminative stimulus. Both the conditioned reinforcer and the explicit reinforcement contingency could maintain collateral-key responding, but the adventitious correlation between collateral-key responses and the delivery of food could not maintain very much collateral-key responding. The pattern of responding on the food-key was determined to a much greater extent by the correlation between the discriminative stimulus and the delivery of food than by the pattern of responding on the collateral key.     

Intermittent reinforcement of an interresponse time

Rats were exposed to schedules in which reinforcement was contingent upon the emission of a 1.0- to 2.0-sec interresponse time. The rate of emission and the temporal distribution of this interresponse time was recorded. Several different contingencies between the emission of the interresponse time and reinforcement were examined. Both the rate of emission and the temporal distribution of the 1.0- to 2.0-sec interresponse time varied as a function of the schedule on which it was reinforced. This finding, which suggests that an interresponse time behaves as other operants, has implications for the analysis of conventional reinforcement schedules in terms of the differential reinforcement of interresponse times.     

Reduction of intraspecies aggression in rats by positive reinforcement of incompatible behaviors

Fighting responses were elicited by response-independent shocks delivered to pairs of rats. Food pellets were presented following different non-fighting responses to shock: some pairs of rats received pellets dependent upon a specific non-fighting response to shock; others received pellets dependent upon any non-fighting response to shock; and control pairs never received pellets. The mean probability of an elicited aggressive response to shock was reduced to 0.2 by food reinforcement for a specific non-fighting response, and to 0.5 by food reinforcement for any kind of non-fighting response. These values contrasted with the 0.8 probability of elicited aggression when pairs of rats received no food reinforcement. Consistent findings were obtained when treatment conditions were changed for individual pairs of rats.     

Effects on responding of mixed and multiple schedules of signalled and unsignalled response-dependent electric-shock delivery

Responding in two rats was maintained under mixed and multiple variable-interval 35-sec variable-interval 35-sec food delivery schedules. Similar rates and patterns of responding occurred in each component of the two schedules. Mixed and multiple variable-interval 65-sec variable-interval 65-sec schedules of response-dependent shock delivery were super-imposed on the mixed and multiple baseline food schedules, respectively. In one component, a 5-sec stimulus was presented on the average of once every 65 sec. Offset of the stimulus arranged that the next response would produce shock. In the other component, no stimulus was presented during the 5-sec period. The mixed schedule of signalled and unsignalled dependent shock delivery yielded similar degrees of response suppression in each component, but the multiple schedule of shock delivery revealed differential degrees of response suppression. Considerably more suppression occurred in the component not associated with the preshock stimulus, thus implicating the discriminative functions of the correlated stimulus.     

A behavioral approach to the control of thumbsucking in the classroom

The purpose of the present study was to determine whether classroom thumbsucking could be controlled by making social reinforcement contingent upon appropriate behaviors incompatible with thumbsucking. The classroom thumbsucking behavior of three 8-yr-old elementary school children was observed and recorded by two observers. Sixteen experimental lessons were divided into sets of baseline, reinforcement, reversal, and representation of reinforcement. The effect of the experimental treatment was determined by the changes in the total thumbsucking level for each subject during each lesson. Results indicated noticeable decreases in thumbsucking rate for all three subjects during lessons in which social reinforcement was made contingent upon appropriate behavior. Two of the three subjects showed sudden increases in thumbsucking time during lessons when social reinforcement for appropriate responses was withheld; the third subject showed continued low thumbsucking time during these lessons.     

Avoidance response rates during a pre-food stimulus in monkeys

Free-operant avoidance responding was maintained by a shock avoidance schedule in three monkeys. The frequency of avoidance responses during a stimulus terminated by response-independent food pellet presentation was dependent upon the method of pellet delivery. Avoidance rates were relatively increased when food retrieval responses followed pellet delivery. Avoidance rates were decreased when retrieval responses preceded pellet delivery.