The effect of S− on observing behavior

Eight pigeons were run on a one-key, discrete-trials observing procedure. Pecks during a trial produced S⁺ and S^?, colored key lights which signalled whether the trial would end with response-independent grain reinforcement or nonreinforcement. S⁺ and S^? were produced on a VI schedule which began operating at the onset of the trial. In the main experimental condition, only a response preceded by at least 6 sec of nonresponding could produce S^? on nonreinforced trials; any response which satisfied the VI requirement produced S⁺ on reinforced trials. This procedure allowed the birds to produce S⁺ on reinforced trials with or without producing S^? on nonreinforced trials. The subjects learned to produce fewer S^?s over sessions, indicating that S^? had a punishing effect on observing. The results were taken as evidence for the conditioned reinforcement hypothesis of observing and against the uncertainty reduction hypothesis.     

Redundant information in an observing-response procedure

In three observing-response experiments relevant to the information hypothesis of conditioned reinforcement, the basic procedure was one in which an observing response produced one stimulus on trials that terminated in non-contingent reinforcement and another stimulus on trials that terminated in a brief timeout. In Experiment I, the observing response consisted of a single peck or a short fixed-ratio schedule (FR 3 or FR 6), depending on the type of trial. If the single peck produced the negative stimulus and the fixed ratio produced the positive stimulus, observing responses were maintained. If the single peck produced the positive stimulus and the fixed-ratio produced the negative stimulus, observing responses were not maintained on negative trials. In the second experiment, the response key was either white or dark at the beginning of a trial, indicating whether it was a positive or negative trial. Observing responses continued to be maintained on positive trials but not on negative trials. In Experiment III, only positive or negative trials were scheduled for several sessions. Observing responses extinguished regardless of whether positive or negative trials were scheduled. The results do not support the hypothesis that making the stimuli produced by observing responses redundant will reduce observing responses.     

The role of information in the emission of observing responses: a test of two hypotheses

Pigeons were trained on a trial procedure in a Skinner box. Each trial began with a fixed-interval schedule. Responding on this schedule produced a stimulus and a delayed trial outcome. The stimulus signalled whether the forthcoming outcome was reinforcement or nonreinforcement. Thus, the response was an observing response. When reinforcement was the outcome on 20% of the trials, response rates in the fixed interval were higher than when reinforcement was the outcome on 80% of the trials. This result is consistent with the hypothesis that observing responses are reinforced by the information associated with the stimulus signalling reinforcement. The result seems inconsistent with the hypothesis that observing responses are also reinforced by the information associated with the stimulus signalling nonreinforcement.     

Timeout from a stimulus correlated with the extinction component of a multiple schedule

Four groups of pigeons were trained to discriminate between green and red. Pecks on a second key produced a timeout from the schedule in effect for 30 sec. For two of the groups, this timeout response turned off all the lights in the chamber (Blackout), while for the other two, only the key-lights were turned off (No Blackout). For one of the Blackout groups and one of the No Blackout groups, responses on the discrimination key during the extinction component (S^?) also resulted in a mild electric shock. Blackout groups produced more timeouts during S^? than did No Blackout groups, but electric shock punishment suppressed, rather than enhanced, timeout responding. These findings suggest a need for reevaluation of the hypothesis that the timeout response is an escape from an aversive S^?.     

Resistance to change and relapse of observing

Four experiments examined relapse of extinguished observing behavior of pigeons using a two-component multiple schedule of observing-response procedures. In both components, unsignaled periods of variable-interval (VI) food reinforcement alternated with extinction and observing responses produced stimuli associated with the availability of the VI schedule (i.e., S+). The components differed in the rate of food arranged (Rich = VI 30 s; Lean = VI 120 s). In Experiment 1, following baseline training, extinction of observing involved removal of both food and S+ deliveries, and reinstatement was examined by presenting either response-independent food or S+ deliveries. In Experiment 2, extinction involved removal of only food deliveries while observing responses continued to produce S+. Reinstatement was examined by delivering food contingent upon the first two food-key responses occurring in the presence of the S+. Experiment 3 assessed ABA renewal of observing by extinguishing food-key and observing responses in the presence of one contextual stimulus (i.e., B) and then returning to the original training context (i.e., A) during continued extinction. Experiment 4 examined resurgence by introducing food reinforcement for an alternative response during extinction, and subsequently removing that alternative source of food. Across experiments, relative resistance to extinction and relapse of observing tended to be greater in the component previously associated with the higher rate of primary reinforcement. Relapse of observing or attending to stimuli associated with primary reinforcement appears to be impacted by frequency of primary reinforcement in a manner similar to responding maintained directly by primary reinforcement.     

Control of observing by antecedent stimuli

Five pigeons were run on a one-key discrete trials observing procedure. Trial onset was signaled by a white or yellow key light. Pecks in white or yellow intermittently produced S⁺ and S⁻, green and red key lights that signaled whether the trial would end with response-independent grain reinforcement or nonreinforcement. In the Redundant conditions, white and yellow were correlated with trial outcome, making S⁺ and S⁻ redundant. In the Informative condition, white and yellow were uncorrelated with trial outcome, so that S⁺ and S⁻ provided new information. During the Informative condition, all birds responded in the formerly positive, now uncorrelated color at higher rates than they did during the preceding or succeeding Redundant conditions, in which that same color was positively correlated with primary reinforcement. This result confirmed the prediction that an animal will observe at higher rates in the absence of reinforcement-correlated cues than in their presence.     

Combinations of response-dependent and response-independent schedule-correlated stimulus presentation in an observing procedure

Pigeons pecked a response key on a variable-interval (VI) schedule, in which responses produced food every 40 s, on average. These VI periods, or components, alternated in irregular fashion with extinction components in which food was unavailable. Pecks on a second (observing) key briefly produced exteroceptive stimuli (houselight flashes) correlated with the component schedule currently in effect. Across conditions within a phase, the dependency between observing and presentation of the stimuli was decreased systematically while the density of stimulus presentation was held constant. Across phases, the proportion of session time spent in the VI component was adjusted from 0.5 to 0.25, and then to 0.75. Results indicate that rate of observing decreased as the dependency between responses and stimulus presentations was decreased. Further, discriminative control by the schedule-correlated stimuli was systematically weakened as dependency was decreased. Increasing the proportion of session time spent in VI decreased the rate of observing. This effect was additive with the manipulation of the dependency between observing and presentation of the stimuli. Overall, these results show that conditioned reinforcers function similarly to unconditioned reinforcers with respect to response-consequence dependencies, and that stimulus control is enhanced under conditions in which the relevant stimuli are produced by an organism's behavior.     

The effects upon food-reinforced pecking and treadle-pressing of auditory and visual signals for response-independent food

Seven pigeons whose key-pecking was maintained by food reinforcement on a differential-reinforcement-of-low-rates 12-sec limited-hold 4-sec schedule and 12 other pigeons whose treadle-pressing was maintained by the same schedule received appetitive Pavlovian conditioning trials superimposed upon the instrumental baseline. Half the birds in each group received a tone as the CS, and the other half received a stimulus change on the key. Each CS was 20 sec long, and was immediately followed by 10-sec access to grain. The visual CS markedly facilitated the rate of pecking on the key for the birds whose baseline response was pecking. The visual CS produced auto-shaping of the key-peck and tended to produce suppression of treadle-pressing for the birds whose baseline response was treadle-pressing. The auditory CS produced inconsistent effects across birds regardless of the baseline response. In all cases the conditioned effects extinguished when response-independent food was omitted.     

Short-term memory in the rhesus monkey: A behavioral analysis of delayed-response performance

This study obtained quantitative data on the bodily orientations of rhesus monkeys in a delayed-response task and determined whether such orientations mediate the correct response in a choice trial. The basic task was a two-key chain schedule with the key leading to food signaled in the initial component. During the subsequent delay interval, the signal was removed, but it was necessary that one of the keys be pressed to advance the schedule to the terminal choice component. The position of the key pressed thus indicated orientation during the delay interval. When the monkeys had free access to the left and right keys, they tended to press the key leading to food throughout the chain schedule components and received food on more than 85% of the trials, even when the delay was extended to 20 seconds. However, when orientation toward the food key was disrupted by forcing the monkeys to press an extraneous center key during the delay, choice performance deteriorated. Requiring the center key presses early, rather than late, in the delay component had a strong disruptive effect. The relation of the results to the mediating coding-response hypothesis is discussed.     

Observing responses and uncertainty reduction

Rhesus monkeys and baboons were placed in an observing response situation where on any trial they could work for food in the presence of an imposed stimulus or make an observing response by pressing a lever to present an alternative stimulus. The reinforcing properties of the alternative stimulus were assessed by placing the observing response on a progressive ratio schedule and were found to vary as a function of the difficulty of the imposed discrimination and the degree to which the imposed discrimination had been learned. The results are consistent with the hypothesis that the reinforcing strength of the alternative stimuli varies with the amount of uncertainty they reduce. Eliminating the response requirement showed that the effect was not simply due to the fact that informative stimuli permitted a saving of responses.     

Observing behavior in squirrel monkeys under a multiple schedule of reinforcement availability

Observing behavior of two squirrel monkeys was examined under a multiple schedule of four components. Lever (observing) responses produced either a stimulus indicating the availability of food or another stimulus indicating food was not available. Key responses in the presence of the food-available stimulus produced food on a continuous reinforcement schedule. In the absence of food-available stimuli, responding on the key had no scheduled consequences. Observing responses produced food-available stimuli according to three different random-interval schedules with mean interstimulus availability times of 1, 2, and 4 min. In the fourth component of the multiple schedule (observing extinction) food-available stimuli never occurred. Each component of the schedule was correlated with a distinctive auditory stimulus. Observing rates decreased with decreasing frequency of the food-available stimulus. Observing rates during extinction continued decreasing when the brief stimulus indicating food unavailability was no longer produced by lever pressing. When the brief stimulus was reinstated response rates increased abruptly.     

Punishment of observing by a stimulus associated with the lower of two reinforcement frequencies

Three pigeons were used to investigate the effects of a stimulus associated with the lower of two reinforcement frequencies on the response producing it. In a three-key chamber, pecking the center key produced grain on alternating variable-interval schedules with mean durations of 2 min or 30 sec. Initially, green illumination of the keys accompanied the more favorable (30-sec) schedule and red accompanied the less favorable (2-min) schedule. Then the keys remained yellow unless the bird pecked one of the side (observing) keys to produce the discriminative stimuli for a 30-sec period. Subsequently, when red was withheld as a possible consequence of pecking a particular side key, the rate on that key increased; when red was restored, the observing rate decreased. Thus the stimulus associated with less frequent reinforcement had a punishing effect on the behavior producing it. When green was withheld on one of the side keys and the other key produced both colors, observing behavior was not maintained on the red-only key, but was maintained on the key that produced both colors.     

The effects of unavoidable shocks on a multiple schedule having an avoidance component

Two dogs were maintained on a multiple schedule having both a food reinforced and an avoidance component (Mult VI 1′ S^Δ Avoid_{SS20 RS20} S^Δ). The effects of superimposing an Estes-Skinner procedure for delivering unavoidable shocks on all components of the multiple schedule were observed. The buzzer-shock pairing of the Estes-Skinner procedure produced an increased rate of responding on the avoidance component of the schedule and also on the S^Δ components. No persistent change in rate was observed on the food component during the pre-shock stimulus. Control performances on all components could be regained by either extinguishing or eliminating the buzzer-shock pairing. Extinction of the avoidance responding had little effect on the increased rates of responding produced by the Estes-Skinner procedure on the S^Δ and avoidance extinction components and did not lead to a conditioned suppression of the food reinforced responding. Rate of responding during the pre-shock stimulus was observed to be relatively independent of changes in the maintaining schedules. Responding during the pre-shock stimulus could be conditioned and maintained after an extensive history of avoidance extinction.     

The generality of selective observing

Four rats obtained food pellets by poking a key and 5-s presentations of the discriminative stimuli by pressing a lever. Every 1 or 2 min, the prevailing schedule of reinforcement for key poking alternated between rich (either variable-interval [VI] 30 s or VI 60 s) and lean (either VI 240 s, VI 480 s, or extinction) components. While the key was dark (mixed-schedule stimulus), no exteroceptive stimulus indicated the prevailing schedule. A lever press (i.e., an observing response), however, illuminated the key for 5 s with either a steady light (S+), signaling the rich reinforcement schedule, or a blinking light (S-), signaling the lean reinforcement schedule. One goal was to determine whether rats would engage in selective observing (i.e., a pattern of responding that maintains contact with S+ and decreases contact with S-). Such a pattern was found, in that a 5-s presentation of S+ was followed relatively quickly by another observing response (which likely produced another 5-s period of S+), whereas exposure to S- resulted in extended breaks from observing. Additional conditions demonstrated that the rate of observing remained high when lever presses were effective only when the rich reinforcement schedule was in effect (S+ only condition), but decreased to a low level when lever presses were effective only during the lean reinforcement component (S- only condition) or when lever presses had no effect (in removing the mixed stimulus or presenting the multiple-schedule stimuli). These findings are consistent with relativistic conceptualizations of conditioned reinforcement and extend the generality of selective observing to procedures in which the experimenter controls the duration of stimulus presentations, the schedule components both offer intermittent food reinforcement, and rats serve as subjects.     

Conditioned behavioral and physiological changes associated with injections of a narcotic antagonist in morphine-dependent monkeys

Environmental stimuli which are repeatedly associated with the nalorphine-induced withdrawal syndrome in morphine-dependent monkeys acquire the ability to produce a variety of conditioned behavioral and physiological responses. Morphine-dependent rhesus monkeys were studied under a fixed-ratio schedule where every tenth lever press produced a food pellet. After several pairings of a stimulus (light or tone) with intravenous injection of a dose of nalorphine which produced an immediate and severe withdrawal syndrome, onset of the stimulus alone produced conditioned suppression of lever pressing, heartrate decrease, vomiting and salivation. Conditioned suppression of responding and conditioned heart-rate changes persisted in post-dependent monkeys for one to four months after termination of chronic morphine treatment. No conditioned electrocardiogram, respiration or temperature changes were ever seen. A second group of morphine-dependent rhesus monkeys was studied under a schedule where every lever press produced an intravenous injection of morphine. After 10 pairings of a light with the intravenous injection of a dose of nalorphine which produced marked withdrawal signs and increased responding for morphine, presentation of the light and injection of sahne produced conditioned increases in responding for morphine. A third group of morphine-dependent rhesus monkeys was studied under a schedule where every nth lever press (n = 1 to 10) terminated a stimulus light associated with periodic injections of nalorphine or naloxone; lever-press responding was engendered and subsequently maintained. Thus, stimuli associated with the nalorphineor naloxone-induced withdrawal syndrome can either suppress, enhance or maintain behavior depending on the schedule conditions.     

A test of the reinforcing properties of stimuli correlated with nonreinforcement

The information hypothesis of conditioned reinforcement predicts that a stimulus that “reduces uncertainty” about the outcome of a trial will acquire reinforcing properties, even when the stimulus reliably predicts nonreinforcement. Four pigeons' key pecks produced one of two 5-sec stimuli with 0.50 probability according to a discriminated variable-interval schedule. One stimulus was followed by reinforcement; a second stimulus was followed by blackout. To the same extent, therefore, both stimuli reduced uncertainty about the possibility that food would arrive at the termination of the schedule interval. When a second key in the chamber was lighted, each peck on it could produce the stimulus preceding reinforcement, the stimulus preceding nonreinforcement, a novel stimulus, or no stimulus, across separate conditions. The stimulus preceding food maintained responding at substantial levels on the second, stimulus-producing, key. Such responding was not maintained by other stimuli. These data, replicated when the stimuli were reversed on the variable-interval schedule, do not support the prediction that uncertainty-reducing stimuli are necessarily conditioned reinforcers.     

Observing stimulus sources that signal food or no food

Pigeons were given a choice between observing a stimulus source that was uncorrelated with food or one that was informative. The informative source was either positive, in which a stimulus change signalled food, or negative, in which change signalled no food. If observing is supported by the reduction of uncertainty, the negative as well as the positive source should be preferred to the uncorrelated source. On the other hand, if observing requires support by conditioned reinforcement, the negative source should not be preferred to the uncorrelated source. Two keys served as stimulus sources in a discrete trial procedure. The keys were lighted together, remained on for a variable length of time, and went off together. A key could change color 1 sec before going off. In the uncorrelated source, the change occurred equally often on trials ending with or without food. In the positive information source, the change occurred only on food trials, whereas in the negative information source, it occurred only on no-food trials. All stimulus changes and food delivery were response independent. As measured by orientation and autoshaped pecking, the positive information source was preferred to the uncorrelated source. However, the uncorrelated source was preferred to the negative information source. The latter result does not support the view that observing behavior is reinforced by the reduction of uncertainty. The positive and negative information sources reduced uncertainty equally but only the positive source provided a signal that could act as a conditioned reinforcer by virtue of its relation to food.     

The role of instrumental responding and contiguity of stimuli in the development of infant secondary reinforcement

Ten-month-old infants received contingent pairings of a tone (T+) and food reinforcer. Groups S^r and S^D received the food on an FI 23-sec schedule for target touching, the former group receiving T+ immediately after the response and 1.5 sec prior to food and the latter group receiving T+ at the end of the intertrial interval. Group SC received food reinforcers 1.5 sec after T+ with no response required. A second tone (Tⁿ) was heard by all groups once during each intertrial interval, at randomly determined points. All groups subsequently were given a spatial discrimination task, receiving T+ for one alternative and Tⁿ for the other. Group S^r gave significantly more responses for T+ than for Tⁿ, but neither of the other two groups produced a superiority for T+. Thus, both contiguity with a primary reinforcer and the presence of an operant during training appear to be necessary for a neutral signal to acquire the ability to enhance responding.     

Arousal, changeover responses, and preference in concurrent schedules

Pigeons were trained on multiple schedules that provided concurrent reinforcement in each of two components. In Experiment 1, one component consisted of a variable-interval (VI) 40-s schedule presented with a VI 20-s schedule, and the other a VI 40-s schedule presented with a VI 80-s schedule. After extended training, probe tests measured preference between the stimuli associated with the two 40-s schedules. Probe tests replicated the results of Belke (1992) that showed preference for the 40-s schedule that had been paired with the 80-s schedule. In a second condition, the overall reinforcer rate provided by the two components was equated by adding a signaled VI schedule to the component with the lower reinforcer rate. Probe results were unchanged. In Experiment 2, pigeons were trained on alternating concurrent VI 30-s VI 60-s schedules. One schedule provided 2-s access to food and the other provided 6-s access. The larger reinforcer magnitude produced higher response rates and was preferred on probe trials. Rate of changeover responding, however, did not differ as a function of reinforcer magnitude. The present results demonstrate that preference on probe trials is not a simple reflection of the pattern of changeover behavior established during training.     

Persistent behavior maintained by unavoidable shocks

Squirrel monkeys were trained on a multiple schedule in which 10-min periods on a continuous shock avoidance schedule, indicated by a yellow light, alternated with 10-min periods on a 1.5-min variable interval schedule of food reinforcement (VI 1.5). A white light indicated that VI 1.5 was in effect, except for the middle 2 min of the period on VI 1.5, in which a blue light appeared and terminated with the delivery of a 0.5-sec unavoidable shock. Stable response rates developed in the avoidance and VI 1.5 components. However, the highest response rates occurred in the blue, preshock stimulus. A series of experiments showed that responding in the blue stimulus persisted even when responding had been extinguished on both the VI schedule of food reinforcement and the shock avoidance schedule. Responding in the blue stimulus ceased when the blue stimulus terminated without shock or when it terminated with a response-contingent shock. Each time responding ceased, it was restored by terminating the blue stimulus with an unavoidable shock. When the blue stimulus was on throughout each session and unavoidable shocks were delivered at regular 10-min intervals, responding was well maintained. These results show that in monkeys that have been trained on a continuous avoidance schedule, unavoidable shocks can maintain responding even under conditions where responses have no programmed consequences.