Human observing: maintained by negative informative stimuli only if correlated with improvement in response efficiency.

Two experiments investigated the effect of observing responses that enabled college students to emit more efficient distributions of reinforced responses. In Experiment 1, the gains of response efficiency enabled by observing were minimized through use of identical low-effort response requirements in two alternating variable-interval schedules. These comprised a mixed schedule of reinforcement; they differed in the number of money-backed points per reinforcer. In each of three choices between two stimuli that varied in their correlation with the variable-interval schedules, the results showed that subjects preferred stimuli that were correlated with the larger average amount of reinforcement. This is consistent with a conditioned-reinforcement hypothesis. Negative informative stimuli--that is, stimuli correlated with the smaller of two rewards--did not maintain as much observing as stimuli that were uncorrelated with amount of reward. In Experiment 2, savings in effort made possible by producing S- were varied within subjects by alternately removing and reinstating the response-reinforcement contingency in a mixed variable-interval/extinction schedule of reinforcement. Preference for an uncorrelated stimulus compared to a negative informative stimulus (S-) decreased for each of six subjects, and usually reversed when observing permitted a more efficient temporal distribution of the responses required for reinforcement; in this case, the responses were pulls on a relatively high-effort plunger. When observing the S- could not improve response efficiency, subjects again chose the control stimulus. All of these results were inconsistent with the uncertainty-reduction hypothesis.     

The generality of selective observing

Four rats obtained food pellets by poking a key and 5-s presentations of the discriminative stimuli by pressing a lever. Every 1 or 2 min, the prevailing schedule of reinforcement for key poking alternated between rich (either variable-interval [VI] 30 s or VI 60 s) and lean (either VI 240 s, VI 480 s, or extinction) components. While the key was dark (mixed-schedule stimulus), no exteroceptive stimulus indicated the prevailing schedule. A lever press (i.e., an observing response), however, illuminated the key for 5 s with either a steady light (S+), signaling the rich reinforcement schedule, or a blinking light (S-), signaling the lean reinforcement schedule. One goal was to determine whether rats would engage in selective observing (i.e., a pattern of responding that maintains contact with S+ and decreases contact with S-). Such a pattern was found, in that a 5-s presentation of S+ was followed relatively quickly by another observing response (which likely produced another 5-s period of S+), whereas exposure to S- resulted in extended breaks from observing. Additional conditions demonstrated that the rate of observing remained high when lever presses were effective only when the rich reinforcement schedule was in effect (S+ only condition), but decreased to a low level when lever presses were effective only during the lean reinforcement component (S- only condition) or when lever presses had no effect (in removing the mixed stimulus or presenting the multiple-schedule stimuli). These findings are consistent with relativistic conceptualizations of conditioned reinforcement and extend the generality of selective observing to procedures in which the experimenter controls the duration of stimulus presentations, the schedule components both offer intermittent food reinforcement, and rats serve as subjects.     

Resistance to change and relapse of observing

Four experiments examined relapse of extinguished observing behavior of pigeons using a two-component multiple schedule of observing-response procedures. In both components, unsignaled periods of variable-interval (VI) food reinforcement alternated with extinction and observing responses produced stimuli associated with the availability of the VI schedule (i.e., S+). The components differed in the rate of food arranged (Rich = VI 30 s; Lean = VI 120 s). In Experiment 1, following baseline training, extinction of observing involved removal of both food and S+ deliveries, and reinstatement was examined by presenting either response-independent food or S+ deliveries. In Experiment 2, extinction involved removal of only food deliveries while observing responses continued to produce S+. Reinstatement was examined by delivering food contingent upon the first two food-key responses occurring in the presence of the S+. Experiment 3 assessed ABA renewal of observing by extinguishing food-key and observing responses in the presence of one contextual stimulus (i.e., B) and then returning to the original training context (i.e., A) during continued extinction. Experiment 4 examined resurgence by introducing food reinforcement for an alternative response during extinction, and subsequently removing that alternative source of food. Across experiments, relative resistance to extinction and relapse of observing tended to be greater in the component previously associated with the higher rate of primary reinforcement. Relapse of observing or attending to stimuli associated with primary reinforcement appears to be impacted by frequency of primary reinforcement in a manner similar to responding maintained directly by primary reinforcement.     

Punishment of observing by a stimulus associated with the lower of two reinforcement frequencies

Three pigeons were used to investigate the effects of a stimulus associated with the lower of two reinforcement frequencies on the response producing it. In a three-key chamber, pecking the center key produced grain on alternating variable-interval schedules with mean durations of 2 min or 30 sec. Initially, green illumination of the keys accompanied the more favorable (30-sec) schedule and red accompanied the less favorable (2-min) schedule. Then the keys remained yellow unless the bird pecked one of the side (observing) keys to produce the discriminative stimuli for a 30-sec period. Subsequently, when red was withheld as a possible consequence of pecking a particular side key, the rate on that key increased; when red was restored, the observing rate decreased. Thus the stimulus associated with less frequent reinforcement had a punishing effect on the behavior producing it. When green was withheld on one of the side keys and the other key produced both colors, observing behavior was not maintained on the red-only key, but was maintained on the key that produced both colors.     

Concurrent performances: rate and accuracy of free-operant oddity responding

In pigeon's oddity performances, maintained by variable-interval reinforcement of pecks on the odd key of three keys in a triangular array, accuracy and response rate varied inversely with the rate of variable-interval reinforcement scheduled concurrently for pecks on a fourth, spatially isolated key. But when variable-interval and extinction components alternated in a multiple schedule for pecks on the spatially isolated key, oddity accuracy was greater during variable-interval components than during extinction components. Oddity response rate was not affected systematically by the alternating components. Changeovers between the oddity keys and the spatially isolated key were frequent during variable-interval components; responding occurred almost exclusively on the oddity keys during extinction components. This difference in performance during the two components was eliminated by arranging stimulus-correlated variable-interval reinforcement in the multiple schedule on the spatially isolated key: a stimulus was presented in the variable-interval components only when reinforcement became available, thereby reducing responding on this key to near-zero levels in both components while maintaining the variable-interval reinforcement. The effect of the multiple-schedule components on oddity accuracy was not altered, however, and thus apparently depended directly on concurrent reinforcement and not on differential sequential properties of concurrent responding during the two components.     

An investigation of peak shift and behavioral contrast for autoshaped and operant behavior.

Instrumental treadle press and nonreinforced key peck responses were monitored during discrimination training and generalization testing in pigeons on positive and negative reinforcement schedules. In Experiment 1, six pigeons pressed a treadle for food on a multiple variable-interval extinction schedule. In Experiment 2, three pigeons pressed a treadle to avoid shock on a multiple free-operant avoidance extinction schedule. Different color keylights signaled S+ and S- components. Some positive behavioral contrast occurred during discrimination training, but the effect was small. Pecking occurred to the S+ keylight in Experiment 1 but not in Experiment 2. On stimulus generalization tests, all subjects displayed a positive peak shift when pressing the treadle for food or to avoid shock. However, peak shift was not found for nonreinforced "autopecks" on the stimulus key, although an area shift was observed in Experiment 1. This is the first demonstration of peak shift for pigeons pressing treadles and the only reliable demonstration of peak shift when negative reinforcement maintained responding. These results, in combination with previous demonstrations of peak shift for rats pressing levers and pigeons pecking keys, indicate that peak shift is a general by-product of operant discrimination learning, since it occurs across a variety of the organisms, responses, and reinforcers.     

Rate of conditioned reinforcement affects observing rate but not resistance to change

The effects of rate of conditioned reinforcement on the resistance to change of operant behavior have not been examined. In addition, the effects of rate of conditioned reinforcement on the rate of observing have not been adequately examined. In two experiments, a multiple schedule of observing-response procedures was used to examine the effects of rate of conditioned reinforcement on observing rates and resistance to change. In a rich component, observing responses produced a higher frequency of stimuli correlated with alternating periods of random-interval schedule primary reinforcement or extinction. In a lean component, observing responses produced similar schedule-correlated stimuli but at a lower frequency. The rate of primary reinforcement in both components was the same. In Experiment 1, a 4:1 ratio of stimulus production was arranged by the rich and lean components. In Experiment 2, the ratio of stimulus production rates was increased to 6:1. In both experiments, observing rates were higher in the rich component than in the lean component. Disruptions in observing produced by presession feeding, extinction of observing responses, and response-independent food deliveries during intercomponent intervals usually were similar in the rich and lean components. When differences in resistance to change did occur, observing tended to be more resistant to change in the lean component. If resistance to change is accepted as a more appropriate measure of response strength than absolute response rates, then the present results provide no evidence that higher rates of stimuli generally considered to function as conditioned reinforcers engender greater response strength.     

Observing behavior in squirrel monkeys under a multiple schedule of reinforcement availability

Observing behavior of two squirrel monkeys was examined under a multiple schedule of four components. Lever (observing) responses produced either a stimulus indicating the availability of food or another stimulus indicating food was not available. Key responses in the presence of the food-available stimulus produced food on a continuous reinforcement schedule. In the absence of food-available stimuli, responding on the key had no scheduled consequences. Observing responses produced food-available stimuli according to three different random-interval schedules with mean interstimulus availability times of 1, 2, and 4 min. In the fourth component of the multiple schedule (observing extinction) food-available stimuli never occurred. Each component of the schedule was correlated with a distinctive auditory stimulus. Observing rates decreased with decreasing frequency of the food-available stimulus. Observing rates during extinction continued decreasing when the brief stimulus indicating food unavailability was no longer produced by lever pressing. When the brief stimulus was reinstated response rates increased abruptly.     

Information on response requirements compared with information on food density as a reinforcer of observing in pigeons

On a variable-interval schedule, pecking the key to the pigeon's right (observing response) produced red or green displays relating to the delivery of grain and its dependence on pecking the key to the left (food key). During various blocks of sessions, mixed (no stimulus change) schedules including the following pairs of components were temporarily converted by the observing response to their corresponding multiple (correlated stimuli) schedules: variable-interval 60-s, extinction; variable-interval 60-s, variable-time (response-independent) 60-s; extinction, variable-time 60-s. Differences in food delivery maintained substantial rates of responding on the observing key, without regard to pecking requirements on the food key. Although stimuli correlated with differences in the response requirement on the food key maintained higher observing rates than those maintained by uncorrelated stimuli, they were much lower than those based on food. The value of predictive stimuli as reinforcers is determined by the value of the events predicted. In particular, the cost of pecking appears to be low, and this may place limitations on the applicability of energy-based and economic models of behavior.     

Punishment of observing by the negative discriminative stimulus

To determine the effect of a negative discriminative stimulus on the response producing it, two pigeons were each studied in a three-key conditioning chamber. During alternating periods of unpredictable duration, pecking the center (food) key either was reinforced with grain on a variable-interval schedule or was never reinforced. On equal but independent variable-interval schedules, pecking either of the side (observing) keys changed the color of all keys for 30 sec from yellow to either green or red. When the schedule on the center key was variable-interval reinforcement, the color was green (positive discriminative stimulus); when no reinforcements were scheduled, the color was red (negative discriminative stimulus). Since pecking the side keys did not affect grain deliveries, changes in the rate of pecking could not be ascribed to changes in the frequency of primary reinforcement. In subsequent sessions, red was withheld as one of the possible consequences of pecking a given side key. When red was omitted, the rate on that key increased, and when red was restored, the rate decreased. It was concluded that red illumination of the keys, the negative discriminative stimulus, had a suppressive effect on the response that produced it.     

Combinations of response-dependent and response-independent schedule-correlated stimulus presentation in an observing procedure

Pigeons pecked a response key on a variable-interval (VI) schedule, in which responses produced food every 40 s, on average. These VI periods, or components, alternated in irregular fashion with extinction components in which food was unavailable. Pecks on a second (observing) key briefly produced exteroceptive stimuli (houselight flashes) correlated with the component schedule currently in effect. Across conditions within a phase, the dependency between observing and presentation of the stimuli was decreased systematically while the density of stimulus presentation was held constant. Across phases, the proportion of session time spent in the VI component was adjusted from 0.5 to 0.25, and then to 0.75. Results indicate that rate of observing decreased as the dependency between responses and stimulus presentations was decreased. Further, discriminative control by the schedule-correlated stimuli was systematically weakened as dependency was decreased. Increasing the proportion of session time spent in VI decreased the rate of observing. This effect was additive with the manipulation of the dependency between observing and presentation of the stimuli. Overall, these results show that conditioned reinforcers function similarly to unconditioned reinforcers with respect to response-consequence dependencies, and that stimulus control is enhanced under conditions in which the relevant stimuli are produced by an organism's behavior.     

The delay-reduction hypothesis of conditioned reinforcement and punishment: Observing behavior

Pigeons responded in an observing-response procedure in which three fixed-interval components alternated. Pecking one response key produced food reinforcement according to a mixed schedule. Pecking the second (observing) key occasionally replaced the mixed-schedule stimulus with the stimulus correlated with the fixed-interval component then in effect. In Experiment 1, observing was best maintained by stimuli correlated with a reduction in mean time to reinforcement. That finding was consistent with the conditioned-reinforcement hypothesis of observing behavior. However, low rates of observing were also maintained by stimuli not representing delay reduction. Experiment 2 assessed the role of sensory reinforcement. It showed that response rate was higher when maintained by stimuli uncorrelated with reinforcement delay than when the stimuli were correlated with a delay increase. This latter result supports a symmetrical version of the conditioned-reinforcement hypothesis that requires suppression by stimuli correlated with an increase in time to reinforcement. The results were inconsistent with hypotheses stressing the reinforcing potency of uncertainty reduction.     

On conditioned reinforcing effects of negative discriminative stimuli

Observing responses by pigeons were studied during sessions in which a food key and an observing key were available continuously. A variable-interval schedule and extinction alternated randomly on the food key. In one condition, food-key pecking during extinction decreased reinforcement frequency during the next variable-interval component, and in the other condition such pecking did not affect reinforcement frequency. Observing responses either changed both keylight colors from white to green (S+) or to red (S−) depending on the condition on the food key, or the observing responses never produced the S+ but produced the S− when extinction was in effect on the food key. Observing responses that produced only S− were maintained only when food-key pecking during extinction decreased reinforcement frequency in the subsequent variable-interval component. The red light conformed to conventional definitions of a negative discriminative stimulus, rendering results counter to previous findings that production of S− alone does not maintain observing. Rather than offering support for an informational account of conditioned reinforcement, the results are discussed in terms of a molar analysis to account for how stimuli acquire response-maintaining properties.     

Separating the effects of salience and disparity on the rate of observing

Pigeons producing deliveries of grain on a mixed variable-interval, extinction schedule by pecking a center key could also produce discriminative stimuli on concurrent variable-interval schedules by pecking the left or right observing key. The stimuli produced by each observing key were varied independently. In the first experiment, the negative discriminative stimulus was at the far end of the spectrum from the key illumination accompanying the mixed schedule and from the positive discriminative stimulus. When the magnitude of the difference between the latter two stimuli (salience) was varied, more pecks occurred on the observing key producing the larger of the two differences than on the key producing the smaller difference. In the second experiment, the stimulus accompanying the mixed schedule was at the far end of the spectrum, and the magnitude of the difference between the two discriminative stimuli (disparity) was varied. The proportion of pecks occurring on each observing key shifted systematically in the direction of the key producing the larger difference. The salience of the discriminative stimuli and their disparity each has an independent influence on the frequency of observing when the other is controlled, but the effect of the salience appears to be the more substantial.     

Differential production of positive and negative discriminative stimuli by normal and retarded children

Using an observing procedure similar to Wyckoff's (1969), two normal and two mentally retarded children were studied. The children were initially trained to earn pennies by pressing a rectangular key according to a multiple variable-interval extinction schedule of reinforcement. After discrimination training, the children were allowed to produce the discriminative stimuli by depressing a second or observing key. The retarded children were different from the normal children in that they produced more of the positive discriminative stimulus (S+) than the negative discriminative stimulus (S?). In the case of the retarded children, S+ observing started high and remained high. In the case of the normal children, S+ observing was initially substantial and then rapidly diminished.     

Variable-interval schedules of timeout from avoidance

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other occasionally produced a 2-min timeout during which the shock-postponement schedule was suspended and its correlated stimuli were removed. Throughout, the shock-postponement schedule maintained proficient levels of avoidance. Nevertheless, in Experiment 1 responding on the timeout lever was established rapidly, was maintained at stable levels on variable-interval schedules, was extinguished by withholding timeout, was reestablished when timeout was reintroduced, and was brought under discriminative control with a multiple variable-interval extinction schedule of timeout. These results are in contrast with Verhave's (1962) conclusion that timeout is an ineffective reinforcer when presented to rats on intermittent schedules. In Experiment 2 the consequence of responding on the timeout lever was altered so that the shock-postponement schedule remained in effect even though the stimulus conditions associated with timeout were produced for 2 min. Responding extinguished, indicating that suspension of the shock-postponement schedule, not stimulus change, was the source of reinforcement. By establishing the reinforcing efficacy of timeout with standard variable-interval schedules, these experiments illustrate a procedure for studying negative reinforcement in the same way as positive reinforcement.     

Pigeons may not remember the stimuli that reinforced their recent behavior

In two experiments the conditioned reinforcing and delayed discriminative stimulus functions of stimuli that signal delays to reinforcement were studied. Pigeons' pecks to a center key produced delayed-matching-to-sample trials according to a variable-interval 60-s (or 30-s in 1 pigeon) schedule (Experiment 1) or a multiple variable-interval 20-s variable-interval 120-s schedule (Experiment 2). The trials consisted of a 2-s illumination of one of two sample key colors followed by delays ranging across phases from 0.1 to 27.0 s followed in turn by the presentation of matching and nonmatching comparison stimuli on the side keys. Pecks to the key color that matched the sample were reinforced with 4-s access to grain. Under some conditions of Experiment 1, pecks to nonmatching comparison stimuli produced a 4-s blackout and the start of the next interval. Under other conditions of Experiment 1 and each condition of Experiment 2, pecks to nonmatching stimuli had no effect and trials ended only when pigeons pecked the other, matching stimulus and received food. The functions relating pretrial response rates to delays differed markedly from those relating matching-to-sample accuracy to delays. Specifically, response rates remained relatively high until the longest delays (15.0 to 27.0 s) were arranged, at which point they fell to low levels. Matching accuracy was high at short delays, but fell to chance at delays between 3.0 and 9.0 s. In Experiment 2, both matching accuracy and response rates remained high over a wider range of delays in the variable-interval 120-s component relative to the variable-interval 20-s component. The difference in matching accuracy between the components was not due to an increased tendency in the variable-interval 20-s component toward proactive interference following short intervals. Thus, under these experimental conditions the conditioned reinforcing and the delayed discriminative functions of the sample stimulus depended on the same variables (delay and variable-interval value), but were nevertheless dissociated.     

Some determinants of inhibitory stimulus control

Interspersed reinforcement and extinction during discrimination learning generate a U-shaped gradient of inhibition about the stimulus correlated with extinction. The present work showed that extinction is not a necessary determinant of inhibitory stimulus control. In Exp. I, a reduction in the rate of reinforcement, through a shift from a multiple variable-interval 1-min variable-interval 1-min schedule to a multiple variable-interval 1-min variable-interval 5-min schedule, resulted in a post-discrimination line orientation gradient of inhibition about the stimulus correlated with the variable-interval 5-min schedule. In Exp. II, the rates of reinforcement, correlated with a pair of stimuli, were held constant during a shift from a multiple variable-interval 1-min variable-interval 1-min schedule to a multiple variable-interval 1-min differential-reinforcement-of-low-rate schedule. Inhibitory stimulus control about the stimulus correlated with the differential reinforcement of low rate was obtained. In both experiments, a reduction in the rate of responding during one stimulus and behavioral contrast during the other stimulus preceded the observation of inhibitory stimulus control.