Observing responses in pigeons

Pigeons were trained on an observing-response procedure in which periods of VR 100 and EXT alternated unpredictably during a white light (mixed stimulus). During VR 100, responses on a food-producing key (the first key) were intermittently reinforced. Responses on the observing key (the second key) produced a green light (positive stimulus) when VR 100 was in effect, and a red light (negative stimulus) for EXT. The birds did not respond on either key during the negative stimulus, but they responded on the food-producing key when the positive stimulus appeared. When observing responses produced the positive or negative stimulus on FR, observing responses were maintained until the FR reached a maximum; beyond this, only food-producing responses occurred. When observing responses did not produce either stimulus, the observing-response rates fell to zero. With prolonged exposure to an FR 20 schedule of observing, observing-response rates during EXT were higher than during VR 100. Chlorpromazine hydrochloride decreased the total response output but markedly increased observing-response rates except when it was administered before sessions of observing response extinction.     

Observing behavior in squirrel monkeys under a multiple schedule of reinforcement availability

Observing behavior of two squirrel monkeys was examined under a multiple schedule of four components. Lever (observing) responses produced either a stimulus indicating the availability of food or another stimulus indicating food was not available. Key responses in the presence of the food-available stimulus produced food on a continuous reinforcement schedule. In the absence of food-available stimuli, responding on the key had no scheduled consequences. Observing responses produced food-available stimuli according to three different random-interval schedules with mean interstimulus availability times of 1, 2, and 4 min. In the fourth component of the multiple schedule (observing extinction) food-available stimuli never occurred. Each component of the schedule was correlated with a distinctive auditory stimulus. Observing rates decreased with decreasing frequency of the food-available stimulus. Observing rates during extinction continued decreasing when the brief stimulus indicating food unavailability was no longer produced by lever pressing. When the brief stimulus was reinstated response rates increased abruptly.     

Extinction of responding maintained by timeout from avoidance

The resistance to extinction of lever pressing maintained by timeout from avoidance was examined. Rats were trained under a concurrent schedule in which responses on one lever postponed shock on a free-operant avoidance (Sidman) schedule (response-shock interval = 30 s) and responses on another lever produced 2 min of signaled timeout from avoidance on a variable-ratio 15 schedule. Following extended training (106 to 363 2-hr sessions), two experiments were conducted. In Experiment 1 two different methods of extinction were compared. In one session, all shocks were omitted, and there was some weakening of avoidance but little change in timeout responding. In another session, responding on the timeout lever was ineffective, and under these conditions timeout responding showed rapid extinction. The within-session patterns produced by extinction manipulations were different than the effects of drugs such as morphine, which also reduces timeout responding. In Experiment 2 shock was omitted for many consecutive sessions. Response rates on the avoidance lever declined relatively rapidly, with noticeable reductions within 5 to 10 sessions. Extinction of the timeout lever response was much slower than extinction of avoidance in all 4 rats, and 2 rats continued responding at baseline levels for more than 20 extinction sessions. These results show that lever pressing maintained by negative reinforcement can be highly resistant to extinction. The persistence of responding on the timeout lever after avoidance extinction is not readily explained by current theories.     

A comparison of two types of extinction following fixed-ratio training

Five groups of pigeons were food reinforced on various schedules. Half of each group were extinguished in the normal manner; the others were presented with a stimulus change, previously paired with reinforcement, each time they completed their respective fixed ratios. Response rate in training was an increasing negatively accelerated function of the FR. Increasing the FR produced transitory rate changes, the amount of which yielded a quantitative index of ratio strain. Cumulative records of extinction performance revealed that the stimulus change exerted discriminative control by maintaining the cohesiveness of FR response units. Nevertheless, neither the absolute number of extinction responses nor extinction response units differed appreciably for the two extinction procedures.     

Response acquisition with delayed reinforcement: a comparison of two-lever procedures.

Groups of 8 experimentally naive rats were exposed during 8-hr sessions to resetting delay procedures in which responses on one lever (the reinforcement lever) produced water after a delay of 8, 16, 32, or 64 s. For rats in one condition, responses on a second (no-consequences) lever had no programmed consequences. For rats in another condition, responses on a second (cancellation) lever during a delay initiated by a response on the reinforcement lever prevented delivery of the scheduled reinforcer; responses on the cancellation lever at other times had no programmed consequences. Under both conditions and at all delays, most subjects emitted more responses on the reinforcement lever than did control rats that never received water emitted on either lever. At 8-s delays, both conditions engendered substantially more responding on the reinforcement lever than on the other lever, and performance closely resembled that of immediate-reinforcement controls. At delays of 16 and 32 s, however, there was clear differential responding on the two levers under the cancellation condition but not under the other condition. When the delay was 64 s, differential responding on the two levers did not occur consistently under either condition. These findings provide strong evidence that the behavior of rats is sensitive to consequences delayed by 8, 16, and 32 s, but only equivocal evidence of such sensitivity to consequences delayed 64 s. They also indicate that acquisition depends, in part, on the measure of performance used to index it.     

Redundant information in an observing-response procedure

In three observing-response experiments relevant to the information hypothesis of conditioned reinforcement, the basic procedure was one in which an observing response produced one stimulus on trials that terminated in non-contingent reinforcement and another stimulus on trials that terminated in a brief timeout. In Experiment I, the observing response consisted of a single peck or a short fixed-ratio schedule (FR 3 or FR 6), depending on the type of trial. If the single peck produced the negative stimulus and the fixed ratio produced the positive stimulus, observing responses were maintained. If the single peck produced the positive stimulus and the fixed-ratio produced the negative stimulus, observing responses were not maintained on negative trials. In the second experiment, the response key was either white or dark at the beginning of a trial, indicating whether it was a positive or negative trial. Observing responses continued to be maintained on positive trials but not on negative trials. In Experiment III, only positive or negative trials were scheduled for several sessions. Observing responses extinguished regardless of whether positive or negative trials were scheduled. The results do not support the hypothesis that making the stimuli produced by observing responses redundant will reduce observing responses.     

Response specificity in animal timing.

The stimuli that control responding in the peak procedure were investigated by training rats, in separate sessions, to make two different responses for food reinforcement. During one type of session, lever pressing was normally reinforced 32 s after the onset of a light. During the other type of session, chain pulling was normally reinforced either 8 s after the onset of one auditory cue or 128 s after the onset of a different auditory cue. For both types of sessions, only the appropriate manipulandum was available, and 20% of the trials lasted 240 s and involved no response-contingent consequences. Rats were then tested with the auditory cues in the presence of the lever and the light in the presence of the chain. If the time of reinforcement associated with each stimulus was learned, response rates should peak at these times during transfer testing. However, if a specific response pattern was learned for each stimulus, little transfer should occur. The results did not clearly support either prediction, leading to the conclusion that both a representation of the time of reinforcement and the rat's own behavior may control responding in this situation.     

Discriminative and reinforcing properties of two types of food pellets

In Experiment I some discriminative functions of food pellets were studied by developing a multiple schedule of reinforcement (mult FR 30 FI 3) in which the delivery of a standard laboratory food pellet as a reinforcer set the occasion for reinforcement on every 30th response (FR 30), and the delivery of a sucrose food pellet as a reinforcer set the occasion for reinforcement after a 3-min interval (FI 3). Discriminative stimulus control by the type of pellet was also demonstrated by reversing the operant discrimination and having the standard pellet control the FI 3 and the sucrose pellet control the FR 30. In Experiment II a mult FR 30 FR 30 with two bars was developed; a standard food pellet was followed by an FR 30 on Bar 1 and extinction (ext) on Bar 2, while a sucrose pellet was followed by an FR 30 on Bar 2 and ext on Bar 1. A control rat was placed, for comparison, on a mixed (mix) FR 30 FR 30 schedule with two bars, but neither bar correlated with the type of food pellet. In Experiments I and II the similarity between pellet controlled multiple schedules and multiple primed schedules was discussed, as was the comparability of transitions and effectiveness of control between pellet controlled multiple schedules and multiple schedules providing continuous exteroceptive stimuli.     

A response duration schedule: effects of training, extinction, and deprivation

Rats were trained to hold down a lever for at least 40 consecutive seconds. When the lever had been held down for 40 sec, white noise came on. Releasing the bar in the presence of the noise turned off the noise and operated a feeder that delivered a pellet of food. At the end of training, frequency distributions of response durations peaked at 40 to 41 sec. If as in training, holding down the lever produced white noise at the end of 40 sec, and release of the lever terminated the noise and operated the feeder, but no food delivery occurred, duration distributions and several other measures were initially not very different from when food was delivered. However, if during extinction white noise was never produced by lever holding, and feeder operation did not occur upon lever release, most responses were shorter than 1 sec in duration, some were much longer than 41 sec, and duration distributions did not peak at 40 to 41 sec. When reinforcement was reinstated after extinction, performance quickly returned to pre-extinction measures. Further sessions at different levels of deprivation produced only temporary disruptions in performance.     

Operant hoarding: a new paradigm for the study of self-control.

In the first of four experiments, rats were exposed to a modified multiple continuous reinforcement-extinction schedule during 15-min daily sessions. In one condition (saves condition) with the cuelight on, a single lever press produced a food pellet, briefly extinguished the cuelight, and started a clock. Saves (additional lever presses with interresponse times less than 1 s) produced an additional food pellet, briefly extinguished the cuelight, and restarted the interresponse time clock. The cuelight was extinguished 1 s after the last lever press and remained off during a 10-s period of extinction, during which no food pellets were delivered. In the other condition (savings account condition), the contingencies were the same except that the cuelight was extinguished and was not reilluminated after the initial lever press, and the delivery of all food pellets in the reinforcement component was delayed until the onset of extinction. In both conditions, rats made saves, but mean saves (total saves divided by the number of reinforcement components) were slightly reduced in the savings account condition. In Experiment 2, using six equally spaced 15-min sessions per day on alternate days, saves were either followed immediately with food and brief cuelight offset (saves condition) or were not reinforced at all. Mean saves were much greater when saves were reinforced. In Experiment 3, during 5-min daily sessions, saves earned a single pellet (savings account condition) or a number of pellets equal to the ordinal number of the lever press (interest condition). Rats made fewer mean saves, with little change in the food rate, when saves earned interest. In Experiment 4, the rats earned all their food in the operant situation during 24 daily 5-min sessions, these separated by 55-min intersession intervals during which no food was available; otherwise, the conditions were the same as in Experiment 3. In Experiment 4, the shift to interest for saves led to an increase in mean daily mean saves (total daily mean saves divided by the number of daily sessions) as well as to an increase in the number of food pellets delivered in each session. The results are discussed in terms of self-control and behavioral economics.     

The conditioned reinforcement of repeated acquisition

Three monkeys were trained to emit a chain of three responses on three separate levers in a set of six levers to obtain food. The chain producing food (correct chain) was changed each day. During a trial, a press on any lever produced a feedback stimulus; a press on a correct lever produced an additional distinctive stimulus; the third correct press produced a food pellet. Test sessions in which either the food or the distinctive stimuli were removed were interspersed with baseline sessions. In tests without food presentations, the subjects acquired the correct chain rapidly, with a level of accuracy comparable to baseline. Removing the distintive stimuli for either the first or second member of the correct chain greatly retarded acquisition of that member of the chain. Removing all distinctive stimuli often reduced accuracy throughout the chain to chance level, even though food was presented following each correct chain. These results were interpreted as evidence that the distinctive stimuli presented after correct responses functioned as conditioned reinforcers. Reductions in accuracy following an omitted distinctive stimulus indicated that they were also discriminative stimuli for correct responding in their presence.     

Effects of schedule of reinforcement on a pentobarbital discrimination in rats.

The purpose of this study was to determine the effects of the schedule of reinforcement on a pentobarbital discrimination in rats. Five rats were trained to discriminate 10 mg/kg pentobarbital from saline under a multiple fixed-interval 180-s fixed-ratio 20 schedule of reinforcement. During both saline and pentobarbital training sessions, subjects emitted a higher percentage of correct responses under the fixed-ratio component as compared to the fixed-interval component of the multiple schedule. Determination of the pentobarbital dose-response curve under the fixed-ratio component resulted in a steep curve characterized by responding on the saline lever at low doses and on the drug lever at higher doses. Under the fixed-interval component, a graded dose-effect curve was produced, with considerable responding on both levers after intermediate doses of pentobarbital. The administration of phencyclidine and MK-801 resulted in an intermediate level of drug-lever responding for some subjects. Administration of d-amphetamine resulted in saline (nondrug) appropriate responding. The results of this study demonstrate that the schedule of reinforcement is a determinant of drug stimulus control, just as it is a determinant of other drug effects.     

Escape from SD associated with fixed-ratio reinforcement

Throughout ascending and descending fixed-ratio (FR) sequences, rats were allowed to terminate the FR stimulus control by pressing a time-out (TO) lever. To minimize chance or accidental responses on this second lever, three presses were required to produce the 30-sec S^Δ period. As FR performance became more “strained,” there was an increased predisposition to escape from the time-in stimulus complex. The generality of this finding was extended by obtaining recoverability (independent of the direction of stimulus change) of the FR-TO function in the descending series. Typically, escapes were produced only during the post-reinforcement pause; however, under a mixed FR FR schedule, their occurrence shifted to a point within the inter-reinforcement interval corresponding to the unreinforced completion of the lower ratio component. It appears that the point where the rat can discriminate the size of the ratio requirement will be the place where TOs are imposed. This inference was supported by a substantial increase in TO frequency accompanying a shift from CRF to extinction on the FR lever. Finally, the escape lever was placed on a progressively increasing FR schedule and later extinguished to demonstrate that the TO condition was in fact reinforcing.     

Duration discrimination: effects of probability of stimulus presentation

Monkeys initiated a stimulus by pressing on the center of three levers and the stimulus terminated independently of behavior 60, 80, 90, or 100 sec later. Presses on the right lever were reinforced with food following the three briefer durations, and presses on the left lever, following the 100-sec duration. Incorrect responses produced a 10-sec timeout. Probability of presenting the 100-sec duration was manipulated in the range from 0.25 to 0.75, with the probabilities of the briefer durations remaining equal and summing to one minus the probability of the 100-sec duration. Percentage of responses on either side lever was functionally related to both the probability of presenting the 100-sec stimulus and to stimulus duration. An analysis of the data based on the theory of signal detection resulted in operating characteristics that were linear when plotted on normal-normal coordinates. The percentage of responses on either lever approximated the optimal values for maximizing reinforcement probability in each condition of the experiment.     

Concurrent schedules of primary and conditioned reinforcement in rats

Rats responded on a fixed-interval schedule during which a 3-sec stimulus preceded each water reinforcement. The stimulus was then scheduled concurrently for responses on the same lever according to either a variable ratio. Although water reinforcement continued on a fixed-interval schedule, the pattern of responding became typical of a variable-interval or variable-ratio schedule. When the 3-sec stimulus was presented on a variable-interval or variable-ratio schedule, but was omitted on the fixed-interval schedule, the response rate decreased. When the stimulus occurred after the same time periods as those of the variable-interval schedule, but at least 7-sec after the last response, the rate decreased. The rate became higher when the fixed-interval schedule was discontinued and each presentation of the 3-sec stimulus was followed by water on a variable-interval schedule. When both water and the 3-sec stimulus were discontinued for a period of time, resulting in extinction of the lever response, and the 3-sec stimulus alone then presented on a variable-interval or variable-ratio schedule after lever responses, rate increased and then gradually decreased.     

Behavior of rats under fixed consecutive number schedules: effects of drugs of abuse.

Four rats responded under a simple fixed consecutive number schedule in which eight or more consecutive responses on the run lever, followed by a single response on the reinforcement lever, produced the food reinforcer. Under this simple schedule, dose-response curves were determined for diazepam, morphine, pentobarbital, and phencyclidine. The rats were then trained to respond under a multiple fixed consecutive number schedule in which a discriminative stimulus signaled when the response requirement on the run lever had been completed in one of the two fixed consecutive number component schedules. Under control conditions, the percentage of reinforced runs under the multiple-schedule component with the discriminative stimulus added was much higher than the percentage of reinforced runs under the multiple-schedule component without the discriminative stimulus. All of the drugs decreased the percentage of reinforced runs under each of the fixed consecutive number schedules by increasing the conditional probability of short run lengths. This effect was most consistently produced by morphine. The drugs produced few differences in responding between the multiple fixed consecutive number components. Responding under the simple fixed consecutive number schedule, however, was affected at lower doses of the drugs than was responding under the same fixed consecutive number schedule when it was a component of the multiple schedule. This result may be due to the difference in schedule context or, perhaps, to the order of the experiments.     

The relationship between observing behavior and food-key response rates under mixed and multiple schedules of reinforcement

Pigeons were trained under an observing response procedure in which pecks on one key (food key) were reinforced under a mixed fixed-interval 30-sec extinction schedule. A response on a second (observing) key replaced the mixed-schedule stimulus with either of two multiple-schedule stimuli (red and green keylights) for 5 sec. Observing response rates were positively correlated with food-key response rates in the presence of multiple-schedule stimuli and inversely related to food-key response rates in the presence of mixed-schedule stimuli. These results suggest that observing response output is controlled not only by the stimuli produced by observing responses but also by the stimuli in the presence of which observing responses occur. The possibility that observing responses alter the probability of reinforcement is advanced.     

Intercurrent and reinforced behavior under multiple spaced-responding schedules

Lever pressing in rats was reinforced with food under a multiple spaced-responding schedule. A lever, food cup, and drinking tube were mounted in a running wheel so that lever pressing, running, and licking could be recorded. Running and licking had no scheduled consequences. Lever pressing was reinforced under a multiple schedule with three spaced-responding components and an extinction component. Each component was associated with a different auditory stimulus. Spaced-responding components reinforced only lever presses terminating interresponse times equal to or greater than 10, 20, or 60 sec, respectively. Rates of lever pressing, reinforcement, and licking all decreased as schedule parameter increased. Efficiency of spaced responding, as measured by reinforcements per response, also decreased. Rate of wheel running either increased or increased and then decreased with increasing schedule parameter. Individual running rates differed substantially. Neither licking nor running rate correlated with individual differences in efficiency. Analysis of conditional probabilities among the several response classes showed that, as the schedule requirement increased, the probability of running after a lever press increased and the probability of licking after a lever press decreased. After reinforcement, one subject always pressed the lever next. In the other subjects, the conditional probability of lever pressing, given reinforcement, increased while the probability of licking, given reinforcement, decreased with increasing schedule requirement. Results are discussed in relation to the concepts of schedule-induced and mediating behavior.     

The distribution of observing responses in a mixed FI-FR schedule,

In Exp I, three pigeons were trained on an observing response procedure where observing responses produced a stimulus correlated either with FI or with FR. Stimulus duration was 30 sec. During FR, the subjects completed the ratio before the stimulus terminated. During FI, the subjects usually observed the stimulus only once. Observing responses occurred immediately after food reinforcement. In Exp II, stimulus duration was shortened to 5 sec and the FR for food was increased. The results were similar to those of Exp 1. During most FIs and FRs, only one observing response occurred. The results of both experiments could be interpreted in a response competition framework. Immediately after food reinforcement, observing behavior is strong. When behavior on the food key begins it competes with further observing behavior.     

Rate of conditioned reinforcement affects observing rate but not resistance to change

The effects of rate of conditioned reinforcement on the resistance to change of operant behavior have not been examined. In addition, the effects of rate of conditioned reinforcement on the rate of observing have not been adequately examined. In two experiments, a multiple schedule of observing-response procedures was used to examine the effects of rate of conditioned reinforcement on observing rates and resistance to change. In a rich component, observing responses produced a higher frequency of stimuli correlated with alternating periods of random-interval schedule primary reinforcement or extinction. In a lean component, observing responses produced similar schedule-correlated stimuli but at a lower frequency. The rate of primary reinforcement in both components was the same. In Experiment 1, a 4:1 ratio of stimulus production was arranged by the rich and lean components. In Experiment 2, the ratio of stimulus production rates was increased to 6:1. In both experiments, observing rates were higher in the rich component than in the lean component. Disruptions in observing produced by presession feeding, extinction of observing responses, and response-independent food deliveries during intercomponent intervals usually were similar in the rich and lean components. When differences in resistance to change did occur, observing tended to be more resistant to change in the lean component. If resistance to change is accepted as a more appropriate measure of response strength than absolute response rates, then the present results provide no evidence that higher rates of stimuli generally considered to function as conditioned reinforcers engender greater response strength.