Ratio requirement and reinforcer effects in concurrent fixed-interval fixed-ratio schedules

The fixed-ratio requirement was varied in concurrent fixed-interval fixed-ratio schedules. Fixed-interval responding was reinforced by food. In different phases, fixed-ratio responding was reinforced by food or water. There was a direct relation between the ratio requirement and interval response rates when both responses were reinforced with food, but essentially no relation when the reinforcers were different. The role of reinforcers in concurrent schedules merits detailed study.     

Fractional punishment of fixed-ratio performance

Key-peck responses of pigeons under a fixed-rate 60 (Exp. I) or fixed-ratio 99 (Exp. II) schedule of positive reinforcement were punished by response-dependent electric shock during a segment of the ratio. The punishing stimulus was scheduled in one of three locations: the first third of the ratio, the middle third, or the final third. At high shock levels, the different loci of punishment differentially affected the typical fixed-ratio performance pattern. Post-reinforcement pauses were lengthened by all punishment conditions but to a greater degree when the responses in initial third of the ratio were punished. Disruption of responses before the punished segment of the ratio was a conspicuous feature of the performances when the middle or final third of the ratio was punished. Two of the punishment conditions produced similar effects on both fixed-ratio baselines but punishing the final third of the ratio suppressed the punished responses of the ratio only with the fixed-ratio 99 schedule. General effects of all punishment conditions included consistent intra-session recoveries of partially suppressed performances, the rapid recovery of the FR performances after the punishment dependency was removed after complete suppression, and the facilitation of overall and/or local response rates of most subjects by low-intensity shock.     

The effect of punishment shock intensity upon responding under multiple schedules

In the first of two experiments, responses of two pigeons were maintained by multiple variable-interval, variable-ratio schedules of food reinforcement. Concurrent punishment was introduced, which consisted of a brief electric shock after each tenth response. The initial punishment intensities had no lasting effect upon responding. Then, as shock intensity increased, variable-ratio response rates were suppressed more quickly than variable-interval response rates. When shock intensity decreased, variable-interval responding recovered more quickly, but the rates under both schedules eventually returned to their pre-punishment levels. In the second experiment, the following conditions were studied in three additional pigeons: (1) With each shock intensity in effect for a number of sessions, punishment shock intensity was gradually increased and decreased and responding was maintained by multiple variable-ratio, fixed-ratio schedules of food reinforcement; (2) Changes in punishment shock intensity as described above with responding maintained by either a variable-ratio or a fixed-ratio schedule, which were presented on alternate days; (3) Session-to-session changes in shock intensity with responding maintained by multiple variable-ratio, fixed-ratio schedules. Responding under the two schedules was suppressed to approximately the same extent by a particular shock intensity. Also, post-reinforcement pauses under the fixed-ratio schedule increased as response suppression increased.     

A choice technique to assess the effects of selective punishment on fixed-ratio performance

The emission of a fixed number of responses by rats was followed by food reinforcement. This fixed number could be accumulated in any way from two continuously available but mutually incompatible response classes, bar pressing, and not bar pressing for a fixed time period. A preference for one response class was arranged by specifying different maximum reinforcement rates for the two classes. Under selective punishment conditions, the preferred response occasionally led to both food and electric shock, while the non-preferred response led only to food. Selective punishment effects were measured through changes in the preference to the two responses in the sequence. The actions of shock intensity, deprivation, the specification of the non-preferred response, and three drugs were investigated. The results were broadly similar to the work reported by Dardano and Sauerbrunn (1964), who found localized increases in interresponse times before punished responses in fixed-ratio schedules. Performance under this procedure was found to be stable and sensitive to each of the experimental variables examined.     

Fixed-ratio reinforcement of spaced responding

Responses by rats were reinforced with food under a second-order schedule involving fixed-ratio reinforcement of temporally spaced responses. Requirements of 20, 8, and 3 responses were examined. The typical characteristic of spaced responding was maintained under the ratio schedules: interresponse time distributions were similar to those typically seen, and were not noticeably affected by the ratio value. Comparison of total response rate, correct response rate, and accuracy showed correct response rate to be the most consistently affected by changes in the ratio value. Substantial evidence of schedule control was seen only for correct responses. Incorrect response records were erratic, but rates generally declined as reinforcement was approached. Correct response records were characterized by increasing rate as reinforcement was approached. It was suggested that the pattern of fixed-ratio performance revealed may be affected by the behavioral unit examined.     

Short-term memory for responses: the "choose-small" effect

Pigeons' short-term memory for fixed-ratio requirements was assessed using a delayed symbolic matching-to-sample procedure. Different choices were reinforced after fixed-ratio 10 and fixed-ratio 40 requirements, and delays of 0, 5, or 20 s were sometimes placed between sample ratios and choice. All birds made disproportionate numbers of responses to the small-ratio choice alternative when delays were interposed between ratios and choice, and this bias increased as a function of delay. Preference for the small fixed-ratio alternative was also observed on "no-sample" trials, during which the choice alternatives were presented without a prior sample ratio. This "choose-small" bias is analogous to results obtained by Spetch and Wilkie (1983) with event duration as the discriminative stimulus. The choose-small bias was attenuated when the houselight was turned on during delays, but overall accuracy was not influenced systematically by the houselight manipulation.     

Selective punishment early and late in fixed-ratio schedules of food reinforcement

Pigeons key pecked for grain on a fixed-ratio 100 schedule; electric shocks occurred intermittently at the fifteenth or eighty-fifth response in the ratio. In Experiment I, shock was at the fifteenth response for two birds, and at the eighty-fifth response for two others, in every sixth, twelfth, or eighteenth ratio. Rate of responding decreased as frequency of shock increased, and the pattern of responding included an increased initial pause and low rates or pause-run sequences that extended further into the ratio when shock was at the fifteenth response than when it was at the eighty-fifth response. Shock early in the ratio engendered longer initial pauses than shock late in the ratio. In Experiment II, four birds responded on a two-component multiple schedule in which shock occurred at the fifteenth response of the third ratio in the presence of a white keylight and at the eighty-fifth response of the third ratio in the presence of a green keylight. The overall rates of responding decreased as shock intensity increased. All four birds responded differentially to the white and green keylights, but with a pattern that varied between birds. In general, punishment reduced the probability of responses that preceded it, regardless of the ordinal position of those responses. Both studies confirm that the probability of responding is reduced less by aversive stimuli produced late in a fixed-ratio than by aversive stimuli produced early in a fixed-ratio.     

Changes in feeding and foraging patterns as an antipredator defensive strategy: a laboratory simulation using aversive stimulation in a closed economy.

The effects of the risk of electric shock on the meal patterns of rats living in an operant chamber were investigated. Rats could obtain food by working on a response lever that provided reinforcement according to chained fixed-ratio continuous reinforcement schedules that allowed the animals control over meal size. Using a two-compartment operant chamber with a safe nesting area and manipulanda area with a grid floor, shock could be correlated with responding on the schedule. Shocks (less than or equal to 1.25 per hour) were scheduled to occur randomly throughout the day, independent of the rat's behavior. Shock caused a reorganization of meal patterns such that the animals took less frequent but larger meals. This pattern reduced the time the animals spent at risk without compromising caloric balance. Similar changes in feeding pattern were obtained in both hooded and albino rats. Exposure to shock in a separate chamber did not produce these behavioral modifications. The magnitude of shock-induced alterations of meal patterns was greater with chained fixed-ratio 90 continuous reinforcement than with chained fixed-ratio 10 continuous reinforcement. Additionally, the rats seemed to be able to reduce food intake but increase caloric efficiency, such that the reduced food intake did not have deleterious effects on maintenance of body weight. These behavioral modifications reduced the number of shocks received from that which would have been expected if meal pattern changes had not occurred. We suggest that this technique may provide a useful laboratory simulation of the impact that the risk of predation has on foraging behavior.     

Schedules using noxious stimuli. IV: An interlocking shock-postponement schedule in the squirrel monkey

Responding was studied under various schedules of electric shock postponement and presentatation in the squirrel monkey. Under an interlocking shock-postponement schedule, successive responses decreased the time by which a response postponed the next scheduled shock until a shock immediately followed the nth response. Some parameters of this schedule, which can be formally related to fixed-interval schedules, engendered a pattern of positively accelerated responding between shocks. This pattern did not occur under comparable parameter values of an alternative fixed-ratio, avoidance schedule under which each response postponed shock by a fixed duration and every nth response produced shock. Subsequently, performances were studied under schedules of shock presentation. Responding was never maintained under fixed-ratio schedules of shock presentation, but was maintained with a pattern of positive acceleration under an alternative fixed-ratio, fixed-interval schedule and under a fixed-interval schedule.     

The effects of escape conditioning and shock intensity on responding during inescapable shock

Eight albino rats, conditioned to press a lever to escape shock, continued to lever press during short inescapable shocks presented subsequently. The rate of this behavior was found to be higher for higher shock intensities regardless of the order in which shock values were presented. Relative to the immediately preceding escape rate, responding during inescapable shock was higher following conditioning at higher fixed-ratio escape requirements. Four subjects not conditioned to escape shock pressed the lever very infrequently during inescapable shock and showed little change with changes in shock intensity. The escape conditioning effects suggest that responding during inescapable shock is superstitious escape behavior. The effects of shock intensity on this behavior appear to be similar to reported effects of shock intensity on escape behavior.     

Compound stimulus control by discriminative stimuli associated with high and moderate response rates

Four rats received water on a fixed-ratio schedule for lever pressing in the presence of a tone (or light) stimulus and on a variable-interval schedule in the presence of a light (or tone) stimulus. Following stabilization of a high response rate during the fixed-ratio component and a moderate response rate during the variable-interval component, brief periods with the light and tone presented simultaneously but with no responses reinforced were inserted into the regular training schedule. Response rates during the compound stimuli were intermediate between the response rates controlled by the individual fixed-ratio and variable-interval associated stimuli.     

Fixed-ratio punishment

Responses were maintained by a variable-interval schedule of food reinforcement. At the same time, punishment was delivered following every nth response (fixed-ratio punishment). The introduction of fixed-ratio punishment produced an initial phase during which the emission of responses was positively accelerated between punishments. Eventually, the degree of positive acceleration was reduced and a uniform but reduced rate of responding emerged. Large changes in the over-all level of responding were produced by the intensity of punishment, the value of the punishment ratio, and the level of food deprivation. The uniformity of response rate between punishments was invariant in spite of these changes in over-all rate and contrary to some plausible a priori theoretical considerations. Fixed-ratio punishment also produced phenomena previously observed under continuous punishment: warm-up effect and a compensatory increase. This type of intermittent punishment produced less rapid and less complete suppression than did continuous punishment.     

The effects of brief-stimulus presentations in fixed-ratio second-order schedules

Pigeons' responses were reinforced according to a three-component multiple schedule. In Component 1, key pecks produced food according to a fixed-ratio second-order schedule with fixed-ratio units. Here, a fixed number of fixed-ratio units produced food, and the brief stimulus terminating each unit also accompanied food. Responses in Component 2 produced food on an identical schedule except that the brief stimulus was not paired with food. Component 3 contained a simple fixed-ratio schedule whose response requirement equaled that of Components 1 and 2. Across conditions the size of the fixed-ratio unit (five, ten, twenty, forty, and eighty responses) and the total number of responses per reinforcement were parametrically manipulated. The highest response rates and shortest preratio pauses were observed in Component 3 (no brief stimulus). The lowest rates and longest pauses were found in the component with paired brief-stimulus presentations, indicating that the food-paired brief stimulus suppressed responding. The suppressive effects were greatest when the fixed-ratio units were small (e.g., fixed-ratio 5) and the total fixed-ratio requirement was large (e.g., fixed-ratio 160). Under no conditions did the paired brief stimulus facilitate responding. The nonpaired brief stimulus also suppressed responding but to a lesser extent. The suppressive effects of nonpaired brief stimuli were greatest when the fixed-ratio units were small and the total response requirement was large. These data suggest that the suppressive effects of the brief stimuli may have masked the conditioned-reinforcing effects reported in other studies, and that conditions that maximize suppression in second-order schedules involve the use of fixed-ratio schedule units and the presentation of many brief stimuli per reinforcer.     

Responding under fixed-ratio and multiple fixed-interval fixed-ratio schedules of electric shock presentation

Squirrel monkeys, initially trained under a schedule of electric shock postponement and then under fixed-interval schedules of electric shock presentation, were studied under multiple fixed-interval fixed-ratio and under fixed-ratio schedules of shock presentation. Under the fixed-interval (10-min) component of the multiple schedule, a pause was followed by a gradual increase in responding to a rate maintained until shock presentation; under the fixed-ratio (3-, 10-, or 30-response) component of the multiple schedule, a brief pause was typically followed by a relatively high and uniform rate of responding until shock was presented. When the 60-sec timeout periods, which usually followed shock presentation, were eliminated from the multiple schedule for one monkey, responding was only transiently affected. In the one monkey studied, responding was maintained under a fixed-ratio schedule alone (with timeout periods), but rates of responding were lower than under the fixed-ratio component of the multiple schedule. Characteristic patterns of responding, similar to those engendered under schedules of food presentation or shock termination, can be maintained under fixed-ratio schedules of shock presentation; further, patterns of responding can be controlled by discriminative stimuli in multiple schedules.     

Second-order schedules with fixed-ratio components: variation of component size

Key pecking by pigeons was reinforced with food under second-order schedules with fixed-ratio units. A constant total number of key pecks was required for reinforcement under each condition, but the size and, inversely, number of fixed-ratio components were varied. The total response requirement of 256 pecks was divided into fixed-ratio units of 128, 64, 32, 8, and 2 responses. A brief stimulus, which always preceded food reinforcement, was presented upon completion of each fixed-ratio unit. Under most conditions, the pattern of within-unit responding was typical of that under simple fixed-ratio schedules. Overall response rate was an inverted U-shaped function of component size. That is, response rates were highest under moderate sized units (fixed ratio 128 and 64). This relationship is consistent with previous determinations of rate as a function of fixed-ratio value for simple fixed-ratio schedules.     

Pentobarbital self-administration in rhesus monkeys: drug concentration and fixed-ratio size interactions.

Performances of three rhesus monkeys were reinforced by the oral delivery of pentobarbital and studied as functions of fixed-ratio size and drug concentration. Pentobarbital solutions and water were concurrently available on identical reinforcement schedules from separate liquid-delivery systems during 3-hour sessions. Under a fixed-ratio 16 schedule of drug availability, a descending series of drug concentrations was tested (4, 2, 1, 0.5, 0.25, 0.125, and 0.0625 mg/ml, followed by a retest at 4 mg/ml). Partial concentration series beginning with the highest concentration were repeated with fixed-ratios of 32 and 64, with a fixed-ratio 128 for two monkeys, and with fixed-ratio 256 for one. At each fixed-ratio value, response rate and number of drug deliveries were inverted U-shaped functions of pentobarbital concentration. Drug intake (mg/kg/session) increased directly with drug concentration. As the fixed-ratio size was increased, the number of liquid deliveries decreased. For each drug concentration, when the numbers of drug deliveries at fixed-ratios of 32, 64, and 128 responses were plotted as percentages of those obtained at fixed-ratio 16, the following orderly relationship emerged: the higher the drug concentration, the less that drug deliveries were decreased by increases in fixed-ratio size. This relationship indicates an increase in reinforcing efficacy with increases in pentobarbital concentration.     

THE DISCRIMINATION OF STIMULUS DURATION BY PIGEONS1

Pigeons were trained to discriminate the duration of a stimulus. One response in a psychophysical choice situation was reinforced, given the immediately prior presentation of a stimulus duration in one class of durations called short durations, and the other response was reinforced given the immediately prior presentation of a stimulus duration in a second class called long durations. Durations of equal logarithmic difference from the cutoff, whether in the short or long class, yielded equal accuracy. Accuracy was a function not only of the properties of the stimuli to be discriminated, but also of the experimental contingencies used. Accuracy was greater in variable-ratio than in fixed-ratio schedules of reinforcement of the discriminative responses, and was lower at the beginning than later in individual fixed ratios. Proportion of short or long responses (response bias) was affected by sequential dependencies among long and short durations and was effectively controlled through the use of asymmetric reinforcement and fixed-ratio contingencies.     

Increased reinforcement when timeout from avoidance includes access to a safe place

Three experiments investigated the reinforcing value of access to a safe place during timeout from an avoidance schedule. Rats were trained on conjoint schedules in which responding both postponed shock on a free-operant avoidance schedule and produced periods of timeout on fixed-ratio schedules. In some conditions, a shelf was inserted into the operant chamber during timeout, enabling subjects to get off the grid floor. The combination of timeout and shelf maintained substantially higher response rates than the baseline avoidance schedule with ratio requirements as high as 90 (Experiment I). Adding the shelf to timeouts in one component of multiple fixed-ratio schedules of timeout resulted in higher response rates in the component where the shelf was included (Experiment II). When timeouts with and without the shelf were arranged on concurrent schedules, the shelf-timeout combination was preferred, even when of shorter duration than timeout alone (Experiment III). In all three experiments, subjects climbed on the shelf, although all shocks were cancelled during timeout periods. The results could not be accounted for solely in terms of the reinforcing properties of changes in shock rates, but required an interpretation that ascribed conditioned reinforcing value to stimuli associated with such changes.     

Quantification of rats' behavior during reinforcement periods

What is treated as a single unit of reinforcement often involves what could be called a reinforcement period during which two or more acts of ingestion may occur, and each of these may have associated with it a series of responses, some reflexive, some learned, that lead up to ingestion. Food-tray presentation to a pigeon is an example of such a “reinforcement period.” In order to quantify this behavior, a continuous-reinforcement schedule was used as the reinforcement period and was chained to a fixed-ratio schedule. Both fixed-ratio size and reinforcement-period duration were manipulated. Rats were used as subjects, food as reinforcement, and a lever press as the operant. Major findings included (a) a rapid decline in response rates during the first 15 to 20 seconds of the reinforcement periods, and (b) a strong positive relationship between these response rates and the size of the fixed ratio. Also revealed was a short scallop not normally found in fixed-ratio response patterns, whose length was a function of fixed-ratio size and reinforcement-period duration. It is suggested that rapidly fluctuating excitatory processes can account for many of these findings and that such processes are functionally significant in terms of behavioral compensation.     

Variable location of punishment in a response sequence

Key pecking of pigeons under a fixed-ratio 100, grain reinforcer schedule was followed by electric shock occurring once in each sequence of 100 responses with the shocked response varying irregularly in successive sequences. Under this shock schedule, a localized suppression of responding in a response sequence was not correlated with the probability of shock at different points in the sequence. High shock levels increased the duration of post-reinforcement pauses and suppressed responding during the first half of the response sequence. This suppression often persisted after the shocked response when shock occurred early in the sequence. The shock schedule did not produce a consistent suppressing effect on responding during the last half of the response sequence.