The role of intermittent food in the induction of attack in pigeons

The present experiments evaluated whether transitions in reinforcer probability are necessary to induce attack in pigeons. In Experiment I, three of six pigeons exposed to response-contingent constant-probability food schedules and a photograph of a conspecific as a target exhibited sustained postreinforcement attack on the target. The postreinforcement pattern of attack developed over the course of the experiment and was accompanied by a reduction in the rate of postreinforcement key pecking and an increase in the postreinforcement pause in key pecking. These effects on key pecking resulted in unprogrammed variations in the probability of reinforcement which may have been responsible for the induction of attack. In Experiment II, the attack-inducing properties of a constant-probability response-independent food schedule were compared to a periodic food schedule matched for overall rate of food delivery and to a no-food condition. In addition to attack, the spatial location of the subjects was monitored during each interfood interval. The periodic and aperiodic food schedules generated very different patterns of spatial location. Postfood attack was induced by both food schedules, although the constant-probability schedule induced attack in fewer birds. The no-food condition was not effective in inducing attack in any birds. These experiments indicate that intermittent food schedules without reductions in reinforcer probability are sufficient to induce attack in some pigeons, although not as effective as schedules with transitions in reinforcer probability.     

Effects of search cost on foraging and feeding: a three-component chain analysis

An experiment determined whether pigeons minimize number of key pecks per food delivery and maintain their baseline intake of food while key pecking on a three-component chain schedule. Pigeons at either 80% or 100% body weight obtained all their food during baseline and contingency sessions. During baseline sessions, pecks on the left and center keys had no consequences; each peck on the right key activated the feeder. During contingency sessions, pigeons key pecked on a three-component chain schedule simulating components of a foraging chain. In the search component either 3, 9 or 15 key pecks (varied parametrically across blocks of sessions) on the left key produced a stimulus on the middle key, indicating an encounter with either the low-cost prey (3 key pecks) or an equally probable high-cost prey (21 key pecks). In the procurement component the pigeon pecked either: (a) the left key once, thus returning to the search component, or (b) the middle key either 3 or 21 times, which activated the right response key. In the handling component one peck on the right key operated the feeder. The pigeons always procured the low-cost prey and minimized the number of key pecks per hopper by procuring the high-cost prey when the search-cost ratio was high (15 key pecks) but not when it was low (3 key pecks). All pigeons maintained their baselines of eating during contingency sessions by key pecking more frequently and eating more efficiently. The 80% body-weight birds produced higher overall rates of key pecking and eating. These results have implications for ecological theories of optimal foraging and for psychological theories of learned performance.     

Reinforcement of behavioral patterns: shaping a scallop

Temporal patterns of key pecking by pigeons were shaped by a schedule in which the delivery of food was contingent upon a measure of the overall extent to which the temporal pattern of behavior within a 5-sec trial conformed to a required pattern. This pattern approximated a constant rate of change in the rate of key pecking throughout the 5-sec trial. In comparison with behavior maintained by a classical fixed-interval 5-sec schedule, the new schedule controlled a better approximation to a "scallop" within individual trials and greatly reduced intersubject variability. These results are consistent with the view that the delivery of a reinforcer after a behavioral pattern a few seconds in duration may strengthen the entire pattern as a unit, or operant. The response topography contiguous with reinforcement may be a negligible fraction of the strengthened operant. One implication of this view is that mean response rate for such brief responses as key pecks and lever presses is a byproduct of whatever patterns are strengthened, and generally will not reveal fundamental controlling relationships, whenever a reinforcer is not contiguous with all the behavior on which it is contingent.     

The Modulation of Operant Variation by the Probability, Magnitude, and Delay of Reinforcement

Recent studies have demonstrated that the expectation of reward delivery has an inverse relationship with operant behavioral variation (e.g., Stahlman, Roberts, & Blaisdell, 2010). Research thus far has largely focused on one aspect of reinforcement – the likelihood of food delivery. In two experiments with pigeons, we examined the effect of two other aspects of reinforcement: the magnitude of the reward and the temporal delay between the operant response and outcome delivery. In the first experiment, we found that a large reward magnitude resulted in reduced spatiotemporal variation in pigeons’ pecking behavior. In the second experiment, we found that a 4-s delay between response-dependent trial termination and reward delivery increased variation in behavior. These results indicate that multiple dimensions of the reinforcer modulate operant response variation.     

THE BASIS OF SUPERSTITIOUS BEHAVIOR: CHANCE CONTINGENCY,STIMULUS SUBSTITUTION,OR APPETITIVE BEHAVIOR?

This research examined three explanations for the "superstitious" behavior of pigeons under frequent fixed-time delivery of food: accidental response-reward contingency, stimulus substitution, and elicited species-typical appetitive behavior. The behavior observed in these studies consisted of occasional postfood locomotion away from the food hopper, and a predominant pattern of activity directed toward the hopper wall (wall-directed behavior), including approaching, stepping side to side, scratching with the feet, bumping with the breast, pendulum movements of the extended neck, and head bobbing, though not pecking. The consistency of these behavior patterns argued against explanation by accidental response contingencies, and the complexity of behavior was incompatible with the classic stimulus-substitution account. These studies also showed that: (1) response contingencies and prior stimulus experience can modify wall-directed behavior, but within definable limits; (2) pecking sometimes can be obtained in birds of specific strains, and by providing extended training; (3) placing the hopper in the floor at the center of a large chamber replaces wall-directed behavior with circling in a manner that resembles ground foraging for food. We conclude that superstitious behavior under periodic delivery of food probably develops from components of species-typical patterns of appetitive behavior related to feeding. These patterns are elicited by a combination of frequent food presentations and the supporting stimuli present in the environment.     

Choosing among multiple alternatives: Relative and overall reinforcer rates

Choice behavior among two alternatives has been widely researched, but fewer studies have examined the effect of multiple (more than two) alternatives on choice. Two experiments investigated whether changing the overall reinforcer rate affected preference among three and four concurrently scheduled alternatives. Experiment 1 trained six pigeons on concurrent schedules with three alternatives available simultaneously. These alternatives arranged reinforcers in a ratio of 9:3:1 with the configuration counterbalanced across pigeons. The overall rate of reinforcement was varied across conditions. Preference between the pair of keys arranging the 9:3 reinforcer ratio was less extreme than the pair arranging the 3:1 reinforcer ratio regardless of overall reinforcer rate. This difference was attributable to the richer alternative receiving fewer responses per reinforcer than the other alternatives. Experiment 2 trained pigeons on concurrent schedules with four alternatives available simultaneously. These alternatives arranged reinforcers in a ratio of 8:4:2:1, and the overall reinforcer rate was varied. Next, two of the alternatives were put into extinction and the random interval duration was changed from 60 s to 5 s. The ratio of absolute response rates was independent of interval length across all conditions. In both experiments, an analysis of sequences of visits following each reinforcer showed that the pigeons typically made their first response to the richer alternative irrespective of which alternative was just reinforced. Performance on these three‐ and four‐alternative concurrent schedules is not easily extrapolated from corresponding research using two‐alternative concurrent schedules.     

Effects of differences between stimuli, responses, and reinforcer rates on conditional discrimination performance

In a discrete-trial conditional discrimination procedure, 4 pigeons obtained food reinforcers by pecking a key with a short latency on trials signaled by one stimulus and by pecking the same key with a long latency on trials signaled by a second stimulus. The physical difference between the two stimuli and the temporal separation between the latency values required for reinforcement were varied factorially over four sets of conditions, and the ratio of reinforcer rates for short and long latencies was varied within each set of conditions. Stimulus discrimination varied directly with both stimulus and response differences and was unaffected by the reinforcer ratio. Sensitivity to reinforcement, estimated by generalized-matching-law fits to the data within each set of conditions, varied directly with the response difference but inversely with the stimulus difference arranged between sets of conditions. Because variations in stimulus differences, response differences, and reinforcer differences did not have equivalent effects, these findings question the functional equivalence of the three terms of the discriminated operant: antecedent stimuli, behavior, and consequences.     

Food-deprivation level alters the effects of morphine on pigeons'' key pecking.

Four pigeons pecked response keys under a multiple fixed-ratio 30 fixed-interval 5-min schedule of food presentation. Components alternated separated by 15-s timeouts; each was presented six times. Pigeons were maintained at 70%, 85%, and greater than 90% of their free-feeding weights across experimental conditions. When response rates were stable, the effects of morphine (0.56 to 10.0 mg/kg) and saline were investigated. Morphine reduced response rates in a dose-dependent manner under the fixed-ratio schedule and at high doses under the fixed-interval schedule. In some cases, low doses of morphine increased rates under the fixed-interval schedule. When pigeons were less food deprived, reductions in pecking rates occurred at lower doses under both schedules for 3 of 4 birds compared to when they were more food deprived. When pigeons were more food deprived, low doses of morphine increased rates of pecking in the initial portions of fixed intervals by a greater magnitude. Thus, food-deprivation levels altered both the rate-decreasing and rate-increasing effects of morphine. These effects may share a common mechanism with increased locomotor activity produced by drugs and with increased drug self-administration under conditions of more severe food deprivation.     

Effects of signaled and unsignaled alternative reinforcement on persistence and relapse in children and pigeons

Three experiments explored the impact of different reinforcer rates for alternative behavior (DRA) on the suppression and post‐DRA relapse of target behavior, and the persistence of alternative behavior. All experiments arranged baseline, intervention with extinction of target behavior concurrently with DRA, and post‐treatment tests of resurgence or reinstatement, in two‐ or three‐component multiple schedules. Experiment 1, with pigeons, arranged high or low baseline reinforcer rates; both rich and lean DRA schedules reduced target behavior to low levels. When DRA was discontinued, the magnitude of relapse depended on both baseline reinforcer rate and the rate of DRA. Experiment 2, with children exhibiting problem behaviors, arranged an intermediate baseline reinforcer rate and rich or lean signaled DRA. During treatment, both rich and lean DRA rapidly reduced problem behavior to low levels, but post‐treatment relapse was generally greater in the DRA‐rich than the DRA‐lean component. Experiment 3, with pigeons, repeated the low‐baseline condition of Experiment 1 with signaled DRA as in Experiment 2. Target behavior decreased to intermediate levels in both DRA‐rich and DRA‐lean components. Relapse, when it occurred, was directly related to DRA reinforcer rate as in Experiment 2. The post‐treatment persistence of alternative behavior was greater in the DRA‐rich component in Experiment 1, whereas it was the same or greater in the signaled‐DRA‐lean component in Experiments 2 and 3. Thus, infrequent signaled DRA may be optimal for effective clinical treatment.     

Cocaine's effects on food-reinforced pecking in pigeons depend on food-deprivation level.

Four pigeons deprived to 80% of their laboratory free-feeding weights pecked keys under a multiple fixed-ratio 30 fixed-interval 5-min schedule of food presentation. Components alternated strictly with 15-s timeouts separating them; each was presented six times. When rates of pecking were stable, 2 pigeons' weights were reduced to 70%, and the other 2 pigeons' weights were increased to 82.5% to 85% of free-feeding levels. Cocaine (1.0, 3.0, 5.6, and 10.0 mg/kg and saline) was administered 5 min prior to sessions. When each dose had been tested twice, pigeons' weights were adjusted to the level that they had not yet experienced, and cocaine was tested again. Cocaine reduced response rates in a dose-dependent manner under the fixed-ratio schedule and under the fixed-interval schedule at high doses, and increased rates under the fixed-interval schedule at low low doses. Reductions in pecking rates occurred at lower doses under both schedules in 3 of 4 pigeons when they were less food deprived compared to when they were more food deprived. Low doses of cocaine increased low baseline rates of pecking in the initial portions of the fixed-interval schedules by a greater magnitude when pigeons were more food deprived. Thus, food-deprivation levels altered both the rate-decreasing and rate-increasing effects of cocaine. The implications of these results for the mechanisms by which food deprivation increases cocaine self-administration and for the dependence of cocaine's effects on the baseline strength of operant behavior are discussed.     

Stimulus-food relations and free-operant postponement of timeout from response-independent food presentation

Grain was briefly presented to food-deprived pigeons intermittently and response-independently except during signaled timeouts. During Experiment 1, key pecks postponed the next timeout for a specified interval. Rates of pecking during time in were inversely related to the length of time pecking postponed the next timeout. Response-independent presentation of temporal distributions of timeouts exactly matched to a preceding postponement condition decreased pecking rates during Experiment 2. These results indicate that key pecking of pigeons can be controlled by response-dependent postponement of timeout, but that responses elicited by stimulus-reinforcer relations inherent in timeout-postponement procedures may substantially modify rates and patterns of pecking.     

Secondary reinforcement and rate of primary reinforcement

Four pigeons were trained to peck at either of two response-keys. Pecking at either key occasionally produced a secondary reinforcer. Then, in the presence of the secondary reinforcer, further pecking occasionally produced the primary reinforcer, food. The relative rate at which each pigeon pecked to obtain a secondary reinforcer equalled the relative rate of primary reinforcement in its presence.     

Multiple schedules,off‐baseline reinforcement shifts,and resistance to extinction

Resistance to extinction in a target multiple‐schedule component varies inversely with the rate of reinforcement arranged in an alternative component during baseline. The present experiment asked whether changing the reinforcer rate in an alternative component would impact extinction of target component responding if those changes occurred in an off‐baseline phase during which the target component was never experienced. Pigeons' key pecking was studied in three types of conditions, and each condition consisted of three phases. In Phase 1, pecking produced food in the target and alternative components of a multiple schedule according to variable‐interval 60‐s schedules. In Phase 2, the alternative‐component stimulus was presented alone in a single schedule. Pecking during this phase produced the same reinforcer rate as in baseline in the Control condition, a higher rate of food (variable‐interval 15 s) in the High‐Rate condition, or was extinguished in the Extinction condition. Extinction of target‐ and alternative‐component key pecking then was assessed in a multiple schedule during the final phase of each condition. Resistance to extinction of target‐component key pecking was the same between the Control and High‐Rate conditions but lower in the Extinction condition. These findings are discussed in terms of discrimination and generalization processes.     

Self-control in pigeons under the Mischel paradigm

Walter Mischel studied self-control in preschool children in the following manner: if the child waited for an interval to end, he or she received the more preferred of two reinforcers; if the child responded to terminate the interval by ringing a bell, the less preferred reinforcer was given. We used an analogous procedure to study self-control in pigeons: if the bird waited for a trial to end, it received the more preferred reinforcer; if the bird terminated the trial by pecking a key, the less preferred reinforcer was given. We explored the effects on self-control of a number of variables analogous to those studied by Mischel and co-workers, e.g., presence versus absence of reinforcers, of alternative responses, and of stimuli during the wait interval; prior experience of the subjects; and test paradigm. The results obtained with pigeons paralleled the results obtained by Mischel with human children.     

Early extinction effects following intermittent reinforcement: Little evidence of extinction bursts

The occurrence of extinction bursts—transient increases in response rate in excess of those observed in baseline during the period immediately following discontinuation of reinforcement of a response—was examined. In Experiment 1, key pecking of pigeons was reinforced according to a multiple schedule in which a variable-ratio schedule alternated with an interval schedule in which the reinforcers were yoked to the preceding variable-ratio component. In Experiments 2 and 3, rats were screened such that the lever-press response rates of different rats maintained by variable-interval schedules were either relatively high or relatively low. Following these baseline conditions, in Experiments 1 and 2 responding was extinguished by eliminating the food reinforcer and in Experiment 3 by removing the response–reinforcer dependency. Responses immediately following extinction implementation were examined. Response increases relative to baseline during the first 20 min of a 324.75-min extinction session (Experiment 1) or during the first 30-min extinction session (Experiments 2 and 3) were rare and unsystematic. The results (a) reinforce earlier meta-analyses concluding that extinction bursts may be a less ubiquitous early effect of extinction than has been suggested and (b) invite further experimentation to establish their generality as a function of preceding reinforcement conditions.     

DOES SENSITIVITY TO MAGNITUDE DEPEND ON THE TEMPORAL DISTRIBUTION OF REINFORCEMENT?

Our research addressed the question of whether sensitivity to relative reinforcer magnitude in concurrent chains depends on the distribution of reinforcer delays when the terminal-link schedules are equal. In Experiment 1, 12 pigeons responded in a two-component procedure. In both components, the initial links were concurrent variable-interval 40-s variable-interval 40-s, and the terminal links were both 20-s interval schedules in which responses were reinforced by either 4-s of grain in one, or 2-s of grain in the other. The only difference between the components was whether the terminal-link schedules were fixed interval or variable intervals. For all subjects, the relative rate of responding in the initial links for the terminal link that produced the 4-s reinforcer was greater when the terminal links were fixed-interval schedules than when they were variable-interval schedules. This result is contrary to the prediction of Grace's (1994) contextual choice model, but is consistent with both Mazur's (2001) hyperbolic value-added model and Killeen's (1985) incentive theory. In Experiment 2, 4 pigeons responded in a concurrent-chains procedure in which 4-s or 2-s reinforcers were provided independently of responding according to equal fixed-time or mixed-time schedules. Preference for the 4-s reinforcer increased as the variability of the intervals comprising the mixed-time schedules was decreased. Generalized-matching sensitivity of initial-link response allocation to relative reinforcer magnitude was proportional to the geometric mean of the terminal-link delays.     

Effects of reinforcement amount on attack induced under a fixed-interval schedule in pigeons

Key pecking by pigeons was maintained on a chained fixed-interval 4-min (12-min for 1 subject) fixed-ratio 1 schedule of food presentation. Attacks toward a restrained and protected conspecific were recorded. In the first experiment, the amount of food presented per interval was manipulated across phases by varying the number of fixed ratios required in the terminal link of the chain. Measures of attack for all pigeons during the fixed-interval component increased monotonically as a function of food amount. In the second experiment, two different food amounts alternated within each experimental session under a multiple schedule. For both pigeons in this experiment, measures of attack were higher during the component that delivered the larger food amount per interval. The differences in levels of attack induced by the two food amounts in Experiment 2, however, were not as great as in Experiment 1; apparently this was because attack during the first interval of each component was controlled in part (P-5626) or entirely (P-7848) by the reinforcement amount delivered at the end of the previous component. Attack was also a function of the location of the interfood interval within the session. For both pigeons, attack tended to decrease throughout the session. The results of both experiments suggest that attack is an increasing function of reinforcement amount under fixed-interval schedules, but that this function may be influenced by the manner in which reinforcement amount is manipulated, by the duration of the interfood interval, and by the location of the interfood interval within the experimental session. In general, these results are compatible with theories of induced attack and other schedule-induced behavior that emphasize aversive after-effects of reinforcement presentation.     

Contingency tracking during unsignaled delayed reinforcement: Effects of delay duration and d‐amphetamine

In two experiments, the role of the response–reinforcer relation in maintaining low‐rate responding under unsignaled delay conditions was investigated. In both experiments pecking by pigeons on one response key, denoted the relevant key, was reinforced under an unsignaled delay‐of‐reinforcement procedure (defined as tandem variable‐interval (VI) differential‐reinforcement‐of‐other behavior [DRO] schedule). Responding on a second key, denoted the irrelevant key, had no programmed consequences. Between sessions, the location of the relevant key varied (after one, two, or three sessions) pseudorandomly. In Experiment 1, the delay (DRO) duration was manipulated parametrically. Overall, proportional relevant‐key response rates (relevant‐key response rates / [relevant‐key response rates + irrelevant key response rates]) increased across 3‐session sequences in which the relevant key remained in the same location and decreased as the DRO duration was changed systematically (2, 5, and 10 s). In Experiment 2, acute administration of d‐amphetamine increased proportional relevant‐key response rates during 1‐day sequences for only the DRO 5‐s duration, and results over 3‐day sequences, once a discrimination had already been established, were inconsistent. Results support that the response–reinforcer relation is the primary determinant of responding, and such discriminations are relatively resistant to disruption or potentiation by behaviorally active doses of d‐amphetamine.     

Matching theory and induction explain operant performance

Matching theory is a general framework for understanding allocation of behavior among activities. It applies to choice in concurrent schedules and was extended to single schedules by assuming that other unrecorded behavior competes with operant behavior. Baum and Davison (2014) found that the competing activities apparently are induced by the “reinforcers” (phylogenetically important events, e.g., food) according to power functions. Combined with power-function induction, matching theory provides new equations with greater explanatory power. Four pigeons were exposed to conditions in which 7 different schedules of food delivery were presented within each experimental session. We replicated earlier results with variable-interval schedules: (a) a negatively accelerated increase of peck rate as food rate increased in the low range of food rates; (b) an upturn in pecking at higher rates; and (c) a downturn in pecking at extremely high food rates. When the contingency between pecking and food was removed, the food continued to induce pecking, even after 20 sessions with no contingency. A ratio schedule inserted in place of 1 variable-interval schedule maintained peck rates comparable to peck rates maintained by short interval schedules. We explained the results by fitting equations that combined matching theory, competition, and induction.     

Escape from serial stimuli leading to food

If the functional relations governing the strength of a conditioned reinforcer correspond to those obtained with other Pavlovian procedures (e.g., Kaplan, 1984), the termination of stimuli appearing early in the interval between successive food deliveries should be reinforcing. During initial training we presented four key colors, followed by food, in a recurrent sequence to each of 6 pigeons. This established a baseline level of autoshaped pecking. In later sessions, we terminated each of these colors or only the first color for a brief period following each peck, replacing the original color with a standard substitute to avoid darkening the key. Pecking decreased in the presence of the last color in the sequence but increased in the presence of the first. In accord with contemporary models of Pavlovian conditioning, these and other data suggest that the behavioral effects of stimuli in a chain may be better understood in terms of what each stimulus predicts, as measured by relative time to the terminal reinforcer, than in the exclusively positive terms of the traditional formulation (Skinner, 1938). The same model may also account for the initial pause under fixed-interval and fixed-ratio schedules of reinforcement.