Leaving patches: An investigation of a laboratory analogue

Five pigeons were trained on a procedure that has been used as a laboratory analogue to natural patch residence. Trials commenced with two responses available. One of these might provide a reinforcer if the patch was a prey patch; the other ended the residence time in the patch and, after a fixed travel time in blackout, produced another patch that might or might not provide a reinforcer. Patch residence also ended, and was followed by the same travel time, after a reinforcer was obtained or after a fixed maximum time was spent in the patch. The dependent variable was patch residence time, from the commencement of the patch to the time at which the subject emitted a response to exit from the patch or until the maximum patch residence time had elapsed. In Parts 1 to 3, the duration of the imposed travel time was varied from 0.25 to 16 s at three different probabilities (.05, .1, and .2) of food per second (λ) in prey patches. As reported in previous research, both increasing travel time and decreasing probabilities of reinforcers per second increased patch residence time. In Parts 4 to 7, the probability of prey trials (ρ) was varied in an irregular order from .1, through .2, .5, and .7, to .9 for different combinations of λ and travel time. Respectively, these were in Part 4, .05 per second and 0.25 s; in Part 5, .05 per second and 16 s; in Part 6, .2 per second and 0.25 s; and in Part 7, .2 per second and 16 s. A previously offered model, based on optimization assumptions, substantially and consistently underpredicted patch residence time. However, a modification of that model, which assumes that the subjects could not accurately discriminate the residence time that provided the minimum interreinforcer interval, described the data well. The same model also described previously reported residence times in a different species with a uniform distribution of prey-arrival times.     

Conjunctive schedules of reinforcement II: response requirements and stimulus effects

Responding of three pigeons was maintained under conjunctive fixed-ratio, fixed-interval schedules where a key peck produced food after both schedule requirements were completed. The individual schedule requirements were then successively removed and reinstated with responding maintained under the following conditions: conjunctive fixed-ratio, fixed-time; fixed-time; and fixed-interval schedules. Patterns of responding changed in accord with the successive removal of the schedule requirements. Compared to the conjunctive fixed-ratio, fixed-interval schedule, pause duration increased and response rate decreased under conjunctive fixed-ratio, fixed-time schedules and under fixed-time schedules alone. Overall mean rates of responding were highest and pause duration lowest under fixed-interval schedules. When changes in the keylight colors were correlated with completion of the fixed-ratio, the end of the fixed-interval, or both of these conditions, the pattern of responding was modified and indicated a greater degree of control by the individual schedules. Although two birds showed large increases in interreinforcement time when they were initially exposed to the conjunctive schedule, when responding stabilized this measure was largely invariant for all birds across most schedule conditions.     

Reinforcement by an imprinting stimulus versus water on simple schedules in ducklings

Ducklings (5 to 28 days old) were trained to peck a pole on fixed-ratio, fixed-interval, and multiple schedules using brief presentation of an imprinting stimulus as the response-contingent event. Other ducklings of the same age were trained similarly except that reinforcement consisted of access to water. With water reinforcement the typical fixed-ratio (“break-run”), fixed-interval (“scallop”), and multiple schedule response patterns were readily established and consistently maintained. With the imprinting stimulus these schedule effects were inconsistent in some subjects and virtually nonexistent in others, despite extended training. Schedule control with the imprinting stimulus was not improved by the use of a reinforcement signaling procedure which enhances responding reinforced by electrical brain stimulation on intermittent schedules. However, the overall rates of responding and the extinction functions generated after reinforcement with water versus the imprinting stimulus were comparable. These findings imply that control by temporal and discriminative stimuli may be relatively weak when a young organism's behavior is reinforced by presentation of an imprinting stimulus.     

Residence time and choice in concurrent foraging schedules

Five pigeons were trained on a concurrent-schedule analogue of the “some patches are empty” procedure. Two concurrently available alternatives were arranged on a single response key and were signaled by red and green keylights. A subject could travel between these alternatives by responding on a second yellow “switching” key. Following a changeover to a patch, there was a probability (p) that a single reinforcer would be available on that alternative for a response after a time determined by the value of λ, a probability of reinforcement per second. The overall scheduling of reinforcers on the two alternatives was arranged nonindependently, and the available alternative was switched after each reinforcer. In Part 1 of the experiment, the probabilities of reinforcement, ρ_red and ρ_green, were equal on the two alternatives, and the arranged arrival rates of reinforcers, λ_red and λ_green, were varied across conditions. In Part 2, the reinforcer arrival times were arranged to be equal, and the reinforcer probabilities were varied across conditions. In Part 3, both parameters were varied. The results replicated those seen in studies that have investigated time allocation in a single patch: Both response and time allocation to an alternative increased with decreasing values of λ and with increasing values of ρ, and residence times were consistently greater than those that would maximize obtained reinforcer rates. Furthermore, both response- and time-allocation ratios undermatched mean reinforcer-arrival time and reinforcer-frequency ratios.     

The role of contingencies and "principles of behavioral variation" in pigeons' pecking

Staddon and Simmelhag's proposal that behavior is produced by “principles of behavioral variation” instead of contingencies of reinforcement was tested in two experiments. In the first experiment pigeons were exposed to either a fixed-interval schedule of response-contingent reinforcement, an autoshaping schedule of stimulus-contingent reinforcement, or a fixed-time schedule of noncontingent reinforcement. Pigeons exposed to contingent reinforcement came to peck more rapidly than those exposed to noncontingent reinforcement. Staddon and Simmelhag's “principles of behavioral variation” included the proposal that patterns (interim and terminal) were a function of momentary probability of reinforcement. In the second experiment pigeons were exposed to either a fixed-time or a random-time schedule of noncontingent reinforcement. Pecking showed a constant frequency of occurrence over postfood time on the random-time schedule. Most behavior showed patterns on the fixed-time schedule that differed in overall shape (i.e., interim versus terminal) from those shown on the random-time schedule. It was concluded that both the momentary probability of reinforcement and postfood time can affect patterning.     

Leaving patches: effects of economy, deprivation, and session duration

Three pigeons pecked keys for food reinforcers in a laboratory analogue of foraging in patches. Half the patches contained food (were prey patches). In prey patches, pecks to one key occasionally produced a reinforcer, followed by a fixed travel time and then the start of a new patch. Pecks to another key were exit responses, and immediately produced travel time and then a new patch. Travel time was varied from 0.25 to 16 s at each of three session durations: 1, 4, and 23.5 hr. This part of the experiment arranged a closed economy, in that the only source of food was reinforcers obtained in prey patches. In another part, food deprivation was manipulated by varying postsession feeding so as to maintain the subjects' body weights at percentages ranging from 85% to 95% of their ad lib weights, in 1-hr sessions with a travel time of 12 s. This was an open economy. Patch residence time, defined as the time between the start of a patch and an exit response, increased with increasing travel time, and consistently exceeded times predicted by an optimal foraging model, supporting previously published results. However, residence times also increased with increasing session duration and, in longer sessions, consistently exceeded previously reported residence times in comparable open-economy conditions. Residence times were not systematically affected by deprivation levels. In sum, the results show that the long residence times obtained in long closed-economy sessions should probably be attributed to session duration rather than to economy or deprivation. This conclusion is hard to reconcile with previous interpretations of longer-than-optimal residence times but is consistent with, in economic terms, a predicted shift in consumption towards a preferred commodity when income is increased.     

Choice, changeover, and travel

Since foraging in nature can be viewed as instrumental behavior, choice between sources of food, known as “patches,” can be viewed as choice between instrumental response alternatives. Whereas the travel required to change alternatives deters changeover in nature, the changeover delay (COD) usually deters changeover in the laboratory. In this experiment, pigeons were exposed to laboratory choice situations, concurrent variable-interval schedules, that were standard except for the introduction of a travel requirement for changeover. As the travel requirement increased, rate of changeover decreased and preference for a favored alternative strengthened. When the travel requirement was small, the relations between choice and relative reinforcement revealed the usual tendencies toward matching and undermatching. When the travel requirement was large, strong overmatching occurred. These results, together with those from experiments in which changeover was deterred by punishment or a fixed-ratio requirement, deviate from the matching law, even when a correction is made for cost of changeover. If one accepted an argument that the COD is analogous to travel, the results suggest that the norm in choice relations would be overmatching. This overmatching, however, might only be the sign of an underlying strategy approximating optimization.     

Competitive fixed-interval performance in humans

Two persons responded in the same session in separate cubicles, but under a single schedule of reinforcement. Each time reinforcement was programmed, only the first response to occur, that is, the response of only one of the subjects, was reinforced. “Competitive” behavior that developed under these conditions was examined in three experiments. In Experiment 1 subjects responded under fixed-interval (FI) 30-s, 60-s, and 90-s schedules of reinforcement. Under the competition condition, relative to baseline conditions, the response rates were higher and the pattern was “break-and-run.” In Experiment 2, subjects were exposed first to a conventional FI schedule and then to an FI competition schedule. Next, they were trained to respond under either a differential-reinforcement-of-low-rate (DRL) or fixed-ratio (FR) schedule, and finally, the initial FI competition condition was reinstated. In this second exposure to the FI competition procedure, DRL subjects responded at lower rates than were emitted during the initial exposure to that condition and FR subjects responded at higher rates. For all subjects, however, responding gradually returned to the break-and-run pattern that had occurred during the first FI competition condition. Experiment 3 assessed potential variables contributing to the effects of the competitive FI contingencies during Experiments 1 and 2. Subjects were exposed to FI schedules where (a) probability of reinforcement at completion of each fixed interval was varied, or (b) a limited hold was in effect for reinforcement. Only under the limited hold was responding similar to that observed in previous experiments.     

Operant and nonoperant vocal responding in the mynah: Complex schedule control and deprivation-induced responding

Several recent studies have been concerned with operant responses that are also affected by nonoperant factors, (e.g., biological constraints, innate behavior patterns, respondent processes). The major reason for studying mynah vocal responding concerned the special relation of avian vocalizations to nonoperant emotional and reflexive systems. The research strategy was to evaluate operant and nonoperant control by comparing the schedule control obtained with the vocal response to that characteristic of the motor responses of other animals. We selected single, multiple, and chain schedules that ordinarily produce disparate response rates at predictable times. In multiple schedules with one component where vocal responding (“Awk”) was reinforced with food (fixed-ratio or fixed-interval schedule) and one where the absence of vocal responding was reinforced (differential reinforcement of other behavior), response rates never exceeded 15 responses per minute, but clear schedule differences developed in response rate and pause time. Nonoperant vocal responding was evident when responding endured across 50 extinction sessions at 25% to 40% of the rate during reinforcement. The “enduring extinction responding” was largely deprivation induced, because the operant-level of naive mynahs under food deprivation was comparable in magnitude, but without deprivation the operant level was much lower. Food deprivation can induce vocal responding, but the relatively precise schedule control indicated that operant contingencies predominate when they are introduced.     

The effects of reinforcement frequency and response requirements on the maintenance of behavior.

Six rats responded under fixed-interval and tandem fixed-interval fixed-ratio schedules of food reinforcement. Basic fixed-interval schedules alternated over experimental conditions with tandem fixed-interval fixed-ratio schedules with the same fixed-interval value. Fixed-interval length was varied within subjects over pairs of experimental conditions; the ratio requirement of the tandem schedules was varied across subjects. For both subjects with a ratio requirement of 10, overall response rates and running response rates typically were higher under the tandem schedules than under the corresponding basic fixed-interval schedules. For all subjects with ratio requirements of 30 or 60, overall response rates and running response rates were higher under the tandem schedules than under the corresponding basic fixed-interval schedules only with relatively short fixed intervals. At longer fixed intervals, higher overall response rates and running rates were maintained by the basic fixed-interval schedules than by the tandem schedules. These findings support Zeiler and Buchman's (1979) reinforcement-theory account of response strength as an increasing monotonic function of both the response requirement and reinforcement frequency. Small response requirements added in tandem to fixed-interval schedules have little effect on reinforcement frequency and so their net effect is to enhance responding. Larger response requirements reduce reinforcement frequency more substantially; therefore their net effect depends on the length of the fixed interval, which limits overall reinforcement frequency. At the longest fixed intervals studied in the present experiment, reinforcement frequency under the tandem schedules was sufficiently low that responding weakened or ceased altogether.     

Formation of transitivity in conditional matching to sample by pigeons

Four homing pigeons were trained over 5 months in a zero-delay, “arbitrary” matching-to-sample procedure with sample and comparison stimuli presented on any of three response keys. Birds were also required to complete a fixed-ratio 10 requirement on both sample and comparison stimuli to terminate their presentation. The procedure resulted in the establishment of relations that were not specifically trained and that can be characterized by the property of transitivity in a stimulus equivalence context. This result was in contrast with the findings obtained from most previous research with nonhuman subjects.     

Free-operant choice behavior: A molecular analysis

Pigeons' pecks to two concurrent initial-link stimuli occasionally produced one of two mutually exclusive terminal links. Further responding to the terminal-link stimulus produced food under fixed-interval or fixed-ratio schedules. In such concurrent chained schedules, investigators rarely use a changeover delay to control superstitious switching, although it is customary to use a changeover delay in simple concurrent schedules in which choice responses produce food directly. When terminal-link fixed-interval schedules were equal or similar in duration and no changeover delay was employed, conditional probabilities of choice for a schedule were found to be lower if the last choice was for that schedule than if the last choice was for the other schedule (“switching dependency”). Imposition of a changeover delay with equal or unequal terminal links produced the opposite pattern: conditional probabilities of choice for a schedule were higher if the last choice was for that schedule than if the last choice was for the other schedule. Turning off all chamber lights during the changeover delay interval attenuated these “repetition dependencies.” The results indicate that excessive switching can complicate the interpretation of data from concurrent chains much as from simple concurrent schedules, and that using blackouts to control switching may be preferable to using a changeover delay.     

Responding under sequence schedules of electric shock presentation

Lever pressing by squirrel monkeys was examined under second-order schedules of electric shock presentation in which different discriminative stimuli were associated with consecutive components (sequence schedules). Components were always two-minute fixed-interval schedules, and three different overall schedules were studied. Under an overall eight-minute fixed-interval schedule, the first component completion after at least eight minutes had elapsed produced electric shock. The number of components actually completed ranged from one to four; thus, different discriminative stimuli were occasionally associated with electric shock presentation. Under an overall “yoked” variable-ratio schedule, electric shock was presented after completion of a variable number of components; the required number and the distribution of components were matched to those obtained under the overall eight-minute fixed-interval schedule. Under an overall fixed-ratio schedule, electric shock was presented after completion of four components (chained schedule). Under all three sequence schedules, responding in early components was characterized by a pause followed by a single response after the end of the two-minute interval; responding in later components was characterized by a shorter pause followed by positively accelerated responding. Manipulation of overall schedules of shock presentation in these complex behavioral situations produced changes in responding comparable to those ordinarily obtained after similar manipulation of dependencies under both single and second-order schedules of food presentation. These experiments extend the range of conditions and levels of complexity under which responding can be maintained by presentation of electric shock.     

RESPONDING MAINTAINED UNDER INTERMITTENT SCHEDULES OF ELECTRIC-SHOCK PRESENTATION: “SAFETY” OR SCHEDULE EFFCTS?

Four experiments were conducted in which lever pressing by squirrel monkeys was maintained under multiple, mixed, or chained schedules of electric-shock presentation. In the first two experiments, a multiple schedule was employed in which a fixed-interval schedule of shock presentation alternated with a signaled two-minute component. Initially, no events were scheduled during the two-minute component (a safety period). In the first experiment, the safety period was “degraded” by introducing and systematically increasing the frequency of periodic shocks presented during that component. In the second experiment, the proportion of overall safe time to unsafe time was decreased by decreasing the value of the fixed-interval schedule while holding constant shock frequency during the two-minute component. In the third experiment, the overall arrangement was changed from a multiple to a mixed schedule in an attempt to determine whether fixed-interval responding would be maintained when a single exteroceptive stimulus was associated with both components. In the fourth experiment, the overall arrangement was changed from a multiple to a chained schedule in an effort to determine whether fixed-interval responding would be maintained when its consequence was presentation of a signaled “unsafe” period. Fixed-interval responding was well maintained under all experimental conditions; the varied relationships obtained lend more support to conceptualizations of shock-maintained behavior as exemplifying schedule-controlled behavior than to suggestions that such behavior may be readily accounted for by “safety theory.”     

Alternative fixed-ratio fixed-interval schedules of reinforcement

Five rats were trained under alternative fixed-ratio fixed-interval schedules, in which food reinforcement was provided for the completion of either a fixed-ratio or a fixed-interval requirement, whichever was met first. Overall response rate and running rate (the rate of responding after the postreinforcement pause) decreased for all subjects as the fixed-ratio value increased. As the proportion of reinforcements obtained from the fixed-ratio component increased and the alternative schedule approached a simple fixed ratio, overall response rate and running rate both increased; conversely, as the proportion of reinforcements obtained from the fixed-interval component increased and the alternative schedule approached a simple fixed interval, response rates decreased. Postreinforcement pause length increased linearly as the average time between reinforcements increased, regardless of the schedule parameters. A break-run pattern of responding was predominant at low- and medium-valued fixed ratios. All subjects displayed at least occasional positively accelerated responding within interreinforcement intervals at higher fixed-ratio values.     

Conditioning history and the control of human avoidance and escape responding

Inter-subject differences in response rates under free-operant avoidance and escape schedules are commonly obtained from humans. Data are presented which demonstrate that such differences can be controlled experimentally by giving subjects different conditioning histories. Subjects given a fixed-ratio history avoided and/or escaped from “point-loss periods” at higher rates than subjects given a differential-reinforcement-of-low-rates history. History related differences in response rates were maintained during 40 hr of escape responding. For low-rates history subjects, response rates under escape contingencies decreased as the rate of point-loss periods decreased.     

A comparison of delays and ratio requirements in self-control choice.

In a discrete-trial procedure, pigeons could choose between 2-s and 6-s access to grain by making a single key peck. In Phase 1, the pigeons obtained both reinforcers by responding on fixed-ratio schedules. In Phase 2, they received both reinforcers after simple delays, arranged by fixed-time schedules, during which no responses were required. In Phase 3, the 2-s reinforcer was available through a fixed-time schedule and the 6-s reinforcer was available through a fixed-ratio schedule. In all conditions, the size of the delay or ratio leading to the 6-s reinforcer was systematically increased or decreased several times each session, permitting estimation of an "indifference point," the schedule size at which a subject chose each alternative equally often. By varying the size of the schedule for the 2-s reinforcer across conditions, several such indifference points were obtained from both fixed-time conditions and fixed-ratio conditions. The resulting "indifference curves" from fixed-time conditions and from fixed-ratio conditions were similar in shape, and they suggested that a hyperbolic equation describes the relation between ratio size and reinforcement value as well as the relation between reinforcer delay and its reinforcement value. The results from Phase 3 showed that subjects chose fixed-time schedules over fixed-ratio schedules that generated the same average times between a choice response and reinforcement.     

Escape from an effortful situation

This experiment investigated the tendency to escape from a situation requiring effortful responding. Five human subjects responded in a situation where the response mechanism required 20-lb force to operate; responses were reinforced according to a variable-interval schedule. A subject escaped from this situation by emitting a vocal response which produced a 60-sec “easy period”. During the easy period the reinforcement contingency was switched to a response mechanism requiring 1 lb to operate. It was found that: (1) Escape responding could be conditioned and maintained by producing the easy period; the easy period did not maintain escape responding when the force requirement in the normal situation was equated with it. (2) The rate of escape responding was a function of the magnitude of the force normally required. (3) When easy periods were scheduled after fixed ratios, pausing from the end of the previous easy period to the first escape response was noted. It was concluded that a situation requiring high-force responding is a negative reinforcer. The pattern of fixed-ratio responding suggests that this reinforcer produces typical schedule control in human subjects.     

Choice for aperiodic versus periodic ratio schedules: A comparison of concurrent and concurrent-chain procedures

Choice between mixed-ratio schedules, consisting of equiprobable ratios of 1 and 99 responses per reinforcement, and fixed-ratio schedules of food reinforcement was assessed by two commonly used procedures: concurrent schedules and concurrent-chains schedules. Rats were trained under concurrent fixed-ratio mixed-ratio schedules, in which both ratio schedules were simultaneously available, and under a concurrent-chains schedule, in which access to one of the mutually exclusive ratio schedules comprising the terminal links was contingent on a single “choice” response. The distribution of responses between the two ratio schedules was taken as the choice proportion under the concurrent procedure, and the distribution of “choice” responses was taken as the choice proportion under the concurrent-chains procedure. Seven of eight rats displayed systematic choice; of those, each displayed nearly exclusive choice for fixed-ratio 35 to the mixed-ratio schedule under the concurrent procedure, but each displayed nearly exclusive choice for the mixed-ratio schedule to fixed-ratio 35 under the concurrent-chains procedure. Thus, preference for a fixed or a mixed schedule of reinforcement depended on the procedure used to assess preference.     

Response and time allocation in concurrent second-order schedules

Six pigeons were trained on two-key concurrent variable-interval schedules in which the required response was the completion of a fixed number of key pecks. When the required number of pecks was equal on the two keys, response- and time-allocation ratios under-matched obtained reinforcement rate ratios. A similar result was found when the required number of pecks was unequal, except that performance, measured in response terms, was biased to the shorter required number of pecks and was less sensitive to reinforcement-rate changes. No such differences were found in the data on time spent responding. When the variable-interval schedules were kept constant and the required numbers of pecks were systematically varied, response ratios changed inversely with the ratio of the required number of pecks, but time-allocation ratios varied directly with the same independent variable. Thus, on response measures, pigeons “prefer” the schedule with the smaller peck requirement, but on time measures they “prefer” the schedule with the larger peck requirement. This finding is inconsistent with a commonsense notion of choice, which sees response and time-allocation measures as equivalent.