Pigeons' memory for number of events: Effects of intertrial interval and delay interval illumination

In Experiment 1, pigeons were trained at a 0-s baseline delay to discriminate sequences of light flashes (illumination of the feeder) that varied in number but not time (2f/4s and 8f/4s). During training, the intertrial interval was illuminated by the houselight for Group Light, but it was dark for Group Dark. Testing conducted with dark delay intervals produced a strong choose-small bias in both groups. All birds then received baseline training with a 5-s dark delay and were subsequently tested at shorter and longer dark delays. A choose-small bias was again observed at delays longer than the training delay, while a choose-large bias occurred at delays shorter than the training delay. Differentiating the ambient chamber illumination during the intertrial interval and the delay interval did not attenuate choose-small or choose-large effects. In Experiment 2, all birds received baseline training with a 5-s illuminated delay and were subsequently tested at shorter and longer illuminated delays. A choose-large bias was observed at delays longer than the training delay, while a choose-small bias occurred at delays shorter than the training delay. In Experiment 3, on intermittent test trials, when the duration of the second flash on small-sample trials was equal to the total flash duration on large-sample trials (i.e., 1600 ms), accuracy fell to approximately chance. These results suggest that pigeons discriminated the sequences of light flashes that varied in number but not in total duration of the sequence by relying on other temporal properties of the sequence rather than by using an event switch to count flashes.     

Asymmetrical retention functions for sequences of light flashes or tone bursts in the rat

In Experiment 1, rats were trained in a symbolic delayed matching-to-sample task to discriminate sample stimuli that consisted of sequences of magazine light flashes. The intertrial interval was illuminated by the houselight for Group Light, and it was dark for Group Dark. Retention functions exhibited a choose-many response bias when the delay interval was illuminated by the houselight in both groups, and no consistent response bias when the delay interval was dark. In Experiment 2, rats were trained to discriminate sample stimuli that consisted of sequences of tone bursts. During delay testing, a different tone (i.e., different frequency and location than the sample tone) was present or absent during the delay interval. The retention functions exhibited a significant choose-many bias when tone was present during the delay and a choose-few bias when tone was absent. Asymmetrical retention functions for tone burst and light flash sequences are due to the similarity between the stimulus conditions of the delay interval and the modality of the sequential event being discriminated. These results are consistent with an instructional ambiguity explanation of response biases in memory for number.     

A comparison of delays and ratio requirements in self-control choice.

In a discrete-trial procedure, pigeons could choose between 2-s and 6-s access to grain by making a single key peck. In Phase 1, the pigeons obtained both reinforcers by responding on fixed-ratio schedules. In Phase 2, they received both reinforcers after simple delays, arranged by fixed-time schedules, during which no responses were required. In Phase 3, the 2-s reinforcer was available through a fixed-time schedule and the 6-s reinforcer was available through a fixed-ratio schedule. In all conditions, the size of the delay or ratio leading to the 6-s reinforcer was systematically increased or decreased several times each session, permitting estimation of an "indifference point," the schedule size at which a subject chose each alternative equally often. By varying the size of the schedule for the 2-s reinforcer across conditions, several such indifference points were obtained from both fixed-time conditions and fixed-ratio conditions. The resulting "indifference curves" from fixed-time conditions and from fixed-ratio conditions were similar in shape, and they suggested that a hyperbolic equation describes the relation between ratio size and reinforcement value as well as the relation between reinforcer delay and its reinforcement value. The results from Phase 3 showed that subjects chose fixed-time schedules over fixed-ratio schedules that generated the same average times between a choice response and reinforcement.     

Control of pigeons' matching-to-sample performance by differential sample response requirements

Pigeons were trained on a matching-to-sample task in which sample hue and required sample-specific observing behavior provided redundant, relevant cues for correct choices. On trials that involved red and yellow hues as comparison stimuli, a fixed-ratio 16 schedule (FR 16) was required to illuminate the comparisons when the sample was red, and a differential-reinforcement-of-low-rates 3-sec schedule (DRL 3-sec) was required when the sample was yellow. On trials involving blue and green hues as comparison stimuli, an FR 16 schedule was required when the sample was blue and a DRL 3-sec schedule was required when the sample was green. For some pigeons, a 0-sec delay intervened between sample offset and comparison onset, whereas other pigeons experienced a random mixture of 0-sec and 2-sec delay trials. Test trial performance at 0-sec delay indicated that sample-specific behavior controlled choice performance considerably more than sample hue did. Test performance was independent of whether original training involved all 0-sec delay trials or a mixture of 0-sec and 2-sec delays. Sample-specific observing response requirements appear to facilitate pigeons' matching-to-sample performance by strengthening associations between the observing response and correct choice.     

Directed forgetting of elements in compound samples.

Pigeons were trained in a delayed matching-to-sample procedure in which the sample stimuli consisted of a compound of color (red or green) and spatial location (left or right). A postsample cue (houselight on or off) signaled whether color matching or location matching would be required following the delay. In Experiment 1, the reduction in performance on probe trials (in which the houselight condition was reversed relative to that on regular trials) was greater for location matching than for color matching. The birds showed overt mediational behavior during the delays on location-matching trials. On color-matching trials, the birds exhibited behavior during delays that might have interfered with that mediational behavior. In Experiment 2, the houselight condition was changed shortly before presentation of the comparison stimuli on probe trials. Accuracy of location matching was reduced when the cue initially signaled color matching and was then changed to signal location matching, whereas matching accuracy was not reduced by a change in the opposite direction. Accuracy of color matching was reduced by a change in illumination level from dark to light, regardless of type of the relevant dimension signaled by houselight illumination. Discussion of these findings focuses on the variables critical to establishment of an effective cue to forget.     

Choice between delayed reinforcers and fixed-ratio schedules requiring forceful responding.

This experiment measured pigeons' choices between delayed reinforcers and fixed-ratio schedules in which a force of approximately 0.48 N was needed to operate the response key. In ratio-delay conditions, subjects chose between a fixed-ratio schedule and an adjusting delay. The delay was increased or decreased several times a session in order to estimate an indifference point--a delay duration at which the two alternatives were chosen about equally often. Each ratio-delay condition was followed by a delay-delay condition in which subjects chose between the adjusting delay and a variable-time schedule, with the components of this schedule selected to match the ratio completion times of the preceding ratio-delay condition. The adjusting delays at the indifference point were longer when the alternative was a fixed-ratio schedule than when it was a matched variable-time schedule, which indicated a preference for the matched variable-time schedules over the fixed-ratio schedules. This preference increased in a nonlinear manner with increasing ratio size. This nonlinearity was inconsistent with a theory that states that indifference points for both time and ratio schedules can be predicted by multiplying the choice response-reinforcer intervals of the two types of schedules by different multiplicative constants. Two other theories, which predict nonlinear increases in preference for the matched variable-time schedules, are discussed.     

A procedure for generating differential "sample" responding without different exteroceptive stimuli

Sidman (1994, 2000) suggested that responses as well as stimuli can join equivalence classes, a hypothesis difficult to test because differential responding typically requires different stimuli. The present experiments describe a procedure with pigeons that avoids this potential confounding effect. In Experiment 1, spacing two responses 3 s apart (a differential-reinforcement-of-low-rate [DRL] schedule) to a white stimulus on some trials produced food or the comparison stimuli in a matching task, whereas pecking 10 or more times with no temporal restrictions (a fixed-ratio [FR] schedule) produced the same effect on other trials. Completing the alternative (unscheduled) requirement terminated the white stimulus and repeated the trial. Following such errors, pigeons learned to switch to the alternative response pattern on the repeat trials. In addition, the correct response pattern functioned as a conditional cue for comparison choice. In Experiment 2, mixed DRL-FR training was preceded by two-sample/two-alternative matching-to-sample with DRL and FR sample-response requirements. In a subsequent transfer test in which the correct response pattern to white served as the sample, pigeons preferentially chose the comparison previously reinforced following that pattern in the baseline task. This "unsignaled response" procedure may be useful for assessing whether differential responses can be members of acquired equivalence classes.     

Response Form, Force, And Number: Effects On Concurrent-schedule Performance

Six hens responded on concurrent variable-interval (key-peck) variable-interval (door-push) schedules of reinforcement in which the second-order (fixed-ratio) requirements on the alternatives (Experiment 1) or the required door forces (Experiment 2) were varied. The key-peck and door-push response (measured as fixed-ratio completion) and time data were well described by the generalized matching law. However, the manipulations of fixed-ratio requirement and required response force differed in their effects. The manipulations of fixed-ratio size affected the response and time measures differently, producing fairly constant, multiplicative biases only in terms of response allocation. It was argued that variations in fixed-ratio size necessarily change the time allocated to that response unit, and thus changes in time bias were not necessarily a fundamental effect of changing the ratio. In contrast, the changes in response bias were a fundamental result of changes in ratio size. The response-force manipulations produced similar bias shifts in terms of response and time allocation, but they appeared to combine with relative reinforcement rate to affect choice interactively. Specifically, behavior appeared to be biased towards the least effortful (i.e., key-peck) response, but the increases in door force had a larger effect on bias when the hens were making this response infrequently (on a lean schedule). The different effects of the fixed-ratio and response-force manipulations on concurrent performance were partially accounted for by the differing times required to complete each response unit under those manipulations, but this would not account for the interaction. The interaction would be consonant with increased response effort decreasing the effective value of the associated reinforcement schedule.     

Sub-optimal choice in pigeons does not depend on avoidance of the stimulus associated with the absence of reinforcement

When pigeons are given a choice between two alternatives, one leading to a stimulus 20% of the time that always signals reinforcement (S+) or another stimulus 80% of the time that signals no reinforcement (S−), and the other alternative leading to one of two stimuli each signaling reinforcement 50% of the time, they show a strong preference for the first alternative. This preference occurs in spite of the fact that, overall, the second alternative provides two and a half times more reinforcement than the first. In the present experiment we tested the hypothesis that the S− is a less effective conditioned inhibitor because as soon as it is recognized, the pigeon may orient away from it, whereas it does not orient away from the other signals. To test this hypothesis, for Group HLS−, we made the S− more salient and less avoidable by using a ceiling mounted houselight. To control for a possible aversion to the houselight we included Group HLS+, a group for which the houselight served as the S+. And the preferences of both groups were compared to those of a standard no houselight group. The pigeons in all three groups showed a strong preference for the lower probability of reinforcement alternative. Thus, reduced peripheral orienting during presentation of the S− stimulus was not likely responsible for pigeons’ sub-optimal choice.     

Theories of probabilistic reinforcement.

In three experiments, pigeons chose between two alternatives that differed in the probability of reinforcement and the delay to reinforcement. A peck at a red key led to a delay of 5 s and then a possible reinforcer. A peck at a green key led to an adjusting delay and then a certain reinforcer. This delay was adjusted over trials so as to estimate an indifference point, or a duration at which the two alternatives were chosen about equally often. In Experiments 1 and 2, the intertrial interval was varied across conditions, and these variations had no systematic effects on choice. In Experiment 3, the stimuli that followed a choice of the red key differed across conditions. In some conditions, a red houselight was presented for 5 s after each choice of the red key. In other conditions, the red houselight was present on reinforced trials but not on nonreinforced trials. Subjects exhibited greater preference for the red key in the latter case. The results were used to evaluate four different theories of probabilistic reinforcement. The results were most consistent with the view that the value or effectiveness of a probabilistic reinforcer is determined by the total time per reinforcer spent in the presence of stimuli associated with the probabilistic alternative. According to this view, probabilistic reinforcers are analogous to reinforcers that are delivered after variable delays.     

Sources of visual interference in delayed matching-to-sample with pigeons

In two experiments, independent groups of pigeons were trained on an identity matching task involving line orientations as sample and comparison stimuli. For some birds an overhead houselight was illuminated continuously throughout each training session. For other birds the houselight was never illuminated during training sessions. During subsequent testing, the lighting conditions during the delay were the same as in training on some trials, but on other trials they were opposite those of training during either the entire delay (Experiment 1) or during a portion of the delay (Experiment 2). In birds trained with the houselight off, turning the houselight on during the delay produced a large and enduring disruption in matching accuracy. On the other hand, in birds trained with the houselight on, turning the houselight off during the delay produced only a moderate and temporary disruption in matching accuracy. These findings are inconsistent with the prevailing view that retroactive interference in pigeons is a function of a change in illumination level relative to that which prevailed during training. In pigeons, as in monkeys, sustained retoactive interference effects obtain only when the level of illumination is increase during the delay interval.     

Rapid acquisition of bias in signal detection: dynamics of effective reinforcement allocation

We investigated changes in bias (preference for one response alternative) in signal detection when relative reinforcer frequency for correct responses varied across sessions. In Experiment 1, 4 rats responded in a two-stimulus, two-response identification procedure employing temporal stimuli (short vs. long houselight presentations). Relative reinforcer frequency varied according to a 31-step pseudorandom binary sequence and stimulus duration difference varied over two values across conditions. In Experiment 2, 3 rats responded in a five-stimulus, two-response classification procedure employing temporal stimuli. Relative reinforcer frequency was varied according to a 36-step pseudorandom ternary sequence. Results of both experiments were analyzed according to a behavioral model of detection. The model was extended to incorporate the effects of current and previous session reinforcer frequency ratios on current-session performance. Similar to findings with concurrent schedules, effects on bias of relative reinforcer frequency were highest for the current session. However, carryover from reinforcer ratios of previous sessions was evident. Generally, the results indicate that bias can come under control of frequent changes in relative reinforcer frequency in both identification and classification procedures.     

Exchange delays and impulsive choice in adult humans.

Choice responding by adult humans in a discrete-trial task was examined as a function of conditions that manipulated either the delay to point delivery or the delay between points and their exchange for money. In point-delay conditions, subjects chose between an "impulsive" alternative that provided a small amount of points immediately and a "self-control" alternative that provided a larger amount of points delayed by 15, 30, or 60 s. Points were exchanged for money immediately following the session. Subjects preferred the self-control alternative. In exchange-delay conditions, subjects chose between a small amount of points exchangeable for money immediately following the session and a larger amount of points exchangeable for money after 1 day, 3 weeks, or 6 weeks. A self-control preference observed for all subjects in the 1-day exchange-delay condition reversed to exclusive impulsive preference for 4 of the 6 subjects when choice conditions involved exchange delays of 3 or 6 weeks. These results show that human choice is sensitive to the manipulation of exchange delays and that impulsive preference can be obtained with exchange delays on the order of weeks.     

Modifying drug-reinforced behavior by altering the economic conditions of the drug and a nondrug reinforcer.

Six rhesus monkeys were trained to self-administer orally delivered phencyclidine (0.25 mg/mL) and saccharin (0.03% wt/vol) under concurrent fixed-ratio 16 schedules. In Condition 1 the fixed-ratio requirement for phencyclidine was changed from 16 to 4, 8, 16, 32, 64, 128 and 16 while the fixed-ratio requirement for saccharin deliveries remained constant at 16. In Condition 2 the fixed-ratio value for saccharin was systematically altered while the fixed-ratio requirement for phencyclidine remained at 16, and in Condition 3 the fixed-ratio requirements for both phencyclidine and saccharin were altered simultaneously. Water was then substituted for saccharin, and the series of fixed-ratio manipulations was replicated. The phencyclidine concentration was reduced to 0.125 mg/mL and Conditions 1 and 3 were repeated. When the fixed-ratio requirement for phencyclidine was increased and the fixed-ratio requirement for saccharin or water remained fixed at 16, phencyclidine deliveries decreased when saccharin (vs. water) was concurrently available. The magnitude of the decrease ranged from 20% to 90% (of the concurrent water condition) as the fixed-ratio requirement for phencyclidine increased from 4 to 128. When the fixed-ratio requirement for phencyclidine remained at 16 and the fixed-ratio requirements for concurrent saccharin or water varied between 4 and 128, phencyclidine deliveries decreased by 30% to 40% due to the concurrent availability of saccharin (vs. water). This decrease occurred only at the three lowest fixed-ratio values when saccharin intake was relatively high. When the fixed-ratio requirements for both phencyclidine and concurrent saccharin or water were varied simultaneously, phencyclidine deliveries were reduced from 20% to 45% when saccharin (vs. water) was concurrently present. There was little effect of reducing the phencyclidine concentration when the data were analyzed in terms of unit price (responses per milligram). Thus, changes in the fixed-ratio requirement or drug concentration were functionally similar, and unit price of phencyclidine was the variable that was influenced by the presence of concurrent saccharin. These data indicate that drug-reinforced behavior is substantially reduced when the environment is enriched with an alternative nondrug reinforcer. The economic context in which these substances are presented is an important determinant of drug-reinforced behavior.     

Effects of previous delay of reward,generalized effort,and deprivation on impulsiveness

Two experiments investigated the effects of learning and motivational variables upon rats' degree of impulsiveness (choice of a small, immediate reward versus a large, delayed reward). In Experiment 1, food was presented in one of four ways for 24 sessions: fixed-ratio lever pressing requirement, continuous-reinforcement lever-pressing requirement, long intervals between free food presentations, or short intervals between free food presentations. Exposure to the long intervals produced less subsequent impulsiveness in 48 choice trials than did the short intervals, there being no effect of the instrumental requirement. In Experiment 2, there were 48 choice trials in which two hunger levels were combined factorially with two delays of reward for the self-control alternative. Hunger level did not affect implusiveness when there was a short delay, but high hunger increased the impulsiveness when there was a long delay. These results indicate that impulsiveness is affected by prior learning and current motivation. Implications for Baum, Rachlin, and Green's matching formula and for frustration theory are discussed.     

Behavioral and pharmacological variables affecting risky choice in rats.

The effects of manipulations of response requirement, intertrial interval (ITI), and psychoactive drugs (ethanol, phencyclidine, and d-amphetamine) on lever choice under concurrent fixed-ratio schedules were investigated in rats. Responding on the "certain' lever produced three 45-mg pellets, whereas responding on the "risky" lever produced either 15 pellets (p = .33) or no pellets (p .67). Rats earned all food during the session, which ended after 12 forced trials and 93 choice trials or 90 min, whichever occurred first. When the response requirement was increased from 1 to 16 and the ITI was 20 s, percentage of risky choice was inversely related to fixed-ratio value. When only a single response was required but the ITI was manipulated between 20 and 120 s (with maximum session duration held constant), percentage of risky choice was directly related to length of the ITI. The effects of the drugs were investigated first at an ITI of 20 s, when risky choice was low for most rats, and then at an ITI of 80 s, when risky choice was higher for most rats. Ethanol usually decreased risky choice. Phencyclidine did not usually affect risky choice when the ITI was 20 s but decreased it in half the rats when the ITI was 80 s. For d-amphetamine, the effects appeared to he related to baseline probability of risky choice; that is, low probabilities were increased and high probabilities were decreased. Although increase in risky choice as a function of the ITI is at variance with previous ITI data, it is consistent with foraging data showing that risk aversion decreases as food availability decreases. The pharmacological manipulations showed that drug effects on risky choice may be influenced by the baseline probability of risky choice, just as drug effects can be a function of baseline response rate.     

Choice for aperiodic versus periodic ratio schedules: A comparison of concurrent and concurrent-chain procedures

Choice between mixed-ratio schedules, consisting of equiprobable ratios of 1 and 99 responses per reinforcement, and fixed-ratio schedules of food reinforcement was assessed by two commonly used procedures: concurrent schedules and concurrent-chains schedules. Rats were trained under concurrent fixed-ratio mixed-ratio schedules, in which both ratio schedules were simultaneously available, and under a concurrent-chains schedule, in which access to one of the mutually exclusive ratio schedules comprising the terminal links was contingent on a single “choice” response. The distribution of responses between the two ratio schedules was taken as the choice proportion under the concurrent procedure, and the distribution of “choice” responses was taken as the choice proportion under the concurrent-chains procedure. Seven of eight rats displayed systematic choice; of those, each displayed nearly exclusive choice for fixed-ratio 35 to the mixed-ratio schedule under the concurrent procedure, but each displayed nearly exclusive choice for the mixed-ratio schedule to fixed-ratio 35 under the concurrent-chains procedure. Thus, preference for a fixed or a mixed schedule of reinforcement depended on the procedure used to assess preference.     

Joint stimulus control in a temporal discrimination task

The ability to identify stimuli that signal important events is fundamental for an organism to adapt to its environment. In the present paper, we investigated how more than one stimulus could be used jointly to learn a temporal discrimination task. Ten pigeons were exposed to a symbolic matching-to-sample procedure with three durations as samples (2, 6, and 18 s of keylight) and two colors as comparisons (red and green hues). A 30-s intertrial interval (ITI), illuminated with a houselight, separated the trials. Both the houselight and the sample keylight could control responding, so two tests were run to assess how these stimuli influenced choice. In the no-sample test, the keylight was not presented; in the dark-ITI test, the houselight was not illuminated. Results suggest that both houselight and keylight controlled choice, and with the exception of one animal, the more a pigeon relied on one of these stimuli, the less it appeared to rely on the other.     

Effects of pre-trial response requirements on self-control choices by rats and pigeons

Parallel experiments with rats and pigeons examined whether the size of a pre-trial ratio requirement would affect choices in a self-control situation. In different conditions, either 1 response or 40 responses were required before each trial. In the first half of each experiment, an adjusting-ratio schedule was used, in which subjects could choose a fixed-ratio schedule leading to a small reinforcer, or an adjusting-ratio schedule leading to a larger reinforcer. The size of the adjusting ratio requirement was increased and decreased over trials based on the subject's responses, in order to estimate an indifference point-a ratio at which the two alternatives were chosen about equally often. The second half of each experiment used an adjusting-delay procedure-fixed and adjusting delays to the small and large reinforcers were used instead of ratio requirements. In some conditions, particularly with the reinforcer delays, the rats had consistently longer adjusting delays with the larger pre-trial ratios, reflecting a greater tendency to choose the larger, delayed reinforcer when more responding was required to reach the choice point. No consistent effects of the pre-trial ratio were found for the pigeons in any of the conditions. These results may indicate that rats are more sensitive to the long-term reinforcement rates of the two alternatives, or they may result from a shallower temporal discounting rate for rats than for pigeons, a difference that has been observed in previous studies.     

Alternative fixed-ratio fixed-interval schedules of reinforcement

Five rats were trained under alternative fixed-ratio fixed-interval schedules, in which food reinforcement was provided for the completion of either a fixed-ratio or a fixed-interval requirement, whichever was met first. Overall response rate and running rate (the rate of responding after the postreinforcement pause) decreased for all subjects as the fixed-ratio value increased. As the proportion of reinforcements obtained from the fixed-ratio component increased and the alternative schedule approached a simple fixed ratio, overall response rate and running rate both increased; conversely, as the proportion of reinforcements obtained from the fixed-interval component increased and the alternative schedule approached a simple fixed interval, response rates decreased. Postreinforcement pause length increased linearly as the average time between reinforcements increased, regardless of the schedule parameters. A break-run pattern of responding was predominant at low- and medium-valued fixed ratios. All subjects displayed at least occasional positively accelerated responding within interreinforcement intervals at higher fixed-ratio values.