The Temporal Dynamics of Cooperation

Parties in real-world conflicts often attempt to punish each other's behavior. If this strategy fails to produce mutual cooperation, they may increase punishment magnitude. The present experiment investigated whether delay-reduction - potentially less harmful than magnitude increase - would generate mutual cooperation as interactions are repeated. Participants played a prisoner's dilemma game against a computer that played a tit-for-tat strategy, cooperating after a participant cooperated, defecting after a participant defected. For half of the participants, the delay between their choice and the computer's next choice was long relative to the delay between the computer's choice and their next choice. For the other half, long and short delays were reversed. The tit-for-tat contingency reinforces the other player's cooperation (by cooperating) and punishes the other player's defection (by defecting). Both rewards and punishers are discounted by delay. Consistent with delay discounting, participants cooperated more when the delay between their choice and the computer's cooperation (reward) or defection (punishment) was relatively short. These results suggest that, in real-world tit-for-tat conflicts, decreasing delay of reciprocation or retaliation may foster mutual cooperation as effectively as (or more effectively than) the more usual tactic of increasing magnitude of reciprocation or retaliation.     

Delay discounting in the pigeon: In search of a magnitude effect

The magnitude effect, where larger outcomes are discounted proportionally less than smaller outcomes, is a well‐established phenomenon in delay discounting by human participants. To this point in the literature magnitude effects have not been reliably evidenced in nonhuman animals. , however, used a concurrent‐chains arrangement with pigeon and found evidence for a magnitude effect. Grace et al. suggested that in many delay discounting experimental arrangements with nonhuman animals (e.g., adjusting amount, adjusting delay) the organism is not given the opportunity to directly compare outcomes of different sizes. They suggest that because of the lack of direct comparison it is difficult for the organism to determine the relative size of each outcome, which in turn mutes the effect of the amount differences between outcomes. As a test of this “comparison hypothesis,” the present experiment was conducted to assess whether the magnitude effect would be evidenced in pigeon when using an adjusting amount procedure where outcomes of different amounts were presented proximally. In the present arrangement, pigeons were presented two choice panels in an operant chamber where each panel was associated with an independent adjusting amount delay discounting task, but with differing outcome amounts (i.e., a 32‐food pellet panel and an 8‐food pellet panel). In this arrangement the choice panels alternated in their availability within a session from trial block to trial block. The present findings indicate no reliable effect of amount, even when the outcomes were proximal and thus readily comparable. This result suggests that the lack of magnitude effect is not driven by the organism's ability to compare the difference in amount between choice alternatives.     

Alan Baron: A pioneer in translational science

Primates take longer to choose between alternatives with smaller differences in value. This effect—a particular instance of the distance effect in symbolic comparisons—has not been replicated in birds. Instead, birds appear to respond independently to each alternative, such that the latency to choose depends primarily on the alternative of highest value. Three experiments tested for the distance effect in pigeons under conditions not previously considered. Experiment 1 presented pigeons with forced‐ and binary free‐choice trials, where each alternative was one of three possible delays to reinforcement (4, 8, and 16 s). Pigeons were exposed to the choice stimuli for different amounts of time and with different sample response requirements prior to the choice response. Experiment 2 added a fourth (0‐s delay) alternative. Experiment 3 substituted the 16‐s delay with a second 4‐s delay. In all experiments, pigeons systematically chose the shortest delay to reinforcement. Latency to choose the 4‐s delay did not vary when choosing against the 8‐s or 16‐s delay, regardless of whether choice stimuli were exposed for the duration of nine pecks (Experiment 1), or whether a 0‐s delay alternative was sometimes present (Experiment 2). Latency to choose the preferred of two identical alternatives (4‐s vs. 4‐s) was shorter than the latency to choose between different alternatives (4‐s vs. 8‐s; Experiment 3); this is the opposite of a distance effect. These results show no evidence of a distance effect in pigeon choice, consistent with the hypothesis that pigeons respond independently to each choice alternative.     

Observational learning of a conditional hue discrimination in pigeons

Three experiments examined a relationship between cooperation and observational learning in pigeons using procedures similar to Skinner (1962). In Experiment 1, dyads cooperated by searching and pecking nearly simultaneously on the correct pair of adjacent keys of a 2 × 2 matrix. The dyads learned to search for the correct keys in a coordinated fashion, but only if they were permitted to watch one another's performance. Experiment 2 disconfirmed the hypothesis that the coordinated performance of cooperating pigeons reflects an unlearned response tendency to peck at locations close to where the other bird is pecking. Experiment 3 demonstrated that prior cooperation training involving an observer and demonstrator pigeon facilitated subsequent observational learning of a conditional hue discrimination. Cooperation training appeared to facilitate observational learning by establishing the demonstrator's peck response as an instructional stimulus which indicated the reward significance of the discriminative stimuli.     

The sunk cost effect in pigeons and humans

The sunk cost effect is the increased tendency to persist in an endeavor once an investment of money, effort, or time has been made. To date, humans are the only animal in which this effect has been observed unambiguously. We developed a behavior-analytic model of the sunk cost effect to explore the potential for this behavior in pigeons as well as in humans. Each trial started out with a short expected ratio, but on some trials assumed a longer expected ratio part way through the trial. Subjects had the (usually preferable) option of "escaping" the trial if the longer expected ratio had come into effect in order to bring on a new trial that again had a short expected ratio. In Experiments 1 through 3, we manipulated two independent variables that we hypothesized would affect the pigeons' ability to discriminate the increase in the expected ratio within a trial: (a) the presence or absence of stimuli that signal an increase in the expected ratio, and (b) the severity of the increase in the expected ratio. We found that the pigeons were most likely to persist nonoptimally through the longer expected ratios when stimulus changes were absent and when the increase in the expected ratio was less severe. Experiment 4 employed a similar procedure with human subjects that manipulated only the severity of the increase in the expected ratio and found a result similar to that of the pigeon experiment. In Experiment 5, we tested the hypothesis that a particular history of reinforcement would induce pigeons to persist through the longer expected ratios; the results suggested instead that the history of reinforcement caused the pigeons to persist less compared to pigeons that did not have that history.     

Commitment and self-control in a prisoner's dilemma game

Humans often make seemingly irrational choices in situations of conflict between a particular smaller-sooner reinforcer and a more abstract, temporally extended, but larger reinforcer. In two experiments, the extent to which the availability of commitment responses-self-imposed restrictions on future choices-might improve self-control in such situations was investigated. Participants played a prisoner's dilemma game against a computer that played a tit-for-tat strategy-cooperating after a participant cooperated, defecting after a participant defected. Defecting produced a small-immediate reinforcer (consisting of points convertible to gift cards) whereas cooperating increased the amount of subsequent reinforcers, yielding a greater overall reinforcer rate. Participants were normally free to cooperate or defect on each trial. Additionally, they could choose to make a commitment response that forced their choice for the ensuing five trials. For some participants, the commitment response forced cooperation; for others, it forced defection. Most participants, with either commitment response available, chose to commit repeatedly despite a minor point loss for doing so. After extended exposure to these contingencies, the commit-to-cooperate group cooperated significantly more than a control group (with no commitment available). The commit-to-defect group cooperated significantly less than the control group. When both commitment alternatives were simultaneously available-one for cooperation and one for defection-cooperation commitment was strongly preferred. In Experiment 2, the commitment alternative was removed at the end of the session; gains in cooperation, relative to the control group, were not sustained in the absence of the self-imposed behavioral scaffold.     

Correspondence as conditional stimulus control: insights from experiments with pigeons.

Correspondence between saying and doing, typically studied in young children and individuals with developmental disabilities, was examined as an instance of conditional stimulus control. In Experiment 1, 3 pigeons were exposed to a two‐component repeated‐trials procedure. In the first—sample or say—component, two response keys transilluminated by different colored lights were presented and the pigeon pecked one of the keys. After 1 s of darkness in the chamber, the second—choice or do—component was presented, in which the two keys again were transilluminated, one by the color selected in the first component and the second by another color. Selecting the color that matched that selected in the say component resulted in access to food. Selecting the other color produced a blackout of the chamber. After an intertrial interval (ITI), the next say component was programmed, and the procedure was repeated. Correspondence remained at chance levels through several manipulations of ITI duration and sample response requirement. When a correction procedure was added such that only the originally selected sample stimulus was re‐presented until a correct choice response occurred, reliable correspondence developed in 2 pigeons. This correspondence was eliminated by making reinforcement independent of correspondence and subsequently was reestablished when reinforcement again depended on correspondence. In Experiment 2, 3 other pigeons rapidly acquired correspondence under the final procedure used in Experiment 1. Increasing the time interval between the say and do components diminished correspondence. The results of the two experiments suggest how correspondence may be considered an instance of conditional stimulus control and that it is possible to construct a homologue of human say‐do correspondence with pigeons.     

Pictorial target control of schedule-induced attack in White Carneaux pigeons

Three pigeons with a history of attacking a mirror target, and two of six pigeons with no prior exposure to targets, attacked a colored photograph of a conspecific during exposure to intermittent schedules of reinforcement for key pecking. Rate of attack on the photograph decreased when the reinforcement schedule was removed. The topography, temporal pattern, and locus of attack on the picture were comparable to schedule-induced attack on live, stuffed, and mirror targets. When silhouette, outline, and plain paper targets were used, schedule-induced attack was more sensitive to a change in target characteristics with a concurrent target-preference procedure than with an analogous successive-testing procedure. The combined results of the two testing procedures indicated that an “upright” white-on-black silhouette of a pigeon with or without an eye was more effective in controlling attack than was a comparable “inverted” silhouette, an outline of a pigeon, or a piece of colored paper.     

How suboptimal is suboptimal choice?

In a frequently used suboptimal‐choice procedure pigeons choose between an alternative that delivers three food pellets with p = 1.0 and an alternative that delivers ten pellets with p = 0.2. Because pigeons reliably choose the probabilistic (suboptimal) alternative, the procedure has been proposed as a nonhuman analog of human gambling. The present experiments were conducted to evaluate two potential threats to the validity of this procedure. Experiments 1 and 2 evaluated if pigeons obtained food at a lower unit price (i.e., pecks per pellet) on the suboptimal alternative than on the optimal alternative. When pigeons worked under this suboptimal procedure they all preferred the suboptimal alternative despite some pigeons paying a higher price for food on that alternative. In Experiment 2, when the unit price ratio more closely approximated the inverse of the expected value ratio, pigeons continued to prefer the suboptimal alternative despite its economic suboptimality. Experiment 3 evaluated if, in accord with the string‐theory of gambling, the valuation of the suboptimal alternative was increased when pigeons misattributed a subset of the suboptimal no‐food trials to the optimal alternative. When trial sequences were arranged to minimize these possible attribution errors, pigeons still preferred the suboptimal alternative. These data remove two threats to the validity of the suboptimal choice procedure; threats that would have suggested that suboptimal choice reflects economic maximization.     

Responding during reinforcement delay in a self-control paradigm.

Eight pigeons chose between a small, immediate reinforcer and a large, increasingly delayed reinforcer. Responding during the large-reinforcer delays was examined. During large-reinforcer delays, pecks on one key produced the small, immediate reinforcer; pecks on the other key had no effect. Thus, a pigeon could reverse its initial choice of the large, delayed reinforcer, or it could maintain its original choice. Pigeons that made a relatively high number of initial large-reinforcer choices tended to maintain these choices, and those pigeons that actually received a relatively high number of large reinforcers, tended to respond more frequently on the ineffective key during the delay periods. The findings suggest that some previous studies of self-control training in pigeons may have resulted in increased self-control partially due to a lack of opportunity for the pigeons to change their choices.     

The influence of prior choices on current choice

Three pigeons chose between random-interval (RI) and tandem, continuous-reinforcement, fixed-interval (crf-FI) reinforcement schedules by pecking either of two keys. As long as a pigeon pecked on the RI key, both keys remained available. If a pigeon pecked on the crf-FI key, then the RI key became unavailable and the crf-FI timer began to time out. With this procedure, once the RI key was initially pecked, the prospective value of both alternatives remained constant regardless of time spent pecking on the RI key without reinforcement (RI waiting time). Despite this constancy, the rate at which pigeons switched from the RI to the crf-FI decreased sharply as RI waiting time increased. That is, prior choices influenced current choice-an exercise effect. It is argued that such influence (independent of reinforcement contingencies) may serve as a sunk-cost commitment device in self-control situations. In a second experiment, extinction was programmed if RI waiting time exceeded a certain value. Rate of switching to the crf-FI first decreased and then increased as the extinction point approached, showing sensitivity to both prior choices and reinforcement contingencies. In a third experiment, crf-FI availability was limited to a brief window during the RI waiting time. When constrained in this way, switching occurred at a high rate regardless of when, during the RI waiting time, the crf-FI became available.     

Variables affecting establishment of schedule-induced attack on pictorial targets in White King pigeons.

White King pigeons exposed to food schedules before introduction of a colored photograph of a pigeon showed sustained schedule-induced attack on that image; additional birds given an early introduction to both the photograph and the schedule subsequently attacked the image at lower rates. Other pigeons attacked a second photograph of a pigeon regardless of whether it was introduced early or late. The late-introduction procedure was also effective in establishing attack on a projected image of a conspecific. The combined results showed that 14 of 17 White King pigeons given a late introduction to a pictorial target exhibited sustained attack against it and that a pigeon's initial reaction to a photograph of a conspecific when introduced early was a good predictor of subsequent schedule-induced attack on it.     

Timing in choice experiments

In Experiment 1, pigeons chose between variable- and fixed-interval schedules. The timer for 1 schedule was reset by a reinforcement on that schedule or on either schedule. In both cases, the pigeons timed reinforcement on each schedule from trial onset. The data further suggest that their behavior reflects 2 independent processes: 1 deciding when a response should be emitted and responsible for the timing of the overall activity, and the other determining what this response should be and responsible for the allocation of behavior between the 2 response keys. Results from Experiment 2, which studied choice between 2 fixed-interval schedules, support those 2 conclusions. These results have implications for the study of operant choice in general.     

Choice and number of reinforcers

Pigeons were exposed to the concurrent-chains procedure in two experiments designed to investigate the effects of unequal numbers of reinforcers on choice. In Experiment 1, the pigeons were indifferent between long and short durations of access to variable-interval schedules of equal reinforcement density, but preferred a short high-density terminal link over a longer, lower density terminal link, even though in both sets of comparisons there were many more reinforcers per cycle in the longer terminal link. In Experiment 2, the pigeons preferred five reinforcers, the first of which was available after 30 sec, over a single reinforcer available at 30 sec, but only when the local interval between successive reinforcers was short. The pigeons were indifferent when this local interval was sufficiently long. The pigeons' behavior appeared to be under the control of local terminal-link variables, such as the intervals to the first reinforcer and between successive reinforcers, and was not well described in terms of transformed delays of reinforcement or reductions in average delay to reinforcement.     

Required pecking alters judgments of the passage of time by pigeons

There is evidence that how humans perceive time is affected by the activity in which they are engaged when they are judging time. In humans, typically, the more demanding the task, the faster time seems to pass. We asked whether a similar effect could be found in pigeons. Pigeons were trained to discriminate between a short-(2-sec) and a long-(10-sec) duration stimulus. Depending on the color of the stimulus (white or blue), the pigeons were required to peck (at least once per second or the trial was aborted) or to refrain from pecking (pecks aborted the trial). Once these tasks had been acquired to a high degree, probe trials involving white and blue stimuli were presented at durations between 2 and 10 sec. On trials in which the pigeons were required to peck, the point of subjective equality (i.e., the point at which pigeons are equally likely to choose the stimulus associated with long stimuli as the stimulus associated with short stimuli) was almost 1 sec longer than on trials in which the pigeons were required to refrain from pecking. In other words, on trials that required pecking, more time passed before the pigeons indicated that the probe duration was at the subjective midpoint between 2 and 10 sec than on trials that did not require pecking. This result suggests that like humans, the pigeons underestimated the passage of time when they were active or when attention to time-related cues had to be shared with attention to satisfying the response rate requirement.     

Conditioned suppression/avoidance as a procedure for testing hearing in birds: The domestic pigeon (Columba livia)

Although the domestic pigeon is commonly used in learning experiments, it is a notoriously difficult subject in auditory psychophysical experiments, even those in which it need only respond when it detects a sound. This is because pigeons tend to respond in the absence of sound—that is, they have a high false-positive rate—which makes it difficult to determine a pigeon’s audiogram. However, false positives are easily controlled in the method of conditioned suppression/avoidance, in which a pigeon is trained to peck a key to obtain food and to stop pecking whenever it detects a sound that signals impending electric shock. Here, we describe how to determine psychophysical thresholds in pigeons using a method of conditioned suppression in which avoidable shock is delivered through a bead chain wrapped around the base of a pigeon’s wings. The resulting audiogram spans the range from 2 to 8000 Hz; it falls approximately in the middle of the distribution of previous pigeon audiograms and supports the finding of Kreithen and Quine (Journal of Comparative Physiology 129:1–4, 1979) that pigeons hear infrasound.     

Radial maze analog for pigeons: Evidence for flexible coding strategies may result from faulty assumptions

Previous research with the radial maze has found evidence that rats can remember both places that they have already been (retrospective coding) and places they have yet to visit (prospective coding; Cook, R. G., Brown, M. F., & Riley, D. A. (1985). Flexible memory processing by rats: Use of prospective and retrospective information in the radial maze. Journal of Experimental Psychology: Animal Behaviour Processes, 11, 453-469). Such dual coding also has been found in pigeons using a radial maze analog in which insertion of a delay at different points during a trial affects performance differentially depending on where in the trial it is inserted. When a delay is interpolated either early or late in a trial, there is minimal disruption of performance compared with when it is interpolated in the middle of the trial. However, the analysis required with this procedure requires the assumption that if errors made on control trials are subtracted from errors made in delay trials, the remaining errors can be directly attributed to the delay. But errors may also be attributed to the changing criterion for making a response as the trial proceeds. Furthermore, the animal’s tendency to choose alternatives in a systematic order may also affect its need to remember the sequence of choices made (and yet to be made) on each trial. In the present research, we avoided having to make this assumption by giving the pigeons a two-alternative choice at the time of testing and by randomly determining for the pigeon the order of predelay choices on each trial. This change in procedure resulted in comparable performance as a function of where in the trial the test occurred on both control and delay trials. The effect of the delay was to produce a general decrement in performance independent of where it occurred in the trial.     

Sample-duration effects on pigeons' delayed matching as a function of predictability of duration

Three experiments assessed the impact of sample duration on pigeons' delayed matching as a function of whether or not the samples themselves signaled how long they would remain on. When duration was uncorrelated with the sample appearing on each matching trial, the typical effect of duration was observed: Choice accuracy was higher with long (15-s) than with short (5-s) durations. By contrast, this difference either disappeared or reversed when the 5- and 15-s durations were correlated with the sample stimuli. Sample duration itself cued comparison choice by some birds in the latter (predictable) condition when duration was also correlated with the reinforced choice alternatives. However, even when duration could not provide a cue for choice, pigeons matched predictably short-duration samples as accurately as, or more accurately than, predictably long-duration samples. Moreover, this result was observed independently of whether the contextual conditions of the retention interval were the same as, or different from, those of the intertrial interval. These results strongly support the view that conditional stimulus control by the samples is partly a function of their conditioned reinforcing properties, as determined by the relative reduction in overall delay to reinforcement that they signal.     

Feature-based search asymmetries in pigeons and humans

Pigeon and human subjects searched for one target item amidst a number of identical distractors. Simple line forms were used. The target differed from the distractors only in terms of the presence or absence of a feature (a line or a gap); in some experimental series, the feature was present in the target; in others, the feature was in the distractors. The pigeons pecked at the target; the human subjects either reported the presence of the target or pointed to it with a light pen. The time between display onset and this response was recorded. Varied across experimental conditions were the number of distractors in the display, the nature of the stimulus forms, and certain procedural parameters; five conditions were run with pigeons and three with humans. Under all test conditions, the results from the human subjects replicated the previously reported search-asymmetry effect. That is, search speed was greater and decreased less with display size when the target bore the feature (line or gap) than when the distractors bore the feature; both yes/no and localization-response conditions yielded this effect. However, pigeons failed to show search asymmetry; neither line nor gap in a target facilitated search. The results suggest that early visual processing differs for pigeons and humans, that pigeon features differ from human features, or that search asymmetry was eliminated by the long practice given the pigeons.     

Pigeons may not use dual coding in the radial maze analog task

Using a radial maze analog task, T. R. Zentall, J. N. Steirn, and P. Jackson-Smith (1990) found evidence that when a delay was interpolated early in a trial, pigeons coded locations retrospectively, but when the delay was interpolated late in the trial, they coded locations prospectively (support for a dual coding hypothesis). In Experiment 1 of the present study, the authors replicated the original finding of dual coding. In Experiments 2 and 3, they used a 2-alternative test procedure that does not require the assumption that pigeons' choice criterion, which changes over the course of the trial, is the same on delay and control trials. Under these conditions, the pigeons no longer showed evidence for dual coding. Instead, there was some evidence that they showed prospective coding, but a more parsimonious account of the results may be that the delay produced a relatively constant decrement in performance at all points of delay interpolation. The original finding of dual coding by Zentall et al. might have been biased by more impulsive choices early in control trials but not in delay trials and by a more stringent choice criterion late in delay trials.