Effects of intertrial reinforcers on self-control choice.

In three experiments, pigeons chose between a small amount of food delivered after a short delay and a larger amount delivered after a longer delay. A discrete-trial adjusting-delay procedure was used to estimate indifference points--pairs of delay-amount combinations that were chosen about equally often. In Experiment 1, when additional reinforcers were available during intertrial intervals on a variable-interval schedule, preference for the smaller, more immediate reinforcer increased. Experiment 2 found that this shift in preference occurred partly because the variable-interval schedule started sooner after the smaller, more immediate reinforcer, but there was still a small shift in preference when the durations and temporal locations of the variable-interval schedules were identical for both alternatives. Experiment 3 found greater increases in preference for the smaller, more immediate reinforcer with a variable-interval 15-s schedule than with a variable-interval 90-s schedule. The results were generally consistent with a model that states that the impact of any event that follows a choice response declines according to a hyperbolic function with increasing time since the moment of choice.     

Context specificity of conditioned-reinforcement effects on discrimination acquisition

Pigeons were trained on a series of reversals of a simultaneous form discrimination in which the trial outcomes were separated from the choice responses by an 8-s delay interval. Different conditions were defined by the stimuli occurring during the two halves of the delay interval. Discrimination learning was greatly facilitated by having differential stimuli during the delay following correct versus incorrect choices. When the differential stimuli appeared only at the midpoint of the delay, some facilitation occurred relative to when no different stimuli occurred, but there was substantially less facilitation than when the differential stimuli occurred immediately contingent on choice. A reversed-stimulus condition, in which the stimulus at the onset of the delay following a correct choice was the same as that during the last segment of the delay following an incorrect choice, and the stimulus at the onset of the delay following an incorrect choice was the same as that preceding food during the last segment of the delay following a correct choice, also facilitated discrimination learning relative to the nondifferential stimulus conditions.     

Instructional versus schedule control of humans' choices in situations of diminishing returns

Four adult humans chose repeatedly between a fixed-time schedule (of points later exchangeable for money) and a progressive-time schedule that began at 0 s and increased by a fixed number of seconds with each point delivered by that schedule. Each point delivered by the fixed-time schedule reset the requirements of the progressive-time schedule to its minimum value. Subjects were provided with instructions that specified a particular sequence of choices. Under the initial conditions, the instructions accurately specified the optimal choice sequence. Thus, control by instructions and optimal control by the programmed contingencies both supported the same performance. To distinguish the effects of instructions from schedule sensitivity, the correspondence between the instructed and optimal choice patterns was gradually altered across conditions by varying the step size of the progressive-time schedule while maintaining the same instructions. Step size was manipulated, typically in 1-s units, first in an ascending and then in a descending sequence of conditions. Instructions quickly established control in all 4 subjects but, by narrowing the range of choice patterns, they reduced subsequent sensitivity to schedule changes. Instructional control was maintained across the ascending sequence of progressive-time values for each subject, but eventually diminished, giving way to more schedule-appropriate patterns. The transition from instruction-appropriate to schedule-appropriate behavior was characterized by an increase in the variability of choice patterns and local increases in point density. On the descending sequence of progressive-time values, behavior appeared to be schedule sensitive, sometimes even optimally sensitive, but it did not always change systematically with the contingencies, suggesting the involvement of other factors.     

Cued and uncued terminal links in concurrent-chains schedules

Pigeons were trained on a concurrent-chains schedule. The initial links were concurrent variable-interval schedules arranged on two side keys. Each terminal link was a fixed-interval schedule arranged on the center key. In cued conditions, different center-key colors signaled the two terminal-link schedules. In uncued conditions, the same center-key color appeared for both terminal links. Experiment 1 arranged unequal initial links and equal terminal links. Preference for the shorter initial-link schedule was greater when the terminal links were uncued. Experiment 2 arranged equal initial links and unequal terminal links. Preference for the shorter terminal-link schedule was greater when the terminal links were cued. Although the results of Experiment 2 successfully replicated previous research, the results of Experiment 1 are not easily reconciled with conditioned-reinforcement or discriminative-stimulus accounts of the role of terminal-link cues. Rather, terminal-link cues appear to decrease sensitivity to initial-link contingencies.     

How to teach a pigeon to maximize overall reinforcement rate

In two experiments deviations from matching earned higher overall reinforcement rates than did matching. In Experiment 1 response proportions were calculated over a 360-response moving average, updated with each response. Response proportions that differed from the nominal reinforcement proportions, by a criterion that was gradually increased, were eligible for reinforcement. Response proportions that did not differ from matching were not eligible for reinforcement. When the deviation requirement was relatively small, the contingency proved to be effective. However, there was a limit as to how far response proportions could be pushed from matching. Consequently, when the deviation requirement was large, overall reinforcement rate decreased and pecking was eventually extinguished. In Experiment 2 a discriminative stimulus was added to the procedure. The houselight was correlated with the relationship between response proportions and the nominal (programmed) reinforcement proportions. When the difference between response and reinforcement proportions met the deviation requirement, the light was white and responses were eligible for reinforcement. When the difference between response and reinforcement proportions failed to exceed the deviation requirement, the light was blue and responses were not eligible for reinforcement. With the addition of the light, it proved to be possible to shape deviations from matching without any apparent limit. Thus, in Experiment 2 overall reinforcement rate predicted choice proportions and relative reinforcement rate did not. In contrast, in previous experiments on the relationship between matching and overall reinforcement maximization, relative reinforcement rate was usually the better predictor of responding. The results show that whether overall or relative reinforcement rate better predicts choice proportions may in part be determined by stimulus conditions.     

Pigeons' preference for variable-interval water reinforcement under widely varied water budgets.

Water budget of pigeons was varied to assess the dependence of risk-sensitive preferences upon economic context such as has been reported for energy-budget manipulations with small animals in behavioral ecology research. Fixed- and variable-interval terminal-link water schedules reinforced choice between equal variable-interval initial-link schedules arranged on two pecking keys. While keeping a severely restrictive budget the same across three phases of the experiment, a contrasting distinct ample budget was arranged in each. To mimic typical methods in behavioral ecology studies, in each ample budget a more than three-fold increase in amount of water per reinforcer presentation was instituted simultaneously with significantly increased overall access to water. Total choice response rates plummeted in the ample budgets, and body weights either increased significantly or remained unchanged in different phases as expected by the nature of the different manipulations. Clear preferences for the variable-interval schedule were found throughout the experiment, except for rare instances of key bias. The results agree with similar operant food-reinforcement studies and extend conditions under which risk preference apparently does not depend upon economic context.     

Delayed matching-to-sample performance: Effects of relative reinforcer frequency and of signaled versus unsignaled reinforcer magnitudes

Six pigeons were trained on a delayed red-green matching-to-sample task that arranged four delays within sessions. Matching responses intermittently produced either 1.5-s access to food or 4.5-s access to food, and nonmatching responses produced either 1.5-s or 4.5-s blackout. Two phases were conducted: a signaled phase in which the reinforcer magnitudes (small and large) were signaled by houselights (positioned either on the left or right of the chamber), and an unsignaled phase in which there was no correlation between reinforcer magnitude and houselight position. In both phases, the relative frequency with which red and green matching responses produced food was varied across five values. Both matching accuracy and the sensitivity of performance to the distribution of reinforcers for matching responses decreased with increasing delays in both phases. In addition, accuracy and reinforcer sensitivity were significantly lower on signaled small-reinforcer trials compared with accuracy and sensitivity values on signaled large-reinforcer trials and on both types of unsignaled trials. These results are discussed in the context of research on both nonhuman animal and human memory.     

Most directed forgetting in pigeons can be attributed to the absence of reinforcement on forget trials during training or to other procedural artifacts.

In research on directed forgetting in pigeons using delayed matching procedures, remember cues, presented in the delay interval between sample and comparisons, have been followed by comparisons (i.e., a memory test), whereas forget cues have been followed by one of a number of different sample-independent events. The source of directed forgetting in delayed matching to sample in pigeons was examined in a 2 x 2 design by independently manipulating whether or not forget-cue trials in training ended with reinforcement and whether or not forget-cue trials in training included a simultaneous discrimination (involving stimuli other than those used in the matching task). Results were consistent with the hypothesis that reinforced responding following forget cues is sufficient to eliminate performance deficits on forget-cue probe trials. Only when reinforcement was omitted on forget-cue trials in training (whether a discrimination was required or not) was there a decrement in accuracy on forget-cue probe trials. When reinforcement is present, however, the pattern of responding established during and following a forget cue in training may also play a role in the directed forgetting effect. These findings support the view that much of the evidence for directed forgetting using matching procedures may result from motivational and behavioral artifacts rather than the loss of memory.     

Conditioned reinforcement and choice with delayed and uncertain primary reinforcers.

In an adjusting-delay choice procedure, pigeons could peck on either a red key or a green key. A peck on the red key always led to a delay associated with red houselights and then food. The delay was adjusted over trials to estimate an indifference point--a delay at which the two keys were chosen about equally often. In some conditions, a peck on the green key led to food on all trials after delays of either 10 s or 30 s, and green houselights were lit during the delays. In other conditions, food was presented on only half of the green-key trials. If the green houselights continued to occur on both reinforcement and nonreinforcement trials, preference for the green key always decreased. Preference for the green key also decreased if half of the trials had 30-s houselights followed by food and the other half had no green houselights and no food. However, preference for the green key actually increased if half of the trials had 10-s green houselights followed by food and the other half had no green houselights followed by no food. The latter condition therefore demonstrated a case in which preference for an alternative increased when food was removed from half of the trials. The results suggest that the red and green houselights served as conditioned reinforcers. A hyperbolic decay model (Mazur, 1989) provided good predictions for all conditions by assuming that the strength of a conditioned reinforcer is inversely related to the total time spent in its presence before food is delivered.