Concurrent choice: Effects of overall reinforcer rate and the temporal distribution of reinforcers

Six pigeons responded on a series of concurrent exponential variable-interval schedules, offering a within-subject comparison with previously published data from concurrent arithmetic variable-interval schedules. Both relative and overall reinforcer rates were varied between conditions. The generalized matching law described the data well, with undermatching much more frequent than strict matching. Time-allocation sensitivity consistently exceeded response-allocation sensitivity for both schedule types, and exponential-schedule sensitivity exceeded arithmetic-schedule sensitivity for both measures of choice. A further set of conditions using variable-interval schedules whose shortest interval was correlated with the mean interval, like arithmetic schedules, but that provided a constant conditional probability of reinforcement, like exponential schedules, produced sensitivities between those produced by conventional arithmetic and exponential schedules. Unlike previous arithmetic-schedule results, exponential sensitivity changed nonmonotonically with changes in overall reinforcer rate. The results clarify our knowledge of the effects of arithmetic and exponential schedules but confuse our understanding of the effects of overall reinforcer rate on concurrent choice.     

Blocking, unblocking, and overexpectation in autoshaping with pigeons

Three experiments used pigeons in an autoshaping procedure and a single-subject design to examine compound stimulus control in classical conditioning. Experiment 1 examined the blocking effect, and Experiment 2 examined the unblocking effect. In both experiments, response-independent food was first delivered intermittently in the presence of one distinctively colored houselight but not another. Then, conventional autoshaping trials were carried out in the presence of each houselight. In Experiment 1, the keylight readily elicited responding in the presence of the houselight that had been negatively correlated with food, but not in the presence of the houselight that had been positively correlated with food. In Experiment 2, the keylight readily elicited responding in the presence of the houselight positively correlated with food, but only when the amount of food used on the autoshaping trials was either greater or less than that previously delivered in the presence of the houselight. Experiment 3 examined the overexpectation effect. Conventional autoshaping trials were first carried out by presenting each of two keylights individually. Then, additional autoshaping trials were carried out by presenting the two keylights as a compound, with either the same amount of food or a greater amount of food per trial. Finally, the keylights were retested by again presenting them individually. The number of responses per trial elicited by the keylights decreased when the amount of food used in compound trials was the same as that used in individual trials. However, the number of responses per trial remained approximately the same when the amount of food used in compound trials was greater than that used in individual trials. Taken together, the results of the three experiments demonstrate (a) the generality of the blocking, unblocking, and overexpectation effects by virtue of their extension to appetitive unconditioned stimuli; (b) the suitability of pigeons as subjects and autoshaping as a procedure for studying classical conditioning; and (c) the appropriateness of single-subject designs.     

Choice with certain and uncertain reinforcers in an adjusting-delay procedure.

A discrete-trials adjusting-delay procedure was used to investigate the conditions under which pigeons might show a preference for partial reinforcement over 100% reinforcement, an effect reported in a number of previous experiments. A peck on a red key always led to a delay with red houselights and then food. In each condition, the duration of the red-houselight delay was adjusted to estimate an indifference point. In 100% reinforcement conditions, a peck on a green key always led to a delay with green houselights and then food. In partial-reinforcement conditions, a peck on the green key led either to the green houselights and food or to white houselights and no food. In some phases of the experiment, statistically significant preference for partial reinforcement over 100% reinforcement was found, but this effect was observed in only about half of the pigeons. The effect was largely eliminated when variability in the delay stimulus colors was equated for 50% reinforcement conditions and 100% reinforcement conditions. Idiosyncratic preferences for certain colors or for stimulus variability may be at least partially responsible for the effect.     

Representing within-session response rates proportionally and entirely.

In this technical article, methods for collecting and representing response rates maintained by schedules of reinforcement are presented. First, the time in a session that each important event (e.g., responses, reinforcers) occurs is collected and stored by a computer. Another computer program is used, then, to convert each response to a percentage of the total responses in a session and to plot these percentages cumulatively as a function of the time in the session that they occurred. In this manner, response rates may be expressed proportionally (i.e., using the same y-axis scale regardless of absolute response rate) without requiring the arbitrary selection of an interval over which responses are aggregated and expressed relative to the entire-session rate. A property of these records is that deviations in the slope of the obtained record from the diagonal, which connects (x, y) = (start of session, 0%) to (x, y) = (end of session, 100%), occurring at any point and for any duration, represent changes in the local response rate from the entire-session rate. This method of representing ongoing responding is illustrated by several records of key pecking of a pigeon on a variable-interval 60-s schedule of food reinforcement. Relative local response rates were also computed from these data at several levels of resolution (i.e., the time over which responses were aggregated), including the level typically employed by those interested in within-session changes in response rates.     

An investigation of the differential-outcomes effect within sessions

The differential-outcomes effect is manifest as more accurate performance of a delayed conditional discrimination when alternative choice responses are followed by different reinforcers than when they are followed by the same reinforcer. In Experiment 1, a differential-outcomes effect was demonstrated within sessions by signaling the duration of food access for correct responses with stimuli appearing in conjunction with the sample stimuli. The delayed matching-to-sample performance of 5 pigeons was more accurate when green choice responses (matching a green sample) were followed by 3.5-s food access and red choice responses (matching a red sample) were followed by 0.5-s food access (different-outcome trials) than when the correct choice responses were both followed by 1.5-s reinforcers (same-outcome trials). In Experiment 2, the acquisition of this differential-outcomes effect was characterized by a progressive decrease in rate of forgetting on different-outcome trials and no change in rate of forgetting on same-outcome trials. In addition, accuracy at the shortest delay intervals for both different-outcome and same-outcome trials increased over acquisition, but to a greater extent for different-outcome trials. These data suggest that both memorial and attentional (time-dependent and time-independent) factors contribute to the differential-outcomes effect.     

Precurrent contingencies: Behavior reinforced by altering reinforcement probability for other behavior

The present study explored the effects of a precurrent contingency in which one (precurrent) activity increased the reinforcement probability for another (current) activity. Four human subjects responded on a two-key computer mouse. Each right-key press was reinforced (points exchangeable for money) with .02 probability. In one condition (no precurrent contingency), pressing the left key had no scheduled consequence; in another condition (precurrent contingency), pressing the left key increased the reinforcement probability for right-key responding to .08 for 15 s. Initial exposure to the precurrent contingency resulted in acquisition of precurrent left-key responding for 3 subjects, but for the 4th subject a special contingency was required. Right-key responding occurred at a high stable rate across the conditions. Changeovers to left-key responding dropped to near zero when the precurrent contingency was absent and were maintained at enhanced levels when the precurrent contingency was present. Contacts with the left key consisted of short response runs. Right-key responses were more frequently emitted within 15 s of a left-key response when the precurrent contingency was present, an efficient adaptation to the contingency. Continued research on precurrent behavior may produce insights into complex phenomena such as autoclitics and self-control.     

Transfer of relational stimulus control in conditional discriminations.

Four adults were trained, using instructions and a matching-to-sample procedure, to select Stimulus B1 in the presence of Stimulus A1, B2 in the presence of A2, and B3 in the presence of A3 (the AB relations). Analogous PQ relations were trained. Afterwards, one stimulus in Set A and another stimulus in Set B appeared together as a sample, and novel Stimuli X1 and X2 were the comparisons. Responses to X1 were reinforced if the two stimuli in the sample had been related in the previous training (e.g., A1 and B1), and responses to X2 were reinforced if the two samples had not been related (e.g., A1 and B2). These were the ABX relations. In a test in which a stimulus of Set P and another of Set Q were the samples and X1 and X2 were the comparisons, 2 subjects selected X1 when the samples were P1 and Q1, P2 and Q2, and P3 and Q3, and selected X2 in the presence of the other six sample combinations (P1Q2, P1Q3, P2Q1, P2Q3, P3Q1, and P3Q2). Another subject showed the same responding after additional training. In the second experiment, 3 adults and an 11-year-old child were trained on AB, PQ, and ABX relations, and they showed the symmetrical relations BA and QP upon testing. Then all 4 of these subjects responded accurately to the PQX test. Results of Experiments 1 and 2 showed novel, consistent comparison selection based on the previously established relation between the two stimuli in the sample. In a third experiment, 3 of the subjects who had shown PQX relations were trained on EFX relations, with pairs of E and F stimuli as samples and X stimuli as comparisons. When the EF relations were tested, all 3 subjects consistently selected F1 in the presence of E1, F2 in the presence of E2, and F3 in the presence of E3 from the first trial. The results of Experiment 3 showed novel stimulus relations after training with a more complex conditional discrimination format.     

Effects of reinforcement history on responding under progressive-ratio schedules of reinforcement.

The effects of experimental history on responding under a progressive-ratio schedule of reinforcement were examined. Sixteen pigeons were divided into four equal groups. Groups 1 to 3 were trained to peck a key for food under a fixed-ratio, variable-ratio, or differential-reinforcement-of-low-rate schedule of reinforcement. After training, these pigeons were shifted to a progressive-ratio schedule, later were shifted back to their original schedule (with decreased rates of reinforcement), and finally were returned to the progressive-ratio schedule. Pigeons in Group 4 (control) were maintained on the progressive-ratio schedule for the entire experiment. To test for potential "latent history" effects, pigeons responding under the progressive-ratio schedule were injected with d-amphetamine and given behavioral-momentum tests of prefeeding and extinction. Experimental histories affected responding in the immediate transition to the progressive-ratio schedule; response rates of pigeons with variable-ratio and fixed-ratio histories were higher than rates of pigeons with differential-reinforcement-of-low-rate and progressive-ratio-only histories. Pigeons with differential-reinforcement-of-low-rate histories, and to a lesser degree pigeons with variable-ratio and fixed-ratio histories, also had shorter postreinforcement pauses than pigeons with only a progressive-ratio history. No consistent long-term effects of prior contingencies on responding under the progressive-ratio schedule were evident. d-Amphetamine and resistance-to-change tests failed to reveal consistent latent history effects. The data suggest that history effects are sometimes transitory and not susceptible to latent influences.     

The behavioral theory of timing: Reinforcer rate determines pacemaker rate

In the behavioral theory of timing, pulses from a hypothetical Poisson pacemaker produce transitions between states that are correlated with adjunctive behavior. The adjunctive behavior serves as a discriminative stimulus for temporal discriminations. The present experiments tested the assumption that the average interpulse time of the pacemaker is proportional to interreinforcer interval. Responses on a left key were reinforced at variable intervals for the first 25 s since the beginning of a 50-s trial, and right-key responses were reinforced at variable intervals during the second 25 s. Psychometric functions relating proportion of right-key responses to time since trial onset, in 5-s intervals across the 50-s trial, were sigmoidal in form. Average interpulse times derived by fitting quantitative predictions from the behavioral theory of timing to obtained psychometric functions decreased when the interreinforcer interval was decreased and increased when the interreinforcer interval was increased, as predicted by the theory. In a second experiment, average interpulse times estimated from trials without reinforcement followed global changes in interreinforcer interval, as predicted by the theory. Changes in temporal discrimination as a function of interreinforcer interval were therefore not influenced by the discrimination of reinforcer occurrence. The present data support the assumption of the behavioral theory of timing that interpulse time is determined by interreinforcer interval.