Choice with delayed and probabilistic reinforcers: effects of variability, time between trials, and conditioned reinforcers.

In a discrete-trials procedure with pigeons, a response on a green key led to a 4-s delay (during which green houselights were lit) and then a reinforcer might or might not be delivered. A response on a red key led to a delay of adjustable duration (during which red houselights were lit) and then a certain reinforcer. The delay was adjusted so as to estimate an indifference point--a duration for which the two alternatives were equally preferred. Once the green key was chosen, a subject had to continue to respond on the green key until a reinforcer was delivered. Each response on the green key, plus the 4-s delay that followed every response, was called one "link" of the green-key schedule. Subjects showed much greater preference for the green key when the number of links before reinforcement was variable (averaging four) than when it was fixed (always exactly four). These findings are consistent with the view that probabilistic reinforcers are analogous to reinforcers delivered after variable delays. When successive links were separated by 4-s or 8-s "interlink intervals" with white houselights, preference for the probabilistic alternative decreased somewhat for 2 subjects but was unaffected for the other 2 subjects. When the interlink intervals had the same green houselights that were present during the 4-s delays, preference for the green key decreased substantially for all subjects. These results provided mixed support for the view that preference for a probabilistic reinforcer is inversely related to the duration of conditioned reinforcers that precede the delivery of food.     

Maximizing versus matching on concurrent variable-interval schedules

Maximization and matching predictions were examined for a time-based analogue of the concurrent variable-interval variable-ratio schedule. One alternative was a variable interval whose time base operated relatively independent of the schedule chosen, and the other was a discontinuous variable interval for which timing progressed only when selected. Pigeons switched between schedules by pecking a changeover key. The maximization hypothesis predicts that subjects will show a bias toward the discontinuous variable interval and undermatching; however the obtained results conformed closely to the predictions of the matching law. Finally, a quantitative comparison was made of the bias and sensitivity estimates obtained in published concurrent variable-interval variable-ratio analogue studies. Results indicated that only the ratio-based analogue of the concurrent variable interval variable ratio studied by Green, Rachlin, and Hanson (1983) produced significant bias toward the variable-ratio alternative and undermatching, as predicted by reinforcement maximization.     

Stimulus effects on concurrent performance in transition

Six experimentally naive pigeons were exposed to concurrent variable-interval variable-interval schedules in a three-key procedure in which food reinforcement followed pecks on the side keys and pecks on the center key served as changeover responses. In Phase 1, 3 birds were exposed to 20 combinations of five variable-interval values, with each variable-interval value consistently associated with a different color on the side keys. Another 3 pigeons were exposed to the same 20 conditions, but with a more standard procedure that used a nondifferential discriminative stimulus on the two side keys throughout all conditions. In Phase 2, the differential and nondifferential stimulus conditions were reversed for each pigeon. Each condition lasted for one 5-hr session and one subsequent 1-hr session. In the last 14 conditions of each phase, the presence of differential discriminative stimuli decreased the time necessary for differential responding to develop and increased the sensitivity of behavior to reinforcement distribution in the 1st hr of training; during the last hours of training in each condition, however, the effects of the differential discriminative stimuli could not be distinguished from the effects of reinforcement distribution per se. These results show the importance of studying transitions in behavior as well as final performance. They may also be relevant to discrepancies in the results of previous experiments that have used nonhuman and human subjects.     

Molecular contingencies in schedules of intermittent punishment

In two experiments, key pecking of pigeons was maintained by a variable-interval 180-s schedule of food presentation. Conjointly, a second schedule delivered response-dependent electric shock. In the first experiment, shocks were presented according to either a variable-interval or a nondifferential interval-percentile schedule. The variable-interval shock schedule differentially delivered shocks following long interresponse times. Although the nondifferential shock schedules delivered shocks less differentially with respect to interresponse times, the two shock schedules equally reduced the relative frequency of long interresponse times. The second experiment differentially shocked long or short interresponse times in different conditions, with resulting decreases in the relative frequency of the targeted interresponse times. These experiments highlight the importance of selecting the appropriate level of analysis for the interaction of behavior and environment. Orderly relations present at one level of analysis (e.g., interresponse times) may not be revealed at other levels of analysis (e.g., overall response rate).     

Inelastic supply: An economic approach to simple interval schedules

Economic theory predicts an inverse relationship between the quantity of a commodity supplied to the marketplace and the equilibrium market price of that commodity. This prediction was tested in three experiments. Pigeons responded on simple variable-interval schedules, and quantity of reinforcement supplied was varied in a different way in each experiment. In Experiment 1, quantity supplied was varied by manipulating reinforcement rate while keeping session length constant. In Experiment 2, quantity supplied was varied by manipulating reinforcement rate while keeping reinforcers per session constant. In Experiment 3, quantity supplied was varied by manipulating reinforcer magnitude while keeping number of reinforcers constant. As predicted by economic theory, the obtained behavioral cost (responses per reinforcer) increased as supply decreased. The results could not be explained by simple artifacts such as satiation and time available to respond. In addition, the function relating response rate to reinforcement rate was bitonic in 7 of 9 animals in Experiments 1 and 2, which supports economic and regulatory theories over more traditional reinforcement theories.     

The autoshaping procedure as a residual block clock

In the first experiment, 4 pigeons were each presented with a recurring sequence of four key colors followed by the delivery of grain (block clock). Once the rate of pecking had stabilized, three of the colors were replaced, during different series of sessions, by a darkening of the key. The rate of pecking was reduced within those segments of the interval between deliveries of food during which the key was dark; when the key was dark during the final portion of the interval, rates were reduced throughout the entire interval. In the second experiment, 3 new pigeons were exposed to a different sequence of colors, and the final stimulus was replaced in successive conditions by a novel color, a darkened key, and a restoration of the original color. The data indicated that darkening the key had a more severe, more extensive, and more persistent effect than did a mere change in color. These results suggest that it may be fruitful to conceptualize the autoshaping procedure as a special version of the block clock in which pecking is suppressed throughout the greater part of the interval by darkening the key. In the final condition, the same stimulus appeared in each of the last three portions of the interval. The rate of pecking was lower during the last two portions than when distinctive colors were presented, with the peak rate now appearing in the fifth of seven equal temporal components.     

Fixed-ratio pausing: Joint effects of past reinforcer magnitude and stimuli correlated with upcoming magnitude

Pigeons responded on fixed-ratio schedules ending in small or large reinforcers (grain presentations of different duration) interspersed within each session. In mixed-schedule conditions, the response key was lit with a single color throughout the session, and pausing was directly related to the past reinforcer (longer pauses after large reinforcers than after small ones). In multiple-schedule conditions, different colors accompanied the ratios ending in small and large reinforcers, and pausing was affected by the upcoming reinforcer as well as the past one. Pauses were shorter before large reinforcers than before small ones, but they continued to be longer after large reinforcers than after small ones. The influence of the past reinforcer was modulated by the magnitude of the upcoming reinforcer; in the presence of the stimulus before the small reinforcer, the effect of the past reinforcer was enhanced relative to its effect in the stimulus before the large reinforcer. These results show that pausing between ratios is jointly determined by two competing factors: past conditions of reinforcement and stimuli correlated with upcoming conditions.     

Discriminability between alternatives in a switching-key concurrent schedule

Six pigeons were trained to discriminate between two intensities of white light in a symbolic matching-to-sample procedure. These stimuli were then used to signal which schedule was available on the main key in a switching-key concurrent schedule. The concurrent schedules led to a symbolic matching-to-sample phase in which the subject identified the concurrent schedule to which it last responded before a reinforcer could be obtained. The concurrent schedules were varied across conditions. Discriminability, measured during the symbolic matching-to-sample performance, was high throughout and did not differ across the two procedures. Performance in the concurrent schedules was like that typically obtained using these schedules. Delays were then arranged between completion of the concurrent schedules and presentations of the symbolic matching-to-sample phase. A series of conditions with an intervening delay of 10 s showed that both concurrent-schedule performance and symbolic matching-to-sample performance were affected by the delay in a similar way; that is, choice responding was closer to indifference.     

Time-place learning by pigeons, Columba livia

In each of two experiments, 2 pigeons received discrimination training in which food reinforcement for key pecking was conditional upon both spatial and temporal cues. In Experiment 1, food was available for periods of 30 s at each of three locations (pecking keys) during trials that lasted 90 s. In Experiment 2, food was available for periods of 15 min at each of four locations (pecking keys) during a 60-min trial. In both experiments, pigeons' key pecking was jointly controlled by the spatial and temporal cues. These data, and other recent experiments, suggest that animals learn relationships between temporal and spatial cues that predict stable patterns of food availability.     

Using item-specific instructional information in achievement modeling

The problem of detecting instructional sensitivity (item basis) in test items is considered. An illustration is given which shows that for tests with many biased items, traditional item bias detection schemes give a very poor assessment of bias. A new method is proposed instead. This method extends item response theory (IRT) by including item-specific auxiliary measurement information related to opportunity-to-learn. Item-specific variation in measurement relations across students with varying opportunity-to-learn is allowed for.This paper was presented at the 1987 AERA meeting in Washington, DC. This research was supported by grant OERI-G-86-003 from the Office of Educational Research and Improvement, Department of Education. The author thanks Michael Hollis and Chih-fen Kao for valuable research assistance, and appreciates valuable comments made by an anonymous reviewer.