The temporal organization of behavior on periodic food schedules.

Various theories of temporal control and schedule induction imply that periodic schedules temporally modulate an organism's motivational states within interreinforcement intervals. This speculation has been fueled by frequently observed multimodal activity distributions created by averaging across interreinforcement intervals. We tested this hypothesis by manipulating the cost associated with schedule-induced activities and the availability of other activities to determine the degree to which (a) the temporal distributions of activities within the interreinforcement interval are fixed or can be temporally displaced, (b) rats can reallocate activities across different interreinforcement intervals, and (c) noninduced activities can substitute for schedule-induced activities. Obtained multimodal activity distributions created by averaging across interreinforcement intervals were not representative of the transitions occurring within individual intervals, so the averaged multimodal distributions should not be assumed to represent changes in the subject's motivational states within the interval. Rather, the multimodal distributions often result from averaging across interreinforcement intervals in which only a single activity occurs. A direct influence of the periodic schedule on the motivational states implies that drinking and running should occur at different periods within the interval, but in three experiments the starting times of drinking and running within interreinforcement intervals were equal. Thus, the sequential pattern of drinking and running on periodic schedules does not result from temporal modulation of motivational states within interreinforcement intervals.     

Preference for less frequent shock under fixed-interval schedules of electric-shock presentation.

Lever pressing by 2 squirrel monkeys was maintained under fixed-interval 6-min and fixed-interval 2-min schedules of electric-shock presentation. Preference for these schedules was assessed during three experimental phases. In all phases, responses on one lever produced shock according to one or the other fixed-interval schedule, and responses on a second, changeover, lever switched between schedules. The opportunity to change over was presented during separate choice periods (during which the fixed-interval schedules did not operate) that followed the first through fourth shocks in each schedule. If no changeover occurred during those choice periods, a changeover automatically occurred following the fifth shock. In Phase I, durations of the choice periods were fixed. In Phase II, the choice periods equaled a proportion of their respective fixed interval. During Phase III (completed with 1 monkey) a response on the changeover lever during a given choice period reinstated the most recent fixed interval, and a failure to respond resulted in a changeover. During each of these phases, distinct preferences developed for the 6-min schedule. These results suggest that the maintenance of lever pressing by fixed-interval presentation of electric shock may not be an example of positive reinforcement, and that the response-maintaining characteristics of shock presentation may derive from other properties of the schedule.     

ERRORLESS LEARNING: REINFORCEMENT CONTINGENCIES AND STIMULUS CONTROL TRANSFER IN DELAYED PROMPTING

Delayed prompting can produce errorless discrimination learning. There is inherent in the procedure a disparity in reinforcement density which favors unprompted over prompted responses. We used three schedules of reinforcement to investigate the impact of reinforcement probability on transfer of stimulus control. One schedule of reinforcement was equal prior to and following a prompt (CRF/CRF), the second favored unprompted responses (CRF/FR3), and the third favored responses following the prompt (FR3/CRF). Experimental questions concerned the probability of errors, the probability of transfer, and the rate of transfer in the context of delayed prompting. Transfer was accelerated when reinforcement probability favored anticipatory responding. The schedule that favored prompted responses did not prevent a shift to unprompted responding. Errors were infrequent across procedures. Reinforcement probability contributes to but does not entirely determine transfer of stimulus control from a delayed prompt.     

Choice for conditioned reinforcers in the signaled absence of primary reinforcement

Pigeons responded in a multiple schedule in which concurrent schedules of brief-stimulus presentation alternated with a component in which food was available (concurrent-chains component). In the initial links of the concurrent-chains component subjects chose either of two stimuli each correlated with the terminal link of one chain. The terminal links involved either variable-interval 30-second or variable-interval 60-second schedules. In the brief-stimulus component subjects chose between 0.5-second presentations of the terminal-link stimuli from the concurrent-chains component. Responding was generally maintained in the brief-stimulus component in two subjects for more than 300 sessions, suggesting that brief stimuli were conditioned reinforcers. During the brief-stimulus component, in 17 of 21 cases for which a minimal number of responses occurred, choice proportions above 0.55 were obtained for the brief-stimulus presentations correlated with the higher rate of primary reinforcement in the concurrent-chains component. These results support the suggestion that choice in conventional concurrent-chains procedures is partially controlled by production of the terminal-link stimuli.     

Pentobarbital self-administration in rhesus monkeys: drug concentration and fixed-ratio size interactions.

Performances of three rhesus monkeys were reinforced by the oral delivery of pentobarbital and studied as functions of fixed-ratio size and drug concentration. Pentobarbital solutions and water were concurrently available on identical reinforcement schedules from separate liquid-delivery systems during 3-hour sessions. Under a fixed-ratio 16 schedule of drug availability, a descending series of drug concentrations was tested (4, 2, 1, 0.5, 0.25, 0.125, and 0.0625 mg/ml, followed by a retest at 4 mg/ml). Partial concentration series beginning with the highest concentration were repeated with fixed-ratios of 32 and 64, with a fixed-ratio 128 for two monkeys, and with fixed-ratio 256 for one. At each fixed-ratio value, response rate and number of drug deliveries were inverted U-shaped functions of pentobarbital concentration. Drug intake (mg/kg/session) increased directly with drug concentration. As the fixed-ratio size was increased, the number of liquid deliveries decreased. For each drug concentration, when the numbers of drug deliveries at fixed-ratios of 32, 64, and 128 responses were plotted as percentages of those obtained at fixed-ratio 16, the following orderly relationship emerged: the higher the drug concentration, the less that drug deliveries were decreased by increases in fixed-ratio size. This relationship indicates an increase in reinforcing efficacy with increases in pentobarbital concentration.     

Bias of phencyclidine discrimination by the schedule of reinforcement.

Pigeons, trained to discriminate phencyclidine from saline under a procedure requiring the bird to track the location of a color, received cumulative doses of phencyclidine, pentobarbital, or d-amphetamine with a variety of schedules of reinforcement in effect (across phases). When the same second-order schedules were used to reinforce responding after either saline or phencyclidine administration, stimulus control by phencyclidine did not depend on the schedule parameter. When different second-order schedules were used that biased responding toward the phencyclidine-correlated key color, pigeons responded on the phencyclidine-correlated key at lower doses of phencyclidine and pentobarbital than when the second-order schedule biased responding toward the saline key color. A similar but less marked effect was obtained with d-amphetamine. Attempts to produce bias by changing reinforcement magnitude (duration of food availability) were less successful. A signal-detection analysis of dose-effect curves for phencyclidine under two of the second-order schedules employed suggested that at low doses of phencyclidine, response bias is a major determinant of responding. As doses were increased, position preferences occurred and response bias decreased; at higher doses both response bias and position preference decreased and discriminability increased. With low doses of pentobarbital, responding again was biased but increasing doses produced position preference with only small increases in discriminability. At low doses of d-amphetamine responding also was biased, but bias did not decrease consistently with dose nor did discriminability increase. These experiments suggest that the schedule of reinforcement can be used to bias responding toward or away from making the drug-correlated response in drug discrimination experiments, and that signal-detection analysis and analysis of responding at a position can be used to separate the discriminability of the drug state from other effects of the drug on responding.     

Preference in pigeons given a choice between sequences of fixed-ratio schedules: Effects of ratio values and duration of food delivery

Pigeons were exposed to schedules that consisted of two sequential fixed ratios, the completion of each followed by food delivery. When each alternative provided two food deliveries per 100 responses, the schedule with the shorter initial fixed-ratio value was consistently preferred. Subsequent attempts were made to shift this established preference by (1) increasing the ratio requirement in the second fixed ratio of the preferred schedule; (2) increasing the duration of food delivery in the second fixed ratio of the nonpreferred schedule; (3) decreasing the duration of food delivery in the first fixed ratio of the preferred schedule; and (4) shortening the second fixed ratio of the nonpreferred schedule. Preference shifted from the schedule with the shorter initial fixed ratio only when the duration of food delivery associated with the first fixed ratio of that schedule was too brief to allow eating. Under all other conditions, pigeons strongly preferred the schedule with the shorter initial fixed ratio even when, overall, that schedule yielded briefer access to food or required more responses to obtain equivalent access.     

Choice and transformed interreinforcement intervals

Pigeons chose between two aperiodic, time-based schedules of reinforcement. The arithmetic mean interreinforcement interval of the first schedule was short, but the harmonic mean was long, whereas the arithmetic mean interreinforcement interval of the second schedule was long, but the harmonic mean was short. The pigeons preferred the schedule with the shorter harmonic mean in a concurrent-chains procedure when a terminal link ended after the first scheduled reinforcer had been gained on a terminal-link entry, but reversed their preferences, such that they preferred the schedule with the shorter arithmetic mean, when the terminal links ended after a fixed duration of exposure to the schedule. Moreover, the pigeons preferred the schedule with the shorter arithmetic mean in a two-key concurrent variable-interval variable-interval procedure, as well as in a concurrent variable-time variable-time, changeover-key procedure. The data suggest that an aggregate property of a schedule may not yield valid information about the responding that schedule will maintain as a choice alternative.     

Optimization versus response-strength accounts of behavior

Pigeons were run in both single-key and concurrent-key experiments in which, over most of the range of response rates, an increase in response rate gave rise to a continuous decrease in reinforcement rate. In spite of the fact that a low response rate would have produced a high reinforcement rate, all birds responded at relatively high rates, thus keeping reinforcement rates substantially below the maximum possible. In the concurrent-key experiment, in addition to responding at relatively high rates, the birds' ratios of responses approximately matched the corresponding ratios of obtained reinforcers. The results are inconsistent with most theories of optimal performance, which assume that organisms behave in ways that either maximize reinforcement value or minimize deviations from a free-behavior point. On the other hand, the results are consistent with the assumption that reinforcement strengthens the tendency to respond.     

Stimulus control of schedule-induced activity in pigeons during multiple schedules

Stimulus control of schedule-induced general activity was demonstrated with pigeons using multiple schedules of response-independent food delivery. In Experiment 1, the introduction of food during a multiple variable-time 30-second variable-time 30-second schedule produced a tenfold increase in activity above the no-food baseline. Each pigeon developed stable differential activity rates during the components (correlated with red and green lights) of a multiple variable-time 30-second extinction schedule. Lengthening the extinction component from 1 to 7 minutes increased the rate differences and produced a reliable pattern of responding during S− (the stimulus correlated with extinction): Activity rate was high immediately following the change from S+ (the stimulus correlated with variable-time 30-second) to S−, then decreased abruptly and remained low throughout the middle of the interval, and subsequently showed a positively accelerated increase until the stimulus changed to S+. In Experiment 2, three pigeons were exposed to a mixed variable-time extinction schedule prior to a multiple variable-time extinction schedule. Auditory rather than visual stimuli were used to determine the generality of Experiment 1 results. The multiple- versus mixed-schedule results indicated that stimulus control of activity occurred for two of the birds, but rate differences between S+ and S− were much less than those demonstrated with visual stimuli. A direct comparison of visual and auditory stimulus control in Experiment 3 supported this conclusion. These parallels between the stimulus control of reinforced responding and that of schedule-induced activity suggest that the stimulus control of induced activity may be a factor in operant stimulus control.