Within-session Response Patterns On Conjoint Variable-interval Variable-time Schedules

Operant responding often changes within sessions, even when factors such as rate of reinforcement remain constant. The present study was designed to determine whether within-session response patterns are determined by the total number of reinforcers delivered during the session or only by the reinforcers earned by the operant response. Four rats pressed a lever and 3 pigeons pecked a key for food reinforcers delivered by a conjoint variable-interval variable-time schedule. The overall rate of reinforcement of the conjoint schedule varied across conditions from 15 to 480 reinforcers per hour. When fewer than 120 reinforcers were delivered per hour, the within-session patterns of responding on conjoint schedules were similar to those previously observed when subjects received the same total number of reinforcers by responding on simple variable-interval schedules. Response patterns were less similar to those observed on simple variable-interval schedules when the overall rate of reinforcement exceeded 120 reinforcers per hour. These results suggest that response-independent reinforcers can affect the within-session pattern of responding on a response-dependent schedule. The results are incompatible with a response-based explanation of within-session changes in responding (e.g., fatigue), but are consistent with both reinforcer-based (e.g., satiation) and stimulus-based (e.g., habituation) explanations.     

Within-session changes in responding during concurrent schedules with different reinforcers in the components.

Rats and pigeons responded on several concurrent schedules that provided different reinforcers in the two components (food and water for rats, Experiment 1; wheat and mixed grain for pigeons, Experiment 2). The rate of responding and the time spent responding on each component usually changed within the session. The within-session changes in response rates and time spent responding usually followed different patterns for the two components of a concurrent schedule. For most subjects, the bias and sensitivity to reinforcement parameters of the generalized matching law, as well as the percentage of the variance accounted for, decreased within the session. Negative sensitivity parameters were sometimes found late in the session for the concurrent food-water schedules. These results imply that within-session changes in responding could cause problems for assessing the validity of quantitative theories of concurrent-schedule responding when the components provide different reinforcers. They question changes in a general motivational state, such as arousal, as a complete explanation for within-session changes in responding. The results are compatible with satiation for, or sensitization-habituation to, the reinforcers as explanations.     

Within-session rates of responding when reinforcer magnitude is changed within the session

In the present experiment, the authors investigated the idea that within-session changes in operant response rates occur because subjects sensitize and then habituate to the reinforcer. If that is true, then altering an aspect of the reinforcer within the session should alter the observed within-session responding. The authors tested that idea by having rats press a lever for 2 food-pellet reinforcers delivered by a variable-interval 120-s schedule during 60-min baseline sessions. In treatment conditions, the magnitude of the reinforcer was halved (1 pellet) or doubled (4 pellets) 10, 20, 30, 40, or 50 min into the session. That magnitude of reinforcement then remained in effect for the rest of the session. Altering reinforcer magnitude altered the rates of responding within the session in a fashion consistent with the habituation explanation, that is, response rates increased, relative to baseline, when the magnitude of reinforcement was increased. They decreased when the magnitude was decreased. Those results were seemingly inconsistent with the competing idea that within-session decreases in responding rates are produced by satiation.     

Positive and negative conditioned suppression in the pigeon: Effects of the locus and modality of the CS

In Experiment I, four pigeons were exposed to trials in which a 12-sec key light illumination was followed by free food. These trials were superimposed upon a baseline of key pecking for food reinforcement on a variable-interval schedule. When the signal for food was on the operant key, response rate was substantially higher during the signal than during the baseline procedure. When the signal was on a second, signal key, operant responding was suppressed during the signal and substantial pecking of the signal key occurred. The sum of signal key and operant key pecks far exceeded the operant baseline rate of responding. An explanation of opposite results obtained with rats and pigeons as subjects in experiments of this type was suggested in terms of the spatial relation between the signal for free food and the operant target which usually characterizes these experiments. Experiment II assessed the importance of signal location when shock rather than food was the US. Suppression of operant key pecking was unaffected by signal location. Experiment III assessed the relative effectiveness of visual and auditory stimuli (clicks) as signals for food and shock, and found that all combinations of signal and US were equally effective in suppressing operant key responding. The three experiments together suggested that the identification of important effects of species—typical behavior in one experimental situation does not imply that there will be like effects in similar situations.     

An investigation of peak shift and behavioral contrast for autoshaped and operant behavior.

Instrumental treadle press and nonreinforced key peck responses were monitored during discrimination training and generalization testing in pigeons on positive and negative reinforcement schedules. In Experiment 1, six pigeons pressed a treadle for food on a multiple variable-interval extinction schedule. In Experiment 2, three pigeons pressed a treadle to avoid shock on a multiple free-operant avoidance extinction schedule. Different color keylights signaled S+ and S- components. Some positive behavioral contrast occurred during discrimination training, but the effect was small. Pecking occurred to the S+ keylight in Experiment 1 but not in Experiment 2. On stimulus generalization tests, all subjects displayed a positive peak shift when pressing the treadle for food or to avoid shock. However, peak shift was not found for nonreinforced "autopecks" on the stimulus key, although an area shift was observed in Experiment 1. This is the first demonstration of peak shift for pigeons pressing treadles and the only reliable demonstration of peak shift when negative reinforcement maintained responding. These results, in combination with previous demonstrations of peak shift for rats pressing levers and pigeons pecking keys, indicate that peak shift is a general by-product of operant discrimination learning, since it occurs across a variety of the organisms, responses, and reinforcers.     

Satiation, capacity, and within-session responding

Responding may change substantially over the course of a session (McSweeney, Hinson, & Cannon, 1996). The role of satiation in this effect was investigated in three experiments. Experiment 1 showed that the capacity of pigeons to consume milo over a 1-hr period was relatively stable across three different methods of measurement. In Experiment 2, pigeons were divided into two groups that differed in their capacity based on one of those measures. Key pecking was then reinforced under a variable-interval 30-s schedule with hopper durations of 2 or 5 s. According to the satiation hypothesis, subjects with small capacities should satiate faster and therefore show greater decreases in food-reinforced responding than would subjects with larger capacities. The results confirmed this prediction and showed that the magnitudes of within-session decreases were better predicted by the amount an animal consumed relative to its capacity than by absolute amount alone. In Experiment 3, each pigeon was prefed 0, 5, 15, or 25 g of milo prior to each session. Consistent with the satiation hypothesis, increases in prefeeding produced lower overall response rates in the smaller capacity subjects than in the larger capacity subjects at each level of prefeeding. These experiments demonstrate the importance of a new variable in the control of behavior, and provide a recommended technique for its measurement.     

Response-independent Events In The Behavior Stream

The metaphor of the behavior stream provides a framework for studying the effects of response-independent food presentations intruded into an environment in which operant responding of pigeons was maintained by variable-interval schedules. In the first two experiments, response rates were reduced when response-independent food was intruded during the variable-interval schedule according to a concomitantly present fixed-time schedule. These reductions were not always an orderly function of the percentage of response-dependent food. Negatively accelerated patterns of key pecking across the fixed-time period occurred in Experiment 1 under the concomitant fixed-time variable-interval schedules. In Experiment 2, positively and negatively accelerated and linear response patterns occurred even though the schedules were similar to those used in Experiment 1. The variable findings in the first two experiments led to three subsequent experiments that were designed to further illuminate the controlling variables of the effects of intruded response-independent events. When the fixed and variable schedules were correlated with distinct operanda by employing a concurrent fixed-interval variable-interval schedule (Experiment 3) or with distinct discriminative stimuli (Experiments 4 and 5), negatively accelerated response patterns were obtained. Even in these latter cases, however, the response patterns were a joint function of the physical separation of the two schedules and the ratio of fixed-time or fixed-interval to variable-interval schedule food presentations. The results of the five experiments are discussed in terms of the contributions of both reinforcement variables and discriminative stimuli in determining the effects of intruding response-independent food into a stream of operant behavior.     

Response strength in multiple periodic and aperiodic schedules

Responding in multiple periodic and aperiodic schedules of equal mean reinforcement rate was examined during extinction, satiation, and in the presence of various free-food schedules. In Experiments I and II, pigeons were trained on multiple variable-interval–fixed-interval schedules. Decreases in the rate of responding due to extinction, satiation, or food schedules were approximately equal regardless of the temporal pattern of reinforcer presentation. In Experiment III, pigeons responded on a two-component multiple schedule in which each component was a two-member homogeneous response chain terminating in a fixed-interval schedule during one component and in a variable-interval schedule during the other. The length of both terminal links was varied over a series of conditions. Initial-link responding in the fixed-interval component was reduced more by increasing terminal-link length than was initial-link responding in the variable-interval component. However, no differences in resistance to satiation and extinction were obtained across the fixed and variable components. If the relative decrease in responding produced by satiation and extinction is used as an index of the “value” of the conditions maintaining responding, then these data suggest that fixed and variable schedules of equal mean length are equally valued. This conclusion, however, is not consistent with findings of preference for variable over fixed schedules obtained in studies using concurrent-chain procedures.     

Interaction of methadone, reinforcement history, and variable-interval performance.

In the present study, we examined how a reinforcement schedule history that generated high or low rates of responding influenced the effects of acute (Experiment 1) and chronic (Experiment 2) methadone administration. Initially, key-peck responses of pigeons were maintained under a variable-interval 90-s schedule of food presentation, and a methadone dose-response curve was determined with doses of 0.6, 1.2, and 2.4 mg/kg. The pigeons were then exposed, for at least 40 sessions, to either a fixed-ratio 50 schedule or a differential-reinforcement-of-low-rate 10-s schedule, or were given continued exposure to the variable-interval schedule. The methadone dose-response curve was redetermined after all pigeons again were responding under the variable-interval schedule. The effects of two different daily methadone doses (9.0 and 12.0 mg/kg/day) and withdrawal precipitated by naloxone also were assessed. Experience with a fixed-ratio or differential reinforcement of low rate schedule did not result in significantly different response rates under the variable-interval schedule and, in general, the acute effects of methadone did not have differential effects correlated with schedule history. However, for 2 of 4 subjects the rate-decreasing effects of methadone on rates of key pecking were greater following a history of low-rate responding, suggesting a possible interaction between schedule history and effects of methadone. Daily methadone administration under the variable-interval schedule revealed that pigeons with experience under the differential reinforcement of low rate schedule developed more rapid and complete tolerance to the rate-decreasing effects of methadone. Three of the 4 subjects in this group showed rate increases above drug-free baselines during chronic methadone dosing. Pigeons with a history of fixed-ratio responding also developed tolerance to the rate-decreasing effects of methadone but without the subsequent rate increases seen by subjects with low-rate histories. No subjects with variable-interval histories showed complete recovery of drug-free baselines, suggesting that interpolated training under other schedules may attenuate the rate-altering effects of chronically administered drugs. Naloxone (1.0 mg/kg), administered during the chronic methadone phase, resulted in greater disruption of responding by pigeons with a history of low-rate responding, as compared to subjects in the other two groups.(ABSTRACT TRUNCATED AT 400 WORDS)     

Resurgence of temporal patterns of responding

The resurgence of temporal patterns of key pecking by pigeons was investigated in two experiments. In Experiment 1, positively accelerated and linear patterns of responding were established on one key under a discrete-trial multiple fixed-interval variable-interval schedule. Subsequently, only responses on a second key produced reinforcers according to a variable-interval schedule. When reinforcement on the second key was discontinued, positively accelerated and linear response patterns resurged on the first key, in the presence of the stimuli previously correlated with the fixed- and variable-interval schedules, respectively. In Experiment 2, resurgence was assessed after temporal patterns were directly reinforced. Initially, responding was reinforced if it approximated an algorithm-defined temporal pattern during trials. Subsequently, reinforcement depended on pausing during trials and, when it was discontinued, resurgence of previously reinforced patterns occurred for each pigeon and for 2 of 3 pigeons during a replication. The results of both experiments demonstrate the resurgence of temporally organized responding and replicate and extend previous findings on resurgence of discrete responses and spatial response sequences.     

Interactions in multiple schedules: negative induction with squirrel monkeys

In Experiment I, lever pressing by squirrel monkeys was maintained under a sequence of variable-interval, multiple variable-interval variable-interval, and multiple variable-interval extinction schedules of food presentation. Negative induction (decreased responding in the unchanged component) occurred when one component of the multiple variable-interval variable-interval schedule was changed to extinction. Negative induction was transient over sessions; responding in the unchanged component usually recovered to a rate similar to that under the multiple variable-interval variable-interval schedule. Negative induction was not accompanied by consistent changes in the patterns of local responding within the unchanged component, and did not depend on whether component schedules were associated with localized (lever lights) or diffuse visual stimuli (houselights), or on whether the unchanged component was a 60- or 180-sec variable-interval schedule. In Experiment II, responding was maintained under a sequence of variable-interval and multiple variable-interval timeout schedules of food presentation. Negative induction occurred when responding declined gradually in the timeout component but not when responding declined abruptly. The nature of interactions in multiple schedules may depend on the species; negative induction was observed with squirrel monkeys under conditions similar to those that produce positive contrast with pigeons.     

Altering Reinforcer Variety or Intensity Changes the Within-Session Decrease in Responding

Operant responding often changes systematically within experimental sessions. McSweeney, Hinson, and Cannon (1996) argued that sensitization and habituation produce within-session changes in responding. The present study tested two predictions of the sensitization–habituation explanation. In two experiments, rats pressed a lever for reinforcers delivered by a multiple variable interval 15-s variable interval 15-s schedule. In Experiment 1, the variety of reinforcers delivered during the session was manipulated by varying the percentage of programmed reinforcers replaced with qualitatively different reinforcers from 0 to 75%, in five different conditions. In Experiment 2, the intensity of the reinforcer was manipulated by varying the concentration of sucrose in the sucrose and water solution used as the reinforcer from 0 to 30%, in five different conditions. Increasing the variety or the intensity of the reinforcers slowed the within-session decrease in responding. The results are consistent with the predictions of a sensitization–habituation explanation of within-session changes in responding.     

Food-paired stimuli as conditioned reinforcers: effects of d-amphetamine.

Seven pigeons were studied in two experiments in which key pecks were reinforced under a second-order schedule wherein satisfaction of variable-interval schedule requirements produced food or a brief stimulus. In the second part of each session, responses produced only the brief stimulus according to a variable-interval schedule (food extinction). For the 4 pigeons in Experiment 1, the response key was red throughout the session. In separate phases, the brief stimulus was either paired with food, not paired with food, or not presented during extinction. d-Amphetamine (0.3 to 10.0 mg/kg) dose-dependently reduced food-maintained responding during the first part of the session and, at intermediate dosages, increased responding during the extinction portion of the session. The magnitude of these increases, however, did not consistently depend on whether the brief stimulus was paired, not paired, or not presented. It was also true that under nondrug conditions, response rates during extinction did not differ reliably depending on pairing operations for the brief stimulus. In Experiment 2, 3 different pigeons responded under a procedure wherein the key was red in the component with food presentations and blue in the extinction component (i.e., multiple schedule). Again, d-amphetamine produced dose-related decreases in responding during the first part of a session and increases in responding in the second part of the session. These increases, however, were related to the pairing operations; larger increases were observed when the brief stimulus was paired with food than when it was not or when it was not presented at all. Under nondrug conditions, the paired brief stimulus controlled higher response rates during extinction than did a nonpaired stimulus or no stimulus. These findings suggest that d-amphetamine can enhance the efficacy of conditioned reinforcers, and that this effect may be more robust if conditioned reinforcers occur in the context of a signaled period of extinction.     

Choice and foraging

In Experiment 1, six naive pigeons were trained on a foraging schedule characterized by different states beginning with a search state in which completion of a fixed-interval on a white key led to a choice state. In the choice state the subject could, by appropriate responding on a fixed ratio of three, either accept or reject the schedule of reinforcement that was offered (either a variable-interval five-second or a variable-interval 20-second). If the subject accepted the schedule, it entered a “handling state” in which the appropriate variable-interval schedule was presented. Completion of the variable-interval schedule produced food. The independent variable was the fixed-interval value in the search state, and the dependent variable was the rate of acceptance of the long variable-interval in the choice state. Experiment 2 was identical except that the search state required completion of a variable-interval, instead of a fixed-interval, schedule. The rate of acceptance of the long variable-interval schedule in both experiments was a direct function of the length of the search state, in accordance with both optimality theory and the delay-reduction hypothesis.     

Behavioral contrast without response-rate reduction

Behavioral contrast was obtained in two experiments, which both employed a standard free-operant successive discrimination (a multiple variable-interval extinction schedule), without the occurrence of reductions of response rate in the extinction component. In Experiment I, one group of four pigeons was trained on a multiple schedule in which one stimulus was associated with a variable-interval schedule and the second stimulus with response-independent reinforcement on a free variable-interval schedule. Though by the end of this training three pigeons were responding very little to the second stimulus, when this stimulus was associated with extinction all subjects showed a contrast effect. In Experiment II, eight pigeons were trained extensively to respond to a single stimulus on a variable-interval schedule, before a second stimulus associated with extinction was introduced. This second stimulus was dissimilar to the initial stimulus and five pigeons never responded in its presence. Nevertheless, all pigeons showed a contrast effect and there was no evidence that the effect was smaller in errorless subjects or smaller than in a subsequent discrimination where all subjects made many errors. Both experiments indicated that response reduction in one component of a multiple schedule is not a necessary condition for the occurrence of contrast.     

Criticisms of the satiety hypothesis as an explanation for within-session decreases in responding

The authors of four papers recently reported that satiation provides a better explanation than habituation for within-session decreases in conditioned responding. Several arguments question this conclusion. First, the contribution of habituation to within-session changes in responding seems clearly established. Information that is consistent with habituation, but that is difficult to reconcile with satiation, is not adequately addressed. Second, the limited evidence offered in support of satiation is ambiguous because the results are just as compatible with habituation as with other satiety variables. Finally, the term satiation is used in an intuitive way that is sometimes contradicted by research about the termination of ingestion. Use of the technical term satiation in a way that differs from its conventional usage will only isolate operant psychology from other areas of psychological research.     

Within-session Changes In Responding During Concurrent Variable-interval Schedules

Five rats and 4 pigeons responded for food delivered by several concurrent variable-interval schedules. The sum of the rates of reinforcement programmed for the two components varied from 15 to 480 reinforcers per hour in different conditions. Rates of responding usually changed within the experimental session in a similar manner for the two components of each concurrent schedule. The within-session changes were similar to previously reported changes during simple schedules that provided rates of reinforcement equal to the sum of all reinforcers obtained from the concurrent schedules. The number of changeovers also changed within sessions in a manner similar to the changes in instrumental responding. These results suggest that changeovers are governed by the same variables that govern instrumental responding. They also suggest that the within-session change in responding during each component of a concurrent schedule is determined by approximately the sum of the reinforcers obtained from both components when both components provide the same type of reinforcer.     

Resistance to change and preference for variable versus fixed response sequences

In Experiment 1, 4 pigeons were trained on a multiple chain schedule in which the initial link was a variable-interval (VI) 20-s schedule signalled by a red or green center key, and terminal links required four responses made to the left (L) and/or right (R) keys. In the REPEAT component, signalled by red keylights, only LRLR terminal-link response sequences were reinforced, while in the VARY component, signalled by green keylights, terminal-link response sequences were reinforced if they satisfied a variability criterion. The reinforcer rate for both components was equated by adjusting the reinforcer probability for correct REPEAT sequences across sessions. Results showed that initial- and terminal-link responding in the VARY component was generally more resistant to prefeeding, extinction, and response-independent food than responding in the REPEAT component. In Experiment 2, the REPEAT and VARY contingencies were arranged as terminal links of a concurrent chain and the relative reinforcer rate was manipulated across conditions. For all pigeons, initial-link response allocation was biased toward the alternative associated with the VARY terminal link. These results replicate previous reports that operant variation is more resistant to change than operant repetition (Doughty & Lattal, 2001), and show that variation is preferred to repetition with reinforcer-related variables controlled. Behavioral momentum theory (Nevin & Grace, 2000) predicts the covariation of preference and resistance to change in Experiments 1 and 2, but does not explain why these aspects of behavior should depend on contingencies that require repetition or variation.     

Conditioned suppression as a sensitive baseline for social facilitation

The key pecking of pigeons maintained on a variable-interval schedule of food reinforcement was suppressed during occasional presentations of a warning stimulus paired with electric shock. On alternate sessions, a co-actor pigeon was visible in an adjoining chamber where it emitted the same food-reinforced key peck during the warning stimulus that signalled shock for the subject. With no shock and at low shock intensities, where the subject's responding was not suppressed or suppressed only slightly, the co-actor had little effect. At the higher shock intensities, where the subject's responding was reduced by at least 40%, the response rate during the warning stimulus was consistently higher when the co-actor was present. One explanation of these results assumes a special relationship between social stimuli and aversive stimuli in which the presence of another animal reduces emotional reactions and thereby allows operant responses to increase. This was not the case here because the mere presence of the co-actor did not maintain social facilitation. Rather, the present results, taken in conjunction with previous findings, suggest that changes in social and non-social variables which affect the rate of food-reinforced responding may produce proportionately larger changes in responding when that responding is suppressed by aversive stimulation than when it is not.     

Within-session Analysis Of Visual Discrimination

Within-session changes in responding by pigeons during a maintained successive discrimination procedure were examined in four experiments. In the first two experiments, which involved discrimination of visual flicker rate, within-session changes in responding were minimal or absent. A third experiment, which examined discrimination of rectangular forms, demonstrated that the absence of within-session changes in responding was not limited to flicker-rate stimuli. A fourth experiment showed that the absence of within-session changes in responding was not due to high task difficulty in the previous experiments. For the group of subjects in each experiment, within-session changes in responding did not influence discrimination performance. Therefore, measures of overall response rate accurately represented responding both within and across sessions. The occasional appearance of within-session decreases in responding for a few birds may be attributable to satiation.