Persistence and relapse of reinforced behavioral variability

The present study examined persistence and relapse of reinforced behavioral variability in pigeons. Pigeons emitted four‐response sequences across two keys. Sequences produced food according to a lag schedule, in which a response sequence was followed by food if it differed from a certain number of previous sequences. In Experiment 1, food was delivered for sequences that satisfied a lag schedule in both components of a multiple schedule. When reinforcement was removed for one component (i.e., extinction), levels of behavioral variability decreased for only that component. In Experiment 2, food was delivered for sequences satisfying a lag schedule in one component of a multiple schedule. In the other component, food was delivered at the same rate, but without the lag variability requirement (i.e., yoked). Following extinction, levels of behavioral variability returned to baseline for both components after response‐independent food delivery (i.e., reinstatement). In Experiment 3, one group of pigeons responded on a lag variability schedule, and the other group responded on a lag repetition schedule. For both groups, levels of behavioral variability increased when alternative reinforcement was suspended (i.e., resurgence). In each experiment, we observed some evidence for extinction‐induced response variability and for variability as an operant dimension of behavior.     

Resistance to change of operant variation and repetition

A multiple chained schedule was used to compare the relative resistance to change of variable and fixed four-peck response sequences in pigeons. In one terminal link, a response sequence produced food only if it occurred infrequently relative to 15 other response sequences (vary). In the other terminal link, a single response sequence produced food (repeat). Identical variable-interval schedules operated in the initial links. During baseline, lower response rates generally occurred in the vary initial link, and similar response and reinforcement rates occurred in each terminal link. Resistance of responding to prefeeding and three rates of response-independent food delivered during the intercomponent intervals then was compared between components. During each disruption condition, initial- and terminal-link response rates generally were more resistant in the vary component than in the repeat component. During the response-independent food conditions, terminal-link response rates were more resistant than initial-link response rates in each component, but this did not occur during prefeeding. Variation (in vary) and repetition (in repeat) both decreased during the response-independent food conditions in the respective components, but with relatively greater disruption in repeat. These results extend earlier findings demonstrating that operant variation is more resistant to disruption than is operant repetition and suggest that theories of response strength, such as behavioral momentum theory, must consider factors other than reinforcement rate. The implications of the results for understanding operant response classes are discussed.     

Drugs and punished responding II: d-amphetamine-induced increases in punished responding

The effects of d-amphetamine on punished responding were studied in two experiments. In Experiment I, pigeons responded under a multiple fixed-ratio 30 response fixed-interval 5-min schedule of food presentation with 60-sec limited holds in both components. Each response was punished with electric shock, the intensity of which was varied systematically. In Experiment II, another group of pigeons responded under a multiple fixed-interval 5-min fixed-interval 5-min schedule of food presentation with 40-sec limited holds. Each response was punished with shock during one component, and every thirtieth response was punished in the other component. d-Amphetamine increased overall rates of punished responding only rarely under any of the punishment conditions; however, response rates within the fixed-interval when rates were low were increased by d-amphetamine when the shock intensity was low (Experiment I), or when responses produced shock intermittently (Experiment II). The data suggest that the effects of d-amphetamine on punished responding depend on the control rate of responding, the punishment intensity, the punishment frequency, and the schedule of food presentation.     

Effects of ethanol on reinforced variations and repetitions by rats under a multiple schedule.

Response sequences emitted by five Long-Evans rats were reinforced under a two-component multiple schedule. In the REPEAT component, food pellets were contingent upon completion of a left-left-right-right (LLRR) sequence on two levers. In the VARY component, pellets were contingent upon variable sequences (i.e., a sequence was reinforced only if it differed from each of the previous five sequences). The rats learned to emit LLRR sequences in the REPEAT component and variable sequences in VARY. Intraperitoneal injections of ethanol (1.25, 1.75, and 2.25 g/kg) significantly increased sequence variability in REPEAT, thereby lowering reinforcement probability, but had little effect on sequence variability in the VARY component. These results extend previous findings that alcohol impairs the performance of reinforced repetitions but not of reinforced variations in response sequences.     

Effects of d-amphetamine on responding under second-order schedules of reinforcement with paired and nonpaired brief stimuli.

Three pigeons were studied under a multiple schedule in which pecks in each component were reinforced according to a variable-interval 120-s second-order schedule with fixed-interval 60-s units. In the first component of the multiple schedule, the completion of a fixed interval produced either food or a 4-s change in key color plus houselight illumination. In the second component an identical schedule was in effect, but the stimulus was a 0.3-s change in key color. Both long and short brief stimuli were not paired with food presentations in Conditions 1 and 3 and were paired with food in Condition 2. There were no consistent differences in response patterns under paired and nonpaired brief-stimulus conditions when the stimulus was a 4-s change in key color accompanied by houselight illumination. However, pairing the 0.3-s key-color change with food presentations resulted in higher indices of curvature and lower response rates in the early segments of the fixed interval than when the stimulus was not paired with food presentations. Low doses of d-amphetamine (0.3 and 1 mg/kg) produced small and inconsistent increases in overall response rates, and higher doses (3 and 10 mg/kg) decreased overall response rates. d-Amphetamine altered response patterns within fixed intervals by decreasing the indices of curvature and increasing response rates in the early segments of the fixed interval. Response rates and patterns under paired and nonpaired brief-stimulus conditions were not differentially affected by d-amphetamine. Thus, evidence for the enhancement of the conditioned reinforcement effects of psychomotor stimulant drugs was not found with the second-order schedules used in the present study.     

The roles of stimulus control and reinforcement frequency in modulating the behavioral effects of d-amphetamine in the rat.

The behavioral effects of d-amphetamine have been shown to be modulated by stimulus control, with less impairment of performance occurring when control is great. When the fixed-consecutive-number schedule is used (on which at least a specified consecutive number of responses must be made on one operandum before a single response on another will produce a reinforcer), response rate tends to be invariant but reinforcement frequency is not. This study asks whether the differences in reinforcement frequency that usually accompany changes in stimulus control could themselves be responsible for the performance differences. Two versions of the fixed-consecutive-number schedule of reinforcement were combined into a multiple schedule within which stimulus control was varied but differences in reinforcement frequency were minimized by omitting some reinforcer deliveries during the component that usually had the higher reinforcement frequency. In one component, a compound discriminative stimulus was added with the eighth consecutive response on the first lever; a single response on the second lever was then reinforced. In the other component, no such stimulus was presented. With no added stimulus, large decreases occurred in the number of runs satisfying the minimum requirement for reinforcement at doses of drug that produced only minimal changes when an added stimulus controlled behavior. Thus, increased stimulus control diminishes the behavioral changes produced by d-amphetamine even when the possible contribution by baseline reinforcement rate is minimized.     

Human d-amphetamine drug discrimination: methamphetamine and hydromorphone.

Standard measures of subjective and discriminative effects of drugs were compared in 5 human volunteers. Subjects responded on a second-order color-tracking procedure, where 30 mg of d-amphetamine served as a discriminative stimulus for one response and its absence as the discriminative stimulus for another response. Self-reported subjective effects were assessed concurrently using the single-dose questionnaire, subscales of the Addiction Research Center Inventory, and several analogue rating scales. On different days following discrimination acquisition, varying doses of d-amphetamine, methamphetamine, and hydromorphone were administered. In these test sessions, either response was reinforced. Methamphetamine and d-amphetamine occasioned dose-related increases in d-amphetamine appropriate responding; hydromorphone did not. Methamphetamine and d-amphetamine occasioned dose-related increases in reports of the drug received being most like "speed"; hydromorphone occasioned dose-related increases in reports of the drug received being most like "dope." All three drugs occasioned dose-related increases in reports of drug liking, and increases in the morphine-benzedrine group, amphetamine, and benzedrine group scales of the Addiction Research Center Inventory. This experiment demonstrated that although explicit discriminative control of behavior by a drug may covary with drug identification, it does not necessarily covary with other self-reported subjective effects. Thus, the complementary nature of the data provided by drug discrimination and standard subjective-effects measures provides quantitative and qualitative data useful in studying both relatively novel compounds and the behavioral biology of psychoactive drugs in general.     

Separate neural sites for d-amphetamine suppression of mouse killing and feeding behavior in rats

The present study was conducted to investigate several possible neural sites for d-amphetamine's effect on mouse killing and feeding behaviors. d-Amphetamine (10, 20, and 30 μg) injected into each lateral ventricle, suppressed mouse kiling, food, and water intake in a dose-dependent manner. Bilateral adminstration of d-amphetamine (20 μg) into the central amygdaloid nucleus abolished mouse killing behavior but did not affect feeding and drinking. By contrast, bilateral amphetamine injections into the substantia nigra, or into the ventral region of the caudate nucleus, did not suppress mouse killing behavior, but significantly decreased food and water intake. The lateral hypothalamus was sensitive to d-amphetamine injections, which suppressed mouse killing and food intake as well as water intake. d-Amphetamine injections into the nucleus accumbens produced inconsistent effects on mouse killing and feeding. Our observations suggest a differentiation of the neural sites that mediate feeding from those underlying mouse killing behavior.     

An experimental analysis of the effects of d-amphetamine and cocaine on the acquisition and performance of response chains in monkeys.

In one component of a multiple schedule of food presentation, monkeys acquired a different four-response chain each session by responding sequentially on three keys in the presence of four geometric forms (learning). In the other component, the four-response chain was the same each session (performance). Both d-amphetamine and cocaine, at the higher doses, disrupted the behavior in the learning component; the overall response rate decreased, the overall accuracy was impaired (i.e., percent errors increased), and there was less within-session error reduction. The performance component was generally less sensitive than the learning component to the disruptive effects of both drugs on rate and accuracy. After pre-feeding or during an extended session, the response rate decreased in both components, but accuracy was generally unaffected. When the four discriminative stimuli in both components were removed, the behavior was disrupted to a greater extent in the performance component. The disruptive effects of both drugs on behavior in the learning component were attenuated when the drugs were administered during the session after the response chain had been acquired. It was concluded that the greater sensitivity of the learning component to disruptive drug effects is related to the relatively weak stimulus control and/or the lower rate of reinforcement associated with that component.     

Resistance to change and preference for variable versus fixed response sequences

In Experiment 1, 4 pigeons were trained on a multiple chain schedule in which the initial link was a variable-interval (VI) 20-s schedule signalled by a red or green center key, and terminal links required four responses made to the left (L) and/or right (R) keys. In the REPEAT component, signalled by red keylights, only LRLR terminal-link response sequences were reinforced, while in the VARY component, signalled by green keylights, terminal-link response sequences were reinforced if they satisfied a variability criterion. The reinforcer rate for both components was equated by adjusting the reinforcer probability for correct REPEAT sequences across sessions. Results showed that initial- and terminal-link responding in the VARY component was generally more resistant to prefeeding, extinction, and response-independent food than responding in the REPEAT component. In Experiment 2, the REPEAT and VARY contingencies were arranged as terminal links of a concurrent chain and the relative reinforcer rate was manipulated across conditions. For all pigeons, initial-link response allocation was biased toward the alternative associated with the VARY terminal link. These results replicate previous reports that operant variation is more resistant to change than operant repetition (Doughty & Lattal, 2001), and show that variation is preferred to repetition with reinforcer-related variables controlled. Behavioral momentum theory (Nevin & Grace, 2000) predicts the covariation of preference and resistance to change in Experiments 1 and 2, but does not explain why these aspects of behavior should depend on contingencies that require repetition or variation.     

Free-operant forgetting: Delayed stimulus control of multiple-schedule performance

Pigeons' responses were reinforced in two components of several multiple variable-interval variable-interval schedules of food reinforcement. In one component, the key was illuminated green for 15 seconds and white for 45 seconds. In the other component, the key was illuminated red for 15 seconds and white for 45 seconds. Values for the exponent of the power functions relating response ratios to reinforcement ratios were higher in the presence of the discriminative stimuli (green or red) than in the presence of white. Sensitivity of response ratios to changes in reinforcement ratios provided an index of the extent to which responding was under delayed stimulus control by prior discriminative stimuli.     

Behavior of rats under fixed consecutive number schedules: effects of drugs of abuse.

Four rats responded under a simple fixed consecutive number schedule in which eight or more consecutive responses on the run lever, followed by a single response on the reinforcement lever, produced the food reinforcer. Under this simple schedule, dose-response curves were determined for diazepam, morphine, pentobarbital, and phencyclidine. The rats were then trained to respond under a multiple fixed consecutive number schedule in which a discriminative stimulus signaled when the response requirement on the run lever had been completed in one of the two fixed consecutive number component schedules. Under control conditions, the percentage of reinforced runs under the multiple-schedule component with the discriminative stimulus added was much higher than the percentage of reinforced runs under the multiple-schedule component without the discriminative stimulus. All of the drugs decreased the percentage of reinforced runs under each of the fixed consecutive number schedules by increasing the conditional probability of short run lengths. This effect was most consistently produced by morphine. The drugs produced few differences in responding between the multiple fixed consecutive number components. Responding under the simple fixed consecutive number schedule, however, was affected at lower doses of the drugs than was responding under the same fixed consecutive number schedule when it was a component of the multiple schedule. This result may be due to the difference in schedule context or, perhaps, to the order of the experiments.     

Drug effects in squirrel monkeys trained on a multiple schedule with a punishment contingency

The behavior of four monkeys trained on a multiple schedule was differentially sensitive to selected pharmacological agents. The three components of the multiple schedule were: (1) a variable-interval schedule in which responses were reinforced on the average of once per minute; (2) a concurrent schedule in which every tenth response was reinforced and every fifteenth response, on the average, was shocked; and, (3) a neutral stimulus in the presence of which responses were neither reinforced nor shocked. Pentobarbital, chlordiazepoxide, and meprobamate increased responding during each of the components. Scopolamine and d-amphetamine decreased variable-interval performance, had minimal effects on performance during the concurrent-schedule component, and increased responding in the presence of the neutral stimulus. Chlorpromazine decreased variable-interval responding and had slight effects on the responding during the other two components.     

Effects of d-amphetamine, diazepam, and pentobarbital on the schedule-controlled pecking and locomotor activity of pigeons

Pigeons were trained on a variant of the autoshaping procedure devised by Matthews and Lerer (1987) in which a keylight stimulus ramp of increasing brightness signaled the passing of a 30-s interfood interval. This procedure generates two distinct behavioral components: key pecking and locomotor activity. The effects of three psychoactive drugs on these behavior classes were measured. d-Amphetamine had negligible effects on both types of behavior, whereas diazepam and pentobartital increased key pecking and decreased activity in a dose-dependent fashion. In Experiment 2, the possibility that drug effects were suppressed by excessively strong stimulus control exerted in Experiment 1 was tested by decreasing the discriminability of the stimulus ramp. The direction of the effects of diazepam and pentobarbital was the same as in Experiment 1 but the magnitude of the effects tended to be larger. The effects of d-amphetamine, however, remained quite small, suggesting that, under these conditions, locomotor activity and key pecking are less sensitive to d-amphetamine. In Experiments 3 and 4, key pecking was eliminated by removing the keylight. Reinforcers were presented at fixed intervals in Experiment 3 and at variable intervals in Experiment 4. The drug effects on activity observed in Experiments 1 and 2 disappeared in both Experiments 3 and 4. The results suggest that diazepam and pentobarbital affect activity indirectly by increasing key-pecking behavior, which, in turn, competitively decreases activity.     

Food-paired stimuli as conditioned reinforcers: effects of d-amphetamine.

Seven pigeons were studied in two experiments in which key pecks were reinforced under a second-order schedule wherein satisfaction of variable-interval schedule requirements produced food or a brief stimulus. In the second part of each session, responses produced only the brief stimulus according to a variable-interval schedule (food extinction). For the 4 pigeons in Experiment 1, the response key was red throughout the session. In separate phases, the brief stimulus was either paired with food, not paired with food, or not presented during extinction. d-Amphetamine (0.3 to 10.0 mg/kg) dose-dependently reduced food-maintained responding during the first part of the session and, at intermediate dosages, increased responding during the extinction portion of the session. The magnitude of these increases, however, did not consistently depend on whether the brief stimulus was paired, not paired, or not presented. It was also true that under nondrug conditions, response rates during extinction did not differ reliably depending on pairing operations for the brief stimulus. In Experiment 2, 3 different pigeons responded under a procedure wherein the key was red in the component with food presentations and blue in the extinction component (i.e., multiple schedule). Again, d-amphetamine produced dose-related decreases in responding during the first part of a session and increases in responding in the second part of the session. These increases, however, were related to the pairing operations; larger increases were observed when the brief stimulus was paired with food than when it was not or when it was not presented at all. Under nondrug conditions, the paired brief stimulus controlled higher response rates during extinction than did a nonpaired stimulus or no stimulus. These findings suggest that d-amphetamine can enhance the efficacy of conditioned reinforcers, and that this effect may be more robust if conditioned reinforcers occur in the context of a signaled period of extinction.     

Drug effects on responding maintained by stimulus-reinforcer and response-reinforcer contingencies.

The effects of pentobarbital and d-amphetamine were assessed on key pecking by pigeons under conventional single-key multiple schedules and under two-key multiple schedules in which discriminative stimuli appeared on one key (stimulus key) while pecks on a second key (constant key) produced food. Pecks on the stimulus key had no scheduled consequences. A 60-second variable-interval schedule operated in one component of each multiple schedule: either extinction or a 60-second variable-time schedule operated in the alternate component. When the alternate-component schedule was extinction, a high rate of responding was maintained in the variable-interval component of the single-key schedule; responding on both keys was maintained in the variable-interval component of the two-key schedule. Pentobarbital increased responding in the variable-interval component of the single-key schedule and increased stimulus-key, but not constant-key responding in that component of the two-key schedule. When the alternate-component schedule was changed to variable time, responding declined in the variable-interval component of the single-key schedule; stimulus-key responding was no longer maintained under the two-key schedule. Pentobarbital decreased responding in the variable-interval component of both schedules. With an exception, d-amphetamine only decreased responding in the variable-interval component of the single- and two-key schedules both when the alternate-component schedule was extinction and when it was variable time. The results suggest that the effects of pentobarbital, but not d-amphetamine, depend on the nature of the contingency (stimulus-reinforcer, response-reinforcer) that maintains responding.     

Effects of d-amphetamine and caffeine on schedule-controlled and schedule-induced responding.

The effects of d-amphetamine and caffeine were studied on rates and patterns of lever pressing and schedule-induced licking under fixed-interval schedules of food pellet presentation. In addition, the effects of caffeine were studied on lever pressing and licking under a multiple fixed-ratio fixed-interval schedule. Caffeine reduced mean overall rates of licking at lower doses than it reduced mean overall rates of pressing under the fixed-interval schedules, but the effects of caffeine on both licking and lever pressing depended largely on the control rate of responding. d-Amphetamine reduced mean overall rates of lever pressing and licking at about the same dose, but the effects of d-amphetamine also were a function of the control rate of responding.     

Effects of promazine, chlorpromazine, d-amphetamine, and pentobarbital on treadle pressing by pigeons under a signalled shock-postponement schedule.

The effects of promazine on treadle pressing to postpone the presentation of electric shock were studied in three pigeons. The effects of chlorpromazine, d-amphetamine, and pentobarbital were studied in two of these pigeons. Each treadle press postponed electric shock for 20 sec and presentation of a preshock stimulus for 14 sec. Selected doses of both promazine and chlorpromazine increased the rates of treadle pressing in all birds. The response-rate increases produced by promazine and chlorpromazine were due to increased conditional probabilities of treadle pressing both before and during the preshock stimulus. d-Amphetamine (1 and 3 mg/kg) slightly increased responding in one of the birds, but not to the extent that promazine or chlorpromazine did. In the other bird, the 10 mg/kg dose of d-amphetamine increased shock rate but did not change response rate. Some doses of d-amphetamine increased the conditional probabilities of responding both in the absence of the preshock signal and during the preshock signal in both birds. Pentobarbital only decreased response rates and increased shock rates.     

Control of responding during stimuli that precede transitions in reinforcement frequency

Pigeons' responses were reinforced on a variant of a mixed variable-interval extinction schedule of reinforcement in which the transition to the higher reinforcement rate was signaled by a trace stimulus projected on the response key prior to the onset of the component correlated with food delivery. In the first of two experiments, the duration of the trace stimulus preceding the component correlated with food delivery was varied from 1.5 to 50.0 s and in the second experiment, the reinforcement frequency in the same component was varied from 10 to 60 reinforcers per hour. Pigeons pecked at the trace stimulus preceding the onset of the component correlated with food delivery even though responding was not reinforced in its presence and only one of the changes in reinforcement rate (i.e., from extinction to reinforcement) was signaled. The rate of pecking during the trace stimulus was a function of its duration but not of the reinforcement frequency in the following component. Higher rates generally occurred at the shorter trace-stimulus durations. Component responding following the offset of the trace stimulus was under discriminative control of the trace stimulus whether or not responding occurred in the presence of the trace stimulus.     

A pharmacological examination of the resistance-to-change hypothesis of response strength.

The effects of d-amphetamine sulfate, sodium pentobarbital, haloperidol, and cholecystokinin-octapeptide were examined within the context of Nevin's (1974, 1979) resistance-to-change hypothesis of response strength. In three experiments, rats' responding was reinforced by delivery of food under chained random-interval 30-s random-interval 30-s, multiple fixed-interval 30-s fixed-interval 120-s, or multiple random-interval 30-s random-interval 120-s schedules. Each rat received several doses of each drug and changes in response rate were measured. The resistance-to-change hypothesis predicts greater disruption of response rate relative to baseline in the initial component of the chained schedule and in the 120-s component of the multiple schedules. In the chained schedule cholecystokinin-octapeptide produced greater reductions in response rate relative to baseline in the initial component. However, no differences between components were observed with haloperidol or sodium pentobarbital, and high doses of d-amphetamine reduced response rate in the terminal component relatively more than in the initial component. In the multiple schedules either no differences were observed between components or response rate was reduced more relative to baseline in the 30-s component. The data fail to support the notion that drugs may be viewed within the same context as other response disruptors such as extinction, satiation, and the presentation of alternative reinforcement.