The effects of cocaine on behavior maintained by timeout from avoidance.

Rats were trained on concurrent schedules under which responses on one lever postponed shock (avoidance) and responses on the other lever produced brief (2-min) periods of signaled timeout from avoidance. For 6 rats, timeout from avoidance was programmed on a variable-interval 45-s schedule that generally resulted in rates that were lower than those on the avoidance lever. For another 6 rats, timeout was arranged on a variable-ratio 15 schedule that produced higher baseline rates. Cocaine (3 to 40 mg/kg) produced large, dose-dependent increases in behavior maintained by timeout in both groups of rats. Avoidance responding was also generally increased by cocaine, but the increases were of lesser magnitude. Increases in response rates were seen across a broad range of doses on behavior maintained by either interval or ratio schedules, an outcome that was unexpected on the basis of most studies of cocaine on food-maintained behavior. These results were similar to those of previous studies of the effects of amphetamine on behavior maintained by timeout from avoidance and suggest that stimulant drugs affect behavior maintained under a shock-postponement schedule differently than they affect behavior maintained by timeout from avoidance.     

Increased reinforcement when timeout from avoidance includes access to a safe place

Three experiments investigated the reinforcing value of access to a safe place during timeout from an avoidance schedule. Rats were trained on conjoint schedules in which responding both postponed shock on a free-operant avoidance schedule and produced periods of timeout on fixed-ratio schedules. In some conditions, a shelf was inserted into the operant chamber during timeout, enabling subjects to get off the grid floor. The combination of timeout and shelf maintained substantially higher response rates than the baseline avoidance schedule with ratio requirements as high as 90 (Experiment I). Adding the shelf to timeouts in one component of multiple fixed-ratio schedules of timeout resulted in higher response rates in the component where the shelf was included (Experiment II). When timeouts with and without the shelf were arranged on concurrent schedules, the shelf-timeout combination was preferred, even when of shorter duration than timeout alone (Experiment III). In all three experiments, subjects climbed on the shelf, although all shocks were cancelled during timeout periods. The results could not be accounted for solely in terms of the reinforcing properties of changes in shock rates, but required an interpretation that ascribed conditioned reinforcing value to stimuli associated with such changes.     

Schedules using noxious stimuli. IV: An interlocking shock-postponement schedule in the squirrel monkey

Responding was studied under various schedules of electric shock postponement and presentatation in the squirrel monkey. Under an interlocking shock-postponement schedule, successive responses decreased the time by which a response postponed the next scheduled shock until a shock immediately followed the nth response. Some parameters of this schedule, which can be formally related to fixed-interval schedules, engendered a pattern of positively accelerated responding between shocks. This pattern did not occur under comparable parameter values of an alternative fixed-ratio, avoidance schedule under which each response postponed shock by a fixed duration and every nth response produced shock. Subsequently, performances were studied under schedules of shock presentation. Responding was never maintained under fixed-ratio schedules of shock presentation, but was maintained with a pattern of positive acceleration under an alternative fixed-ratio, fixed-interval schedule and under a fixed-interval schedule.     

Variable-ratio schedules of timeout from avoidance: effects of d-amphetamine and morphine.

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other lever produced periods of signaled timeout from avoidance on variable-ratio schedules. These procedures generated high rates of timeout-reinforced responding and provided a baseline for studying the effects of drugs on behavior maintained by different types of negative reinforcement (shock postponement vs. timeout). Morphine (2.5 to 10.0 mg/kg) reduced behavior maintained by timeout at doses that increased or had no effect on avoidance responding. In contrast, d-amphetamine (0.125 to 2.0 mg/kg) produced large increases in timeout responding at doses that had minimal effect on avoidance in rats trained on variable-interval and variable-ratio schedules. Thus, the event-dependent effects of morphine, observed in previous studies in which timeout responding was maintained at low rates by interval schedules, were replicated with high timeout rates maintained by variable-ratio schedules. The effects of d-amphetamine could also be described as "event dependent" because timeout responding was stimulated more than avoidance regardless of the maintenance schedule or baseline rate.     

Effects of cycle length on performance on a temporally defined avoidance schedule

Three rats were trained on a temporally defined avoidance schedule logically similar to a fixed-interval, limited-hold positive reinforcement schedule. This avoidance schedule was composed of time periods during which responses had no scheduled consequences alternating with time periods during which a response precluded shock. As with fixed-interval length and response rate on positive reinforcement schedules, an inverse relationship was obtained between the length of the no-consequence interval and response rate during the no-consequence interval. An inverse relationship was also obtained between the length of the no-consequence interval and the per cent of shocks avoided. A rate increase within the no-consequence interval, similar to that typically produced by fixed-interval positive reinforcement procedures, was displayed by one of the rats where the no-consequence interval was at intermediate values and frequency of shock was relatively high. The introduction of a discriminative stimulus correlated with the avoidance interval produced typical discriminated avoidance behavior as well as alterations in temporal patterning of responses during the no-consequence interval in the two rats exposed to this procedure. These alterations in temporal patterning disappeared when the discriminative stimulus was removed. The results were consonant with those reported in the literature involving food reinforcement and fixed-interval, limited-hold schedules.     

Concurrent avoidance of shocks by pigeons pecking a key

Three pigeons were studied on concurrent, unsignaled, avoidance schedules in a two-key procedure. Shock-shock intervals were two seconds in both schedules. The response-shock interval on one key was always 22 seconds, while the response-shock interval associated with the other key was varied from 7 to 52 seconds in different experimental conditions. Response rates on the key associated with the varied schedule tended to decrease when the response-shock interval length was increased. Responding on the key associated with the constant schedule was not systematically affected.     

Variable-interval schedules of timeout from avoidance

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other occasionally produced a 2-min timeout during which the shock-postponement schedule was suspended and its correlated stimuli were removed. Throughout, the shock-postponement schedule maintained proficient levels of avoidance. Nevertheless, in Experiment 1 responding on the timeout lever was established rapidly, was maintained at stable levels on variable-interval schedules, was extinguished by withholding timeout, was reestablished when timeout was reintroduced, and was brought under discriminative control with a multiple variable-interval extinction schedule of timeout. These results are in contrast with Verhave's (1962) conclusion that timeout is an ineffective reinforcer when presented to rats on intermittent schedules. In Experiment 2 the consequence of responding on the timeout lever was altered so that the shock-postponement schedule remained in effect even though the stimulus conditions associated with timeout were produced for 2 min. Responding extinguished, indicating that suspension of the shock-postponement schedule, not stimulus change, was the source of reinforcement. By establishing the reinforcing efficacy of timeout with standard variable-interval schedules, these experiments illustrate a procedure for studying negative reinforcement in the same way as positive reinforcement.     

Choice of a terminating over a non-terminating signal in free-operant avoidance

Rats chose between two signalled avoidance schedules. Under one schedule, responses in the presence of the signal terminated it and resulted in avoidance of shock; in the other, responses in the presence of the signal resulted in shock avoidance but signal termination was delayed for 10 sec. Pressing a second (changeover) lever produced change from one schedule to the other for 1 min. Once this 1-min period timed out, subjects could remain under the schedule in effect or could reinstate the other schedule for another 1-min period. All four subjects continuously changed over from the non-terminating to the terminating signal schedule. Changeover responding was not maintained when the termination contingency was removed. When changeover responding resulted in a change from a terminating to a non-terminating signal schedule, changeover responding did not occur.     

Performance of humans in variable-interval avoidance schedules programmed singly, and concurrently with variable-interval schedules of positive reinforcement

Performance maintained under single variable-interval avoidance schedules, single variable-interval schedules of positive reinforcement, and concurrent schedules consisting of a variable-interval avoidance component and a variable-interval positive reinforcement component, was studied in three human subjects, using points exchangeable for money as the reinforcer. Response rate in the single variable-interval avoidance schedules was an increasing function of the frequency of monetary loss avoidance. Response rate in the single variable-interval positive reinforcement schedules was an increasing function of the frequency of obtained monetary reinforcement. In the concurrent avoidance/reinforcement schedules, the rate of responding in the avoidance component increased, and the rate of responding in the positive reinforcement schedule decreased (with one exception) as a function of the frequency of loss avoidance in the avoidance component. The logarithms of the ratios of the response rates in the two components, and the logarithms of the ratios of the times spent in the two components, were linearly related to the logarithms of the ratios of the frequency of loss avoidance in the avoidance component to the frequency of reinforcement in the positive reinforcement component. All three subjects exhibited marked undermatching of response rate ratios to reinforcement frequency ratios. The results are discussed in the context of Herrnstein's quantitative model of operant performance.     

The influence of positive and negative reinforcement on selective attention in the rat

In Experiments 1 and 2 rats were trained under two multiple schedules of reinforcement. In one, bar pressing during a tone-light compound stimulus was reinforced under a variable-interval food reinforcement schedule. In the other multiple schedule, bar pressing avoided grid shock on a free-operant schedule. In both multiple schedules, a discrimination was maintained by an extinction schedule that was operative during the absence of the tone-light compound. In Experiments 1 and 2 the intensity of the tone-light compound was manipulated over three levels. Subsequent extinction tests revealed that light was attended to, almost exclusively of the tone, when food reinforcement had maintained bar pressing. On the other hand, the tone gained considerable attentional control under the shock avoidance schedule. This stimulus-reinforcer interaction was maintained for all three levels of the compound intensity. In Experiment 3 it was investigated whether this interaction was associative by presenting shock during the absence of the tone-light compound when food reinforcement maintained responding, and food during the absence of the compound when shock avoidance maintained responding. Since both food and shock were presented during a single session for both schedules, nonassociative effects of the reinforcing stimuli were equivalent across the schedules. Nevertheless, the stimulus-reinforcer interaction was maintained, indicating that the interaction was an associative effect.     

Long-Term conditioned fear modification in the dog as measured by changes in urinary 11-hydroxycorticosteroids. Heart rate and behavior

Heart rate (HR), operant key pressing, and urinary 11-hydroxycorticosteroids (11-OH-CS) were recorded while dogs were subjected to various aversive conditioning schedules over a six-month period. The schedules consisted of Sidman avoidance, followed by a Sidman schedule which paired unavoidable shocks (US) with offset of seven discrete conditioned stimuli (CS). Introduction of the Sidman avoidance schedule led to an increase in HR. Changes in the experimental schedule always results in an increase in urinary 11-OH-CS. Neither HR nor urinary 11-OH-CS were correlated with rate of operant key pressing. Differences in the dependent variables reflected the different topological characteristics of the dogs studied. This study demonstrated: (a) a causal independence of physiological and behavioral responses conditioned to the same stimulus complex, and (b) that longterm increases in HR can be maintained if the experimental situation is manipulated to maintain the fear eliciting characteristics of the conditioning situation.     

Punishment and the potential for negative reinforcement with histamine injection

The present study examined punishment of responding with histamine injection, and its potential to generate avoidance of punishment. Sprague–Dawley rats were trained under concurrent schedules in which responses on one lever (the punishment lever) produced food under a variable‐interval schedule, and under some conditions intermittent injections of histamine, which suppressed behavior. Responses on a second (avoidance) lever prevented histamine injections scheduled on the punishment lever. After stabilization of punished responding, a variable‐interval 15‐s schedule of cancellation of histamine (avoidance) was added for responding on the second/avoidance lever, without subsequent acquisition of responding on that lever. Progressive decreases in the length of the punishment variable‐interval schedule increased suppression on the punishment lever without increases in response rates on the avoidance lever. Exchanging contingencies on the levers ensured that response rates on the avoidance lever were sufficiently high to decrease the histamine injection frequency; nonetheless response rates on the avoidance lever decreased over subsequent sessions. Under no condition was responding maintained on the avoidance lever despite continued punishing effectiveness of histamine throughout. The present results suggest that avoidance conditioning is not a necessary condition for effective punishment, and confirm the importance of empirical rather than presumed categorization of behavioral effects of stimulus events.     

Maintenance of responding by squirrel monkeys under a concurrent shock-postponement, fixed-interval shock-presentation schedule.

A chain-pulling response was initially developed under a shock-postponement (avoidance) schedule with two squirrel monkeys. Few responses occurred on a lever where responding initially had no scheduled consequence or, subsequently, when a 3-minute fixed-interval shock-presentation schedule was concurrently arranged for lever responses. Appropriate rates and patterns of lever responding developed and were later maintained under the fixed-interval 3-minute shock-presentation schedule alone when the chain and shock-postponement schedule were removed. When both the shock-postponement and shock-presentation schedules were again simultaneously in effect, steady rates of chain pulling were maintained by the shock-postponement schedule and positively accelerated rates and patterns were maintained on the lever by the shock-presentation schedule. Response rates under both schedules were directly related to shock intensity. A history of exposure to a shock-postponement schedule, even though with a topographically different response and manipulandum, was sufficient for the development and eventual maintenance of responding by the presentation of shock. Further, differential performances can be maintained simultaneously by the presentation and postponement of electric shock.     

Acquisition and maintenance of postshock response pattern in nondiscriminated avoidance with rats

Five experimentally naive albino rats were placed under a nondiscriminated lever-press avoidance schedule in which the delay to the next shock for responses after a shock was longer than the delay for responses after a response. Four rats acquired the postshock response pattern and maintained it for a prolonged period. The results revealed that postshock responding was under operant control and was not purely shock-elicited. It was suggested that the two kinds of response-shock interval, i.e. the shock-response-shock interval and the response-response-shock interval, could and should be independently controlled in nondiscriminated avoidance schedules.     

Development of a complex multiple schedule in the chimpanzee

The development of chimpanzee behavior on a four-component, three-lever multiple schedule is described. Component schedules included the Sidman avoidance procedure with a concurrent discriminated avoidance schedule on a second lever, fixed ratio performance for food, differential reinforcement of low rate for water requiring a dual response chain, and a symbol discrimination task for continuous food reinforcement using three levers. The avoidance component of this schedule was employed during the January 31, 1961 suborbital space flight of the chimpanzee “Ham.” On November 29, 1961, the chimpanzee “Enos” performed on the multiple schedule during three orbits around the earth in a Mercury capsule.     

Variable-interval schedules of timeout from avoidance: effects of chlordiazepoxide, CGS 8216, morphine, and naltrexone.

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other occasionally (variable-interval schedule) produced a 2-min timeout during which the shock-postponement schedule was suspended and its correlated stimuli were removed. These procedures provided a baseline for studying the effects of drugs on behavior maintained by different sources of negative reinforcement (shock avoidance and timeout from avoidance). Experiment 1 studied a benzodiazepine agonist, chlordiazepoxide, and antagonist, CGS 8216. Chlordiazepoxide (2.5-30 mg/kg) had little effect on avoidance responding except at higher doses, when it reduced responding. By comparison, responding on the timeout lever was increased in 5 of 6 rats. These effects were reversed by CGS 8216 (2.5-5 mg/kg) in the 2 rats tested, but CGS 8216 had no effect by itself. Experiment 2 studied an opiate agonist, morphine, and antagonist, naltrexone, with 3 rats. Morphine's (2.5-20 mg/kg) effects were opposite those of chlordiazepoxide: At doses that either increased or had no effect on avoidance responding, morphine depressed timeout responding. Naltrexone (5 mg/kg) reversed these actions but had no effect by itself.     

Schedules of electric shock presentation in the behavioral control of imprinted ducklings

The behavioral effects of various schedules of electric shock presentation were investigated during and after the imprinting of Peking ducklings to moving stimuli. The behavior of following a moving imprinted stimulus was differentially controlled by a multiple schedule of punishment and avoidance that respectively suppressed and maintained following behavior. Pole-pecking, reinforced by presentations of the imprinted stimulus, was suppressed by response-produced shock (punishment); various schedules of response-independent shock and delayed punishment had an overall minimal effect. The delivery of response-independent shock in the presence of one of two stimuli, both during and after imprinting, resulted in a marked reduction in choice of the stimulus paired with shock. The experiments provide no support for a differentiation of imprinting from learning on the basis of the behavioral effects of aversive stimuli. Instead, as is the case with other organisms, the schedule under which shock is delivered to imprinted ducklings appears to be an important determinant of the temporal patterning of subsequent behavior.     

Reductions in shock frequency and response effort as factors in reinforcement by timeout from avoidance

Rats' presses on one lever canceled shocks programmed after variable cycles, while presses on a second lever occasionally produced a 2-min timout during which the shock-delection schedule was suspended and its correlated stimuli removed. These concurrent schedules of avoidance and timeout were embedded in a multiple schedule whose components differed, within and across conditions, in terms of the programmed shock rate associated with the shock-deletion schedule. Analyses based on the generalized matching law suggest that the reduction in the response requirement correlated with termination of the avoidance schedule was a more important factor in the reinforcing effectiveness of timeout than was shock-frequency reduction, at least in 2 of 3 rats. After training in each condition, responding on the timeout lever was extinguished by withholding timeouts in both components over seven sessions. Resistance to extinction varied directly with the rates of both shock-frequency reduction and avoidance-response reduction experienced during training. Although reduction in response effort appeared to dominate shock-frequency reduction in the maintenance of responding, neither factor had a clear advantage in predicting the course of extinction.     

Punishment contras during free-operant avoidance

Punishment of bar-pressing responses of rhesus monkeys with electric shock in one component of a multiple free-operant avoidance schedule suppressed responding in that component. These decreases were concomitant with response rate increases in the unpunished component (punishment contrast). Response rates in both components increased when punishment was removed and decreased in successive sessions. These effects of punishment on unpunished responding were similar to those obtained during single and multiple schedules of positive reinforcement and they suggest a further similarity in the development of discriminations during positive and negative reinforcement schedules.     

Choice, changeover, and travel: A quantitative model

Six pigeons were trained on concurrent variable-interval schedules in which responding on fixed-interval schedules was required to give access to the alternate schedule. Responding on the concurrent schedules was not allowed, after changing over had commenced, until the changeover schedule had been completed. In Parts 1 to 3 of the experiment, the changeover fixed-interval schedules were equal and were 0 s, 10 s, and 20 s, respectively. In each part, the relative frequency of reinforcement obtained on the concurrent schedules was varied over at least five conditions. In Part 4, the concurrent schedules were equal, and one changeover fixed-interval schedule was twice the other. Under these conditions, the absolute sizes of the changeover schedules were varied. Increasing the changeover requirement from 0 s to 10 s (Parts 1 and 2) resulted in increases in the sensitivity of behavior allocation to reinforcers obtained, but no further increase was obtained when the changeover schedules were increased to 20 s (Part 3). In Part 4, performance was biased towards the concurrent schedule that took less time to enter. These results are consistent with a subtractive punishment model of travel in which the degree of punishment is measured by the number of reinforcers apparently lost from a schedule when the subject changes to that schedule. Absolute times spent on the main keys could be accurately described by a previous model of changeover performance.