Some effects of methylphenidate on stimulus control of ratio avoidance behavior in the rat

After exposure to an avoidance schedule which included a warning signal, a rat was placed on a multiple schedule in which the first component was the same as before, i.e., a single response reset the response-shock interval, delaying shock, and the second component differed only in that four bar-presses were required to postpone shock. A fixed ratio requirement of four responses (FR 4) generated behavior resembling a fixed ratio requirement of one response (FR 1) since responding was controlled by the warning signal but more shocks were received. At a dosage of 2 mg/kg, methylphenidate given intraperitoneally decreased shock frequency during FR 4 periods while FR 1 behavior was not affected; at 4 mg/kg, stimulus control of avoidance responding was impaired during both components. Results at 4 mg/kg were partially confirmed by two animals exposed to an FR 4 avoidance schedule which included a warning signal but with different parameters. Response distributions showed that methylphenidate increased response rates in the absence of the warning signal, i.e., stimulus control of ratio-avoidance behavior was impaired although the increased response rates reduced shock frequency. One hour later responses again occurred more frequently during the signal than in its absence but shocks were less frequent than during control (non-drug) periods.     

Free operant avoidance as a function of the response-shock=shock-shock interval

Two temporal parameters of free operant or Sidman avoidance behavior are the interval by which responses postpone shocks (Response-Shock interval) and the interval between shocks when no responses occur (Shock-Shock interval). Avoidance behavior was examined in three white rats under conditions where the Response-Shock and Shock-Shock intervals were always equal. With intervals from 10 to 60 sec response rates and shock rates were similar, decreasing, negatively accelerated functions of increasing Response-Shock=Shock-Shock interval. Over this range, response and shock rates were linearly related to the reciprocal of the Response-Shock=Shock-Shock interval. It was shown, however, that this relation cannot hold at extremely long intervals. Both the ratio of responses emitted to shocks received and the percentage of shocks possible which were avoided increased at long Response-Shock=Shock-Shock intervals. These findings may be related to the fact that long intervals provide optimal conditions for conditioning avoidance behavior in the rat.     

Discriminated punishment: avoidable and unavoidable shock

Warning stimuli for two punishment conditions were alternated with periods of appetitive responding by rats. In either warning stimulus, the first response produced a brief shock, terminated the stimulus, and started an interval during which the baseline appetitive schedule was in effect. Not responding resulted in stimuli of random duration, which terminated with a shock under one condition and without a shock under the other. Each subject was exposed to several shock intensities, with trials for the two conditions programmed during alternate portions of the session. In general, response frequency in the warning signal for either condition decreased with increasing intensity; however, at a given intensity, responding was more frequent in the stimulus invariably terminating with shock than in the stimulus terminating without shock when no response was made. The frequency difference was greatest at intensities intermediate between those producing minimal and maximal suppression.     

Varying temporal location of a conditioned stimulus in heart rate conditioning ofMacaca mulatta

The several functions that a stimulus can assume were investigated in a Pavlovian conditioning procedure. The subjects were six rhesus monkeys; the response under observation was heart rate. The conditioning began with a temporal separation of zero between a signal and a regularly repeating electric shock; the signal was then moved to a series of earlier locations in the inter-shock interval. After six sessions at each location, two sessions followed in which only the shock was delivered periodically. The findings included: (1) A two-phased conditioned cardiac rate response seen at the first location became more multiphasic and irregular during longer intervals between signal and shock; (2) the location where the conditioned response peaked became increasingly variable as the signal was moved back, but this variability maintained a constant proportion to the signal-shock interval; and, (3) heart rate during a presignal period, and during a comparable period in shock only sessions, was generally deceleratory early in training and acceleratory thereafter. Sessions with the signal showed heart rate in the presignal period to have become acceleratory earlier in training than sessions with shock only. The data pertain to stimulus control over heart rate as a function of: (A) the temporal proximity of a signal to an aversive stimulus; and, (B) the presence or absence of the signal. The use of appropriate response units in cardiac conditioning is also discussed.     

Relative durations of conditioned stimulus and intertrial interval in conditioned suppression.

The effects of the relative durations of the conditional stimulus and the intertrial interval on bar pressing during a conditioned-suppression procedure were examined as a function of two additional variables--type of operant baseline schedule and rate of shock presentation. In Experiment 1, response suppression was compared across components of a multiple fixed-ratio, random-ratio, fixed-interval, random-interval schedule, at relative conditioned-stimulus/intertrial-interval durations of 1/1, 1/4, and 1/9. In Experiment 2, relative conditioned-stimulus/intertrial-interval duration (1/5, 3/3, or 5/1) was manipulated across groups, while shock frequency (2, 6, or 10 shocks/hr) was manipulated within groups. In both experiments, suppression during the signal was virtually complete at all relative durations. Responding was also suppressed during the intertrial interval, but that suppression varied as a function of experimental manipulations. In Experiment 1, intertrial-interval response rates were higher when relative signal duration was 1/9 than when it was 1/1, although both relative signal duration and shock frequency, which covaried, could have contributed to the difference. In Experiment 2, the patterning of response rates between successive shocks was affected by relative duration, absolute rates during the intertrial interval varied as a function of shock frequency, and differences between suppression during the signal and suppression during the intertrial interval were affected by both relative duration and shock frequency. The data support an analysis based upon relationships between shock-correlated and intertrial-interval stimuli and, as assessed by the relative-delay-to-reinforcement metric, are comparable to results that have been reported from experiments using similar manipulations under the autoshaping paradigm.     

Free operant avoidance in rats under increased nitrogen pressures.

Rats performed on a free operant avoidance schedule with a response-shock interval of 20 sec. and a shock-shock interval of 2 sec. Avoidance response rates increased and shock frequency decreased when the rats were exposed to elevated pressures of both air and a nitrogen-oxygen mixture in a hyperbaric chamber. Increases in response rates were related to raised partial pressures of nitrogen at 89.0 psi and 111.3 psi. Conditional probabilities of interresponse times indicated that increases in response rates were not due to disruption of temporal discrimination. Increased avoidance rates under pressure suggested direct excitatory effects of high pressures of nitrogen.     

Selective punishment early and late in fixed-ratio schedules of food reinforcement

Pigeons key pecked for grain on a fixed-ratio 100 schedule; electric shocks occurred intermittently at the fifteenth or eighty-fifth response in the ratio. In Experiment I, shock was at the fifteenth response for two birds, and at the eighty-fifth response for two others, in every sixth, twelfth, or eighteenth ratio. Rate of responding decreased as frequency of shock increased, and the pattern of responding included an increased initial pause and low rates or pause-run sequences that extended further into the ratio when shock was at the fifteenth response than when it was at the eighty-fifth response. Shock early in the ratio engendered longer initial pauses than shock late in the ratio. In Experiment II, four birds responded on a two-component multiple schedule in which shock occurred at the fifteenth response of the third ratio in the presence of a white keylight and at the eighty-fifth response of the third ratio in the presence of a green keylight. The overall rates of responding decreased as shock intensity increased. All four birds responded differentially to the white and green keylights, but with a pattern that varied between birds. In general, punishment reduced the probability of responses that preceded it, regardless of the ordinal position of those responses. Both studies confirm that the probability of responding is reduced less by aversive stimuli produced late in a fixed-ratio than by aversive stimuli produced early in a fixed-ratio.     

ASSESSING THE ROLE OF EFFORT REDUCTION IN THE REINFORCING EFFICACY OF TIMEOUT FROM AVOIDANCE

Rats responded on concurrent schedules of shock‐postponement or deletion (avoidance) and timeout from avoidance. In Experiment 1, 3 rats' responses on one lever postponed shocks for 20 s and responses on a second lever produced a 1‐min timeout according to a variable‐interval 45‐s schedule. Across conditions, a warning signal (white noise) was presented 19.5 s, 16 s, 12 s, 8 s, or 4 s before an impending shock. Raising the duration of the warning signal increased both avoidance and timeout response rates. Timeout responding, although positively correlated with avoidance responding, was not correlated with the prevailing shock rate. In Experiment 2, 3 rats' responses on one lever deleted scheduled shocks according to a variable‐cycle 30‐s schedule and responses on a second lever produced a 2‐min timeout as described above. After this baseline condition, the avoidance lever was removed and noncontingent shocks were delivered at intervals yoked to the receipt of shocks in the baseline sessions. Timeout responding decreased when the avoidance lever was removed, even though the shock‐frequency reduction afforded by the timeout remained constant. These results suggest that a key factor in the reinforcing efficacy of timeout is suspension of the requirement to work to avoid shock, rather than the reduction in shock frequency associated with timeout.     

The puzzle of responding maintained by response-contingent shock.

Four squirrel monkeys were first exposed to a sequence of procedures that reliably generate responding maintained by brief response-contingent electric shocks arranged according to a fixed-interval schedule. After responding had become stable on the fixed-interval schedule, additional contingencies were added in tandem, whereby after completion of the interval, the spacing of responses affected shock delivery. In one procedure, responses had to be spaced more widely than their previous median value if shock were to be delivered. In the other procedure, responses had to be spaced more closely to produce shock. On the first of these procedures, decreased but stable responses rates would indicate that shock functioned as a positive reinforcer; on the second, increased response rates would indicate the positively reinforcing function. Instead, response rates accelerated on the procedure that targeted more widely spaced responses for shock delivery, and decelerated or ceased on the procedure that arranged for shocks to be produced by more closely spaced responses. Consistent with other recent findings, these results question the interpretation of performances maintained by response-contingent shock as engendered by positive reinforcement and are consistent with aversive-control interpretations. The details of that aversive control are not entirely clear, however, and these same procedures would be informative if applied to shock-maintained behavior that is generated in other ways.     

Some factors involved in the comparison of response systems: acquisition, extinction, and transfer of head-poke and lever-press Sidman avoidance

Head poking, a suggested natural escape reaction to shock for the rat, was compared to lever pressing in a Sidman avoidance study. Both responses could be emitted at any time, but only one was effective in a given session. Acquisition and extinction of the two responses were compared under both signalled and unsignalled avoidance. Then, a test for transfer was conducted in which acquisition conditions were re-instated, but the effectiveness of the responses was reversed. Three differences between responses were noted: (a) head poking was superior in reducing shock rates under signalled conditions; (b) head poking was more resistant to extinction, especially under signalled conditions; (c) under unsignalled conditions, animals were unable to learn to head poke if they had previously learned to lever press. Findings a and c were pursued in later experiments. Finding a depended on the location of the warning signal with respect to the response system. When the lever press required approach to the warning signal, the head poke was superior. But when the head poke required approach to the warning signal, the two responses were equally effective. Finding c depended on the absence of feedback for head poke during transfer. Two conclusions are offered: first, the two responses appear to obey the same laws when their topographical differences are taken into account. Second, response feedback appears to be more critical in transfer than in original acquisition.     

The effect of punishment shock intensity upon responding under multiple schedules

In the first of two experiments, responses of two pigeons were maintained by multiple variable-interval, variable-ratio schedules of food reinforcement. Concurrent punishment was introduced, which consisted of a brief electric shock after each tenth response. The initial punishment intensities had no lasting effect upon responding. Then, as shock intensity increased, variable-ratio response rates were suppressed more quickly than variable-interval response rates. When shock intensity decreased, variable-interval responding recovered more quickly, but the rates under both schedules eventually returned to their pre-punishment levels. In the second experiment, the following conditions were studied in three additional pigeons: (1) With each shock intensity in effect for a number of sessions, punishment shock intensity was gradually increased and decreased and responding was maintained by multiple variable-ratio, fixed-ratio schedules of food reinforcement; (2) Changes in punishment shock intensity as described above with responding maintained by either a variable-ratio or a fixed-ratio schedule, which were presented on alternate days; (3) Session-to-session changes in shock intensity with responding maintained by multiple variable-ratio, fixed-ratio schedules. Responding under the two schedules was suppressed to approximately the same extent by a particular shock intensity. Also, post-reinforcement pauses under the fixed-ratio schedule increased as response suppression increased.     

The effects of reinforcement frequency and response requirements on the maintenance of behavior.

Six rats responded under fixed-interval and tandem fixed-interval fixed-ratio schedules of food reinforcement. Basic fixed-interval schedules alternated over experimental conditions with tandem fixed-interval fixed-ratio schedules with the same fixed-interval value. Fixed-interval length was varied within subjects over pairs of experimental conditions; the ratio requirement of the tandem schedules was varied across subjects. For both subjects with a ratio requirement of 10, overall response rates and running response rates typically were higher under the tandem schedules than under the corresponding basic fixed-interval schedules. For all subjects with ratio requirements of 30 or 60, overall response rates and running response rates were higher under the tandem schedules than under the corresponding basic fixed-interval schedules only with relatively short fixed intervals. At longer fixed intervals, higher overall response rates and running rates were maintained by the basic fixed-interval schedules than by the tandem schedules. These findings support Zeiler and Buchman's (1979) reinforcement-theory account of response strength as an increasing monotonic function of both the response requirement and reinforcement frequency. Small response requirements added in tandem to fixed-interval schedules have little effect on reinforcement frequency and so their net effect is to enhance responding. Larger response requirements reduce reinforcement frequency more substantially; therefore their net effect depends on the length of the fixed interval, which limits overall reinforcement frequency. At the longest fixed intervals studied in the present experiment, reinforcement frequency under the tandem schedules was sufficiently low that responding weakened or ceased altogether.     

Lick-shock contingencies in the rat

Hungry rats were allowed to lick an 8% sucrose solution and then one of four lick-shock contingency conditions was superimposed on the licking baseline. These conditions were: free-operant avoidance, free shock, punishment, and no shock. From highest to lowest response rates, the groups fell in the order-avoidance, no shock, free shock, and punishment. Lick rates adjusted rapidly to introduction and removal of the contingencies. Post-shock responding was lowest in the punishment condition and highest in the free shock condition. No method was found simultaneously to equate shock frequency and separate response rates for the three shock contingency conditions. Only small, or no, reductions in shock rate occurred over sessions under the free-operant avoidance schedule when the shock-shock interval was 10 sec but large reductions occurred when the shock-shock interval was reduced to either 1 or 2 sec.     

Length of experimental session as a parameter of response rate under a non-discriminated

Following the stabilization of response rate under an avoidance schedule which was defined by two temporal parameters, the shock-shock interval and the interval by which each response postponed the onset of shock, the length of the experimental session was changed. It was found that after the subjects had been exposed to a longer session of avoidance schedule, their rates of response were considerably increased without a corresponding reduction in the number of shocks received.

In recent years considerable use has been made of an avoidance training technique in which the performance of the response functions to postpone the onset of an aversive stimulus, usually shock, by a fixed period. In the absence of the required response the aversive stimulus is programmed to occur at regular intervals. Experiments by Sidman (1953) have shown that the critical independent variable controlling the rate of avoidance response, is the shock postponement interval (R*S). All other things being equal, the rat in the lever pressing situation will respond at a rate which is inversely related to the R*S interval, low intervals generating high response rates and high intervals generating low response rates. However, under very low values of R*S, the response rate may break down altogether. The animal then receives shock at the rate determined by the shock-shock interval parameter.

As a result of an apparatus failure, Sidman, Herrnstein and Conrad (1957) discovered that the response rate can also be increased by occasionally shocking the animal in spite of its avoidance responses. An apparatus failure has also been responsible for the isolation of yet another parameter of response rate in the shock-postponement avoidance situation and is reported here. Briefly, it was found that a change in the duration of an experimental session influences the response rate on subsequent sessions.     

Timeout postponement without increased reinforcement frequency

Three experiments were conducted to examine pigeons' postponement of signaled extinction periods (timeouts) from a schedule of food reinforcement when such responding neither decreased overall timeout frequency nor increased the overall frequency of food reinforcement. A discrete-trial procedure was used in which a response during the first 5 s of a trial postponed an otherwise immediate 60-s timeout to a later part of that same trial but had no effect on whether the timeout occurred. During time-in periods, responses on a second key produced food according to a random-interval 20-s schedule. In Experiment 1, the response-timeout interval was 45 s under postponement conditions and 0 s under extinction conditions (responses were ineffective in postponing timeouts). The percentage of trials with a response was consistently high when the timeout-postponement contingency was in effect and decreased to low levels when it was discontinued under extinction conditions. In Experiment 2, the response-timeout interval was also 45 s but postponement responses increased the duration of the timeout, which varied from 60 s to 105 s across conditions. Postponement responding was maintained, generally at high levels, at all timeout durations, despite sometimes large decreases in the overall frequency of food reinforcement. In Experiment 3, timeout duration was held constant at 60 s while the response-timeout interval was varied systematically across conditions from 0 s to 45 s. Postponement responding was maintained under all conditions in which the response-timeout interval exceeded 0 s (the timeout interval in the absence of a response). In some conditions of Experiment 3, which were designed to control for the immediacy of food reinforcement and food-correlated (time-in) stimuli, responding postponed timeout but the timeout was delayed whether a response occurred or not. Responding was maintained for 2 of 3 subjects, suggesting that behavior was negatively reinforced by timeout postponement rather than positively reinforced by the more immediate presentation of food or food-correlated (time-in) stimuli.     

Avoidance response rates during a pre-food stimulus in monkeys

Free-operant avoidance responding was maintained by a shock avoidance schedule in three monkeys. The frequency of avoidance responses during a stimulus terminated by response-independent food pellet presentation was dependent upon the method of pellet delivery. Avoidance rates were relatively increased when food retrieval responses followed pellet delivery. Avoidance rates were decreased when retrieval responses preceded pellet delivery.     

Responding of pigeons under variable-interval schedules of signaled-delayed reinforcement: effects of delay-signal duration.

Two experiments with pigeons examined the relation of the duration of a signal for delay ("delay signal") to rates of key pecking. The first employed a multiple schedule comprised of two components with equal variable-interval 60-s schedules of 27-s delayed food reinforcement. In one component, a short (0.5-s) delay signal, presented immediately following the key peck that began the delay, was increased in duration across phases; in the second component the delay signal initially was equal to the length of the programmed delay (27 s) and was decreased across phases. Response rates prior to delays were an increasing function of delay-signal duration. As the delay signal was decreased in duration, response rates were generally higher than those obtained under identical delay-signal durations as the signal was increased in duration. In Experiment 2 a single variable-interval 60-s schedule of 27-s delayed reinforcement was used. Delay-signal durations were again increased gradually across phases. As in Experiment 1, response rates increased as the delay-signal duration was increased. Following the phase during which the signal lasted the entire delay, shorter delay-signal-duration conditions were introduced abruptly, rather than gradually as in Experiment 1, to determine whether the gradual shortening of the delay signal accounted for the differences observed in response rates under identical delay-signal conditions in Experiment 1. Response rates obtained during the second exposures to the conditions with shorter signals were higher than those observed under identical conditions as the signal duration was increased, as in Experiment 1. In both experiments, rates and patterns of responding during delays varied greatly across subjects and were not systematically related to delay-signal durations. The effects of the delay signal may be related to the signal's role as a discriminative stimulus for adventitiously reinforced intradelay behavior, or the delay signal may have served as a conditioned reinforcer by virtue of the temporal relation between it and presentation of food.     

Drugs and punished responding II: d-amphetamine-induced increases in punished responding

The effects of d-amphetamine on punished responding were studied in two experiments. In Experiment I, pigeons responded under a multiple fixed-ratio 30 response fixed-interval 5-min schedule of food presentation with 60-sec limited holds in both components. Each response was punished with electric shock, the intensity of which was varied systematically. In Experiment II, another group of pigeons responded under a multiple fixed-interval 5-min fixed-interval 5-min schedule of food presentation with 40-sec limited holds. Each response was punished with shock during one component, and every thirtieth response was punished in the other component. d-Amphetamine increased overall rates of punished responding only rarely under any of the punishment conditions; however, response rates within the fixed-interval when rates were low were increased by d-amphetamine when the shock intensity was low (Experiment I), or when responses produced shock intermittently (Experiment II). The data suggest that the effects of d-amphetamine on punished responding depend on the control rate of responding, the punishment intensity, the punishment frequency, and the schedule of food presentation.     

Postreinforcement signal processing

Postreinforcement signal processing by rats was demonstrated in six experiments that used a discrete-trials choice procedure. Experiment 1 assessed the extent to which rats are able to transfer knowledge about associations between postreinforcement signal durations and choice responses to conditions where a particular signal duration preceded the opportunity to make a choice response. In Experiment 2 the generality of the transfer effect was demonstrated by using both signal duration and signal modality as relevant stimulus attributes for the postreinforcement signals. The role of the relative durations of the reinforcement-signal gap and the intertrial interval was investigated in Experiment 3. In order to assess the effects of within-trial and between-trial signal relations on the acquisition of a temporal discrimination, both pre-and postreinforcement signals were presented on each trial in Experiments 4 and 5. The effects of pre- and postreinforcement signal relations on the steady-state performance of a temporal bisection task across three different signal ranges were studied in Experiment 6. The conclusion is that rats readily process various stimulus attributes of postreinforcement signals and that relations between postreinforcement signals, choice responses, and prereinforcement signals are major determinants of choice behavior.     

Selective punishment of interresponse times

Lever pressing by two squirrel monkeys was maintained under a variable-interval 60-second schedule of food presentation. When response-dependent electric shock was made contingent on comparatively long interresponse times, response rate increased, and further increases were obtained when the minimum interresponse-time requirement was decreased. When an equal proportion of responses produced shock without regard to interresponse time, rates decreased. Thus, shock contingent on long interresponse times selectively decreased the relative frequency of those interresponse times, and increased the relative frequency of shorter interresponse times, whereas shock delivered independent of interresponse times decreased the relative frequency of shorter interresponse times while increasing the frequency of longer ones. The results provide preliminary evidence that interresponse times may be differentiated by punishment, further supporting the notion that interresponse times may be considered functional units of behavior.