Some effects of methylphenidate on stimulus control of ratio avoidance behavior in the rat

After exposure to an avoidance schedule which included a warning signal, a rat was placed on a multiple schedule in which the first component was the same as before, i.e., a single response reset the response-shock interval, delaying shock, and the second component differed only in that four bar-presses were required to postpone shock. A fixed ratio requirement of four responses (FR 4) generated behavior resembling a fixed ratio requirement of one response (FR 1) since responding was controlled by the warning signal but more shocks were received. At a dosage of 2 mg/kg, methylphenidate given intraperitoneally decreased shock frequency during FR 4 periods while FR 1 behavior was not affected; at 4 mg/kg, stimulus control of avoidance responding was impaired during both components. Results at 4 mg/kg were partially confirmed by two animals exposed to an FR 4 avoidance schedule which included a warning signal but with different parameters. Response distributions showed that methylphenidate increased response rates in the absence of the warning signal, i.e., stimulus control of ratio-avoidance behavior was impaired although the increased response rates reduced shock frequency. One hour later responses again occurred more frequently during the signal than in its absence but shocks were less frequent than during control (non-drug) periods.     

Methylphenidate and stimulus control of avoidance behavior

The introduction of a warning signal that preceded a scheduled shock modified the temporal distribution of free-operant avoidance responses. With response-shock and shock-shock intervals held constant, response rates increased only slightly when the response-signal interval was reduced. The result is consistent with Sidman's (1955) findings under different conditions, but at variance with Ulrich, Holz, and Azrin's (1964) findings under similar conditions. Methylphenidate in graded doses increased response rates, modifying frequency distributions of interresponse times. Drug treatment may have disrupted a "temporal discrimination" formed within the signal-shock interval. More simply, methylphenidate influenced response rates by increasing short response latencies after signal onset; this effect was more prominent than the drug's tendency to increase the frequency of pre-signal responses. When signal-onset preceded shock by 2 sec, individual differences in performance were marked; methylphenidate suppressed responding in one rat as a function of increasing dose levels to a greater degree than in a second animal, but both subjects received more shocks than under control conditions.     

The response-shock-shock-shock interval and unsignalled avoidance in goldfish

Goldfish were trained to swim back and forth in a shuttle tank to avoid unsignalled shocks. The response-shock interval and the shock-shock interval were always of equal duration; both were either 15, 30, 45, or 60 sec. Response rates varied inversely with response-shock–shock-shock interval duration, as has been found with rats. Percentage of shocks avoided was somewhat lower at the 15 sec response-shock–shock-shock interval, but otherwise did not vary systematically with changes in the interval. As the response-shock–shock-shock interval increased, the fish made increasingly more responses than necessary to avoid all shocks. Interresponse-time distributions showed that response probability rose to a maximum at about 15 to 25 sec after a response, regardless of the response-shock–shock-shock interval. Thus, at the longer intervals the fish were responding too early in the response-shock–shock-shock interval to minimize response rates.     

Key pecking as a function of response-shock and shock-shock intervals in unsignalled avoidance

Five pigeons were exposed to an unsignalled avoidance procedure where key pecks were maintained through shock postponement. Functions obtained showed an inverse relationship between rate of responding and length of the response-shock interval, while changes in the shock-shock interval had no systematic effect on response rates. The rate of shocks delivered generally decreased with increases in length of both response-shock and shock-shock intervals. Results show that key pecking in pigeons, maintained through an unsignalled avoidance procedure, was affected by changes in response-shock and shock-shock intervals in the same manner as other responses in pigeons and in rats.     

Nondiscriminated avoidance of shock by pigeons pecking a key

Four pigeons were trained to avoid shock by pecking a key on a free-operant avoidance schedule in which no exteroceptive stimulus signalled impending shock. Response rate was an inverse function of response-shock interval when shock-shock interval was held constant at 2 sec and response-shock intervals varied from 5 to 40 sec. Amphetamine increased response rates in two subjects and reserpine markedly reduced responding in one.     

Effects of response-shock interval and shock intensity on free-operant avoidance responding in the pigeon

Two experiments investigated free-operant avoidance responding with pigeons using a treadle-pressing response. In Experiment I, pigeons were initially trained on a free-operant avoidance schedule with a response-shock interval of 32 sec and a shock-shock interval of 10 sec, and were subsequently exposed to 10 values of the response-shock parameter ranging from 2.5 to 150 sec. The functions relating response rate to response-shock interval were similar to the ones reported by Sidman in his 1953 studies employing rats, and were independent of the order of presentation of the response-shock values. Shock rates decreased as response-shock duration increased. In Experiment II, a free-operant avoidance schedule with a response-shock interval of 20 sec and a shock-shock interval of 5 sec was used, and shock intensities were varied over five values ranging from 2 to 32 mA. Response rates increased markedly as shock intensity increased from 2 to 8 mA, but rates changed little with further increases in shock intensity. Shock rates decreased as intensity increased from 2 to 8 mA, and showed little change as intensity increased from 8 to 32 mA.     

Signalled free-operant avoidance of shock by pigeons pecking a key

Two pigeons were trained to peck a key under a free-operant avoidance schedule. Then, changes in key color signalled the beginning (safe period) and the end (warning period) of the response-shock interval, with a response required to change the key color. Finally, a change in key color signalled the warning period and either a response or a shock reinstated the safe stimulus. During signalled avoidance, response rate was higher during the warning stimulus than during the safe stimulus. More responding tended to occur in the warning stimulus when it was terminated by either a response or a shock than by only a response. In either procedure, response latency during the warning stimulus was a function of the duration of the warning stimulus. In general, response and shock rate were higher during unsignalled than during signalled avoidance. When the warning stimulus was brief, the results were similar to those of unsignalled avoidance. These results confirm previous findings with pigeons, are in general agreement with data provided by other species in studies of signalled avoidance, and thereby indicate the transituationality of the key-pecking operant.     

Stress-induced breakdown of an appetitive discrimination

Rats trained to discriminate between S^D and S^Δ for food reinforcement showed marked impairments in this discrimination when strong, unavoidable shocks occurred at the termination of a third stimulus. The predominant feature of this impairment was a supernormal rate of unreinforced (S^Δ) behavior. Shocks delivered without exteroceptive warning also led to a discriminative breakdown. The effect was a direct function of shock intensity. When behavior was strongly suppressed in the third stimulus by response-correlated shock (“punishment”), instead of unavoidable shock, breakdowns were only temporary; as soon as responding recovered from its overall suppression, discriminative performance returned to normal. The discriminative deterioration may be interpreted as an emotional by-product of frequent aversive stimulation, but accidental contingencies could also have played a role.     

Free-operant avoidance conditioning in individual and paired human subjects

Male, medical and graduate students were subjected to a non-discriminated avoidance regimen with shock-shock and response-shock intervals of 10 sec. Using a yoked-chair procedure it was found that acquisition of the button-pressing avoidance response was influenced by the social environment in which the conditioning occurred. There was a significantly greater number of “learners” among subjects conditioned individually than among those exposed to the conditioning procedures in the presence of a second person.     

Partial avoidance contingencies: Absolute omission and punishment probabilities

Avoidance contingencies were defined by the absolute probability of the conjunction of responding or not responding with shock or no shock. The “omission” probability (ρ₀₀) is the probability of no response and no shock. The “punishment” probability (ρ₁₁) is the probability of both a response and a shock. The traditional avoidance contingency never omits shock on nonresponse trials (ρ₀₀=0) and never presents shock on response trials (ρ₁₁=0). Rats were trained on a discrete-trial paradigm with no intertrial interval. The first lever response changed an auditory stimulus for the remainder of the trial. Shocks were delivered only at the end of each trial cycle. After initial training under the traditional avoidance contingency, one group of rats experienced changes in omission probability (ρ₀₀>0), holding punishment probability at zero. The second group of rats were studied under different punishment probability values (ρ₁₁>0), holding omission probability at zero. Data from subjects in the omission group looked similar, showing graded decrements in responding with increasing probability of omission. These subjects approximately “matched” their nonresponse frequencies to the programmed probability of shock omission on nonresponse trials, producing a very low and approximately constant conditional probability of shock given no response. Subjects in the punishment group showed different sensitivity to increasing absolute punishment probability. Some subjects decreased responding to low values as punishment probability increased, while others continued to respond at substantial levels even when shock was inevitable on all trials (noncontingent shock schedule). These results confirm an asymmetry between two dimensions of partial avoidance contingencies. When the consequences of not responding included occasional omission of shock, all subjects showed graded sensitivity to changes in omission frequency. When the consequences of responding included occasional shock delivery, some subjects showed graded sensitivity to punishment frequency while others showed control by overall shock frequency as well.     

The effects of aversive conditioned stimuli on the timing of unsignalled avoidance responding in rats

Rats received unsignalled avoidance training in a shuttlebox. After acquisition, classical conditioning sessions were administered. The Excitatory Group received signals paired with shock, the Inhibitory Group received signals explicitly unpaired with shock, while the Random Control Group received signals and shocks randomly in time. When subsequently superimposed on the avoidance baseline, the excitatory signal increased response rate, the inhibitory signal decreased rate and the random signal had no effect compared to signal-only control data. Unsignalled avoidance generates temporal gradients of responding; the major purpose of this study was to determine how these rate changes were reflected in the baseline temporal behavior. Very clear and simple summation rules emerged: In the presence of the excitatory signal, rats acted as if they were ahead of their actual location in the response-shock interval regardless of where the signal occurred; the inhibitory signal had the opposite effect while the random signal did not change the temporal behavior relative to signal-only control data. These results were interpreted within a motivational framework.     

Extinction of Sidman avoidance behavior

Extinction of Sidman avoidance behavior by eliminating the noxious stimulus was studied in Sprague-Dawley rats with bar-pressing as the response. Each of three subjects was trained and extinguished on each of the following schedules in a different order: nondiscriminated, response-shock interval = 20 sec, shock-shock interval = 5 sec; nondiscriminated, response-shock interval = 40 sec, shock-shock interval = 5 sec; discriminated, response-white noise interval = 15 sec, noise-shock interval = 5 sec, shock-shock interval = 5 sec. Less than one 4-hr session was required for extinction for all procedures. When a warning stimulus was present, resistance to extinction increased. Subjects did not, however, respond to avoid the signal. Only small differences in extinction were found after training on different schedules with no warning signal.     

ASSESSING THE ROLE OF EFFORT REDUCTION IN THE REINFORCING EFFICACY OF TIMEOUT FROM AVOIDANCE

Rats responded on concurrent schedules of shock‐postponement or deletion (avoidance) and timeout from avoidance. In Experiment 1, 3 rats' responses on one lever postponed shocks for 20 s and responses on a second lever produced a 1‐min timeout according to a variable‐interval 45‐s schedule. Across conditions, a warning signal (white noise) was presented 19.5 s, 16 s, 12 s, 8 s, or 4 s before an impending shock. Raising the duration of the warning signal increased both avoidance and timeout response rates. Timeout responding, although positively correlated with avoidance responding, was not correlated with the prevailing shock rate. In Experiment 2, 3 rats' responses on one lever deleted scheduled shocks according to a variable‐cycle 30‐s schedule and responses on a second lever produced a 2‐min timeout as described above. After this baseline condition, the avoidance lever was removed and noncontingent shocks were delivered at intervals yoked to the receipt of shocks in the baseline sessions. Timeout responding decreased when the avoidance lever was removed, even though the shock‐frequency reduction afforded by the timeout remained constant. These results suggest that a key factor in the reinforcing efficacy of timeout is suspension of the requirement to work to avoid shock, rather than the reduction in shock frequency associated with timeout.     

Ratio responding as a function of concurrent avoidance schedules, yoked shocks, and ratio value

Fixed-ratio food-reinforced responding in rats was studied alone and with concurrent shock avoidance or with concurrent response-independent shocks matched to those that occurred in the avoidance condition. Under each condition, fixed-ratio size was increased over successive daily sessions. Fixed-ratio response rate generally passed through a maximum as a function of fixed-ratio size. Decreased fixed-ratio responding at values beyond the maximum occurred when (1) the time to complete a fixed ratio approximated the response-shock interval of the avoidance schedule, (2) the shock rate increased, and/or (3) the ratio requirements were so high that ratio strain occurred. Avoidance rates decreased slightly as fixed-ratio size increased.     

Order and chaos in fixed-interval schedules of reinforcement

Fixed-interval schedule performance is characterized by high levels of variability. Responding is absent at the onset of the interval and gradually increases in frequency until reinforcer delivery. Measures of behavior also vary drastically and unpredictably between successive intervals. Recent advances in the study of nonlinear dynamics have allowed researchers to study irregular and unpredictable behavior in a number of fields. This paper reviews several concepts and techniques from nonlinear dynamics and examines their utility in predicting the behavior of pigeons responding to a fixed-interval schedule of reinforcement. The analysis provided fairly accurate a priori accounts of response rates, accounting for 92.8% of the variance when predicting response rate 1 second in the future and 64% of the variance when predicting response rates for each second over the entire next interreinforcer interval. The nonlinear dynamics account suggests that even the “noisiest” behavior might be the product of purely deterministic mechanisms.     

Performances on ratio and interval schedules of reinforcement: Data and theory

Two differences between ratio and interval performance are well known: (a) Higher rates occur on ratio schedules, and (b) ratio schedules are unable to maintain responding at low rates of reinforcement (ratio “strain”). A third phenomenon, a downturn in response rate at the highest rates of reinforcement, is well documented for ratio schedules and is predicted for interval schedules. Pigeons were exposed to multiple variable-ratio variable-interval schedules in which the intervals generated in the variable-ratio component were programmed in the variable-interval component, thereby “yoking” or approximately matching reinforcement in the two components. The full range of ratio performances was studied, from strained to continuous reinforcement. In addition to the expected phenomena, a new phenomenon was observed: an upturn in variable-interval response rate in the midrange of rates of reinforcement that brought response rates on the two schedules to equality before the downturn at the highest rates of reinforcement. When the average response rate was corrected by eliminating pausing after reinforcement, the downturn in response rate vanished, leaving a strictly monotonic performance curve. This apparent functional independence of the postreinforcement pause and the qualitative shift in response implied by the upturn in variable-interval response rate suggest that theoretical accounts will require thinking of behavior as partitioned among at least three categories, and probably four: postreinforcement activity, other unprogrammed activity, ratio-typical operant behavior, and interval-typical operant behavior.     

Avoidance of risk as a determinant of cooperation

Pairs of subjects could either cooperate or respond on a lower paying individual task. Whenever both subjects chose to cooperate, either subject could make a response that took $1.00 of the other's earnings. In Exp. I, a stimulus signalled when a “take” response had been made. Either subject could avoid the loss by switching to the individual task within 5 sec after the stimulus appeared. Rates of cooperation were high when losses could be avoided but decreased again when the avoidance condition was removed. In Exp. II, a response prevented “takes” from occurring for a specified time interval after the response. This procedure also maintained cooperation. When each avoidance response subtracted from earnings, both avoidance responding and cooperation were eliminated.     

Effect of reinforcement duration on fixed-interval responding

Five different reinforcement durations occurred randomly within each session on fixed interval 60-sec. Postreinforcement pause was directly related (and “running” rate inversely related) to the duration of reinforcement initiating each fixed interval.     

College students' responding to and rating of contingency relations: The role of temporal contiguity

Two experiments investigated the role of temporal contiguity in college students' responding to and rating of contingency relations during operant conditioning. Schedules were devised that determined when but not whether appetitive or aversive events would occur. Subjects' reports concerning the schedules were obtained by means of a 200-point rating scale, anchored by the phrases “prevents the light from occurring” (−100) and “causes the light to occur” (+100). When tapping a telegraph key advanced the time of point gain, responding was maintained or increased and subjects gave positive ratings. When tapping a telegraph key advanced the time of point loss, subjects also gave positive ratings, but responding now decreased. When key tapping delayed the time of point gain, responding decreased and subjects gave negative ratings. When key tapping delayed the time of point loss, subjects also gave negative ratings, but responding now increased. These findings implicate response-outcome contiguity as an important contributor to causal perception and to reinforcement and punishment effects. Other accounts—such as those stressing the local probabilistic relation between response and outcome or the molar correlation between response rate and outcome rate—were seen to be less preferred interpretations of these and other results.