Parameters affecting the maintenance of negatively reinforced key pecking

Three negative reinforcement experiments employing a key-peck response are described. In Experiment I, pigeons shocked on the average of twice per minute (imposed condition) could produce, by pecking a key, an alternate condition with correlated stimuli. Delayed shocks were added, across sessions, to the alternate condition until pecking stopped. Two of three pigeons continued to peck despite a 100% increase in shock frequency. In Experiment II, pigeons were shocked in the imposed condition four times per minute. The postresponse delay to shock was held constant by delivering, in the alternate condition, the next shock, or the next two, three, or four shocks from the imposed-condition shock schedule. All three subjects continued to peck with no change in delay to the first two postresponse shocks but with a 75% reduction in shock frequency. In Experiment III, a response produced an immediate shock followed by a shock-free period. Three of four subjects continued to respond despite reduced delay to shock. Delay-to-shock or shock-frequency reduction was sufficient to maintain key pecking, but neither was necessary. The conditions that negatively reinforce the pigeon's key peck were similar to conditions that negatively reinforce the rat's bar press.     

Integrated delays to shock as negative reinforcement

Rats were shocked at the rate of two per minute until they pressed a lever. In Experiment I, shocks were delivered at variable-time intervals averaging 30 sec; in Experiment II, shocks were delivered at fixed-time intervals of 30 sec. A response produced an alternate condition for a fixed-time period. The shock frequency following a response, calculated over the whole alternate condition, was two per minute. The pattern of shocks in the alternate condition was controlled so that the first shock occurred at the same time as it would have occurred had the response not been emitted; the remaining shocks were delayed until near the end of the alternate condition. Bar pressing was acquired in both experiments. This finding is not explained by two-factor theories of avoidance and is inconsistent with the notion that overall shock-frequency reduction is necessary for negative reinforcement. The data imply that responding is determined by the integrated delays to each shock following a response versus the integrated delays to shock in the absence of a response.     

Experimental self-punishment and superstitious escape behavior

Rats were trained to escape from shock by pressing a bar. Bar holding was subsequently punished with very brief shocks. This treatment failed to depress bar-holding behavior. In some cases, although the escape shocks were delivered very infrequently, bar holding was maintained and resulted in the delivery of several thousand punishments per session. These and other effects of the punishment treatment were investigated. Finally, some of the possibilities of superstitious escape responding were explored by presenting inescapable shocks to rats that had been trained to escape shock by lever pressing. Although responding during these shocks had no programmed consequences, responding was sustained.     

Extinction of responding maintained by timeout from avoidance

The resistance to extinction of lever pressing maintained by timeout from avoidance was examined. Rats were trained under a concurrent schedule in which responses on one lever postponed shock on a free-operant avoidance (Sidman) schedule (response-shock interval = 30 s) and responses on another lever produced 2 min of signaled timeout from avoidance on a variable-ratio 15 schedule. Following extended training (106 to 363 2-hr sessions), two experiments were conducted. In Experiment 1 two different methods of extinction were compared. In one session, all shocks were omitted, and there was some weakening of avoidance but little change in timeout responding. In another session, responding on the timeout lever was ineffective, and under these conditions timeout responding showed rapid extinction. The within-session patterns produced by extinction manipulations were different than the effects of drugs such as morphine, which also reduces timeout responding. In Experiment 2 shock was omitted for many consecutive sessions. Response rates on the avoidance lever declined relatively rapidly, with noticeable reductions within 5 to 10 sessions. Extinction of the timeout lever response was much slower than extinction of avoidance in all 4 rats, and 2 rats continued responding at baseline levels for more than 20 extinction sessions. These results show that lever pressing maintained by negative reinforcement can be highly resistant to extinction. The persistence of responding on the timeout lever after avoidance extinction is not readily explained by current theories.     

Variable-interval escape from stimuli accompanied by shocks

Individual performances of three rats were examined under a procedure in which steady rates of bar pressing were maintained by conditioned aversive stimulation. Originally neutral visual and auditory stimuli were accompanied by widely and irregularly spaced pulses of shock; they were terminated on a variable-interval schedule by pressing a bar. The contingencies between behavior and shock were also duplicated in a control procedure in which no visual or auditory stimuli were provided. Pressing observed under the control procedure was attributed to differences in the aversiveness of pressing and nonpressing behavior engendered by differences in the incidence of shock following the two classes of behavior. Increased rates with visual and auditory stimuli were attributed to termination of conditioned aversive stimulation. Control rates declined more rapidly than did experimental rates as the mean interval between successive shocks was lengthened; both rates tended to decline when less than 60 sec was allowed as time out from shocks following the successful response. In the control procedure, discrimination between the continuation and discontinuation of the shock series, as measured by relative rates, depended on the relative length of the interval between shocks and the time-out period. Regular warm-up accelerations in rate were noted following an initial delay in responding at the beginning of each session. The length of time required for the warm-up depended on the length of the mean interval between shocks, indicating that exposure to a certain amount of shock was required to establish a supporting state for the observed performance.     

Acceleration and suppression of rats' responding to avoid foot shock and tail shock

Signalled response-independent shocks were superimposed on rats' wheel-turn responding to avoid shock administered to their feet through a grid floor or to their tails through fixed electrodes. In Experiment I, a tone paired with response-independent foot shock increased responding in three of four rats; a tone paired with tail shock increased responding in only one of four rats and suppressed responding in two rats. In Experiment II, a tone presented randomly with respect to response-independent shock had no reliable effect on responding to avoid foot shock or tail shock. In Experiment III, tail shock and foot shock were compared in a within-subject design while the temporal pattern of responding during conditioned stimuli was recorded. Responding during the conditioned stimulus preceding foot shock was characterized by initial suppression of responding at tone onset, followed by increased responding just before response-independent shock. Responding was suppressed throughout the conditioned stimulus preceding tail shock. Foot shock elicited bursts of responding, but tail shock did not.     

Responding under schedules combining response-dependent and response-independent shock delivery

Lever pressing of three squirrel monkeys with experience under continuous avoidance schedules was maintained by response-produced shock under a 5-minute variable-interval schedule. Responding decreased when half of the scheduled shocks were delivered independently of lever pressing and decreased further when all shocks were independent of lever pressing. Responding was lowest when all shocks were eliminated. When the proportion of response-dependent shocks increased, responding increased. This relation occurred even though the frequency and temporal distribution of shock delivery remained the same. Responding of two monkeys increased in a graded fashion as the frequency of shock was increased by arranging variable-time 5-minute, 2-minute, and 1-minute schedules jointly with the variable-interval 5-minute schedule. Thus, increasing the proportion of response-independent shocks decreased responding when the overall frequency of shocks stayed the same, but increased responding when the overall frequency of shock delivery increased.     

Characteristics and response-displacement effects of shock-generated responding during negative reinforcement procedures: pre-shock responding and post-shock aggressive responding

Bar-pressing (Experiment I) or key-pressing (Experiments II and III) responses of monkeys were reinforced according to a fixed-interval schedule of negative reinforcement: the first response after a fixed interval of time terminated regularly spaced shocks for a fixed time designated as the reinforcement period. During extinction, shocks continued during the reinforcement period. That there were two types of responding generated by shock alone was indicated by (1) the level of responding maintained during extinction relative to conditions without shock, (2) the stability of two between-shock response patterns across reinforcement and extinction conditions, and (3) the development of these two between-shock patterns without a history of reinforcement. Subjects developed either a pre-shock or a post-shock response pattern when only the bar was available. However, when both a bite tube, an operandum requiring an aggressive topography, and a recessed key, an operandum that did not require an aggressive topography, were provided, the post-shock pattern was observed in tube biting and the pre-shock pattern was observed in key pressing. Removal of the bite tube produced post-shock key responding similar to that observed when only the bar was available. The displacement of post-shock, aggression-motivated responding confirmed the confounding effect of shock-generated responding in negative reinforcement procedures, and suggests that the use of concurrent response alternatives would reduce such confounding.     

Autocontingencies: Suppressive and accelerative effects of pairs of shocks superimposed on a positively reinforced operant baseline

Previous research has shown that unsignaled shock may accelerate positively reinforced operant responding if each shock signals a subsequent shock-free period. In order to explore the boundary conditions of this effect, two experiments were performed. In Experiment 1, pairs of unsignaled shocks separated by 15, 30, 60, or 120 seconds resulted in suppressed responding during the briefest intershock interval, and in accelerated responding during the longer intervals. When the second shock in each pair signaled a shock-free period of at least 3 minutes, accelerated responding also followed offset of the second shock in all but the 30-second condition. In Experiment 2, the addition of a conditioned stimulus prior to each pair of shocks restored baseline responding, and eliminated accelerative control following the second shock only under the briefest inter-shock interval. The results are discussed in terms of the similarity between autocontingencies (shock/no-shock relations; Davis, Memmott, & Hurwitz, 1975) and recent modifications of the feature-positive procedure (e.g., Reberg & Memmott, 1979), which stress stimulus control by shock/no-shock relationships.     

Effect of a signalled delay between an action and outcome on human judgement of causality

Three experiments examined the influence of a stimulus inserted between an action (pressing the space bar of a computer) and an associated delayed outcome (a triangle lighting up on the screen) on ratings of the causal effectiveness of that action. In Experiment 1, subjects rated an action that produced an outcome immediately as more causally effective than an action that produced a delayed outcome. The rating of the causal effectiveness of an action that produced a delayed outcome in which the delay was filled by a stimulus was higher than the rating of actions that produced the outcome after an unsignalled delay. In Experiment 2, the effect of a stimulus presented uncorrelated with the action or the outcome was examined and found not to enhance the rating of causal effectiveness of an action that produced a delayed outcome. In Experiment 3, the degree of correlation between the stimulus and the consequence was examined. Ratings of the causal effectiveness of the action were greater the higher percentage of delay intervals that were signalled.     

The preference-for-signalled-shock phenomenon: Signalling shock is reinforcing only if shock is modifiable

Inescapable, unavoidable, 5-sec. shocks were preceded by 5-sec. white-noise signals contingent on previously food-trained (FR 40) bar pressing of 20 rats. Twenty yoked-control subjects receiving the same, but non-contingent, series of signals indicated that the signal was reliably reinforcing only if the shocks were delivered through an alternate, rather than scrambled, floor-grid system. Implications are discussed for previous reports of the preference for signalled shock phenomenon.     

ASSESSING THE ROLE OF EFFORT REDUCTION IN THE REINFORCING EFFICACY OF TIMEOUT FROM AVOIDANCE

Rats responded on concurrent schedules of shock‐postponement or deletion (avoidance) and timeout from avoidance. In Experiment 1, 3 rats' responses on one lever postponed shocks for 20 s and responses on a second lever produced a 1‐min timeout according to a variable‐interval 45‐s schedule. Across conditions, a warning signal (white noise) was presented 19.5 s, 16 s, 12 s, 8 s, or 4 s before an impending shock. Raising the duration of the warning signal increased both avoidance and timeout response rates. Timeout responding, although positively correlated with avoidance responding, was not correlated with the prevailing shock rate. In Experiment 2, 3 rats' responses on one lever deleted scheduled shocks according to a variable‐cycle 30‐s schedule and responses on a second lever produced a 2‐min timeout as described above. After this baseline condition, the avoidance lever was removed and noncontingent shocks were delivered at intervals yoked to the receipt of shocks in the baseline sessions. Timeout responding decreased when the avoidance lever was removed, even though the shock‐frequency reduction afforded by the timeout remained constant. These results suggest that a key factor in the reinforcing efficacy of timeout is suspension of the requirement to work to avoid shock, rather than the reduction in shock frequency associated with timeout.     

Response-shock delay as a reinforcer in avoidance behavior

After rats received preliminary training to avoid shock on a discrete-trial retractable-bar avoidance procedure, the procedure was changed such that responses retracted the lever but did not affect the rate of shock. Responses only delayed the onset of shock. About half of the animals under these procedures responded consistently on almost 100% of the discrete-trial cycles over days. When short latencies maximized the response-shock delay, animals tended to make short-latency responses. When long latencies maximized the response-shock delay, animals tended to make long-latency responses. When all response latencies produced the same response-shock delay, animals made differing average-latency responses. And, when responses did not delay shock, most of the animals primarily engaged in shock-elicited responding while the other animals engaged in preshock responding.     

Choice and the dependability of stimuli that predict shock and safety

This study assessed whether choosing a signalled shock condition over an unsignalled one is controlled by a stimulus that predicts the presence of shock (Experiment I), or by a stimulus that predicts the absence of shock (Experiment II). The dependability of these stimuli as predictors of either the presence or the absence of shock was parametrically varied over a wide range, and subjects (rats) were given an option to change from an unsignalled to a signalled condition. In the first experiment, all shocks were preceded by signals; however, the probability of a signal being followed by shock varied from 1.0 to 0.02. The data obtained indicate that the dependability of the signal as a predictor of shock is unimportant. Rats changed to the signalled condition when the signal was completely dependable (all signals followed by shock) and when the dependability of the signal was systematically degraded. In the second experiment, all signals were followed by shock; however, some shocks were not preceded by a signal. The data show that the dependability of a stimulus predicting the absence of shock is important in that, as dependability decreases, changing to the signalled condition also decreases.     

Effect of CS-US pairings on shock-elicited aggression as a function of US intensity

In two experiments with paired rats, the effect of superimposing CS-US pairings on a baseline of shock-elicited aggression was studied. Baseline shocks (3.0 mA, 0.125-sec duration) occurred at a rate of 20 shocks per min throughout each session. In Experiment I, each independent group of two pairs of subjects received (in addition to baseline shocks) US shocks of 1.0, 3.0, or 5.0 mA and 5-sec duration, each shock signalled by a 1-min CS. At all three US intensities, aggression increased during the CS. In Experiment II, pairs of subjects received each unconditioned stimulus intensity in a within-subjects design. This procedure revealed a direct relationship between rate of responding and unconditioned shock intensity.     

SOME RELATIONS BETWEEN CLASSICALLY CONDITIONED AGGRESSION AND CONDITIONED SUPPRESSION IN SQUIRREL MONKEYS

During three experiments with squirrel monkeys, stimulus and shock pairings were given in the presence of a bite tube. Experiments 1 and 2 used a conditioned-suppression procedure in which bar pressing was reinforced with food. A transparent shield prevented biting of the bar. When the stimulus was paired with shock, bar pressing decreased (conditioned suppression) and tube biting increased during the stimulus (classically conditioned aggression). When the bite tube was removed on alternate sessions in Experiment 2, there was more suppression when the tube was present, thus suggesting that biting competed with bar pressing. However, this simple competing-response interpretation was complicated by the findings of Experiment 3 where, with naive monkeys, bar pressing was never reinforced with food, yet bar pressing was induced during the stimulus and was highest when the bite tube was absent. The fact that stimulus-induced bar pressing developed indicates that bar pressing in conditioned-suppression procedures, suppressed or not, may be maintained by two types of control—the food reinforcer and induced CS control. The higher rate of induced bar pressing during the stimulus with the bite tube absent confounds a simple competing response interpretation of conditioned suppression. It suggests that shock-induced responses during conditioned suppression could be both contributing to and competing with responding maintained by food, with the net effect depending on specific but ill-defined features of the situation.     

The influence of positive and negative reinforcement on selective attention in the rat

In Experiments 1 and 2 rats were trained under two multiple schedules of reinforcement. In one, bar pressing during a tone-light compound stimulus was reinforced under a variable-interval food reinforcement schedule. In the other multiple schedule, bar pressing avoided grid shock on a free-operant schedule. In both multiple schedules, a discrimination was maintained by an extinction schedule that was operative during the absence of the tone-light compound. In Experiments 1 and 2 the intensity of the tone-light compound was manipulated over three levels. Subsequent extinction tests revealed that light was attended to, almost exclusively of the tone, when food reinforcement had maintained bar pressing. On the other hand, the tone gained considerable attentional control under the shock avoidance schedule. This stimulus-reinforcer interaction was maintained for all three levels of the compound intensity. In Experiment 3 it was investigated whether this interaction was associative by presenting shock during the absence of the tone-light compound when food reinforcement maintained responding, and food during the absence of the compound when shock avoidance maintained responding. Since both food and shock were presented during a single session for both schedules, nonassociative effects of the reinforcing stimuli were equivalent across the schedules. Nevertheless, the stimulus-reinforcer interaction was maintained, indicating that the interaction was an associative effect.     

Density and delay of punishment of free-operant avoidance

In two experiments, the free-operant shock-avoidance behavior of rats was punished by electric shock. Two aspects of the schedule of response-produced shock were varied: the frequency of punishment over time (punishment density) and the temporal interval between the punished response and the punishment (punishment delay). The general finding was that response-produced shock suppressed avoidance responding under most of the density-delay combinations studied, and suppression increased as a function of increases in density and decreases in delay. Rate increases of small magnitude also were observed, usually as an initial reaction to the lesser densities and longer delays. Response suppression, while decreasing the number of punishment shocks received, also increased the number of avoidance shocks, so that the total number of shocks received usually was greater than the minimal number possible. The results were discussed from the standpoint of similarities between the effects of punishing positively and negatively reinforced behavior. The finding that subjects did not minimize the total number of shocks suggested that when avoidance behavior is punished, responding is controlled more by the local consequences of responding than by overall shock frequencies during the course of the session.     

Selective punishment of interresponse times: The roles of shock intensity and scheduling

Two experiments investigated the roles of shock intensity and scheduling in selective punishment of interresponse times. In each experiment the punishment contingencies were imposed on a background of rats' responding maintained by a variable-interval schedule of food presentation. In Experiment 1 all interresponse times greater than 8 seconds produced shock. In Experiment 2 all interresponse times greater than 8 seconds but less than 12 seconds produced shock. In each experiment shock intensity was initially 0.3 milliamperes (mA) and then was varied through an ascending sequence ranging from 0.1 mA to 0.4 mA, in 0.1-mA increments. Experiment 1 produced response-rate increases at low intensities (0.1 and 0.2 mA) but eliminated responding at the remaining intensities. Experiment 2 produced response-rate increases only with 0.1-mA shock, although responding was maintained at all shock parameters investigated. Analysis of the interresponse times per opportunity showed differential suppression of the targeted responses in all cases except the high-intensity shock phases of Experiment 1. The current data support and extend previous studies of selective interresponse-time-dependent shock schedules but suggest that response-rate increases are not a necessary outcome of this type of procedure. The view that variable-interval schedules of shock presentation selectively target long interresponse times was also supported.