Avoidance of timeout from response-independent reinforcement

Responses on a lever by rats postponed scheduled timeouts, or periods during which the delivery of response-independent food was withheld. The effects of a number of experimental variables were examined and the conclusions drawn are that the functional relations describing free-operant avoidance of timeout from response-independent reinforcement are similar to those for avoidance of electric shock and that both phenomena are sensitive to the same parametric manipulations. Results suggest that high frequency of food delivery in timein maintains a higher rate of timeout avoidance than low frequency. The evidence argues against an interpretation in terms of adventitious food-reinforcement of the timeout avoidance response. Finally, the effects of scheduling timeouts independently of responding and of omitting timeouts confirm the view that timeouts can be aversive and may act as punishment for responding.     

Lever attacking by rats during free-operant avoidance

Rats pressed a lever to avoid shock on a free-operant avoidance schedule. Some subjects were also exposed to extinction in which the response-shock contingency was eliminated while the shock-shock contingency remained in effect. A specially constructed lever was used that registered not only presses, but also biting attacks on the lever. Throughout various phases of the study, shocks often elicited lever biting as well as post-shock responding. The results suggested that shock-elicited attacks that are forceful enough to activate the operandum might account for some of the responding that occurs in experiments on free-operant avoidance behavior. In particular, shock-elicited operandum attacking might account for post-shock response bursting during free-operant avoidance and the extreme persistence of responding sometimes noted when shocks are delivered during the extinction of avoidance behavior. To the extent that this is true, these phenomena should not be characterized as operant behavior in interpreting the results of experiments on free operant avoidance.     

Free-operant avoidance in the pigeon using a treadle response

A free-operant avoidance schedule was used to establish and maintain foot-treadle responding by two Homing, one White King, and two Carneaux pigeons. In the absence of responding, the interval between shocks equaled 10 sec. Each time a treadle response occurred the shock was postponed for 32 sec. Pigeons appear to learn the treadle response more quickly and use it to avoid shock more successfully than do rats bar pressing on similar schedules. The treadle response becomes highly stereotyped and interresponse time distributions obtained from terminal behavior appear very similar to data obtained from rats. It is concluded that the difficulty in training pigeons to avoid electric shock is not in establishing avoidance behavior but in attempting to evaluate such behavior with the key-peck response.     

Extinction of responding maintained by timeout from avoidance

The resistance to extinction of lever pressing maintained by timeout from avoidance was examined. Rats were trained under a concurrent schedule in which responses on one lever postponed shock on a free-operant avoidance (Sidman) schedule (response-shock interval = 30 s) and responses on another lever produced 2 min of signaled timeout from avoidance on a variable-ratio 15 schedule. Following extended training (106 to 363 2-hr sessions), two experiments were conducted. In Experiment 1 two different methods of extinction were compared. In one session, all shocks were omitted, and there was some weakening of avoidance but little change in timeout responding. In another session, responding on the timeout lever was ineffective, and under these conditions timeout responding showed rapid extinction. The within-session patterns produced by extinction manipulations were different than the effects of drugs such as morphine, which also reduces timeout responding. In Experiment 2 shock was omitted for many consecutive sessions. Response rates on the avoidance lever declined relatively rapidly, with noticeable reductions within 5 to 10 sessions. Extinction of the timeout lever response was much slower than extinction of avoidance in all 4 rats, and 2 rats continued responding at baseline levels for more than 20 extinction sessions. These results show that lever pressing maintained by negative reinforcement can be highly resistant to extinction. The persistence of responding on the timeout lever after avoidance extinction is not readily explained by current theories.     

Self-punitive behavior in humans: Effects of a self-fulfilling prophecy

Human subjects were exposed to contingencies which programmed aversive tones (100 db). Two types of contingencies were employed: self-confirming (i.e., self-fulfilling prophecies), in which the aversive tone was occasioned by the prediction it was about to occur; and self-disconfirming, in which the tone was probable when subjects predicted it would not occur. Experiments 1 and 2 used a modified classical conditioning paradigm, and demonstrated that a self-confirming contingency maintained reliable self-punitive responding, i.e., subjects consistently predicted and therefore obtained tones on every trial. Subjects in Experiment 3 were instructed to express predictions continuously throughout four sessions to ensure adequate sampling of the various predictions. Subjects exposed to a self-disconfirming contnngency reliably evidenced awareness of the contingency in effect (judged by answers on a postexperimental questionnaire), whereas subjects exposed to a self-confirming contingency failed to show effective avoidance behavior or contingency awareness. Experiment 4 investigated free-operant self-punitive behavior, utilizing a single prediction response button, which subjects depressed repeatedly. Subjects were exposed to either periodic or aperiodic punishment schedules over as many as four sessions. In general, more persistent self-punitive responding was found in the groups receiving periodic punishment. The results from the four experiments show that self-confirming contingencies can effectively prolong self-punitive responding in human subjects. The findings are consistent with a blocking interpretation of self-punitive behavior, which asserts that when an aversive event is already predicted by stimuli in the situation (including temporal cues), the association between a response and punishment is impaired, and self-punitive responding is likely to be maintained. An integration of human and animal self-punitive research is proposed.     

Concurrent schedules of positive and negative reinforcement: differential-impact and differential-outcomes hypotheses

Considerable evidence from outside of operant psychology suggests that aversive events exert greater influence over behavior than equal-sized positive-reinforcement events. Operant theory is largely moot on this point, and most operant research is uninformative because of a scaling problem that prevents aversive events and those based on positive reinforcement from being directly compared. In the present investigation, humans' mouse-click responses were maintained on similarly structured, concurrent schedules of positive (money gain) and negative (avoidance of money loss) reinforcement. Because gains and losses were of equal magnitude, according to the analytical conventions of the generalized matching law, bias (log b (double dagger) 0) would indicate differential impact by one type of consequence; however, no systematic bias was observed. Further research is needed to reconcile this outcome with apparently robust findings in other literatures of superior behavior control by aversive events. In an incidental finding, the linear function relating log behavior ratio and log reinforcement ratio was steeper for concurrent negative and positive reinforcement than for control conditions involving concurrent positive reinforcement. This may represent the first empirical confirmation of a free-operant differential-outcomes effect predicted by contingency-discriminability theories of choice.     

Compounding discriminative stimuli controlling free-operant avoidance

The performances of three rats were stabilized on a multiple schedule that maintained responding by a free-operant avoidance schedule during independent presentations of tone and light. The simultaneous absence of these stimuli signalled shock-free periods and controlled response cessation. Subsequently, test sessions were administered consisting of independent presentations of each stimulus and these stimuli compounded (tone-plus-light). During an extinction test, additive summation was observed to the compounded stimuli, i.e., more responses were emitted to the compound than to either tone or light. During a series of 28 maintenance-test sessions in which the shock schedule remained operative, the compounded stimuli produced a generally enhanced response rate and fewer pauses terminating with shock than either single stimulus condition. These results extend the generality of free-operant additive summation to responding maintained by aversive control. In addition, a comparison of the present study with previous experiments reporting additive summation of positively reinforced responding indicates that similar variables—rate and aversive differences between training stimulus conditions—should be considered in accounting for response distributions during stimulus compounding when responding is controlled by either positive or negative contingencies.     

Summation of food and shock based responding

In a multiphase experiment, dogs first received discriminative, discretetrial, barrier-jumping training with two tones (S^D, S^Δ) in a shuttle box reinforced by either shock avoidance (Group I) or by food (Group II). Then the dogs were trained on free-operant barrier-jumping reinforced by the qualitatively opposite reinforcer—food in Group I and shock avoidance in Group II. Finally, test presentations of the tone stimuli were superimposed on the free-operant behavior. The tone S^Ds markedly facilitated responding in all animals. This experiment demonstrates a summation of responding maintained by shock avoidance and food reinforcement and casts doubt on explanations of conditioned suppression outcomes that appeal solely to incompatible motivational states within the organism.     

Punishment contras during free-operant avoidance

Punishment of bar-pressing responses of rhesus monkeys with electric shock in one component of a multiple free-operant avoidance schedule suppressed responding in that component. These decreases were concomitant with response rate increases in the unpunished component (punishment contrast). Response rates in both components increased when punishment was removed and decreased in successive sessions. These effects of punishment on unpunished responding were similar to those obtained during single and multiple schedules of positive reinforcement and they suggest a further similarity in the development of discriminations during positive and negative reinforcement schedules.     

The discriminative control of free-operant avoidance despite exposure to shock during the stimulus correlated with nonreinforcement

Four rats were trained in darkness on a free-operant avoidance procedure in which shocks occurred randomly, but lever presses could reduce their frequency. Discrimination training followed, during which responses in light continued to reduce shock frequency, but responses in darkness had no effect. During each cycle, the light period was 4 min, while darkness lasted only until a 20-sec interval had elapsed without a response. This no-response requirement was increased to 40 sec for three animals and eventually to 60 sec for two of them. Discriminative control developed, despite a greater shock density in the dark, with response rate and number of responses per shock maintained or increasing during light and decreasing to very low values in darkness. Two animals were later exposed to a procedure in which shock density was unaffected by responding either in light or darkness. A 60-sec no-response requirement was continued in the dark. Discriminative control persisted through 42 sessions for one animal and required 45 sessions to approach extinction for the other animal. The role of the light as a potential conditioned reinforcer of other behavior in the dark was implicated in the development and persistence of discriminative control. These data support shock-frequency reduction as reinforcement for avoidance behavior.     

Generalization peak shift in rats under conditions of positive reinforcement and avoidance

Rats were trained to discriminate between two click frequencies. One frequency was associated with either variable-interval food reinforcement (Experiment 1) or free-operant avoidance (Experiment 2). The other frequency was associated with the absence of food in Experiment 1 and the absence of shock in Experiment 2. On a click frequency generalization test, the rats in both experiments showed positive peak shift with the shape of the relative gradients being very similar. This is the first reported instance of peak shift in rats when responding was maintained by an avoidance contingency. Nondifferentially trained controls showed that this shift was due exclusively to associative processes, with nonassociative stimulus factors in themselves apparently making no contribution to increased rates at particular stimulus values. These results show the comparability of appetitive and aversive control and support the position that gradient differences do not result from approach versus avoidance per se.     

Behavioral regulation of gravity: schedule effects under escape-avoidance procedures

Squirrel monkeys were restrained in a centrifuge capsule and trained to escape and avoid increases in artificial gravity. During escape-avoidance, lever responses reduced centrifugally simulated gravity or postponed scheduled increases. The effect of variation in the interval of postponement (equal to the duration of decrease produced by escape responses) was studied under a multiple schedule of four components. Three components were gravity escape-avoidance with postponement times of 20, 40, and 60 sec. The fourth component was extinction. Each component was associated with a different auditory stimulus. Rate of responding decreased with increasing postponement time and higher mean g-levels occurred at shorter intervals of postponement. Effects of the schedule parameter on response rate and mean g-level were similar to effects of the schedule on free-operant avoidance and on titration behavior maintained by shock.     

Variable-ratio schedules of timeout from avoidance: effects of d-amphetamine and morphine.

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other lever produced periods of signaled timeout from avoidance on variable-ratio schedules. These procedures generated high rates of timeout-reinforced responding and provided a baseline for studying the effects of drugs on behavior maintained by different types of negative reinforcement (shock postponement vs. timeout). Morphine (2.5 to 10.0 mg/kg) reduced behavior maintained by timeout at doses that increased or had no effect on avoidance responding. In contrast, d-amphetamine (0.125 to 2.0 mg/kg) produced large increases in timeout responding at doses that had minimal effect on avoidance in rats trained on variable-interval and variable-ratio schedules. Thus, the event-dependent effects of morphine, observed in previous studies in which timeout responding was maintained at low rates by interval schedules, were replicated with high timeout rates maintained by variable-ratio schedules. The effects of d-amphetamine could also be described as "event dependent" because timeout responding was stimulated more than avoidance regardless of the maintenance schedule or baseline rate.     

On the independence of stimulus evocation of fear and fear evocation of responses

Dogs in an experimental group were first trained in a shuttlebox on a free-operant avoidance response; then a second, discriminative avoidance response was trained in a second apparatus to a tone. Following criterion, the second avoidance response was “extinguished” through unsignaled. response-independent US presentations. Finally, the tone was presented in the shuttlebox as a test for residual fear. The tone produced as much facilitation of the free-operant avoidance responding as it did in two control groups; one control had not had the second avoidance response extinguished, while the other had received only Pavlovian conditioning in the second apparatus. Thus, the elimination of a discriminative avoidance response to a CS does not impair the CS's ability to control a second independent avoidance response. This result reflects and confirms the separability and independence of the CS-fear and fearavoidance response links hypothesized by two-process theories.     

Density and delay of punishment of free-operant avoidance

In two experiments, the free-operant shock-avoidance behavior of rats was punished by electric shock. Two aspects of the schedule of response-produced shock were varied: the frequency of punishment over time (punishment density) and the temporal interval between the punished response and the punishment (punishment delay). The general finding was that response-produced shock suppressed avoidance responding under most of the density-delay combinations studied, and suppression increased as a function of increases in density and decreases in delay. Rate increases of small magnitude also were observed, usually as an initial reaction to the lesser densities and longer delays. Response suppression, while decreasing the number of punishment shocks received, also increased the number of avoidance shocks, so that the total number of shocks received usually was greater than the minimal number possible. The results were discussed from the standpoint of similarities between the effects of punishing positively and negatively reinforced behavior. The finding that subjects did not minimize the total number of shocks suggested that when avoidance behavior is punished, responding is controlled more by the local consequences of responding than by overall shock frequencies during the course of the session.     

Timing in free-operant and discrete-trial avoidance

A procedure (“discrete-trial” avoidance) was devised to differentiate between the two main theories of responding in Sidman's “free-operant” avoidance procedure. One theory, a version of two-factor theory, holds that responding is reinforced by the removal of a conditioned aversive stimulus. The conditioned aversive stimulus is held to be temporal, which accounts for the spaced responding, or timing, that Sidman's procedure produces. The other theory holds that the reinforcement for both responding and timing is shock-frequency reduction. The new procedure eliminated this reinforcement for timing, but retained the conditions for the formation of conditioned aversive temporal stimuli. According to one theory, the new procedure should have sustained timing as well as Sidman's, while according to the other, it should have sustained no timing. The results confirmed neither theory. Timing was found with both procedures, but unequally in degree and kind. Large variations in the precision of timing did not appear to be correlated with successful avoidance for either procedure.     

Responding changes systematically within sessions during conditioning procedures.

When the procedure is held constant within an experimental session, responding often changes systematically within that session. Many of these within-session changes in responding cannot be dismissed as learning curves or by-products of satiation. They have been observed in studies of positive reinforcement, avoidance, punishment, extinction, discrimination, delayed matching to sample, concept formation, maze and alley running, and laboratory analogues of foraging, as well as in the unconditioned substrates of conditioned behavior. When aversive stimuli are used, responding usually increases early in the session. When positive reinforcers are used, responding changes in a variety of ways, including increasing, decreasing, and bitonic functions. Both strong and minimal reinforcement procedures produce within-session decreases in positively reinforced behavior. Within-session changes in responding have substantial theoretical and methodological implications for research in conditioning.     

Reinforcement magnitude and responding during treatment with differential reinforcement

Basic findings indicate that the amount or magnitude of reinforcement can influence free-operant responding prior to and during extinction. In this study, the relation between reinforcement magnitude and adaptive behavior was evaluated with 3 children as part of treatment with differential reinforcement. In the first experiment, a communicative response was shaped and maintained by the same reinforcer that was found to maintain problem behavior. Two reinforcement magnitudes (20-s or 60-s access to toys or escape from demands) were compared and found to be associated with similar levels of resistance to extinction. The relation between reinforcement magnitude and response maintenance was further evaluated in the second experiment by exposing the communicative response to 20-s or 300-s access to toys or escape. Results for 2 participants suggested that this factor may alter the duration of postreinforcement pauses.     

Nondiscriminated avoidance of shock by pigeons pecking a key

Four pigeons were trained to avoid shock by pecking a key on a free-operant avoidance schedule in which no exteroceptive stimulus signalled impending shock. Response rate was an inverse function of response-shock interval when shock-shock interval was held constant at 2 sec and response-shock intervals varied from 5 to 40 sec. Amphetamine increased response rates in two subjects and reserpine markedly reduced responding in one.