Negative reinforcement as shock-frequency reduction

Is a conditioned aversive stimulus necessary in avoidance conditioning? Or is a reduction in the rate of aversive stimulation alone sufficient to generate and maintain an avoidance response? Rats were subjected to an avoidance procedure in which shocks occurred randomly in time, but a response could reduce the overall rate of shock. Fifteen acquisition curves, obtained from 16 animals, showed both immediate and delayed, rapid and gradual increases in response rate; there was no representative acquisition curve. Response rates were directly related to the amount by which the response reduced shock frequency. In extinction, when shock rates were not affected by responding, the response total was inversely related to the amount by which the response had reduced shock frequency during prior conditioning, with as many as 20,000 extinction responses when the shock frequency reduction had been relatively small. Responding on this procedure shows that avoidance conditioning can occur without benefit of either classical exteroceptive stimuli or covert stimuli inferred from the temporal constancies of a procedure. It also shows that reduction in shock rate is alone sufficient to maintain avoidance.     

Secondary reinforcement value of stimuli associated with shock reduction

An experiment is described in which 32 rats were tested to determine if an escape box which had been previously associated with shock reduction had secondary reinforcement value. Subjects were tested in both a fear and a neutral situation. Significant secondary reinforcement effects were found in both situations. However, contary to expectations, greater secondary reinforcement effects were obtained in the neutral than in the fear situation.     

Negative reinforcement with shock-frequency increase

Two avoidance-conditioning experiments in which responding delayed shocks are reported. Rats receiving an average of two shocks per minute (imposed condition) could produce, by pressing a bar, a 3-min alternate condition. Six (Experiment I) or more (Experiment II) shocks occurred in the alternate condition. All shocks in the alternate condition were delayed and delivered at 1-sec intervals. With long delays, all subjects produced the alternate condition and spent a large percentage of each session in the alternate condition. The first experiment demonstrated that the longer the delay from onset of the alternate condition to onset of the shocks, the more session time spent in the alternate condition. The second experiment indicated that despite increased shock frequency, behavior is acquired and maintained when responding leads to sufficient delay. Individual subjects produced the alternate condition by bar pressing in essentially one of two patterns. One pattern, termed postshock, involved bar pressing immediately after shock; the other, termed posttransition, involved responding immediately after the transition from the alternate to the imposed condition. These results indicate that shock-frequency reduction is not necessary for avoidance conditioning; delay to shock onset is sufficient.     

Negative effects of positive reinforcement

Procedures classified as positive reinforcement are generally regarded as more desirable than those classified as aversive-those that involve negative reinforcement or punishment. This is a crude test of the desirability of a procedure to change or maintain behavior. The problems can be identified on the basis of theory, experimental analysis, and consideration of practical cases. Theoretically, the distinction between positive and negative reinforcement has proven difficult (some would say the distinction is untenable). When the distinction is made purely in operational terms, experiments reveal that positive reinforcement has aversive functions. On a practical level, positive reinforcement can lead to deleterious effects, and it is implicated in a range of personal and societal problems. These issues challenge us to identify other criteria for judging behavioral procedures.     

Negative reinforcement and choice in humans

Participants chose between reinforcement schedules differing in delay and/or duration of noise offset. In Experiment 1 it was found that (1) immediate reinforcement was preferred to delayed reinforcement when amounts (durations) of reinforcement were equal; (2) a relatively large reinforcer was preferred to a smaller one when both reinforcers were obtained immediately; and (3) preference for an immediate, small reinforcer versus a delayed, large reinforcer increased as the delay preceding the large reinforcer increased, a sign of “impulsivity”. In Experiment 2, the schedules differed in amount or delay and equal intervals were added either to the constant parameter or the varied parameter. A shift from virtually exclusive preference to indifference occurred in the latter case but not the former, a result supporting a model of self-control that assumes that the value of a schedule depends on the ratio of amount and delay, and that choice between schedules depends on the ratio of these values.     

Negative priming without ignoring

Negative priming is conventionally defined by slowed responses to a target item that appeared previously as a distractor. As a result, it is widely assumed that negative priming is caused by an act of ignoring. Three experiments are reported in which novel abstract shapes were studied with either “shallow” or “deep” encoding instructions. This study phase was followed by asame-different discrimination task similar to that employed by DeSchepper and Treisman (1996). Same-different discrimination was slower for old than for new target shapes, and this negative priming effect depended on the difficulty of the discrimination task. The results suggest that negative priming may not be caused by the ignoring of a prime stimulus.     

Delayed reinforcement of fixed-ratio performance without mediating exteroceptive conditioned reinforcement

The performance of pigeons was studied under conditions in which the completion of a fixed-ratio requirement was not contiguous with the presentation of a reinforcer. Timein and timeout periods alternated throughout the experimental sessions. Responses made by an experimental bird during the timein period were accumulated, and when a fixed-ratio requirement had been met, grain was presented to the experimental bird and a yoked control following their first response in the next timein period. Across most manipulations of the fixed-ratio requirement and of the duration of the timeout period, the response rates of the experimental birds were considerably higher than those of their controls, suggesting that the response-reinforcer dependency controlled the behavior of the experimental bird in the absence of a close temporal association between responding on the ratio schedule and reinforcer presentations.     

Negative priming without probe selection

Responses to an object may be slower or less accurate if that object shares attributes with a recently ignored object(negative priming). Some studies have found negative priming only if the probe trial required selection against a distractor stimulus. In the present experiment, subjects responded to the location of a target (O), ignoring a distractor (X) if it appeared in another location. Reaction time was slower to probe targets that appeared in the same location as the prime distractor, regardless of whether or not the probe target was accompanied by a distractor.     

Integrated delays to shock as negative reinforcement

Rats were shocked at the rate of two per minute until they pressed a lever. In Experiment I, shocks were delivered at variable-time intervals averaging 30 sec; in Experiment II, shocks were delivered at fixed-time intervals of 30 sec. A response produced an alternate condition for a fixed-time period. The shock frequency following a response, calculated over the whole alternate condition, was two per minute. The pattern of shocks in the alternate condition was controlled so that the first shock occurred at the same time as it would have occurred had the response not been emitted; the remaining shocks were delayed until near the end of the alternate condition. Bar pressing was acquired in both experiments. This finding is not explained by two-factor theories of avoidance and is inconsistent with the notion that overall shock-frequency reduction is necessary for negative reinforcement. The data imply that responding is determined by the integrated delays to each shock following a response versus the integrated delays to shock in the absence of a response.     

The intercalated reinforcement technique: Learning without differential total reinforcement between groups

An intercalated reinforcement technique is described. The method equates number of trials, response topography, sensory exposure, and frequency of reinforcement between reversal Ss overtrained on a discrimination and then trained on reversals and nonreversal Ss similarly overtrained but continued on the same discrimination. In extending previous successful work in the rat, monkeys were trained on spatial discrimination and reversal tasks in the WGTA at 50%, 75%, and 100% reinforcement in order to test the efficacy of learning by this procedure, and to compare the method with learning obtained by a more classical procedure. Learning rates were the same by both procedures. This technique, therefore, appears useful for investigating neurochemical (or other) correlates of learning unconfounded by performance or sensory effects.     

Uncertainty reduction, conditioned reinforcement, and observing

In a concurrent-chains procedure, pigeons chose between equivalent mixed and multiple fixed-interval schedules of reinforcement. In the first experiment, preference for the multiple schedule was higher when the probability of the shorter fixed interval was less than .50 than for complementary points, an outcome consistent with the delay-reduction hypothesis of conditioned reinforcement and observing, but inconsistent with the uncertainty-reduction hypothesis which requires symmetrical preferences with a maximum when the two intervals are equiprobable. A second experiment assessed preference for equivalent mixed and multiple schedules when each choice outcome resulted in two reinforcements, one on the longer and one on the shorter fixed interval. The order of the two fixed intervals was determined probabilistically. Pigeons again preferred multiple to mixed schedules, although multiple-schedule preference did not vary systematically with the likelihood of the shorter fixed interval occurring first. The results from these choice procedures are consistent with those from the observing-response literature in suggesting that the strength of a stimulus cannot be well described as a function of the degree of uncertainty reduction the stimulus provides about reinforcement.     

Some effects of correlation between response-contingent shock and reinforcement

Rats responded on a two-component chain schedule in which a response-contingent electric shock at the end of the first component was either positively correlated, negatively correlated, or uncorrelated with reinforcement availability in the second component. With 0.4-ma shocks, rate in the first component depended on the shock-reinforcement correlation: when shock and reinforcement availability were positively correlated, after extended exposure to the contingencies, rates exceeded those in the absence of shock; when shock and reinforcement availability were negatively correlated, responding was generally suppressed throughout. The discriminative control of shock over responding in the second component, in which reinforcement was available 50% of the time, also depended somewhat on correlation. However, rate change in the first component was not specifically related to discrimination in the second component. With 0.8-ma shocks, responding was substantially suppressed in the first component at all three values of shock-reinforcement correlations.     

Chaining and secondary reinforcement based on escape from shock

Three white rats were trained to press a bar while being shocked. This produced a white noise. After 30 sec they were allowed to terminate both the shock and the noise by nosing a pigeon key. Comparison of the rates of pressing before and after the onset of the noise indicated that the noise itself was the immediate reinforcing agent for pressing. Furthermore, control tests showed that pressing was maintained only if it produced the noise: either omission of the noise or elimination of the dependency of the noise on the occurrence of the response led to a gradual abolition of pressing. When automatic termination of the shock was substituted for the key nosing requirement, however, only the key nosing extinguished. This indicated that the effectiveness of the noise as a reinforcer did not depend on its status as a discriminative stimulus for some other form of operant behavior.     

Reduction of shock frequency as reinforcement for avoidance behavior

An avoidance technique was used in which rats had two levers available, with independent shock schedules associated with each. Behavioral patterns in initial conditioning and in the maintenance of the responses with various response-shock intervals led to the suggestion that reduction of shock density be considered an important variable in avoidance behavior.     

Behavioral contrast without response-rate reduction

Behavioral contrast was obtained in two experiments, which both employed a standard free-operant successive discrimination (a multiple variable-interval extinction schedule), without the occurrence of reductions of response rate in the extinction component. In Experiment I, one group of four pigeons was trained on a multiple schedule in which one stimulus was associated with a variable-interval schedule and the second stimulus with response-independent reinforcement on a free variable-interval schedule. Though by the end of this training three pigeons were responding very little to the second stimulus, when this stimulus was associated with extinction all subjects showed a contrast effect. In Experiment II, eight pigeons were trained extensively to respond to a single stimulus on a variable-interval schedule, before a second stimulus associated with extinction was introduced. This second stimulus was dissimilar to the initial stimulus and five pigeons never responded in its presence. Nevertheless, all pigeons showed a contrast effect and there was no evidence that the effect was smaller in errorless subjects or smaller than in a subsequent discrimination where all subjects made many errors. Both experiments indicated that response reduction in one component of a multiple schedule is not a necessary condition for the occurrence of contrast.