Negative reinforcement as shock-frequency reduction

Is a conditioned aversive stimulus necessary in avoidance conditioning? Or is a reduction in the rate of aversive stimulation alone sufficient to generate and maintain an avoidance response? Rats were subjected to an avoidance procedure in which shocks occurred randomly in time, but a response could reduce the overall rate of shock. Fifteen acquisition curves, obtained from 16 animals, showed both immediate and delayed, rapid and gradual increases in response rate; there was no representative acquisition curve. Response rates were directly related to the amount by which the response reduced shock frequency. In extinction, when shock rates were not affected by responding, the response total was inversely related to the amount by which the response had reduced shock frequency during prior conditioning, with as many as 20,000 extinction responses when the shock frequency reduction had been relatively small. Responding on this procedure shows that avoidance conditioning can occur without benefit of either classical exteroceptive stimuli or covert stimuli inferred from the temporal constancies of a procedure. It also shows that reduction in shock rate is alone sufficient to maintain avoidance.     

Secondary reinforcement value of stimuli associated with shock reduction

An experiment is described in which 32 rats were tested to determine if an escape box which had been previously associated with shock reduction had secondary reinforcement value. Subjects were tested in both a fear and a neutral situation. Significant secondary reinforcement effects were found in both situations. However, contary to expectations, greater secondary reinforcement effects were obtained in the neutral than in the fear situation.     

Negative reinforcement with shock-frequency increase

Two avoidance-conditioning experiments in which responding delayed shocks are reported. Rats receiving an average of two shocks per minute (imposed condition) could produce, by pressing a bar, a 3-min alternate condition. Six (Experiment I) or more (Experiment II) shocks occurred in the alternate condition. All shocks in the alternate condition were delayed and delivered at 1-sec intervals. With long delays, all subjects produced the alternate condition and spent a large percentage of each session in the alternate condition. The first experiment demonstrated that the longer the delay from onset of the alternate condition to onset of the shocks, the more session time spent in the alternate condition. The second experiment indicated that despite increased shock frequency, behavior is acquired and maintained when responding leads to sufficient delay. Individual subjects produced the alternate condition by bar pressing in essentially one of two patterns. One pattern, termed postshock, involved bar pressing immediately after shock; the other, termed posttransition, involved responding immediately after the transition from the alternate to the imposed condition. These results indicate that shock-frequency reduction is not necessary for avoidance conditioning; delay to shock onset is sufficient.     

Negative effects of positive reinforcement

Procedures classified as positive reinforcement are generally regarded as more desirable than those classified as aversive-those that involve negative reinforcement or punishment. This is a crude test of the desirability of a procedure to change or maintain behavior. The problems can be identified on the basis of theory, experimental analysis, and consideration of practical cases. Theoretically, the distinction between positive and negative reinforcement has proven difficult (some would say the distinction is untenable). When the distinction is made purely in operational terms, experiments reveal that positive reinforcement has aversive functions. On a practical level, positive reinforcement can lead to deleterious effects, and it is implicated in a range of personal and societal problems. These issues challenge us to identify other criteria for judging behavioral procedures.     

Negative reinforcement and choice in humans

Participants chose between reinforcement schedules differing in delay and/or duration of noise offset. In Experiment 1 it was found that (1) immediate reinforcement was preferred to delayed reinforcement when amounts (durations) of reinforcement were equal; (2) a relatively large reinforcer was preferred to a smaller one when both reinforcers were obtained immediately; and (3) preference for an immediate, small reinforcer versus a delayed, large reinforcer increased as the delay preceding the large reinforcer increased, a sign of “impulsivity”. In Experiment 2, the schedules differed in amount or delay and equal intervals were added either to the constant parameter or the varied parameter. A shift from virtually exclusive preference to indifference occurred in the latter case but not the former, a result supporting a model of self-control that assumes that the value of a schedule depends on the ratio of amount and delay, and that choice between schedules depends on the ratio of these values.     

Function transformation without reinforcement

In studies of function transformation, participants initially are taught to match stimuli in the presence of a contextual cue, X; the stimuli to be matched bear some formal relation to each other, for example, a relation of opposition or difference. In a second phase, the participants are taught to match arbitrary stimuli (say, A and B) in the presence of X. In a final test, A often displays behavioral functions that differ from those of B, and can be predicted from the nature of the relation associated with X in the initial training phase. Here we report function-transformation effects in the absence of selection responses and of their reinforcers. In three experiments with college students, exposure to relations of difference or identity modified the responses given to later stimuli. In Experiment 1, responses to a test stimulus A varied depending on preexposure to pairs of colors that were distinct from A but exemplified relations of difference or identity. In Experiment 2, a stimulus A acquired distinct functions, depending on its previous pairing with a contextual cue X that had itself been paired with identity or difference among colors. Experiment 3 confirmed the results of Experiment 2 with a modified design. Our data are consistent with the notion that relations of identity or difference can serve as stimuli for Pavlovian processes, and, in compound with other cues, produce apparent function-transformation effects.     

Negative priming without ignoring

Negative priming is conventionally defined by slowed responses to a target item that appeared previously as a distractor. As a result, it is widely assumed that negative priming is caused by an act of ignoring. Three experiments are reported in which novel abstract shapes were studied with either “shallow” or “deep” encoding instructions. This study phase was followed by asame-different discrimination task similar to that employed by DeSchepper and Treisman (1996). Same-different discrimination was slower for old than for new target shapes, and this negative priming effect depended on the difficulty of the discrimination task. The results suggest that negative priming may not be caused by the ignoring of a prime stimulus.     

Delayed reinforcement of fixed-ratio performance without mediating exteroceptive conditioned reinforcement

The performance of pigeons was studied under conditions in which the completion of a fixed-ratio requirement was not contiguous with the presentation of a reinforcer. Timein and timeout periods alternated throughout the experimental sessions. Responses made by an experimental bird during the timein period were accumulated, and when a fixed-ratio requirement had been met, grain was presented to the experimental bird and a yoked control following their first response in the next timein period. Across most manipulations of the fixed-ratio requirement and of the duration of the timeout period, the response rates of the experimental birds were considerably higher than those of their controls, suggesting that the response-reinforcer dependency controlled the behavior of the experimental bird in the absence of a close temporal association between responding on the ratio schedule and reinforcer presentations.     

Negative priming without probe selection

Responses to an object may be slower or less accurate if that object shares attributes with a recently ignored object(negative priming). Some studies have found negative priming only if the probe trial required selection against a distractor stimulus. In the present experiment, subjects responded to the location of a target (O), ignoring a distractor (X) if it appeared in another location. Reaction time was slower to probe targets that appeared in the same location as the prime distractor, regardless of whether or not the probe target was accompanied by a distractor.     

Timeout postponement without increased reinforcement frequency

Three experiments were conducted to examine pigeons' postponement of signaled extinction periods (timeouts) from a schedule of food reinforcement when such responding neither decreased overall timeout frequency nor increased the overall frequency of food reinforcement. A discrete-trial procedure was used in which a response during the first 5 s of a trial postponed an otherwise immediate 60-s timeout to a later part of that same trial but had no effect on whether the timeout occurred. During time-in periods, responses on a second key produced food according to a random-interval 20-s schedule. In Experiment 1, the response-timeout interval was 45 s under postponement conditions and 0 s under extinction conditions (responses were ineffective in postponing timeouts). The percentage of trials with a response was consistently high when the timeout-postponement contingency was in effect and decreased to low levels when it was discontinued under extinction conditions. In Experiment 2, the response-timeout interval was also 45 s but postponement responses increased the duration of the timeout, which varied from 60 s to 105 s across conditions. Postponement responding was maintained, generally at high levels, at all timeout durations, despite sometimes large decreases in the overall frequency of food reinforcement. In Experiment 3, timeout duration was held constant at 60 s while the response-timeout interval was varied systematically across conditions from 0 s to 45 s. Postponement responding was maintained under all conditions in which the response-timeout interval exceeded 0 s (the timeout interval in the absence of a response). In some conditions of Experiment 3, which were designed to control for the immediacy of food reinforcement and food-correlated (time-in) stimuli, responding postponed timeout but the timeout was delayed whether a response occurred or not. Responding was maintained for 2 of 3 subjects, suggesting that behavior was negatively reinforced by timeout postponement rather than positively reinforced by the more immediate presentation of food or food-correlated (time-in) stimuli.     

Integrated delays to shock as negative reinforcement

Rats were shocked at the rate of two per minute until they pressed a lever. In Experiment I, shocks were delivered at variable-time intervals averaging 30 sec; in Experiment II, shocks were delivered at fixed-time intervals of 30 sec. A response produced an alternate condition for a fixed-time period. The shock frequency following a response, calculated over the whole alternate condition, was two per minute. The pattern of shocks in the alternate condition was controlled so that the first shock occurred at the same time as it would have occurred had the response not been emitted; the remaining shocks were delayed until near the end of the alternate condition. Bar pressing was acquired in both experiments. This finding is not explained by two-factor theories of avoidance and is inconsistent with the notion that overall shock-frequency reduction is necessary for negative reinforcement. The data imply that responding is determined by the integrated delays to each shock following a response versus the integrated delays to shock in the absence of a response.     

The intercalated reinforcement technique: Learning without differential total reinforcement between groups

An intercalated reinforcement technique is described. The method equates number of trials, response topography, sensory exposure, and frequency of reinforcement between reversal Ss overtrained on a discrimination and then trained on reversals and nonreversal Ss similarly overtrained but continued on the same discrimination. In extending previous successful work in the rat, monkeys were trained on spatial discrimination and reversal tasks in the WGTA at 50%, 75%, and 100% reinforcement in order to test the efficacy of learning by this procedure, and to compare the method with learning obtained by a more classical procedure. Learning rates were the same by both procedures. This technique, therefore, appears useful for investigating neurochemical (or other) correlates of learning unconfounded by performance or sensory effects.     

Differential reinforcement with and without instructional fading

We evaluated a differential-reinforcement-based treatment package for the reduction of problem behavior during instructional situations. Differential reinforcement of alternative behavior (DRA; compliance) was implemented across two conditions. During one condition, instructions were presented approximately once every other minute. This condition was considered the terminal goal for treatment. During the second condition, the rate of instructions was gradually increased (beginning at zero and ending when instruction rate was similar to the first condition). Results indicated that DRA with instructional fading resulted in less problem behavior than DRA without instructional fading. These results are similar to previous studies regarding the utility of instructional fading.     

Uncertainty reduction, conditioned reinforcement, and observing

In a concurrent-chains procedure, pigeons chose between equivalent mixed and multiple fixed-interval schedules of reinforcement. In the first experiment, preference for the multiple schedule was higher when the probability of the shorter fixed interval was less than .50 than for complementary points, an outcome consistent with the delay-reduction hypothesis of conditioned reinforcement and observing, but inconsistent with the uncertainty-reduction hypothesis which requires symmetrical preferences with a maximum when the two intervals are equiprobable. A second experiment assessed preference for equivalent mixed and multiple schedules when each choice outcome resulted in two reinforcements, one on the longer and one on the shorter fixed interval. The order of the two fixed intervals was determined probabilistically. Pigeons again preferred multiple to mixed schedules, although multiple-schedule preference did not vary systematically with the likelihood of the shorter fixed interval occurring first. The results from these choice procedures are consistent with those from the observing-response literature in suggesting that the strength of a stimulus cannot be well described as a function of the degree of uncertainty reduction the stimulus provides about reinforcement.     

Chaining and secondary reinforcement based on escape from shock

Three white rats were trained to press a bar while being shocked. This produced a white noise. After 30 sec they were allowed to terminate both the shock and the noise by nosing a pigeon key. Comparison of the rates of pressing before and after the onset of the noise indicated that the noise itself was the immediate reinforcing agent for pressing. Furthermore, control tests showed that pressing was maintained only if it produced the noise: either omission of the noise or elimination of the dependency of the noise on the occurrence of the response led to a gradual abolition of pressing. When automatic termination of the shock was substituted for the key nosing requirement, however, only the key nosing extinguished. This indicated that the effectiveness of the noise as a reinforcer did not depend on its status as a discriminative stimulus for some other form of operant behavior.     

Some effects of correlation between response-contingent shock and reinforcement

Rats responded on a two-component chain schedule in which a response-contingent electric shock at the end of the first component was either positively correlated, negatively correlated, or uncorrelated with reinforcement availability in the second component. With 0.4-ma shocks, rate in the first component depended on the shock-reinforcement correlation: when shock and reinforcement availability were positively correlated, after extended exposure to the contingencies, rates exceeded those in the absence of shock; when shock and reinforcement availability were negatively correlated, responding was generally suppressed throughout. The discriminative control of shock over responding in the second component, in which reinforcement was available 50% of the time, also depended somewhat on correlation. However, rate change in the first component was not specifically related to discrimination in the second component. With 0.8-ma shocks, responding was substantially suppressed in the first component at all three values of shock-reinforcement correlations.