Reinforcement and the organization of behavior in golden hamsters: Punishment of three action patterns

Golden hamsters received free shock (Experiment 1) or they were partially (Experiments 2 and 3) or continuously (Experiments 4 and 5) punished with mild electric shock for face washing, open rearing, or scrabbling. Punishment immediately and enduringly reduced the time engaged in scrabbling and the number and length of its bouts. Face washing was also clearly associated with shock, since effects specific to the contingency between face washing and shock were retained when shock was discontinued, but under some conditions punishment increased the number of bouts of face washing. Open rearing was the most refractory of the three activities to suppression and showed least evidence of retaining the effects of punishment. This pattern of results differs from that found previously when food reinforcement was given for the same activities. It is discussed in terms of constraints on performance of learned associations by a variety of response-specific and reinforcer-specific motivational factors.     

Drugs and punished responding II: d-amphetamine-induced increases in punished responding

The effects of d-amphetamine on punished responding were studied in two experiments. In Experiment I, pigeons responded under a multiple fixed-ratio 30 response fixed-interval 5-min schedule of food presentation with 60-sec limited holds in both components. Each response was punished with electric shock, the intensity of which was varied systematically. In Experiment II, another group of pigeons responded under a multiple fixed-interval 5-min fixed-interval 5-min schedule of food presentation with 40-sec limited holds. Each response was punished with shock during one component, and every thirtieth response was punished in the other component. d-Amphetamine increased overall rates of punished responding only rarely under any of the punishment conditions; however, response rates within the fixed-interval when rates were low were increased by d-amphetamine when the shock intensity was low (Experiment I), or when responses produced shock intermittently (Experiment II). The data suggest that the effects of d-amphetamine on punished responding depend on the control rate of responding, the punishment intensity, the punishment frequency, and the schedule of food presentation.     

Shock preexposure and the reduced effectiveness of shock

In three experiments experience with shock was shown to reduce the effectiveness of shock as a reinforcer or motivator. In Experiment 1 rats were given signaled shock in a box separate from the runway where they were subsequently punished. These rats were less suppressed by shock punishment than rats that had no previous shock experience. In Experiment 2 preshocked rats were less suppressed by punishment and were slower to learn an escape-avoidance response than nonpreshocked rats, whether the preshock was signaled or unsignaled. In Experiment 3 as number of CS-shock pairings increased, fear of the CS decreased as did fear of the context. These results suggest that some central adaptation process produced by experience with shock reduces the effectiveness of shock as a reinforcer whenever shock is used repeatedly. This is independent of other effects, such as context blocking, that can affect responding after shock preexposure.     

Predator videos and electric shock function as punishers for zebrafish (Danio rerio)

Zebrafish (Danio rerio) are a promising animal model for studying the effects of gene–environment interactions on behavior. Two experiments were conducted to assess punishment effects of presenting predator videos (Indian leaf fish; Nandus nandus) and electric shock on operant approach responses in zebrafish. In Experiment 1, the predator video and shock stimuli were presented upon a response maintained by a single variable‐interval schedule of food reinforcement in different groups of fish. In Experiment 2, the predator video and shock stimuli were presented upon one of two response alternatives maintain by concurrently available variable‐interval schedules of food reinforcement in different groups of fish. Responding decreased when the predator video and shock stimuli were presented relative to their absence in both experiments. Moreover, responding on an unpunished alternative did not reliably decrease in Experiment 2. These results indicate that the decrease in responding resulted from the punishment contingency rather than from elicited species‐specific defense responses or conditioned avoidance. Thus, the predator video and electric shock functioned as punishers of operant behavior for zebrafish. Identifying punishers for this species could lead to research on how gene–environment interactions influence individual differences in sensitivity to punishment.     

The effects of a signal for free or for earned reward: Implications for the role of response-reinforcer associations in instrumental performance

Pearce and Hall (1978) investigated the effects of making a brief flash of light contingent upon response in rats lever-pressing for food on a variable-interval (VI) schedule. When this signal occurred in conjunction only with reinforced responses the response rate was lowered with respect to a condition in which an equal number of light flashes occurred uncorrelated with reinforcement. The experiments reported here compared these effects with those produced by signalling “free” food deliveries in a similar way. Experiments I and II compared the effects of presenting correlated and uncorrelated schedules of light and food to rats given no opportunity to lever-press. The different schedules did not produce differences in response rate when the levers were made available. In Experiment III, free food was delivered to rats responding on a VI schedule. Signalling the delivery of earned food pellets produced a low response rate in comparison with a condition in which the free pellets were signalled. It is concluded that signalling food delivery is effective only when the rat must respond to earn the food and it is argued that the signal has its effect by overshadowing a response-reinforcer association.     

Conditioned suppression, punishment, and aversion

Three experiments were conducted to assess the aversive properties of a visual stimulus in the presence of which one group of birds received response-contingent shock (discriminated punishment) while a yoked group of birds received non-contingent shocks (conditioned suppression). In Experiment 1, presentation of the visual stimulus contingent on key pecking reduced the response rate (conditioned punishment effect) for birds under the conditioned suppression procedure but did not reduce the response rate of birds under the discriminative punishment procedure. Non-contingent shocks also produced greater suppression of responding maintained by positive reinforcement in the presence of a visual stimulus than did response-contingent shocks. In Experiment 2, a greater shock intensity (2 mA) was used. All the differences between the two groups found in Experiment 1 were also found in Experiment 2. Experiment 3 demonstrated that response-contingent shock did not result in a conditioned punishment effect even when positive reinforcers were unavailable during the discriminative punishment schedule. The exteroceptive stimulus that was paired with shock in the conditioned suppression procedure acquired the ability to punish behavior. The exteroceptive stimulus in the discriminative punishment schedule did not acquire this ability.     

The role of the response-punishment contingency in the suppression of a positively-reinforced operant

Two experiments tested the prediction from current theories of punishment that a response-contingent procedure is superior to a comparable noncontingent procedure for the administration of punishment. Results demonstrated that the two procedures produced comparable contiguity effects and did not contain artifacts present in previous research. Three different measures of suppression corroborated the findings that more response suppression occurred in the contingent procedure. Results were shown to be stable across eight days of shock administration (Experiment 1), and three levels of shock intensity (Experiment 2). In both experiments, response suppression occurred during contingent shocks associated with shock at offset, as well as after conditioned stimulus (CS) offset. The contingent procedure was associated with more suppression during both of these periods and also longer runs of CSs with complete suppression. The relevance of the present findings to current theories of punishment was discussed.     

Recovery of an overshadowed association achieved by extinction of the overshadowing stimulus

Three experiments are reported in which conditioned lick suppression by water-deprived rats was used as an index of associative strength. In Experiment 1, overshadowing of a light by a tone was observed when the light-tone compound stimulus was paired with foot shock. After initial compound pairings, the tone-shock association was extinguished in one group of subjects. Subsequently, these animals demonstrated significantly higher levels of suppression to the light relative to a control group in which the tone had not been extinguished. Experiment 2 replicated this effect while failing to find evidence to support the possibilities that extinction presentations of the overshadowing tone act as retrieval cues for the light-shock association, or that, via second-order conditioning, the light-shock association is actually formed during extinction of the tone. Experiment 3 determined that the recovery from overshadowing observed in Experiments 1 and 2 was specific to the extinction of the overshadowing stimulus rather than the extinction of any excitatory cue. Collectively, these results suggest that the debilitated response to an overshadowed stimulus does not represent an acquisition failure, but rather the failure of an acquired association to be manifest in behavior.     

The effect of punishment shock intensity upon responding under multiple schedules

In the first of two experiments, responses of two pigeons were maintained by multiple variable-interval, variable-ratio schedules of food reinforcement. Concurrent punishment was introduced, which consisted of a brief electric shock after each tenth response. The initial punishment intensities had no lasting effect upon responding. Then, as shock intensity increased, variable-ratio response rates were suppressed more quickly than variable-interval response rates. When shock intensity decreased, variable-interval responding recovered more quickly, but the rates under both schedules eventually returned to their pre-punishment levels. In the second experiment, the following conditions were studied in three additional pigeons: (1) With each shock intensity in effect for a number of sessions, punishment shock intensity was gradually increased and decreased and responding was maintained by multiple variable-ratio, fixed-ratio schedules of food reinforcement; (2) Changes in punishment shock intensity as described above with responding maintained by either a variable-ratio or a fixed-ratio schedule, which were presented on alternate days; (3) Session-to-session changes in shock intensity with responding maintained by multiple variable-ratio, fixed-ratio schedules. Responding under the two schedules was suppressed to approximately the same extent by a particular shock intensity. Also, post-reinforcement pauses under the fixed-ratio schedule increased as response suppression increased.     

An analysis of overshadowing and blocking

In classical aversive conditioning experiments, rats do not always learn about all aspects of a compound stimulus predicting shock. A strong stimulus may overshadow a weaker one; and pretraining on one component may block learning about a second component. These results have been explained either by appealing to a notion of selective attention, or by assuming that learning about one component is a function of prior response strength to the entire compound of which it forms a part. In Experiment I, overshadowing was demonstrated on the first trial of conditioning, i.e. before either component had acquired any response strength. In Experiment II, pretraining on one component resulted in complete failure to learn about a second component during compound training, but did not prevent additional learning about the first component. Both results were interpreted as supporting an attentional analysis of blocking and overshadowing.     

Selective punishment early and late in fixed-ratio schedules of food reinforcement

Pigeons key pecked for grain on a fixed-ratio 100 schedule; electric shocks occurred intermittently at the fifteenth or eighty-fifth response in the ratio. In Experiment I, shock was at the fifteenth response for two birds, and at the eighty-fifth response for two others, in every sixth, twelfth, or eighteenth ratio. Rate of responding decreased as frequency of shock increased, and the pattern of responding included an increased initial pause and low rates or pause-run sequences that extended further into the ratio when shock was at the fifteenth response than when it was at the eighty-fifth response. Shock early in the ratio engendered longer initial pauses than shock late in the ratio. In Experiment II, four birds responded on a two-component multiple schedule in which shock occurred at the fifteenth response of the third ratio in the presence of a white keylight and at the eighty-fifth response of the third ratio in the presence of a green keylight. The overall rates of responding decreased as shock intensity increased. All four birds responded differentially to the white and green keylights, but with a pattern that varied between birds. In general, punishment reduced the probability of responses that preceded it, regardless of the ordinal position of those responses. Both studies confirm that the probability of responding is reduced less by aversive stimuli produced late in a fixed-ratio than by aversive stimuli produced early in a fixed-ratio.     

An inhibitory within-compound association attenuates overshadowing

According to the comparator hypothesis (Miller & Matzel, 1988), cue competition depends on the association between a target stimulus (X) and a competing cue (e.g., an overshadowing cue [A]). Thus, it was expected that overshadowing would be reduced by establishing an inhibitory-like relationship between X and A before compound conditioning. In three lever press suppression experiments with rats, this expectation was supported. Experiment 1 showed that establishing an inhibitory X-A relationship reduced overshadowing. In Experiment 2, degrading the inhibitory-like relationship before conditioning allowed reinforced AX compound trials to result in overshadowing. Experiment 3 replicated the results of Experiment 2 when the inhibitory relationship was degraded after compound conditioning. The results support the view that within-compound associations are necessary not only for retrospective revaluation, but also for conventional cue competition.     

The role of response-reward correlation in stimulus-response overshadowing

Rats pressed levers for food reward which was delivered, when appropriate, 0·4 s after the response. For one group, the delay interval was filled by a light cue; for the other group, the same number of lights was given but they were not correlated with food delivery. In Experiment I, all lever presses were reinforced and there were no differences in response rate between groups. In Experiments II and III, lever pressing was rewarded according to a VI and VR schedule respectively. Group differences were observed in Experiment II but they disappeared in Experiment III. The results of Experiments I and II show that a reward-related stimulus does not overshadow a lever response unless the stimulus is a better predictor of reward. Differences in salience or competition from sign-tracking behaviors were ruled out as causes of this phenomenon. Experiment III demonstrated, however, that a weak response-reward correlation is not a sufficient condition for the overshadowing effect. A fourth experiment replicated the results of Experiment III using naive animals. The results of these last two experiments are not consistent with an information theory approach unless (a) a response-units concept is adopted or (b) the cue involved in overshadowing is not the pre-food light but the end of a temporal interval, whose salience is enhanced by the light.     

SUPPRESSIVE AND FACILITATIVE EFFECTS OF SHOCK INTENSITY AND INTERRESPONSE TIMES FOLLOWED BY SHOCK

Although response‐dependent shock often suppresses responding, response facilitation can occur. In two experiments, we examined the suppressive and facilitative effects of shock by manipulating shock intensity and the interresponse times that produced shock. Rats' lever presses were reinforced on a variable‐interval 40‐s schedule of food presentation. Shock followed either long or short interresponse times. Shock intensity was raised from 0.05 mA to 0.4 mA or 0.8 mA. Overall, shock contingent on long interresponse times punished long interresponse times and increased response rates. Shock contingent on short interresponse times punished short interresponse times and decreased response rates. In Experiment 1, raising the range of interresponse times that produced shock enhanced these effects. In Experiment 2, the effects of shock intensity depended on the interresponse times that produced shock. When long interresponse times produced shock, low intensities increased response rates. High intensities decreased response rates. When short interresponse times produced shock, high shock intensities punished short interresponse times and decreased response rates more than low intensities. The results may explain why punishment procedures occasionally facilitate responding and establish parameters for future studies of punishment.     

Postinjection suppression of drinking is modified by the presence of conditioned contextual cues: implications for both anticipatory and posttreatment nausea in humans

In three experiments, we set out to determine whether the response of rats to an injection of LiCl would be modified by the presence of an environmental context that had previously been paired with LiCl. Experiment 1 confirmed that one feature of the malaise produced by LiCl is a reduced tendency to consume an otherwise palatable flavor. Experiment 2 showed that the size of this response was enhanced if it was measured in the presence of a conditioned context. In Experiment 3, we investigated the possibility that the postinjection response could be modified by an overshadowing treatment given during the conditioning phase. The significance of these findings for the understanding of chemotherapy-induced nausea in the clinical population is discussed.     

Stimulus-Response Compatibility for Vertically Oriented Stimuli and Horizontally Oriented Responses: Evidence for Spatial Coding

It has previously been shown that, when stimuli positioned above or below a central fixation point (“up” and “down” stimuli) are assigned to left and right responses, the stimulus-response mapping up-left/down-right is more compatible than the mapping up-right/down-left for responses executed by the left hand in the left hemispace, but this relation is reversed for responses executed by the right hand in the right hemispace. In Experiment 1, each hand responded at locations in both hemispaces to dissociate the influence of hand identity from response location, and response location was found to be the determinant of relative compatibility. In Experiment 2 responses were made at the sagittal midline, and an inactive response switch was placed to the left or right to induce coding of the active switch as right or left, respectively. This manipulation of relative location had an effect similar to, although of lesser magnitude than, that produced by physically changing location of the response switch in Experiment 1. It is argued that these results are counter to predictions of a movement-preference account and consistent with the view that spatial coding underlies compatibility effects for orthogonally oriented stimulus and response sets.     

Pavlovian processes and response competition as determinants of avoidance response-prevention effects

Two experiments investigated the facilitation of avoidance extinction by exposure to lengthy (5-sec) shock during avoidance response prevention. In Experiment 1, animals exposed to light only or to light-shock pairings during response prevention showed equal facilitation of extinction relative to shock-only animals or to animals receiving no response prevention. Preshock rearing, directly antagonistic to the avoidance response, developed for shocked animals during response prevention and persisted during extinction for light-shock animals. Immediately before extinction, half of each group was permitted a single escape from a light-shock compound by means of the response previously required for avoidance. The only effect was upon the extinction performance of light-shock animals. Rearing was eliminated and extinction responding increased to a level far above that for any of the other animals. Experiment 2 demonstrated that the shock-only treatment affected the extinction performance and rearing of nonescape and escape animals in a manner entirely equivalent to the effects of the light-shock treatment of Experiment 1, provided stimulus conditions (light absent) were the same for all experimental phases. Thus, lengthy shock during avoidance response prevention simultaneously leads to the acquisition of competing behavior and enhances control by a warning signal or contextual stimuli over the avoidance response. Implications for the CS-only response-prevention treatment and the transfer of aversive control are discussed.     

Punishment of an extinguishing shock-avoidance response by time-out from positive reinforcement

Two experiments were conducted to determine the effects of punishment by time-out from positive reinforcement on the extinction of discriminated shock-avoidance responding. Subjects were trained initially to bar press for food on an intermittent schedule of reinforcement and, concurrently, to avoid shock at the onset of a warning signal. Experiment I compared avoidance extinction performance under no punishment and when avoidance responding resulted in a 30-sec TO from reinforced appetitive responding. In Exp II, the contingent use of TO punishment was compared with its random, or noncontingent use. The results of both experiments showed that in the absence of punishment, avoidance extinction was characterized by short latencies and nearly 100% avoidance responding. Avoidance responding in extinction was little affected by noncontingent TO punishment. When TO was made contingent upon avoidance responding, however, avoidance latencies immediately increased and the frequency of avoidance responses subsequently decreased to zero.     

After-effects of responding and of withholding responses in C.RT tasks

The effects of response repetition on choice RT were compared in b-reaction and in c-reaction tasks [Experiments I(a) and I(b)]. The difference in RTs for repeated and for non-repeated responses was found to be less for c-reaction than for b-reaction tasks. This seemed to be because in c-reaction tasks subjects can prepare themselves to make the same response on every trial, so that there is little further RT reduction consequent on immediate response repetition. In b-reaction tasks subjects cannot always prepare to make the same response, so that the difference between response repetition RT and responce alteration RT is greater. Experiment II examined transitions between events in a serial, self-paced C.RT task in which subjects made a different response to each of two signals but withheld any response to the onset of a third. In this task responses were faster when they followed other, different responses than when they followed “no go” trials. The results of these experiments allow us to reject, even for very elementary tasks, a simple “S--R connection network” model for the processes involved in the identification of signals and the production of responses to them.     

Continuous punishment of free-operant avoidance in the rat

Three groups of albino rats were trained under a free-operant avoidance (Sidman) procedure with equal shock-shock and response-shock intervals. After stable performance was achieved, the animals were concurrently exposed to a brief electric shock after each response. The procedures were as follows: Punishment Schedule I: punishment shock was introduced at an intensity approximately one quarter that of avoidance shock; increments of nearly this same size were made as stable performance was achieved at succeeding punishment shock intensities. Punishment Schedule II: punishment shock was introduced at approximately one-half the intensity of avoidance shock; after stable performance, punishment shock was increased to the same intensity as avoidance shock. Punishment Schedule III: punishment shock was introduced and maintained at the same intensity as avoidance shock. Punishment was continued for all groups until one of two suppression criteria was attained. All animals made fewer responses and received more avoidance shocks as a function of increasing punishment shock. Half of the animals under Punishment Schedule I required punishment shock higher than avoidance shock to meet their assigned suppression criterion. A comparison of all procedures showed that suppression was greater when punishment shock was initially at high intensity.