The effects of 100 dB 1‐kHz and 22‐kHz tones as punishers on lever pressing in rats

Aversive control is an important yet understudied process of learning. One reason aversive control may be relatively understudied is ethical concerns about painful stimuli (e.g., electric shock). High decibel broad‐band noise and 22‐kHz vocalizations both demonstrably affect rodent behavior while not necessarily being painful. The goal of this study was to determine if 100‐dB 22‐kHz‐pure tones were differentially more effective in reducing operant response rates in rats. We examined whether 22‐kHz pure tones would function as aversive stimuli, specifically as positive punishers. The effects of response‐dependent as well as continuously presented 22‐kHz and 1‐kHz tones on rate of response maintained by variable interval 30‐s food deliveries were assessed across several conditions. We found that response rates were lower when tones were presented response dependently than when tones were presented continuously throughout a session. We also found that the lower response rates obtained with response‐dependent 22‐kHz tones were not significantly different from response rates obtained with response‐dependent 1‐kHz tones. The primary conclusion of this experiment is that both 1‐kHz and 22‐kHz tones functioned as punishers, but that the 22‐kHz tones were not differentially more effective in reducing response rate.     

COMPARING PLEASURE AND PAIN: THE FUNDAMENTAL MATHEMATICAL EQUIVALENCE OF REWARD GAIN AND SHOCK REDUCTION UNDER VARIABLE INTERVAL SCHEDULES

The relationship between positive and negative reinforcement and the symmetry of Thorndike's law of effect are unresolved issues in operant psychology. Here we show that, for a given pattern of responding on variable interval (VI) schedules with the same programmed rate of food rewards (positive reinforcement VI) or electric shocks (negative reinforcement VI), there is a fundamental mathematical equivalence between reward gain and shock reduction. We also provide the first normative account of how animals should respond on a negative VI schedule, showing that it is better to space responses evenly than to respond with a variable interresponse time (IRT). Published data from rats, however, indicate that these animals respond irregularly, often with a burst of activity immediately following a shock. While this is irrational in the experimental setting, it may represent an appropriate response to the heterogeneity of stimuli commonly encountered in natural environments. We discuss the broader implications of our analysis for understanding how animals evaluate appetitive and aversive stimuli.     

RESISTANCE TO CHANGE AND FREQUENCY OF RESPONSE‐DEPENDENT STIMULI UNCORRELATED WITH REINFORCEMENT

Stimuli uncorrelated with reinforcement have been shown to enhance response rates and resistance to disruption; however, the effects of different rates of stimulus presentations have not been assessed. In two experiments, we assessed the effects of adding different rates of response‐dependent brief stimuli uncorrelated with primary reinforcement on relative response rates and resistance to change. In both experiments, pigeons responded on variable‐interval 60‐s schedules of food reinforcement in two components of a multiple schedule, and brief response‐dependent keylight‐color changes were added to one or both components. Although relative response rates were not systematically affected in either experiment, relative resistance to presession feeding and extinction were. In Experiment 1, adding stimuli on a variable‐interval schedule to one component of a multiple schedule either at a low rate (1 per min) for one group or at a high rate (4 per min) for another group similarly increased resistance to disruption in the components with added stimuli. When high and low rates of stimuli were presented across components (i.e., within subjects) in Experiment 2, however, relative resistance to disruption was greater in the component presenting stimuli at a lower rate. These results suggest that stimuli uncorrelated with food reinforcement do not strengthen responding in the same way as primary reinforcers.     

Positive and negative conditioned suppression in the pigeon: Effects of the locus and modality of the CS

In Experiment I, four pigeons were exposed to trials in which a 12-sec key light illumination was followed by free food. These trials were superimposed upon a baseline of key pecking for food reinforcement on a variable-interval schedule. When the signal for food was on the operant key, response rate was substantially higher during the signal than during the baseline procedure. When the signal was on a second, signal key, operant responding was suppressed during the signal and substantial pecking of the signal key occurred. The sum of signal key and operant key pecks far exceeded the operant baseline rate of responding. An explanation of opposite results obtained with rats and pigeons as subjects in experiments of this type was suggested in terms of the spatial relation between the signal for free food and the operant target which usually characterizes these experiments. Experiment II assessed the importance of signal location when shock rather than food was the US. Suppression of operant key pecking was unaffected by signal location. Experiment III assessed the relative effectiveness of visual and auditory stimuli (clicks) as signals for food and shock, and found that all combinations of signal and US were equally effective in suppressing operant key responding. The three experiments together suggested that the identification of important effects of species—typical behavior in one experimental situation does not imply that there will be like effects in similar situations.     

Responding maintained under fixed-interval and fixed-time schedules of electric shock presentation

Following initial histories under a schedule of electric shock postponement, lever pressing in squirrel monkeys was maintained under fixed-interval and fixed-time schedules of electric shock presentation. No difference in either rate or pattern of responding was obtained when these schedules were presented as components of a multiple schedule. When they were presented singly for long periods of time, the fixed-interval schedule consistently maintained a higher response rate than the fixed-time schedule. The pattern of responding under both schedules was similar, typically consisting of a pause at the beginning of each interval followed by either a steady or a positively accelerating rate of responding. The results suggest that the response-shock dependency is of critical importance in the maintenance of high rates of responding under schedules of electric shock presentation, and support the general view that such responding may be conceptualized as operant behavior under control of many of the same variables that control responding under comparable schedules of food or water reinforcement.     

Effect of signaled reinforcement on response variability

Three experiments examined the effect of signaling reinforcement on rats' lever pressing on contingencies that reinforced variable responding to extend the exploration of signaled reinforcement to a schedule that has previously not been examined in this respect. In Experiment 1, rats responding on a lag-8 variability schedule with signaled reinforcement displayed greater levels of variability (U values) than rats on the same schedule lacking a reinforcement signal. In Experiment 2, rats responding on a differential reinforcement of least frequent responses schedule also displayed greater operant variability with a signal for reinforcement compared with rats without a reinforcement signal. In Experiment 3, a reinforcement signal decreased the variability of a response sequence when there was no variability requirement. These results offer empirical corroboration that operant variability responds to manipulations in the same manner as do other forms of operant response and that a reinforcement signal facilitates the emission of the required operant.     

EFFECTS OF POINT‐LOSS PUNISHERS ON HUMAN SIGNAL‐DETECTION PERFORMANCE

Three experiments using human participants varied the distribution of point‐gain reinforcers or point‐loss punishers in two‐alternative signal‐detection procedures. Experiment 1 varied the distribution of point‐gain reinforcers for correct responses (Group A) and point‐loss punishers for errors (Group B) across conditions. Response bias varied systematically as a function of the relative reinforcer or punisher frequencies. Experiment 2 arranged two conditions — one where an unequal ratio of reinforcement (5:1 or 1:5) was presented without punishment (R‐only), and another where the same reinforcer ratio was presented with an equal distribution of point‐loss punishers (R+P). Response bias was significantly greater in the R‐only condition than the R+P condition, supporting a subtractive model of punishment. Experiment 3 varied the distribution of point‐gain reinforcers for correct responses across four unequal reinforcer ratios (5:1, 2:1, 1:2, 1:5) both without (R‐only) and with (R+P) an equal distribution of point‐loss punishers for errors. Response bias varied systematically with changes in relative reinforcer frequency for both R‐only and R+P conditions, with 5 out of 8 participants showing increases in sensitivity estimates from R‐only to R+P conditions. Overall, the results indicated that punishers have similar but opposite effects to reinforcers in detection procedures and that combined reinforcer and punisher effects might be better modeled by a subtractive punishment model than an additive punishment model, consistent with research using concurrent‐schedule choice procedures.     

SUPPRESSIVE AND FACILITATIVE EFFECTS OF SHOCK INTENSITY AND INTERRESPONSE TIMES FOLLOWED BY SHOCK

Although response‐dependent shock often suppresses responding, response facilitation can occur. In two experiments, we examined the suppressive and facilitative effects of shock by manipulating shock intensity and the interresponse times that produced shock. Rats' lever presses were reinforced on a variable‐interval 40‐s schedule of food presentation. Shock followed either long or short interresponse times. Shock intensity was raised from 0.05 mA to 0.4 mA or 0.8 mA. Overall, shock contingent on long interresponse times punished long interresponse times and increased response rates. Shock contingent on short interresponse times punished short interresponse times and decreased response rates. In Experiment 1, raising the range of interresponse times that produced shock enhanced these effects. In Experiment 2, the effects of shock intensity depended on the interresponse times that produced shock. When long interresponse times produced shock, low intensities increased response rates. High intensities decreased response rates. When short interresponse times produced shock, high shock intensities punished short interresponse times and decreased response rates more than low intensities. The results may explain why punishment procedures occasionally facilitate responding and establish parameters for future studies of punishment.     

Interactions between the effects of food and water motivating operations on food‐ and water‐reinforced responding in mice

Two experiments examined interactions between the effects of food and water motivating operations (MOs) on the food‐ and water‐reinforced operant behavior of mice. In Experiment 1, mice responded for sucrose pellets and then water reinforcement under four different MOs: food deprivation, water deprivation, concurrent food and water deprivation, and no deprivation. The most responding for pellets occurred under food deprivation and the most responding for water occurred under water deprivation. Concurrent food and water deprivation decreased responding for both reinforcers. Nevertheless, water deprivation alone increased pellet‐reinforced responding and food deprivation alone likewise increased water‐reinforced responding relative to no deprivation. Experiment 2 demonstrated that presession food during concurrent food and water deprivation increased in‐session responding for water relative to sessions where no presession food was provided. Conversely, presession water during concurrent food and water deprivation did not increase in‐session responding for pellets. These results suggest that a) the reinforcing value of a single stimulus can be affected by multiple MOs, b) a single MO can affect the reinforcing value of multiple stimuli, and c) reinforcing events can also function as MOs. We consider implications for theory and practice and suggest strategies for further basic research on MOs.     

The effect of punishment shock intensity upon responding under multiple schedules

In the first of two experiments, responses of two pigeons were maintained by multiple variable-interval, variable-ratio schedules of food reinforcement. Concurrent punishment was introduced, which consisted of a brief electric shock after each tenth response. The initial punishment intensities had no lasting effect upon responding. Then, as shock intensity increased, variable-ratio response rates were suppressed more quickly than variable-interval response rates. When shock intensity decreased, variable-interval responding recovered more quickly, but the rates under both schedules eventually returned to their pre-punishment levels. In the second experiment, the following conditions were studied in three additional pigeons: (1) With each shock intensity in effect for a number of sessions, punishment shock intensity was gradually increased and decreased and responding was maintained by multiple variable-ratio, fixed-ratio schedules of food reinforcement; (2) Changes in punishment shock intensity as described above with responding maintained by either a variable-ratio or a fixed-ratio schedule, which were presented on alternate days; (3) Session-to-session changes in shock intensity with responding maintained by multiple variable-ratio, fixed-ratio schedules. Responding under the two schedules was suppressed to approximately the same extent by a particular shock intensity. Also, post-reinforcement pauses under the fixed-ratio schedule increased as response suppression increased.     

HUMAN RESPONDING ON RANDOM‐INTERVAL SCHEDULES OF RESPONSE‐COST PUNISHMENT: THE ROLE OF REDUCED REINFORCEMENT DENSITY

An experiment with adult humans investigated the effects of response‐contingent money loss (response‐cost punishment) on monetary‐reinforced responding. A yoked‐control procedure was used to separate the effects on responding of the response‐cost contingency from the effects of reduced reinforcement density. Eight adults pressed buttons for money on a three‐component multiple reinforcement schedule. During baseline, responding in all components produced money gains according to a random‐interval 20‐s schedule. During punishment conditions, responding during the punishment component conjointly produced money losses according to a random‐interval schedule. The value of the response‐cost schedule was manipulated across conditions to systematically evaluate the effects on responding of response‐cost frequency. Participants were assigned to one of two yoked‐control conditions. For participants in the Yoked Punishment group, during punishment conditions money losses were delivered in the yoked component response independently at the same intervals that money losses were produced in the punishment component. For participants in the Yoked Reinforcement group, responding in the yoked component produced the same net earnings as produced in the punishment component. In 6 of 8 participants, contingent response cost selectively decreased response rates in the punishment component and the magnitude of the decrease was directly related to the punishment schedule value. Under punishment conditions, for participants in the Yoked Punishment group response rates in the yoked component also decreased, but the decrease was less than that observed in the punishment component, whereas for participants in the Yoked Reinforcement group response rates in the yoked component remained similar to rates in the no‐punishment component. These results provide further evidence that contingent response cost functions similarly to noxious punishers in that it appears to suppress responding apart from its effects on reinforcement density.     

An evaluation of a punisher assessment for decreasing automatically reinforced problem behavior

We extended research on the identification and evaluation of potential punishers for decreasing automatically reinforced problem behavior in four individuals with autism spectrum disorder. A punisher selection interview was conducted with lead clinicians to identify socially acceptable punishers. During the treatment evaluation, treatment phases were introduced sequentially and included noncontingent reinforcement (NCR), NCR and differential reinforcement of alternative behavior (DRA), and NCR‐and‐DRA with punishment. During the NCR‐and‐DRA with punishment phase, four to five potential punishers were evaluated using a multielement design. Dependent measures included the target problem behavior, appropriate item engagement, and emotional responding. For all participants, NCR‐and‐DRA was not effective and punishment was necessary. However, the most effective punisher identified in the context of NCR‐and‐DRA differed across participants.     

Drugs and punished responding II: d-amphetamine-induced increases in punished responding

The effects of d-amphetamine on punished responding were studied in two experiments. In Experiment I, pigeons responded under a multiple fixed-ratio 30 response fixed-interval 5-min schedule of food presentation with 60-sec limited holds in both components. Each response was punished with electric shock, the intensity of which was varied systematically. In Experiment II, another group of pigeons responded under a multiple fixed-interval 5-min fixed-interval 5-min schedule of food presentation with 40-sec limited holds. Each response was punished with shock during one component, and every thirtieth response was punished in the other component. d-Amphetamine increased overall rates of punished responding only rarely under any of the punishment conditions; however, response rates within the fixed-interval when rates were low were increased by d-amphetamine when the shock intensity was low (Experiment I), or when responses produced shock intermittently (Experiment II). The data suggest that the effects of d-amphetamine on punished responding depend on the control rate of responding, the punishment intensity, the punishment frequency, and the schedule of food presentation.     

Discriminative properties of shock-correlated reinforcement in an operant context

A two-choice discrete operant procedure was devised for the study of shock-correlated reinforcement effects in rats. In the presence of one auditory stimulus, responding on one response lever was reinforced with food; with another auditory stimulus, responding on a second lever was reinforced. It was found that discrimination performance of one group, relative to appropriate control groups, was facilitated when electric shock was correlated with reinforcement on one lever and not on the other. Further, relative discrimination levels were found to be higher on the lever correlated with the shock than on the alternate lever. The significance of the results for operant within-S studies and for a mediational theory of shock-correlated reinforcement was discussed.     

Schedules of food postponement: II. Maintenance of behavior by food postponement and effects of the schedule parameter

In Experiment I, food-deprived, feeder-trained squirrel monkeys pressed a lever to postpone brief electric shocks (Response-Shock=Shock-Shock interval=30 seconds). Forty-one three-hour sessions of shock postponement were followed by 120 sessions of concurrent shock and food postponement. The shock schedule was unchanged and the food schedule was Response-food interval–20 seconds, Food-food interval 10 seconds. After concurrent shock and food postponement, the shock schedule was discontinued and 40 sessions of food postponement ensued, followed by 53 sessions of extinction. After extinction, food postponement was resumed for 11 sessions. Stable responding with low food rates was maintained under food-postponement after the concurrent schedule. Responding decreased to low levels under extinction and recovered immediately to previous levels when the food-postponement schedule was re-instated. In Experiment II, a parameter of the food-postponement schedule was studied sequentially. Using the same subjects, the Response-food–Food-food interval was manipulated from four seconds to 80 seconds with several orders of presentation. Relations of response rates and food rates to the parameter were similar to those seen under shock postponement. Exposure to very short postponement times (four seconds), resulting in very high food rates, decreased but did not abolish subsequent responding at longer postponement times. Results are discussed from the point of view that reinforcing functions of stimuli consequent on responding depend on a prior history of scheduled contact with those stimuli.     

Induced attack during fixed-ratio and matched-time schedules of food presentation

Adjunctive or induced behavior is generated during a variety of schedules of reinforcement. Several theoretical conceptualizations suggest that rate of reinforcement is the primary variable controlling the strength or levels of induced behavior. The operant response requirement within the schedule context has not been extensively studied as a determinant of induced responding. In the present study, levels of induced attack by food-deprived pigeons against restrained conspecifics were compared during response-dependent and response-independent schedules of food presentation equated or yoked interval-by-interval for reinforcement frequency. Experiment 1 compared levels of attack induced by fixed-ratio schedules of key pecking and yoked "matched-time" schedules. Experiment 2 similarly compared chained fixed-ratio 1 fixed-ratio 74 and yoked chained matched-time matched-time schedules. In both experiments, the response-dependent schedules generated greater levels (amount and probability) of induced attack than the response-independent time-based schedules. Thus, the ratio response requirement may be an important determinant of levels of induced responding, and the lower levels of attack observed during the response-independent condition may not be due to the absence of stimuli predicting food presentations. It is concluded that rate of reinforcement is not the sole variable determining levels of induced responding and that response-based and time-based schedules differ in their generation of induced responding.     

Control of responding by sounds of different quality: an evolutionary analysis.

Two experiments investigated the acquisition of discriminations between two acoustic stimuli of different quality (noise bursts vs. a 2-kHz pulsed signal) when features of the everyday environment were incorporated into the experiments. In Experiment 1, rats were trained, using food, to press a lever. Throughout all sessions, 5-s trials of noise bursts (the random stimulus) were presented, after variable intertrial intervals, through a remote speaker mounted outside the experimental enclosure. The noise burst occurred randomly with respect to reinforcement of lever pressing and had no programmed relationship to the animal's behavior. When lever pressing was established, the 2-kHz signal was presented through a speaker adjacent to the response lever according to a different set of variable intertrial intervals. A response in the presence of the 2-kHz signal terminated the trial and was reinforced. The 2-kHz signal acquired control of responding within the first few trials, whereas the random stimulus exerted no control of responding. In Experiment 2, rats were trained to press the lever in the presence of the 2-kHz signal presented through the adjacent speaker on a variable intertrial interval. After 14 sessions, 5-s trials of noise bursts (random stimulus) were presented through the remote speaker on the second variable intertrial interval. The random stimulus initially elicited exploratory behavior, which then rapidly declined. Subsequently, the random stimulus exerted no or weak control of responding. The introduction of the random stimulus had no effect on responding in the presence of the adjacent stimulus. In Experiments 3 and 4 the random stimulus was presented through the adjacent speaker, and the stimulus correlated with reinforcement was presented through the remote speaker. In both experiments, there was persistent control of responding by the random stimulus and slow development of control by the stimulus correlated with reinforcement. In Experiment 5, both stimuli were presented through the adjacent speaker. There was persistent control of responding by the random stimulus.     

Schedule interactions involving punishment with pigeons and humans

The principal aim of the present experiments was to assess whether punishment increased or decreased the rate of unpunished behavior (contrast and induction, respectively) for which reinforcement rate was held constant, with physical and nonphysical punishers (electric shock and response cost), pigeon and human subjects, signaled and unsignaled components (multiple and mixed schedules), and the presence or absence of a blackout period between components. Across the three experiments there were 20 punishment conditions. Induction was found in nine of those, less consistent response-rate reduction was found in three, contrast was found in four, and in four there was no change in responding from conditions without punishment. Contrast occurred consistently only with multiple schedules during the first exposure to electric-shock punishment. Induction and no change, however, were found with every combination of the independent variables studied. Four conclusions regarding the interactions between punished and unpunished responding emerged from the present results: (a) Both contrast and induction occurred with the reinforcement rate held constant and a blackout between components, (b) induction was more common than contrast, (c) contrast occurred only in the presence of a stimulus different from that correlated with the punisher, and (d) contrast diminished with prolonged exposure to punishment. None of the current theoretical accounts of punishment contrast can explain the present results.     

Response acquisition by zebrafish (Danio rerio) with delayed reinforcement

Zebrafish (Danio rerio) is a common vertebrate animal model in biomedical research and is a promising species for studying how genes interact with environmental factors in determining behavior. The present study investigated how reinforcement parameters affect zebrafish behavior by assessing response acquisition with delayed reinforcement, which has been studied with other species (e.g., rats, pigeons, humans, etc.) but not with zebrafish. Twenty‐four experimentally naïve subjects were exposed to a tandem fixed‐ratio 1 differential‐reinforcement‐of‐other‐behavior x‐s schedule of reinforcement, where x varied across subjects. There were six different delay‐to‐reinforcement durations and sets of four fish were assigned to each delay duration. All of the fish assigned to a 0‐, 0.5‐, or 1‐s delay acquired responding. Two fish acquired responding with a 3‐s delay and one fish appeared to have acquired it with a 6‐s delay although the latter result was less clear. None acquired responding with a 12‐s delay. These results suggest that zebrafish behavior is sensitive to delays to reinforcement and the time frame over which reinforcement is effective may be limited approximately to 6 s. This time frame is shorter than that found with other species. Practical and theoretical implications of the present finding are discussed.     

Conditioned suppression of drl responding: Effect of ucs intensity, schedule parameter, and schedule context

In Experiment I, groups of rats were trained to press a lever for food reinforcement on differential reinforcement of low rate (DRL) schedules which differed in parameter value. A stimulus which terminated with either a 0.5-mA or 2.0-mA electric shock was then superimposed upon each DRL baseline. In general, the magnitude of conditioned suppression was an inverse function of DRL schedule parameter and a direct function of shock intensity. Experiment II demonstrated that the rate of responding maintained by the DRL component of a multiple DRL-extinction schedule decreased during a stimulus preceding a 0.5-mA shock, whereas the rate of responding maintained by the DRL component of a multiple DRL-variable interval schedule showed little change or increased slightly during a stimulus preceding a 0.5-mA shock.