Response latency as a function of amount of reinforcement

Food-deprived rats were trained to press and hold down a telegraph key in the presence of a light. Subsequent release of the key during a tone was followed by 0.15 ml of a 20-percent sucrose solution as reinforcement. The Ss were subsequently shifted to a 0-percent and to a 5-percent solution from the 20-percent base line. The median RT and the variability of RT increased markedly as a result of the shift to the lower sucrose concentrations. For all Ss, the change in median and variability was greater for the shift to the 0-percent solution than for the shift to the 5-percent solution. It is probable that median RT and variability of RT are inversely related to amount of reinforcement.     

Preference for signalled reinforcement

Key pecking was reinforced on a two-component multiple schedule. A variable-interval schedule controlled reinforcement in both components. During one component, access to reinforcement was preceded by a tone; in the other component, a standard unsignalled schedule was in effect. After performance stabilized, subjects were given a choice between the signalled and unsignalled schedules. They were placed in the chamber with the unsignalled schedule in effect on the right key. A single response on the left, or changeover, key produced the signalled schedule for 1 min. Both pigeons in Experiment I pecked the changeover key at a rate sufficient to remain under the signalled schedule for over 90% of the session. Removing and reintroducing the tone demonstrated that the changeover-key responses were due to the occurrence of the tone. In Experiment II, when pecking the changeover key produced the unsignalled schedule, pecking the changeover key declined. The results may be explained either in terms of Hendry's information hypothesis or as escape from an intermittent positive reinforcement schedule.     

Alternative reinforcement effects on fixed-interval performance

Pigeons' key pecks were reinforced with food on a fixed-interval schedule. Food also was available at variable time periods either independently of responding or for not key pecking (a differential-reinforcement-of-other-behavior schedule). The latter condition arranged reinforcement following the first pause of t seconds after it became available according to a variable-time schedule. This schedule allowed separation of the effects of pause requirements ≤ five-seconds and reinforcement frequency. The time spent pausing increased as the duration of the pause required for reinforcement increased from 0 to 30 seconds and as the frequency of reinforcement for pausing increased from 0 to 2 reinforcers per minute. Key pecking was more evenly distributed within each fixed interval with shorter required pauses and with more frequent reinforcement for pausing. The results complement those obtained with other concurrent schedules in which the same operant response was reinforced in both components.     

Auditory stimulus control in pigeons: Jenkins and Harrison (1960) revisited

Pigeons were trained to peck a key in the presence of a 1000-Hz tone on a variable-interval one-minute schedule of reinforcement. One group was trained with an illuminated key; the other was trained in a totally dark chamber. During a generalization test on tonal frequency, subjects trained and tested with the key illuminated produced rather shallow gradients around the training value; subjects trained and tested in the dark produced steeper generalization gradients. These data replicate Jenkins and Harrison's (1960) finding that tone acquires relatively little control over responding and demonstrate that this absence of control is a function of the presence of the keylight.     

Concurrent assessment of schedule and intensity control across successive discriminations

Two chinchillas were trained on a series of two-valued auditory intensity discriminations. Lever presses were reinforced when no tone was present and not reinforced in the presence of a four-kiloHertz tone. The intensity of the nonreinforced tone was successively decreased, increasing the difficulty of the discrimination, until differential responding resembled that on a mixed schedule (no-tone–no-tone). Response data were partitioned in such a way as to provide a continuing assessment of the relative amounts of control exerted by the reinforcement schedule and the sound intensity, respectively. Control by reinforcement density was a direct function of discrimination difficulty, whereas the control exerted by intensity was inversely related to difficulty. For these chinchillas, the absolute threshold value obtained at four kiloHertz was about two decibels referenced to 20 microNewtons per meter squared.     

Antecedent reinforcement contingencies in the stimulus control of an auditory discrimination

In order to assess possible confounding of discriminative stimulus effects with those produced by the reinforcing stimulus, three groups of four rats each were trained for 45 hr on a variable-interval 1-min reinforcement program. Two groups were run on a multiple variable-interval extinction schedule in which the reinforcement stimulus (S^D) and the nonreinforcement stimulus (S^Δ) were two intensities of a 4-kHz (cps) tone separated by 40 or 10 db. The third group was run on a mixed schedule with a single intensity constantly present. The mixed-schedule animals showed no discrimination of the reinforcement program. Under the multiple schedule, the highest S^Δ rates were obtained after S^D intervals, regardless of the reinforcement availability in the S^D interval. These local rate variations in S^Δ were small in proportion to those produced by the S^D versus S^Δ intensities.     

Transfer of control of the pigeon's key peck from food reinforcement to avoidance of shock

Eight pigeons were initially trained to peck a white key for food under a variable-interval 1-min schedule of reinforcement. Then, a shock-avoidance schedule was initiated and food was no longer available in the experimental situation. Under the avoidance schedule, each peck on the key postponed shock for 40 sec. A warning signal, consisting of tone and red houselights, was presented after 30 sec without a response. If no response occurred, a shock was delivered 10 sec after warning-signal onset. Shocks were delivered every 10 sec in the presence of the warning signal until a response was made. The warning signal was terminated only by a response. Key pecking of all eight pigeons came under control of the avoidance schedule and responding continued throughout the 20-day avoidance training period.     

Conditioned acceleration and conditioned suppression in pigeons

Two experiments were performed to investigate the effects on pigeons' keypecking behavior of stimuli that signal different kinds of aversive events: time-out from positive reinforcement, electric shock, loud noise, and loud tone. Behavior maintained by a variable-interval schedule of reinforcement was suppressed by a stimulus before shock, was accelerated by a stimulus before time-out from positive reinforcement, and was unchanged by a stimulus before loud noise or a stimulus before loud tone. Conditioned acceleration with time-out from positive reinforcement and conditioned suppression with shock were obtained regardless of whether a response contingent or response-independent procedure was employed.     

Schedule-induced mirror responding in the pigeon

Two pigeons that were previously exposed to a multiple schedule of reinforcement in the presence of a stuffed and a live pigeon, and two of three naive pigeons, responded on a mirror during exposure to multiple fixed-ratio, fixed-ratio schedules of reinforcement for key pecking. Both the topography and temporal pattern of mirror responding were comparable to schedule-induced “attack” on live and stuffed targets. Rate of target responding was reduced when either the mirror was covered with paper or when the multiple schedule was removed. A reversal in the relationship between reinforcement schedules and discriminative stimuli demonstrated that mirror responding was controlled by the stimulus correlated with the higher fixed-ratio schedule. With one component of the multiple schedule held constant at fixed ratio 25 and the ratio requirement of the other component varying from 25 to 150, there was an inverted U-shaped relationship between rate of mirror responding and fixed-ratio schedule in the varied component. As in Flory's study (1969b) there was an inverted U-shaped relationship between target responding and inter-food intervals. The combined results of these studies suggest that the relationship between rate of target responding and reinforcement schedules is controlled primarily by the inter-food intervals resulting from the schedules.     

The influence of positive and negative reinforcement on selective attention in the rat

In Experiments 1 and 2 rats were trained under two multiple schedules of reinforcement. In one, bar pressing during a tone-light compound stimulus was reinforced under a variable-interval food reinforcement schedule. In the other multiple schedule, bar pressing avoided grid shock on a free-operant schedule. In both multiple schedules, a discrimination was maintained by an extinction schedule that was operative during the absence of the tone-light compound. In Experiments 1 and 2 the intensity of the tone-light compound was manipulated over three levels. Subsequent extinction tests revealed that light was attended to, almost exclusively of the tone, when food reinforcement had maintained bar pressing. On the other hand, the tone gained considerable attentional control under the shock avoidance schedule. This stimulus-reinforcer interaction was maintained for all three levels of the compound intensity. In Experiment 3 it was investigated whether this interaction was associative by presenting shock during the absence of the tone-light compound when food reinforcement maintained responding, and food during the absence of the compound when shock avoidance maintained responding. Since both food and shock were presented during a single session for both schedules, nonassociative effects of the reinforcing stimuli were equivalent across the schedules. Nevertheless, the stimulus-reinforcer interaction was maintained, indicating that the interaction was an associative effect.     

Conditioned suppression or facilitation as a function of the behavioral baseline

Rats were exposed to a multiple schedule of reinforcement. During one component, a bar-press was followed by reinforcement only if it occurred between 15 and 20 sec after the previous response. This differential-reinforcement-of-low-rate (DRL) schedule produced a typical slow rate of responding. During the other component, reinforcement followed the first response to be emitted during limited periods of time which occurred at fixed intervals. These fixed-interval schedules with a limited hold produced higher response rates, described as `interval' or `ratio-like' behavior. Responding during the DRL component increased in frequency during a tone which ended with an unavoidable shock of low intensity, but decreased during the tone when the shock intensity was raised. The `interval' and `ratio-like' responding decreased in frequency during the tone at all shock intensities. Initial acceleration of the DRL responding appeared to be due to adventitious punishment of collateral behavior which was observed between the bar-presses. The more severe conditioned suppression during the fixed-interval components might be the result of the lower probability of reinforcement after any single response.     

The effect of reinforcement differences on choice and response distribution during stimulus compounding

In Experiments I and II, rats were trained to respond on one lever during light and another during tone. The absence of tone and light controlled response cessation. In the multiple schedule of Experiment I, all reinforcements were received for responding in tone or light; in the chain schedule of Experiment II, all reinforcements were received in no tone + no light for not responding. Experiment I subjects, for which tone and light were associated with response and reinforcement increase, responded significantly more to tone-plus-light than to tone or light alone (additive summation). Experiment II subjects, for which tone and light were associated with response increase and reinforcement decrease, responded comparably to tone, light, and tone + light. Thus, additive summation was observed when stimulus-response and stimulus-reinforcer associations in tone and light were both positive, but not when they were conflicting. All subjects in both experiments responded predominantly on the light-correlated lever during tone + light, even when light intensity was reduced in testing. Furthermore, when a light was presented to a subject engaged in tone-associated responding, all subjects immediately switched the locus of responding to the light-correlated lever. No change in locus occurred when a tone was presented to a subject engaged in light-associated responding, irrespective of the stimulus-reinforcer association conditioned to tone. The light-lever preference in tone + light indicates that the heightened responding observed in Experiment I was not the summation of tone-associated behavior with light-associated behavior. Rather, it appears to be the result of a facilitation of one operant (light-associated responding) by the reinforcement-associated cue for the other.     

The effect of signaled reinforcement on rats' fixed-interval responding

Four experiments examined the effect on rats' response rate of presenting a brief (500 ms) stimulus simultaneously with the delivery of food on fixed-interval (FI) schedules. In Experiment 1, reinforcement signals that were spatially diffuse (both tones and lights) elevated rates of responding, but responding was attenuated by localized visual stimuli. The remaining experiments examined the signal-induced potentiation of responding. In Experiment 2, a tone reinforcement signal potentiated response rates on an FI schedule, but attenuated response rates on a variable-interval (VI) schedule. This difference was obtained even though the overall rate of responding was equated on the two schedules before the introduction of the signal. Signal-induced potentiation of responding occurred over a range of FI values employed in Experiment 3. In Experiment 4, presenting a reinforcement signal when high local rates of response had occurred immediately before reinforcement resulted in potentiated rates of responding on an FI schedule. The opposite effect on response rate occurred when the reinforcement signal followed only low local rates of response. These results indicate that a variety of factors influence the effects of a reinforcement signal. They imply, however, that the local rate of response at the time of reinforcement is a key factor in establishing the nature of the signaling effect.     

Response-dependent shock in second-order fixed-ratio schedules of food presentation

Three pigeons key pecked under second-order schedules in which the completion of two successive fixed-ratio 50 components constituted a reinforcement cycle. Tandem, chained, and brief-stimulus second-order schedules were studied when completion of the initial fixed-ratio 50 component delivered brief intense electric shock in every nth reinforcement cycle and n assumed values between one and nine. During sessions without shock, the brief-stimulus (unpaired with food) schedule generated higher rates of responding in the initial component than did the tandem schedule. Electric shock engendered increased time to the fifth response and a repeated pause-run pattern of not responding and responding, particuuarly in the initial component, even with shock scheduled in every ninth reinforcement cycle. The results were consistent with those reported for shock of a shorter duration scheduled in every reinforcement cycle. The overall rate of responding decreased as a function of increasing shock density and was lower in brief-stimulus than in tandem schedules.     

Concurrent responses during multiple schedules in rats

Abstract.— Rats' lever pressing was reinforced with a sucrose solution. In the presence of a tone pressing one lever was reinforced on a FR 40 schedule, in the absence of the tone pressing another lever was reinforced on a FR 20 schedule. The components alternated every fourth minute. In both components, the duration of pauses in lever pressing after reinforcement was positively correlated with the duration of licking the empty sucrose-delivery mechanism. In the FR 40 component, the duration of pauses was also positively correlated with the number of presses on the non-reforced lever. Licking and non-reinforced lever pressing were decreased when the rate of reinforced lever pressing was high. The response interactions were similar to those described for concurrent reinforcement schedules.     

Constituents of response rates

Response rate and the proportion of time pigeons allocated to a key-pecking activity were measured on several basic types of reinforcement schedules. Reinforcement frequency was varied within each type of basic schedule, and the effects on two constituents of response rate were noted. Propensity, the proportion of time the birds spent on a platform in front of the key, showed very consistent effects as reinforcement frequency varied: in general, it decreased when reinforcement frequency markedly decreased and it increased when reinforcement frequency increased. Speed, key pecks per unit of time spent on the platform, showed inconsistent effects when reinforcement frequency varied. Consequently, response rate showed less consistent effects than did propensity. Cumulative response records demonstrated the existence of several different types of transitions or boundary states between the key-pecking activity and other activities. The types of transitions that occurred between activities depended on both the type of reinforcement schedule and the frequency of reinforcement. The propensity data support the position that general laws of behavior can be based on temporal measures of behavior. The speed data suggest that, if a complete assessment of the dynamic properties of behavior is to be achieved, measures of behavior must incorporate the structural variations in the operant unit.     

Differential responding without differential reinforcement: Intensity difference, continuum position, and reinforcement density effects

The response rates of five groups of rats were observed during exposure to different intensities of a four kilohertz tone within a two-component multiple schedule of nondifferential reinforcement. Response rates were found to be higher during the multiple schedule component which contained the higher intensity tone. Larger differences in response rates between the two multiple schedule components occurred with greater intensity separations (30 versus 20 decibels). At the 30 decibel separation a low absolute magnitude produced larger response rate differences than a high absolute magnitude, while at the 20 decibel separation a high absolute magnitude produced larger response rate differences. Increases in reinforcement density were accompanied by decreases in response rate differences between high and low intensity components only when over-all response rates also increased.     

The influence of partial reinforcement on serial autoshaping with pigeons

The effect of partial reinforcement on the rate of responding during the first element of a serial compound was investigated using autoshaping in pigeons. Experiment I employed the illumination of a response key by two different colours as the elements of the compound. Responding during the first element was faster when this stimulus was intermittently paired with the second element and the unconditioned stimulus than when a continuous reinforcement schedule was employed. Experiment II demonstrated that this effect of partial reinforcement is unaffected by maniuplating the associative strength of the second element at the outset of compound conditioning. A similar effect of partial reinforcement was also found in Experiment III which used a tone as the first element of the serial compound.     

Stimulus generalization and the response-reinforcement contingency

Generalization gradients along a line-tilt continuum were obtained from groups of pigeons that had been trained to peck a key on different schedules of reinforcement. In Exp. I, gradients following training on a differential-reinforcement-of-low-rate (DRL) schedule proved to be much flatter than gradients following the usual 1-min variable interval (VI) training. In Exp. II, the value of the VI schedule itself was parametrically studied; Ss trained on long VI schedules (e.g., 4-min) produced much flatter gradients than Ss trained on short VI schedules (30-sec; 1-min). The results were interpreted mainly in terms of the relative control exerted by internal, proprioceptive cues on the different reinforcement schedules. Several implications of the results for other problems in the field of stimulus generalization are discussed.