Secondary reinforcement and rate of primary reinforcement

Four pigeons were trained to peck at either of two response-keys. Pecking at either key occasionally produced a secondary reinforcer. Then, in the presence of the secondary reinforcer, further pecking occasionally produced the primary reinforcer, food. The relative rate at which each pigeon pecked to obtain a secondary reinforcer equalled the relative rate of primary reinforcement in its presence.     

Response rate, reinforcement frequency, and conditioned suppression

In the first of two experiments, periods of noise were terminated with unavoidable shock to 36 rats. The rats' continuously reinforced responding was later completely suppressed during the noise when it was introduced without shock. The rats were then assigned to nine experimental groups. Each group was exposed to different paced variable-interval schedules of reinforcement, which independently controlled response rate and reinforcement frequency. Periods of the noise were periodically superimposed on these schedules, and loss of response suppression was studied. Differences between the groups were assessed statistically. The second experiment used a steady-state design. Six rats were exposed to paced schedules which generated two alternating response rates but gave constant reinforcement frequencies, and six rats to schedules which maintained the same response rates throughout, but in which the reinforcement frequency was alternately high and low. Response suppression was studied during a pre-shock stimulus superimposed on each rat's two behavioral baselines. Both experiments suggest that (1) conditioned suppression is affected by rate of operant responding, high rates being most suppressed, and (2) the frequency of reinforcements obtained also has an effect, most suppression occurring when frequency is low.     

Response rate, reinforcement frequency, and behavioral contrast

Pigeons responded for food on a multiple schedule in which periods of green-key illumination alternated with periods of red-key illumination. When behavior had stabilized with a variable-interval 2-min schedule of reinforcement operating during both stimuli, low rates of responding (interresponse times greater than 2 sec) were differentially reinforced during the green component. Conditions during the red stimulus were unchanged. Response rates during the green component fell without changing the frequency of reinforcement but there were no unequivocal contrast effects during the red stimulus. The frequency of reinforcement during the green component was then reduced by changing to a variable-interval 8-min schedule without reducing the response rates in that component, which were held at a low level by the spacing requirement. Again, the conditions during the red stimulus were unchanged but response rates during that stimulus increased. These results show that reductions in reinforcement frequency, independently of response rate, can produce interactions in multiple schedules.     

Choice, rate of reinforcement, and the changeover delay

Pigeons distribute their responses on concurrently available variable-interval schedules in the same proportion as reinforcements are distributed on the two schedules only when a changeover delay is used. The present study shows that this equality between proportions of responses and proportions of reinforcements (“matching”) is obtained when the value of the changeover delay is varied. When responses are partitioned into the set of rapid response bursts occurring during the delay interval and the set of responses occurring subsequently, the proportion of neither set of responses matches the proportion of reinforcements. Instead, each set deviates from matching but in opposite directions. Matching on the gross level results from the interaction of two patterns evident in the local response rates: (I) the lengthening of the changeover delay response burst is accompanied by a commensurate decrease in the number of changeovers; (2) the changeover delay response burst is longer than the scheduled delay duration. When delay responses are eliminated by introducing a blackout during the delay interval, response matching is eliminated; the pigeon, however, continues to match the proportion of time spent responding on a key to the proportion of reinforcements obtained on that key.     

Choice and rate of reinforcement

Pigeons' responses in the presence of two concurrently available (initial-link) stimuli produced one of two different (terminal-link) stimuli. The rate of reinforcement in the presence of one terminal-link stimulus was three times that of the other. Three different pairs of identical but independent variable-interval schedules controlled entry into the terminal links. When the intermediate pair was in effect, the pigeons distributed their (choice) responses in the presence of the concurrently available stimuli of the initial links in the same proportion as reinforcements were distributed in the mutually exclusive terminal links. This finding was consistent with those of earlier studies. When either the pair of larger or smaller variable-interval schedules was in effect, however, proportions of choice responses did not match proportions of reinforcements. In addition, matching was not obtained when entry into the terminal links was controlled by unequal variable-interval schedules. A formulation consistent with extant data states that choice behavior is dependent upon the amount of reduction in the expected time to primary reinforcement, as signified by entry into one terminal link, relative to the amount of reduction in expected time to reinforcement signified by entry into the other terminal link.     

Contiguity, reinforcement rate and the law of effect

There is an ambiguity in formulations of the Law of Effect which stress the importance of the correlation of rate of responding with frequency of reinforcement. The main problem is that such theories have not specified precisely how the correlation of response and reinforcement rates should be determined, with the result that the theories can become irrefutable. Two experiments were carried out which expose some of the problems created by this ambiguity. “Free” food reinforcers were delivered to rats in the absence of responding. Lever responses intermittently provided immediate (contiguous) reinforcement, but cancelled some of the following “free” reinforcements. Responding was established and maintained even when the overall rate of responding was negatively correlated with the overall frequency of reinforcement. Several ways in which correlational theories could attempt to accommodate these results are discussed but rejected as unsatisfactory, either because they severely limit the scope of the theories or because they lose their most important feature: that of treating behaviour at a molar rather than a molecular level.     

The concurrent reinforcement of two interresponse times: absolute rate of reinforcement

Three pigeons obtained food on a one-key schedule of reinforcement for two concurrent, discriminated interresponse times. The overall rate of reinforcement was determined by a family of variable-interval schedules and by a continuous reinforcement schedule. The average frequency of reinforcement varied from 1.1 to 300 reinforcements per hour; the relative frequency of reinforcement for each of the two interresponse times was 0.5 throughout the experiment. The number of responses per minute increased sharply as the number of reinforcements per hour increased from 1 to 20. Beyond 30 reinforcements per hour, the curve was approximately flat, although it sometimes decreased slightly at the highest reinforcement rates. The relative frequency of the shorter interresponse time also increased sharply as the number of reinforcements per hour increased from 1 to 20. The asymptote of the relative frequency function approximately equalled the relative reciprocal of the length of the shorter interresponse time for reinforcement rates greater than 30 or 40 reinforcements per hour. This approximation was obscured by the response-rate function.     

Punishment and rate of positive reinforcement

This experiment investigated the effect of several punishment intensities on two responses maintained by contrasting rates of reinforcement. The responses were concurrently reinforced according to two different variable-interval schedules. Because these schedules were independent of one another and programmed different rates of reinforcement, the two responses occurred at dissimilar rates. When responses were simultaneously suppressed by punishment, both rates were reduced proportionately until suppression was virtually complete. In other words, the per cent suppression resulting from punishment was independent of the rate at which the response was reinforced. Phenomena found in single-response studies were duplicated here. Responding tended to increase both within and between punishment sessions at mild and moderate punishment intensities. Cessation of punishment led to a "compensatory" overshooting beyond the prepunished response rate.     

Modulation of the conflict monitoring intensity: The role of aversive reinforcement,cognitive demand,and trait-BIS

According to Botvinick’s (2007) integrative account, conflict monitoring is aversive because individuals anticipate cognitive demand, whereas the revised reinforcement sensitivity theory (rRST) predicts that conflict processing is aversive because individuals anticipate aversive reinforcement of erroneous responses. Because these accounts give different reasons for the aversive aspects of conflict, we manipulated cognitive demand and the aversive reinforcement as a consequence of wrong choices in a go/no-go task. Thereby, we also aimed to investigate whether individual differences in conflict sensitivity (i.e., in trait anxiety, linked to high sensitivity of the behavioral inhibition system [trait-BIS]) represent the effects of aversive reinforcement and cognitive demand in conflict tasks. We expected that these manipulations would have effects on the frontal N2 component representing activity of the anterior cingulate cortex. Moreover, higher-trait-BIS individuals should be more sensitive than lower-trait-BIS individuals to aversive effects in conflict situations, resulting in a more negative frontal N2 for higher-trait-BIS individuals. In Study 1, with N = 104 students, and Study 2, with N = 47 students, aversive reinforcement was manipulated in three levels (within-subjects factor) and cognitive demand in two levels (between-subjects factor). The behavioral findings from the go/no-go task with noncounterbalanced reinforcement levels (Study 1) could be widely replicated in a task with counterbalanced reinforcement levels (Study 2). The frontal mean no-go N2 amplitude and the frontal no-go N2 dipole captured predicted reinforcement-related variations of conflict monitoring, indicating that the anticipation of aversive reinforcement induces variations in conflict monitoring intensity in frontal brain areas. The aversive nature of conflict was underlined by the more pronounced conflict monitoring in higher- than in lower-trait-BIS individuals.     

Temporal inhibition: effects of changes in rate of reinforcement and rate of responding

Pigeons were trained to key peck on several multiple schedules in which the first of two components was always a simple fixed-interval schedule. The rate of responding at the beginning of the constant fixed-interval schedule was found to decrease with increases in the rate of reinforcement associated with the other component of the multiple schedule, but remained unchanged with decreases in the rate of responding associated with the other component. These results were interpreted as being consistent with the view that the presence and magnitude of the temporal inhibitory effects observed in a given fixed-interval schedule are a function of the properties of reinforcing stimuli, rather than of changes in the rate of responding associated with the time interval immediately preceding the fixed interval in question.     

Bouts of responding: the relation between bout rate and the rate of variable-interval reinforcement

By nose poking a lighted key, rats obtained food pellets on either a variable-interval schedule of reinforcement or a schedule that required an average of four additional responses after the end of tile variable-interval component (a tandem variable-interval variable-ratio 4 schedule). With both schedule types, the mean variable interval was varied between blocks of sessions from 16 min to 0.25 min. Total rate of key poking increased similarly as a function of the reinforcer rate for the two schedule types, but response rate was higher with than without the four-response requirement. Analysis of log survivor plots of interresponse times showed that key poking occurred in bouts. The rate of initiating bouts increased as a function of reinforcer rate but was either unaffected or was decreased by adding the four-response requirement. Within-bout response rate was insensitive to reinforcer rate and only inconsistently affected by the four-response requirement. For both kinds of schedule, the ratio of bout time to between-bout pause time was approximately a power function of reinforcer rate, with exponents above and below 1.0.     

Observing behavior: effects of rate and magnitude of primary reinforcement

Four experiments examined the free-operant observing behavior of rats. In Experiment 1, observing was a bitonic function of random-ratio schedule requirements for the primary reinforcer. In Experiment 2, decreases in the magnitude of the primary reinforcer decreased observing. Experiment 3 examined observing when a random-ratio schedule or a yoked random-time schedule of primary reinforcement was in effect across conditions. Removing the response requirement for the primary reinforcer increased observing, suggesting that the effects of the random-ratio schedule in Experiment 1 likely were due to an interaction between observing and responding for the primary reinforcer. In Experiment 4, decreasing the rate of primary reinforcement by increasing the duration of a random-time schedule decreased observing monotonically. Overall, these results suggest that observing decreases with decreases in the rate or magnitude of the primary reinforcer, but that behavior related to the primary reinforcer can affect observing and potentially affect measurement of conditioned reinforcing value.