Secondary reinforcement and number of primary reinforcements

Pigeons' pecks on either of two concurrently available response keys produced secondary reinforcers according to independent one-minute variable-interval schedules. Different secondary reinforcers, in the presence of which the rates of primary reinforcement were equal, were associated with each key. The rate of pecking maintained by each secondary reinforcer varied directly, but nonproportionally, with the number of primary reinforcements given in the presence of the secondary reinforcer.     

Choice and rate of reinforcement

Pigeons' responses in the presence of two concurrently available (initial-link) stimuli produced one of two different (terminal-link) stimuli. The rate of reinforcement in the presence of one terminal-link stimulus was three times that of the other. Three different pairs of identical but independent variable-interval schedules controlled entry into the terminal links. When the intermediate pair was in effect, the pigeons distributed their (choice) responses in the presence of the concurrently available stimuli of the initial links in the same proportion as reinforcements were distributed in the mutually exclusive terminal links. This finding was consistent with those of earlier studies. When either the pair of larger or smaller variable-interval schedules was in effect, however, proportions of choice responses did not match proportions of reinforcements. In addition, matching was not obtained when entry into the terminal links was controlled by unequal variable-interval schedules. A formulation consistent with extant data states that choice behavior is dependent upon the amount of reduction in the expected time to primary reinforcement, as signified by entry into one terminal link, relative to the amount of reduction in expected time to reinforcement signified by entry into the other terminal link.     

Response rate and reinforcement rate in Pavlovian conditioning

Four experiments used delay conditioning of magazine approach in rats to investigate the relationship between the rate of responding, R, to a conditioned stimulus (CS) and the rate, r, at which the CS is reinforced with the unconditioned stimulus (US). Rats were concurrently trained with four variable-duration CSs with different rs, either as a result of differences in the mean CS-US interval or in the proportion of CS presentations that ended with the US. In each case, R was systematically related to r, and the relationship was very accurately characterized by a hyperbolic function, R = Ar/(r +c). Accordingly, the reciprocal of these two variables-response interval, I (= 1/R), and CS-US interval, i (= 1/r) - were related by a simple affine (straight line) transformation, I = mi+b. This latter relationship shows that each increment in the time that the rats had to wait for food produced a linear increment in the time they waited between magazine entries. We discuss the close agreement between our findings and the Matching Law (Herrnstein, 1970) and consider their implications for both associative theories (e.g., Rescorla & Wagner, 1972) and nonassociative theories (Gallistel & Gibbon, 2000) of conditioning. (PsycINFO Database Record (c) 2011 APA, all rights reserved).     

Punishment and rate of positive reinforcement

This experiment investigated the effect of several punishment intensities on two responses maintained by contrasting rates of reinforcement. The responses were concurrently reinforced according to two different variable-interval schedules. Because these schedules were independent of one another and programmed different rates of reinforcement, the two responses occurred at dissimilar rates. When responses were simultaneously suppressed by punishment, both rates were reduced proportionately until suppression was virtually complete. In other words, the per cent suppression resulting from punishment was independent of the rate at which the response was reinforced. Phenomena found in single-response studies were duplicated here. Responding tended to increase both within and between punishment sessions at mild and moderate punishment intensities. Cessation of punishment led to a "compensatory" overshooting beyond the prepunished response rate.     

Reinforcement rate and force proportional reinforcement

The research examines the consequences of making two reinforcement contingencies simultaneously available for the same bar-pressing response. Specifically, two experiments are described which concern the effects of reinforcement rate upon force proportional reinforcement (FPR). In Experiment 1, reinforcement rate was constrained by combining differential reinforcement of low rate with a force-proportional reinforcement contingency. The group of rats exposed to this regimen eventually showed a significant increase in response force compared to control groups. Experimental vs control response rates were not significantly different. In Experiment 2, reinforcement rate was affected through the expedient of relating the probability of obtaining a single pellet to the response force. The group for which the probability-force relationship was discontinuous came to exert higher forces than either a group for which the relationship was fairly continuous or an ordinary variable ratio control group. Response rate was significantly higher for the control group.     

Response rate and sensitivity to the molar feedback function relating response and reinforcement rate on VI+ schedules of reinforcement

In 5 experiments, the author examined rats' sensitivity to the molar feedback function relating response rate to reinforcement rate on schedules of reinforcement. These studies demonstrated that, at lower rates of responding, rats' performance on variable ratio (VR), variable interval (VI), and variable interval with linear feedback loop (VI+) schedules was determined largely by reinforcement of interresponse times; response rates were faster on VR than on both VI and VI+ schedules. In contrast, when procedures were adopted to maintain high rates of response, rats showed sensitivity to the molar characteristics of the schedules; they responded as fast on a VI+ schedule as on a VR schedule and faster on both of these schedules than on a yoked VI schedule. When the variance of response rate was manipulated, this factor was noted as an important element in determining sensitivity to the molar characteristics of the schedule.     

The concurrent reinforcement of two interresponse times: absolute rate of reinforcement

Three pigeons obtained food on a one-key schedule of reinforcement for two concurrent, discriminated interresponse times. The overall rate of reinforcement was determined by a family of variable-interval schedules and by a continuous reinforcement schedule. The average frequency of reinforcement varied from 1.1 to 300 reinforcements per hour; the relative frequency of reinforcement for each of the two interresponse times was 0.5 throughout the experiment. The number of responses per minute increased sharply as the number of reinforcements per hour increased from 1 to 20. Beyond 30 reinforcements per hour, the curve was approximately flat, although it sometimes decreased slightly at the highest reinforcement rates. The relative frequency of the shorter interresponse time also increased sharply as the number of reinforcements per hour increased from 1 to 20. The asymptote of the relative frequency function approximately equalled the relative reciprocal of the length of the shorter interresponse time for reinforcement rates greater than 30 or 40 reinforcements per hour. This approximation was obscured by the response-rate function.     

Signalled reinforcement in differential-reinforcement-of-low rate schedules

At several fixed and variable minimum reinforced interresponse times, a stimulus was added to differential-reinforcement-of-low-rate schedules to signal the availability or nonavailability of reinforcement. As the minimum reinforced interresponse time increased, the rate of unreinforced responding decreased. Changing from fixed to variable minimum interresponse time in the basic differential-reinforcement-of-low-rate schedule further decreased the rate of unreinforced responding. Both effects were to some degree reversible. For fixed minimum reinforced interresponse times of 30 sec or shorter, most unreinforced responses terminated interresponse times just short of that required for reinforcement. The minimum reinforced interresponse time and the number of short response latencies (≤0.5 sec) to the onset of the signal were negatively correlated. Both of these analyses suggested that at values of 30 sec or shorter, the subjects discriminated the availability of the reinforcer more on the basis of time than on the basis of presence or absence of the signal.     

Relative frequency of reinforcement and rate of punished behavior

After training on a multiple variable-interval variable-interval schedule of reinforcement, each response in one component of the schedule was followed by a brief electric shock. When the rate of punished responses stabilized, the frequency of reinforcement in the other component was first decreased and then increased from the baseline frequency. The effects of these manipulations were consistent with reports of interactions in multiple schedules involving only unpunished behavior, i.e., the rate of punished responses increased with a higher relative frequency of reinforcement in the punishment component and decreased with a lower relative frequency of reinforcement in that component. The relevance of such findings to a further generality of behavioral contrast effects is discussed.     

Some effects of relative reinforcement rate and changeover delay in response-independent concurrent schedules of reinforcement

Reinforcements were arranged independently of the pigeon's behavior by concurrent variable-interval schedules. The reinforcements arranged by one of the schedules occurred when the chamber was illuminated with amber light, and the reinforcements arranged by the other schedule occurred when the chamber was illuminated with blue light. Both schedules functioned concurrently, but reinforcers were delivered by each only in the presence of the appropriate stimulus condition. A response on a white key, the only key in the chamber, alternated the stimulus condition and the effective schedule. The results of this procedure were similar to those obtained with concurrent response-dependent variable-interval schedules of reinforcement. The proportion of the total session time spent in the presence of a schedule component approximated the proportion of the total number of reinforcements in the component. Changeover rate was a decreasing function of the changeover delay and of the difference between the relative rates of reinforcement for each pair of concurrent schedules.     

Contiguity, reinforcement rate and the law of effect

There is an ambiguity in formulations of the Law of Effect which stress the importance of the correlation of rate of responding with frequency of reinforcement. The main problem is that such theories have not specified precisely how the correlation of response and reinforcement rates should be determined, with the result that the theories can become irrefutable. Two experiments were carried out which expose some of the problems created by this ambiguity. “Free” food reinforcers were delivered to rats in the absence of responding. Lever responses intermittently provided immediate (contiguous) reinforcement, but cancelled some of the following “free” reinforcements. Responding was established and maintained even when the overall rate of responding was negatively correlated with the overall frequency of reinforcement. Several ways in which correlational theories could attempt to accommodate these results are discussed but rejected as unsatisfactory, either because they severely limit the scope of the theories or because they lose their most important feature: that of treating behaviour at a molar rather than a molecular level.