Matching and conditioned reinforcement rate

Attempts to examine the effects of variations in relative conditioned reinforcement rate on choice have been confounded by changes in rates of primary reinforcement or changes in the value of the conditioned reinforcer. To avoid these problems, this experiment used concurrent observing responses to examine sensitivity of choice to relative conditioned reinforcement rate. In the absence of observing responses, unsignaled periods of food delivery on a variable-interval 90-s schedule alternated with extinction on a center key (i.e., a mixed schedule was in effect). Two concurrently available observing responses produced 15-s access to a stimulus differentially associated with the schedule of food delivery (S+). The relative rate of S+ deliveries arranged by independent variable-interval schedules for the two observing responses varied across conditions. The relation between the ratio of observing responses and the ratio of S+ deliveries was well described by the generalized matching law, despite the absence of changes in the rate of food delivery. In addition, the value of the S+ deliveries likely remained constant across conditions because the ratio of S+ to mixed schedule food deliveries remained constant. Assuming that S+ deliveries serve as conditioned reinforcers, these findings are consistent with the functional similarity between primary and conditioned reinforcers suggested by general choice theories based on the concatenated matching law (e.g., contextual choice and hyperbolic value-added models). These findings are inconsistent with delay reduction theory, which has no terms for the effects of rate of conditioned reinforcement in the absence of changes in rate of primary reinforcement.     

Rate of conditioned reinforcement affects observing rate but not resistance to change

The effects of rate of conditioned reinforcement on the resistance to change of operant behavior have not been examined. In addition, the effects of rate of conditioned reinforcement on the rate of observing have not been adequately examined. In two experiments, a multiple schedule of observing-response procedures was used to examine the effects of rate of conditioned reinforcement on observing rates and resistance to change. In a rich component, observing responses produced a higher frequency of stimuli correlated with alternating periods of random-interval schedule primary reinforcement or extinction. In a lean component, observing responses produced similar schedule-correlated stimuli but at a lower frequency. The rate of primary reinforcement in both components was the same. In Experiment 1, a 4:1 ratio of stimulus production was arranged by the rich and lean components. In Experiment 2, the ratio of stimulus production rates was increased to 6:1. In both experiments, observing rates were higher in the rich component than in the lean component. Disruptions in observing produced by presession feeding, extinction of observing responses, and response-independent food deliveries during intercomponent intervals usually were similar in the rich and lean components. When differences in resistance to change did occur, observing tended to be more resistant to change in the lean component. If resistance to change is accepted as a more appropriate measure of response strength than absolute response rates, then the present results provide no evidence that higher rates of stimuli generally considered to function as conditioned reinforcers engender greater response strength.     

Effect of intermittent reinforcement on acquisition and retention in delayed matching-to-sample in pigeons

Experiments 1 and 2 involved independent groups that received primary reinforcement after a correct match with a probability of 1.0, .50 or .25. Correct matches that did not produce primary reinforcement produced a conditioned reinforcer. Both experiments revealed little evidence that acquisition or retention was adversely affected by use of intermittent reinforcement. Experiment 3 involved a group that received 100% reinforcement and two others that received 25% reinforcement, one of which received conditioned reinforcement and the other did not. Following acquisition and retention testing, birds in group 100% and group 25% with conditioned reinforcement were exposed to 25% reinforcement and no conditioned reinforcement. Results revealed that conditioned reinforcement was important in promoting acquisition but was irrelevant in maintaining performance. It was concluded that intermittent reinforcement, especially when combined with conditioned reinforcement during acquisition, supports levels of acquisition and retention comparable to that of continuous reinforcement. Theoretically, the findings are consistent with an extension of Blough's instance-based theory of discrimination performance and, practically, they suggest that use of intermittent reinforcement could result in increased efficiency and economy in labs using delayed matching.     

Classical skin conductance response conditioning: effects of intermittent reinforcement and information about schedule contingencies.

Skin conductance responses were differentially conditioned using reinforcement schedules of 25%, 50%, 75%, and 100%, manipulated between subjects. Half of the subjects were informed about schedule contingencies, and half were uninformed. The interstimulus interval was 6 sec. Discrimination of first-interval responses (1.0-3.5 sec after conditioned stimulus [CS] onset) by informed subjects did not vary with the ratio variable, but that by uninformed subjects improved with increasing reinforcement ratio because of diminished response levels to the nonreinforced CS (CS-). Discrimination of second-interval responses (3.6-7.0 sec after CS onset) improved as a function of increasing reinforcement ratio because of elevated response levels to the reinforced CS (CS+), but the effect was not persistent across trials in informed subjects. Performance in the first and second intervals did not reflect sequential increments and decrements as a function of reinforced and nonreinforced trials. Third-interval responses (7.1-9.9 sec after CS on nonreinforced trials) were not affected by schedule manipulations, but unconditioned responses diminished with increasing reinforcement ratio. Information about schedule contingencies led to superior discrimination of first-, second-, and third-interval responses and to suppression of unconditioned responses.     

Operant conditioning in redwinged blackbirds

An operant conditioning technique for use with passerine birds is described. Two redwinged blackbirds were successfully conditioned to perch-hop for food reinforcement. Continuous reinforcement and fixed-ratio schedules involving substantial ratio requirements were used to maintain this response. The behavior of the two redwinged blackbirds was comparable to that of more conventional organisms working on similar schedules of reinforcement.     

Some notes on conditioned suppression and reinforcement schedules

Two pigeons were trained on an FR 150 schedule of reinforcement. An Estes-Skinner conditioned suppression procedure was then superimposed on this performance at varied intervals. If the CS occurred during the early stages of the ratio run, complete suppression resulted. If the CS occurred during the later stages of the run, the birds continued to respond until the reinforcement was obtained, which was then followed by complete suppression.     

Using progressive ratio schedules to evaluate tokens as generalized conditioned reinforcers

The properties of operant reinforcers are dynamic and dependent on a number of variables, such as schedule and effort. There has been sparse research on the generalized conditioned properties of token reinforcement. We evaluated leisure items, edible items, and tokens using a progressive ratio schedule with three children with diagnoses of ASD and developmental delays. The highest break points occurred during the token reinforcement condition for two out of three participants, but response rates tended to be higher with edibles. We then evaluated the effects of presession access to edibles on the break points of edible items and tokens with two participants. Break points decreased only in the edible reinforcement condition, and the participants chose to work for leisure items rather than edibles when presession access to edibles was in place. These findings suggest that the tokens functioned as generalized conditioned reinforcers.     

The effect of conditioned reinforcement rate on choice: a review

We review the nature of conditioned reinforcement, including evidence that conditioned reinforcers maintain choice behavior in concurrent schedules and that they elevate responding in the terminal links of concurrent‐chains schedules. A question has resurfaced recently: Do theories of choice in concurrent‐chains schedules need to include a term reflecting greater preference for higher rates of conditioned reinforcement? The review of several studies addressing this point suggests that such a term is inappropriate. Elevated rates of conditioned reinforcement (and responding) in the terminal links of concurrent‐chains schedules do not lead to greater preference in the initial link leading to the higher rate of conditioned reinforcement. If anything, the opposite preference is likely to occur. This result is not surprising, since the additional putative conditioned reinforcers in the terminal link are not correlated with a reduction in time to primary reinforcement nor with an increase in value.     

Effects of conditioned reinforcement frequency in an intermittent free-feeding situation,

Key-pecking intermittently produced a set of brief exteroceptive stimulus changes under two-component multiple schedules of conditioned reinforcement. Throughout the study, free access to grain was concurrently provided on an intermittent basis via a variable-interval tape. Free food presentations scheduled by the tape were delivered if no peck had been emitted for 6 sec, and the brief stimulus changes produced by responding under the multiple schedules were those which accompanied food presentation. The second component of each multiple schedule was always associated with a 1-min, variable-interval schedule of conditioned reinforcement. The schedule associated with the first component was systematically varied and conditioned reinforcement was either absent (extinction) or programmed on a 1-, 3-, 6-, or 12-min variable-interval schedule. Under these conditions, rate of responding in the manipulated component decreased monotonically with a decrease in the frequency of conditioned reinforcement. In addition, contrast effects were often obtained in the constant, second component. These results are similar to those obtained with similar multiple schedules of primary reinforcement.     

Differential effects of omission contingencies on various components of Pavlovian appetitive conditioned responding in rats

Five experiments with rat subjects investigated the effects of omission and partial reinforcement contingencies on five individual behaviors evoked by visual and auditory conditioned stimuli paired with a food unconditioned stimulus. The effects of omission depended on the behavior on which that contingency was placed: One behavior was eliminated, one was unaffected, and three were reduced relative to the performance of yoked controls. Partial reinforcement resulted in lower frequencies of three behaviors and higher frequencies of two behaviors, compared with performance under consistent reinforcement. A partial reinforcement extinction effect was noted with one behavior but not with the others. These results are related to the possible role of instrumental conditioning contingencies in generating conditioned behavior in this appetitive conditioning preparation and to the independence of individual components of a complex conditioned response.     

Conditioned suppression of an avoidance response by a stimulus paired with food

Three food-deprived Long-Evans rats were exposed to a non-discriminated shock avoidance procedure. Superimposed upon this operant avoidance baseline were periodic presentations of a conditioned stimulus that was paired with food, the unconditioned stimulus. These pairings resulted in increases in the rate of shock over that recorded when the conditioned stimulus was not present. A traditional suppression ratio failed to reveal any differential effect of the conditioned stimulus on the overall rate of avoidance responding, although all subjects showed a consistent pattern of pausing and postshock response bursts during presentations of the conditioned stimulus. When food was withheld during a final extinction phase, the conditioned stimulus ceased to occasion increases in shock rates and disruptive postshock response bursts were eliminated. An analysis of conditioned suppression procedures is proposed that stresses not only operant-Pavlovian or appetitive-aversive incompatibility, but also the manner in which the baseline schedule of reinforcement affects operant behavior changes that are elicited by the superimposed Pavlovian procedure.     

Conditioned reinforcement value and resistance to change

Three experiments examined the effects of conditioned reinforcement value and primary reinforcement rate on resistance to change using a multiple schedule of observing-response procedures with pigeons. In the absence of observing responses in both components, unsignaled periods of variable-interval (VI) schedule food reinforcement alternated with extinction. Observing responses in both components intermittently produced 15 s of a stimulus associated with the VI schedule (i.e., S+). In the first experiment, a lower-valued conditioned reinforcer and a higher rate of primary reinforcement were arranged in one component by adding response-independent food deliveries uncorrelated with S+. In the second experiment, one component arranged a lower valued conditioned reinforcer but a higher rate of primary reinforcement by increasing the probability of VI schedule periods relative to extinction periods. In the third experiment, the two observing-response components provided similar rates of primary reinforcement but arranged different valued conditioned reinforcers. Across the three experiments, observing-response rates were typically higher in the component associated with the higher valued conditioned reinforcer. Resistance to change was not affected by conditioned reinforcement value, but was an orderly function of the rate of primary reinforcement obtained in the two components. One interpretation of these results is that S+ value does not affect response strength and that S+ deliveries increase response rates through a mechanism other than reinforcement. Alternatively, because resistance to change depends on the discriminative stimulus-reinforcer relation, the failure of S+ value to impact resistance to change could have resulted from a lack of transfer of S+ value to the broader discriminative context.     

A conditioned reinforcer maintained by temporal association with the termination of shock

Two experiments were conducted to determine whether a stimulus can be established as a positive conditioned reinforcer by associating it with the termination of shock, but without training the animal to make any response in its presence. In the first, six rats were conditioned to press a bar to terminate shock on a variable ratio schedule; white noise was then substituted as the immediate consequence, with the shock terminating 30 sec after the last press in its presence. It was found that the rate of pressing in the absence of noise depended on the contingency between the pressing and the noise. The second experiment sought to determine whether the difference in rates before and after the onset of the noise was due to the reinforcement of prior responding by the onset of the noise or to the suppression of subsequent responding by differential reinforcement of competing behavior. Six more rats were trained in the same manner, but with shock terminating 30 sec after the onset of the noise, regardless of what the animal did in its presence. Again the rate was higher before the onset of the noise, indicating that pressing was indeed maintained by the noise as a conditioned reinforcer.     

Contiguity and conditioned reinforcement in probabilistic choice

In a baseline condition, pigeons chose between an alternative that always provided food following a 30-s delay (100% reinforcement) and an alternative that provided food half of the time and blackout half of the time following 30-s delays (50% reinforcement). The different outcomes were signaled by different-colored keylights. On average, each alternative was chosen approximately equally often, replicating the finding of suboptimal choice in probabilistic reinforcement procedures. The efficacy of the delay stimuli (keylights) as conditioned reinforcers was assessed in other conditions by interposing a 5-s gap (keylights darkened) between the choice response and one or more of the delay stimuli. The strength of conditioned reinforcement was measured by the decrease in choice of an alternative when the alternative contained a gap. Preference for the 50% alternative decreased in conditions in which the gap preceded either all delay stimuli, both delay stimuli for the 50% alternative, or the food stimulus for the 50% alternative, but preference was not consistently affected in conditions in which the gap preceded only the 100% delay stimulus or the blackout stimulus for the 50% alternative. These results support the notion that conditioned reinforcement underlies the finding of suboptimal preference in probabilistic reinforcement procedures, and that the signal for food on the 50% reinforcement alternative functions as a stronger conditioned reinforcer than the signal for food on the 100% reinforcement alternative. In addition, the results fail to provide evidence that the signal for blackout functions as a conditioned punisher.     

Conditioned suppression and variable ratio reinforcement

Conditioned suppression is a decrease in response rate during a relatively short duration stimulus that terminates independently of the animal's behavior and coincidentally with a brief unavoidable shock. The degree of conditioned suppression was measured for each of three birds on three variable ratio schedules; that is, the number of responses required for food reinforcement was varied around a mean of 50, 100, or 200. The results indicated a slight and possibly negligible decrease in the degree of suppression as the mean number of responses required on the schedule was increased from 50, to 100, and 200. In general, it was found that all of the variable ratio schedules tested were quite insensitive to the conditioned suppression procedure, although almost complete suppression was obtained on a few occasions. Since the reinforcement was contingent upon the emission of responses, the birds typically displayed a high rate of response during the pre-shock stimulus on all schedules. In addition, the rate during the pre-shock stimulus often changed abruptly independent of the presentation of a reinforcement. As a result of the high rate of response and the abrupt changes in rate, the degree of suppression from trial to trial was quite variable. A clear analysis of an experimental variable on this baseline is thus difficult.     

Choice and conditioned reinforcement.

A potential weakness of one formulation of delay-reduction theory is its failure to include a term for rate of conditioned reinforcement, that is, the rate at which the terminal-link stimuli occur in concurrent-chains schedules. The present studies assessed whether or not rate of conditioned reinforcement has an independent effect upon choice. Pigeons responded on either modified concurrent-chains schedules or on comparable concurrent-tandem schedules. The initial link was shortened on only one of two concurrent-chains schedules and on only one of two corresponding concurrent-tandem schedules. This manipulation increased rate of conditioned reinforcement sharply in the chain but not in the tandem schedule. According to a formulation of delay-reduction theory, when the outcomes chosen (the terminal links) are equal, as in Experiment 1, choice should depend only on rate of primary reinforcement; thus, choice should be equivalent for the tandem and chain schedules despite a large difference in rate of conditioned reinforcement. When the outcomes chosen are unequal, however, as in Experiment 2, choice should depend upon both rate of primary reinforcement and relative signaled delay reduction; thus, larger preferences should occur in the chain than in the tandem schedules. These predictions were confirmed, suggesting that increasing the rate of conditioned reinforcement on concurrent-chains schedules may have no independent effect on choice.     

Naloxone decreases responding for conditioned reinforcement in rats

That opioids can mediate unconditioned reinforcement is well established, but there is little evidence indicating whether they modify conditioned reinforcement. Here, a tone which initially served as a discriminative stimulus for the availability of water reinforcement was established as a conditioned stimulus. When later given a choice between pressing a lever producing the tone (but not water) or one which produced no effect, rats chose the tone-producing lever 66% of the time. Naloxone (3.0 mg/kg) reduced overall responding and completely eliminated the preference for the tone-producing lever. Results are briefly discussed in terms of the importance of understanding mechanisms serving conditioned reinforcement.     

Conditioned reinforcement: Experimental and theoretical issues

The concept of conditioned reinforcement has received decreased attention in learning textbooks over the past decade, in part because of criticisms of its validity by major behavior theorists and in part because its explanatory function in a variety of different conditioning procedures has become uncertain. Critical data from the major procedures that have been used to investigate the concept (second-order schedules, chain schedules, concurrent chains, observing responses, delay-of-reinforcement procedures) are reviewed, along with the major issues of interpretation. Although the role played by conditioned reinforcement in some procedures remains unresolved, the results taken together leave little doubt that the underlying idea of conditioned value is a critical component of behavior theory that is necessary to explain many different types of data. Other processes (marking, bridging) may also operate to produce effects similar to those of conditioned reinforcement, but these clearly cannot explain the full domain of experimental effects ascribed to conditioned reinforcement and should be regarded as complements to the concept rather than theoretical competitors. Examples of practical and theoretical applications of the concept of conditioned reinforcement are also considered.     

Stimulus functions in some chained fixed-ratio schedules of reinforcement

For two pigeons, behavior was compared in equivalent two-link chained and tandem schedules in which 100 responses were required for food reinforcement but the responses required in the two links of the chained schedule were varied over the values 2-98, 50-50, and 98-2. Initial pausing, response rate in each link, and the total time to complete each ratio were recorded. In chained components, initial-link pausing (for both birds) and total time to complete the ratio (for one bird) were generally shorter when the response requirement was 2-98 than when it was 50-50 or 98-2; for terminal-link pausing this relationship was reversed for both birds. There were also systematic changes in behavior in tandem components, and in the relationship between behavior in chained and tandem components. The results are discussed in terms of conditioned reinforcement.     

The effects of token reinforcement on attending in a young child with autism

The benefits of token economies have been widely established, however there are very few empirical studies on the effects of token reinforcement on the behavior of young children with autism. The establishment of conditioned reinforcers such as tokens may be important in interventions for children with autism. Token reinforcement was used to increase the attending behavior of a young child with autism during discrete trials instruction for academic and communication skills. A reversal design showed that token reinforcement sustained attending only when the back‐up reinforcer was available and was accessed immediately. The results extend and corroborate findings on token reinforcement reported in other applied settings and are also consistent with findings from basic research on conditioned reinforcement. Copyright © 2006 John Wiley & Sons, Ltd.