Effects on choice of reinforcement delay and conditioned reinforcement

Pigeons chose between fixed-interval schedules of different durations presented in the terminal links of concurrent-chains schedules. The pair of schedules was always in the ratio of 2:1, but the absolute duration of the fixed intervals varied. In one set of conditions, the different terminal-link schedules were associated with different keylight stimuli (cued conditions). In a second set of conditions, the different terminal-link schedules were associated with the same stimulus (uncued conditions). Results from the cued conditions replicated previous findings that preference for the shorter fixed-interval schedule increased with fixed-interval duration. Preferences in the uncued conditions were lower than in the corresponding cued conditions but also increased with fixed-interval length. In addition, the degree of control under the uncued conditions was correlated with the extent to which the schedule during the terminal link was discriminated immediately upon entry into the terminal link. The pattern of results in both conditions was inconsistent with the notion that choice behavior matches relative immediacy of reinforcement. Reanalysis of previous evidence for matching (Chung and Herrnstein, 1967) showed that matching in fact did not occur, as the preferences of their subjects for the shorter of two delays also increased with the absolute size of the delays.     

Second-order schedules with paired auditory brief stimuli

Pigeons were exposed to multiple second-order schedules of paired and unpaired brief stimuli in which responding on the main key was reinforced according to a fixed-interval thirty-second schedule by a brief stimulus (a tone in the paired schedule) and advancement to the next segment of the second-order schedule. In Experiment 1, a response on the second key was required during the tone in its fourth and final presentation to produce food. Responses during earlier brief stimuli indicated the extent to which the final brief stimulus was discriminated from preceding ones. Responding was comparable during all tones, extending prior findings with visual paired brief stimuli and weakening explanations of subjects' failure to discriminate between brief-stimulus presentations in terms of elicited responding. In Experiment 2 the number of fixed-interval segments comprising the second-order schedules varied from one through eight. Although main-key response rates increased across segments in both experiments, they increased much less sharply with a variable number of segments. These results suggest that the increase in main-key response rates across segments is due primarily to a degree of temporal discrimination not reflected on the second key. Main-key response rates were higher on paired auditory brief-stimulus schedules than on unpaired visual brief-stimulus schedules, especially in Experiment 2, thus further extending findings with visual brief stimuli to second-order schedules with auditory brief stimuli.     

Conditioned reinforcement as a function of duration of stimulus

Pigeons were provided with three keys. Pecking the center key produced grain on a schedule that alternated at unpredictable times between a variable-interval component and extinction. On concurrent variable-interval schedules, pecking either side key produced a stimulus associated with the variable-interval component on the center key provided that said schedule was currently in effect. The independent variable was the length of time this stimulus remained on the keys. Pecking one side key produced the stimulus for 27 seconds, whereas the duration produced by pecking the other key varied for successive blocks of sessions. For the first four birds, the values tested were 3, 9, 27, and 81 seconds. For the second group, numbering three birds, the values tested were 1, 3, 9, and 27 seconds. The dependent variable was the proportion of total side key pecks that occurred on the variable key. For all birds, the function was positive in slope and negative in acceleration. This finding supports a formulation that ascribes the maintenance of observing responses in a normal setting to the fact that the subject exposes itself to the positive discriminative stimulus for a longer mean duration than it does to the negative stimulus.     

Conditioned reinforcement value and choice.

The delay-reduction hypothesis of conditioned reinforcement states that the reinforcing value of a food-associated stimulus is determined by the delay to primary reinforcement signaled by the onset of the stimulus relative to the average delay to primary reinforcement in the conditioning situation. In contrast, most contemporary models of conditioned reinforcement strength posit that the reinforcing strength of a stimulus is some simple function only of the delay to primary reinforcement in the presence of stimulus. The delay-reduction hypothesis diverges from other conditioned reinforcement models in that it predicts that a fixed-duration food-paired stimulus will have different reinforcing values depending on the frequency of its presentation. In Experiment 1, pigeons' key pecks were reinforced according to concurrent-chains schedules with variable-interval 10-second and variable-interval 20-second terminal-link schedules. The initial-link schedule preceding the shorter terminal link was always variable-interval 60 seconds, and the initial-link schedule requirement preceding the longer terminal link was varied between 1 second and 60 seconds across conditions. In Experiment 2, the initial-link schedule preceding the longer of two terminal links was varied for each of three groups of pigeons. The terminal links of the concurrent chains for the three groups were variable-interval 10 seconds and 20 seconds, variable-interval 10 seconds and 30 seconds, and variable-interval 30 seconds and 50 seconds. In both experiments, preference for the shorter terminal link was either a bitonic function or an inverse function of the initial-link schedule preceding the longer terminal-link schedule. Consistent with the predictions of the delay-reduction hypothesis, the relative values of the terminal-link stimuli changed as a function of the overall frequency of primary reinforcement. Vaughan's (1985) melioration model, which was shown to be formally similar to Squires and Fantino's (1971) delay-reduction model, can be modified so as to predict these results without changing its underlying assumptions.     

Substitutability between conditioned and primary reinforcers in discrimination acquisition.

Rats and pigeons were trained on a series of reversals of a conditional simultaneous discrimination. The percentage of reinforcement for correct trials was varied across reversals. When nonreinforced correct trials produced the same feedback as incorrect trials, the number of errors to reach an acquisition criterion was greater for smaller percentages of reinforcement, but the number of reinforcers required was either approximately constant or smaller for the smaller percentages. When a stimulus paired with food (the conditioned reinforcer) was added on nonreinforced correct trials, both measures were substantially decreased. When the same stimulus was presented, but without a history of food pairing, learning rate was similar to when no stimulus was presented on nonreinforced trials. The results provide direct evidence that conditioned reinforcers may substitute, although imperfectly, for a primary reinforcer, and that pairing with the primary reinforcer is a necessary condition for such substitutability to occur.     

Choice and conditioned reinforcement.

A potential weakness of one formulation of delay-reduction theory is its failure to include a term for rate of conditioned reinforcement, that is, the rate at which the terminal-link stimuli occur in concurrent-chains schedules. The present studies assessed whether or not rate of conditioned reinforcement has an independent effect upon choice. Pigeons responded on either modified concurrent-chains schedules or on comparable concurrent-tandem schedules. The initial link was shortened on only one of two concurrent-chains schedules and on only one of two corresponding concurrent-tandem schedules. This manipulation increased rate of conditioned reinforcement sharply in the chain but not in the tandem schedule. According to a formulation of delay-reduction theory, when the outcomes chosen (the terminal links) are equal, as in Experiment 1, choice should depend only on rate of primary reinforcement; thus, choice should be equivalent for the tandem and chain schedules despite a large difference in rate of conditioned reinforcement. When the outcomes chosen are unequal, however, as in Experiment 2, choice should depend upon both rate of primary reinforcement and relative signaled delay reduction; thus, larger preferences should occur in the chain than in the tandem schedules. These predictions were confirmed, suggesting that increasing the rate of conditioned reinforcement on concurrent-chains schedules may have no independent effect on choice.     

Choice with probabilistic reinforcement: effects of delay and conditioned reinforcers.

Two experiments measured pigeons' choices between probabilistic reinforcers and certain but delayed reinforcers. In Experiment 1, a peck on a red key led to a 5-s delay and then a possible reinforcer (with a probability of .2). A peck on a green key led to a certain reinforcer after an adjusting delay. This delay was adjusted over trials so as to estimate an indifference point, or a duration at which the two alternatives were chosen about equally often. In all conditions, red houselights were present during the 5-s delay on reinforced trials with the probabilistic alternative, but the houselight colors on nonreinforced trials differed across conditions. Subjects showed a stronger preference for the probabilistic alternative when the houselights were a different color (white or blue) during the delay on nonreinforced trials than when they were red on both reinforced and nonreinforced trials. These results supported the hypothesis that the value or effectiveness of a probabilistic reinforcer is inversely related to the cumulative time per reinforcer spent in the presence of stimuli associated with the probabilistic alternative. Experiment 2 tested some quantitative versions of this hypothesis by varying the delay for the probabilistic alternative (either 0 s or 2 s) and the probability of reinforcement (from .1 to 1.0). The results were best described by an equation that took into account both the cumulative durations of stimuli associated with the probabilistic reinforcer and the variability in these durations from one reinforcer to the next.     

Initial excitatory conditioning with the truly random control procedure in rats: The effects of density of the conditioned stimulus

The present experiments were designed to test whether initial excitatory conditioning to a stimulus was obtained, using a within-subject design of the truly random control (TRC) procedure, and to examine the effects of the density of the conditioned stimulus on it. In Experiment 1, the effect of the three levels of CS density (50%, 25%, 20%) was assessed at six different stages of training, using the conditioned suppression of licking in rats. Although all of the groups given the TRC procedure showed evidence of initial excitatory conditioning, it was more marked in the group with the highest CS density than in the other groups. Experiment 2 confirmed the results of Experiment 1, using a between-subject design. The findings are in line with predictions from the Rescorla-Wagner model, and are discussed with reference to other contemporary theories of associative learning.     

电击信号应激对大鼠体液免疫及内分泌功能的影响

研究了足电击及以电击装置为信号刺激所诱发的情绪应激对大鼠原发性体液免疫反应及内分泌的影响。 结果表明每天１０分钟,共６天的足电击对大鼠抗特异性抗原ＯＶＡ的原发性体液免疫反应无明显作用,而此电击作 用结合每天１０分钟,共８天的情绪应激则可显著降低大鼠体液免疫反应及脾脏指数。同时该应激可显著提高大鼠 血儿茶酚胶和皮质酮水平。该研究证明了情绪应激对大鼠体液免疫功能的调节作用,并对交感神经系统和下丘脑－ 垂体－肾上腺（ＨＰＡ）轴在其中所起的作用进行了初步探讨。     

Roger T. Kelleher, behavior analyst

Roger T. Kelleher, rightly known as one of the foremost contributors to behavioral pharmacology, also made many important contributions to the experimental analysis of behavior. He participated significantly in the development of the discipline, through both his research and his editorial contributions to thisjournal. This article summarizes his contributions to the field of behavior analysis. His most significant empirical and conceptual contributions to behavior analysis came in two domains-conditioned reinforcement and the power of schedules of reinforcement. His accomplishments in these two domains still serve as principal foundations for modern research and theory.