Stimulus Equivalence: Effects Of A Default-response Option On Emergence Of Untrained Stimulus Relations

Human subjects were exposed to a concurrent-chains schedule in which reinforcer amounts, delays, or both were varied in the terminal links, and consummatory responses were required to receive points that were later exchangeable for money. Two independent variable-interval 30-s schedules were in effect during the initial links, and delay periods were defined by fixed-time schedules. In Experiment 1, subjects were exposed to three different pairs of reinforcer amounts and delays, and sensitivity to reinforcer amount and delay was determined based on the generalized matching law. The relative responding (choice) of most subjects was more sensitive to reinforcer amount than to reinforcer delay. In Experiment 2, subjects chose between immediate smaller reinforcers and delayed larger reinforcers in five conditions with and without timeout periods that followed a shorter delay, in which reinforcer amounts and delays were combined to make different predictions based on local reinforcement density (i.e., points per delay) or overall reinforcement density (i.e., points per total time). In most conditions, subjects' choices were qualitatively in accord with the predictions from the overall reinforcement density calculated by the ratio of reinforcer amount and total time. Therefore, the overall reinforcement density appears to influence the preference of humans in the present self-control choice situation.     

Choice in a self-control paradigm: Quantification of experience-based differences

Previous quantitative models of choice in a self-control paradigm (choice between a larger, more-delayed reinforcer and a smaller, less-delayed reinforcer) have not described individual differences. Two experiments are reported that provide additional quantitative data on experience-based differences in choice between reinforcers of varying sizes and delays. In Experiment 1, seven pigeons in a self-control paradigm were exposed to a fading procedure that increased choices of the larger, more-delayed reinforcer through gradually decreasing the delay to the smaller of two equally delayed reinforcers. Three control subjects, exposed to each of the small-reinforcer delays to which the experimental subjects were exposed, but for fewer sessions, demonstrated that lengthy exposure to each of the conditions in the fading procedure may be necessary in order for the increase to occur. In Experiment 2, pigeons with and without fading-procedure exposure chose between reinforcers of varying sizes and delays scheduled according to a concurrent variable-interval variable-interval schedule. In both experiments, pigeons with fading-procedure exposure were more sensitive to variations in reinforcer size than reinforcer delay when compared with pigeons without this exposure. The data were described by the generalized matching law when the relative size of its exponents, representing subjects' relative sensitivity to reinforcer size and delay, were grouped according to subjects' experience.     

Self-Control and Impulsiveness in Adult Human Females: Effects of Food Preferences

Two experiments explored the effects of food preferences on humans’ choice in a self-control paradigm. Reinforcers consisted of programmed periods of access time to drinking juice during the experimental session. Experiment 1 used 14 adult women and Experiment 2 used 9 adult women. In Experiment 1, the women demonstrated significantly less sensitivity to reinforcer amount relative to sensitivity to reinforcer delay (a measure of self-control) when they reported a higher preference for the juice received as the less delayed, smaller reinforcer than for the juice received as the more delayed, larger reinforcer. Conversely in Experiment 2, the women demonstrated significantly more sensitivity to reinforcer amount relative to sensitivity to reinforcer delay when they reported a higher preference for the juice received as the more delayed, larger reinforcer than for the juice received as the less delayed, smaller reinforcer. Together, the results show that participants’ food preferences can influence self-control for food reinforcers.     

Self-control in adult humans: variation in positive reinforcer amount and delay.

In five experiments, choice responding of female human adults was examined, as a function of variations in reinforcer amount and reinforcer delay. Experiment 1 used a discrete-trials procedure, and Experiments 2, 3, 4, and 5 used a concurrent variable-interval variable-interval schedule. Reinforcer amount and reinforcer delay were varied both separately and together. In contrast to results previously reported with pigeons, the subjects in the present experiments usually chose the larger reinforcers even when those reinforcers were delayed. Together, the results from all the experiments suggest that the subjects followed a maximization strategy in choosing reinforcers. Such behavior makes it easy to observe self-control and difficult to observe impulsiveness in traditional laboratory experiments that use adult human subjects.     

Self-control and choice in humans: Effects of video game playing as a positive reinforcer

Participants chose between reinforcement schedules differing in delay of reinforcement (interval between a choice response and onset of a video game) and/or amount of reinforcement (duration of access to the game). Experiment 1 showed that immediate reinforcement was preferred to delayed reinforcement with amount of reinforcement held constant, and a large reinforcer was preferred to a small reinforcer when both were obtainable immediately. Imposing a delay before the large reinforcer produced a preference for the immediate, small reinforcer in 40% of participants. This suggested a limited degree of “impulsivity.” In Experiment 2, unequal delays were extended by equal intervals, the amounts being kept equal. Preference for the shorter delay decreased, an effect that presumably makes possible the “preference reversal” phenomenon in studies of self-control. Overall, the results demonstrate that video game playing can produce useful, systematic data when used as a positive reinforcer for choice behavior in humans.     

Effects of reinforcement rate and reinforcer magnitude on choice behavior of humans

Two experiments with human subjects investigated the effects of rate of reinforcement and reinforcer magnitude upon choice. In Experiment 1, each of five subjects responded on four concurrent variable-interval schedules. In contrast to previous studies using non-human organisms, relative response rate did not closely match relative rate of reinforcement. Discrepancies ranged from 0.03 to 0.43 (mean equal to 0.19). Similar discrepancies were found between relative amount of time spent responding on each schedule and the corresponding relative rates of reinforcement. In Experiment 2, in which reinforcer magnitude was varied for each of five subjects, similar discrepancies ranging from 0.05 to 0.50 (mean equal to 0.21), were found between relative response rate and relative proportion of reinforcers received. In both experiments, changeover rates were lower on the long-interval concurrent schedules than on the short-interval ones. The results suggest that simple application of previous generalizations regarding the effects of reinforcement rate and reinforcer magnitude on choice for variable-interval schedules does not accurately describe human behavior in a simple laboratory situation.     

Effects of reinforcer consumption and magnitude on response rates during noncontingent reinforcement

Results of previous research on the effects of noncontingent reinforcement (NCR) have been inconsistent when magnitude of reinforcement was manipulated. We attempted to clarify the influence of NCR magnitude by including additional controls. In Study 1, we examined the effects of reinforcer consumption time by comparing the same magnitude of NCR when session time was and was not corrected to account for reinforcer consumption. Lower response rates were observed when session time was not corrected, indicating that reinforcer consumption can suppress response rates. In Study 2, we first selected varying reinforcer magnitudes (small, medium, and large) on the basis of corrected response rates observed during a contingent reinforcement condition and then compared the effects of these magnitudes during NCR. One participant exhibited lower response rates when large-magnitude reinforcers were delivered; the other ceased responding altogether even when small-magnitude reinforcers were delivered. We also compared the effects of the same NCR magnitude (medium) during 10-min and 30-min sessions. Lower response rates were observed during 30-min sessions, indicating that the number of reinforcers consumed across a session can have the same effect as the number consumed per reinforcer delivery. These findings indicate that, even when response rate is corrected to account for reinforcer consumption, larger magnitudes of NCR (defined on either a per-delivery or per-session basis) result in lower response rates than do smaller magnitudes.     

Local preference in concurrent schedules: the effects of reinforcer sequences

We investigated the effects that sequences of reinforcers obtained from the same response key have on local preference in concurrent variable-interval schedules with pigeons as subjects. With an overall reinforcer rate of one every 27 s, on average, reinforcers were scheduled dependently, and the probability that a reinforcer would be arranged on the same alternative as the previous reinforcer was manipulated. Throughout the experiment, the overall reinforcer ratio was 1:1, but across conditions we varied the average lengths of same-key reinforcer sequences by varying this conditional probability from 0 to 1. Thus, in some conditions, reinforcer locations changed frequently, whereas in others there tended to be very long sequences of same-key reinforcers. Although there was a general tendency to stay at the just-reinforced alternative, this tendency was considerably decreased in conditions where same-key reinforcer sequences were short. Some effects of reinforcers are at least partly to be accounted for by their signaling subsequent reinforcer locations.     

Effects of social context, reinforcer probability, and reinforcer magnitude on humans' choices to compete or not to compete.

In the first two experiments, subjects' choices to earn points (exchangeable for money) either by competing with a fictitious opponent or by not competing were studied. Buskist, Barry, Morgan, and Rossi's (1984) competitive fixed-interval schedule was modified to include a second response option, a noncompetitive fixed-interval schedule. After choosing to enter either option, the opportunity for reinforcers became available after the fixed-interval's duration had elapsed. Under the no-competition condition, points were always available after the interval had elapsed. Under the competition condition, points were available based on a predetermined probability of delivery. Experiments 1 and 2 examined how reinforcer probabilities and reinforcer magnitudes affected subjects' choices to compete. Several general conclusions can be made about the results: (a) Strong preferences to compete were observed at high and moderate reinforcer probabilities; (b) competing was observed even at very low reinforcer probabilities; (c) response rates were always higher in the competition component than in the no-competition component; and (d) response rates and choices to compete were insensitive to reinforcer-magnitude manipulations. In Experiment 3, the social context of this choice schedule was removed to determine whether the high levels of competing observed in the first two experiments were due to a response preference engendered by the social context provided by the experimenters through instructions. In contrast to the first two experiments, these subjects preferred the 60-s fixed-interval schedule (formerly the no-competition option), indicating that the instructions themselves were responsible for the preference to compete. This choice paradigm may be useful to future researchers interested in the effects of other independent variables (e.g., drugs, social context, instructions) on competitive behavior.     

Choice in a self-control paradigm with human subjects: Effects of changeover delay duration

The present experiment examined the choices of human subjects as a function of changeover delay (COD) duration. A self-control paradigm was used; subjects chose between larger, more delayed and smaller, less delayed reinforcers. The COD durations were 1 s, 15 s, and 30 s. The results indicated that at the 1-s COD, the subjects distributed their responses approximately equally between the two response alternatives. However, at the 15-s and 30-s COD durations, the subjects tended to demonstrate virtually exclusive preference for the larger, more delayed reinforcer. Furthermore, increasing the COD duration significantly increased the subjects' sensitivity to variation in reinforcer delay. Increasing the COD duration also increased the subjects' sensitivity to reinforcer amount, but this effect was not significant. The results are qualitatively consistent with an interpretation that the subjects followed a strategy which attempted to maximize overall amount of reinforcement.     

Sensitivity to reinforcer duration in a self-control procedure

In a concurrent-chains procedure, pigeons' responses on left and right keys were followed by reinforcers of different durations at different delays following the choice responses. Three pairs of reinforcer delays were arranged in each session, and reinforcer durations were varied over conditions. In Experiment 1 reinforcer delays were unequal, and in Experiment 2 reinforcer delays were equal. In Experiment 1 preference reversal was demonstrated in that an immediate short reinforcer was chosen more frequently than a longer reinforcer delayed 6 s from the choice, whereas the longer reinforcer was chosen more frequently when delays to both reinforcers were lengthened. In both experiments, choice responding was more sensitive to variations in reinforcer duration at overall longer reinforcer delays than at overall shorter reinforcer delays, independently of whether fixed-interval or variable-interval schedules were arranged in the choice phase. We concluded that preference reversal results from a change in sensitivity of choice responding to ratios of reinforcer duration as the delays to both reinforcers are lengthened.     

Self-control and impulsiveness in children and adults: Effects of food preferences

Experiment 1 used 6 preschool boys and Experiment 2 used 6 adult women to explore the effects of food preference on humans' choice in self-control paradigms. The boys showed a higher proportion of responses for more delayed, larger reinforcers (a measure of self-control) when those choices resulted in receipt of the most preferred food compared to when those choices resulted in the least preferred food. Further, the boys chose the less delayed, smaller reinforcers significantly more often when only those choices, as opposed to both choices, resulted in the most preferred food. Conversely, they chose the more delayed, larger reinforcers significantly more often when only those choices, as opposed to both choices, resulted in the most preferred food. Finally, the women demonstrated significantly less sensitivity to reinforcer amount relative to sensitivity to reinforcer delay (another measure of self-control) when they had a higher preference for the juice received as the less delayed, smaller reinforcer than for the juice received as the more delayed, larger reinforcer. Together, the results show that subjects' food preferences can influence self-control for food reinforcers.     

Some effects of response-correlated increases in reinforcer magnitude on human behavior

After training under short or long fixed-interval schedules, humans responded under a modified fixed-interval schedule in which magnitude of reinforcement (X or 2X) was minimally correlated with response frequency. Response frequencies that equaled or exceeded a minimum response criterion were followed by the larger reinforcer at the end of the interval; otherwise, the smaller reinforcer was delivered. The modified schedule alternated with the baseline schedule across conditions. In a control condition, the reinforcer magnitudes produced by control subjects were yoked to those of experimental subjects. Experimental subjects, but not control subjects, showed increased responding. In addition to the baseline and modified fixed-interval schedules used in Experiment 1, subjects in Experiment 2 also responded under a second modified fixed-interval contingency in which increases in reinforcer magnitude were more highly correlated with response frequency. Experimental subjects, but not control subjects, showed increased responding under both procedures. Direct comparison of these two procedures showed that the high-correlation procedure produced greater increases in responding than did the low-correlation procedure.     

Determination of discount functions in rats with an adjusting-amount procedure.

An adjusting-amount procedure was used to measure discounting of reinforcer value by delay. Eight rats chose between a varying amount of immediate water and a fixed amount of water given after a delay. The amount of immediate water was systematically adjusted as a function of the rats' previous choices. This procedure was used to determine the indifference point at which each rat chose the immediate amount and the delayed amount with equal frequency. The amount of immediate water at this indifference point was used to estimate the value of the delayed amount of water. In Experiment 1, the effects of daily changes in the delay to the fixed reinforcer (100 microliters of water delivered after 0, 2, 4, 8, or 16 s) were tested. Under these conditions, the rats reached indifference points within the first 30 trials of each 60-trial session. In Experiment 2, the effects of water deprivation level on discounting of value by delay were assessed. Altering water deprivation level affected the speed of responding but did not affect delay discounting. In Experiment 3, the effects of varying the magnitude of the delayed water (100, 150, and 200 microliters) were tested. There was some tendency for the discounting function to be steeper for larger than for smaller reinforcers, although this difference did not reach statistical significance. In all three experiments, the obtained discount functions were well described by a hyperbolic function. These experiments demonstrate that the adjusting-amount procedure provides a useful tool for measuring the discounting of reinforcer value by delay.     

On the effects of component durations and component reinforcement rates in multiple schedules

Four experiments, each using the same six pigeons, investigated the effects of varying component durations and component reinforcement rates in multiple variable-interval schedules. Experiment 1 used unequal component durations in which one component was five times the duration of the other, and the shorter component was varied over conditions from 120 seconds to 5 seconds. The schedules were varied over five values for each pair of component durations. Sensitivity to reinforcement rate changes was the same at all component durations. In Experiment 2, both component durations were 5 seconds, and the schedules were again varied using both one and two response keys. Sensitivity to reinforcement was not different from the values found in Experiment 1. In Experiment 3, various manipulations, including body-weight changes, reinforcer duration changes, blackouts, hopper lights correlated with keylights, and overall reinforcement rate changes were carried out. No reliable increase in reinforcement sensitivity resulted from any manipulation. Finally, in Experiment 4, reinforcement rates in the two components were kept constant and unequal, and the component durations were varied. Shorter components produced significantly increased response rates normally in the higher reinforcement rate component, but schedule reversals at short component durations eliminated the response rate increases. The effects of component duration on multiple schedule performance cannot be interpreted as changing sensitivity to reinforcement nor to changing bias.     

Every reinforcer counts: reinforcer magnitude and local preference

Six pigeons were trained on concurrent variable-interval schedules. Sessions consisted of seven components, each lasting 10 reinforcers, with the conditions of reinforcement differing between components. The component sequence was randomly selected without replacement. In Experiment 1, the concurrent-schedule reinforcer ratios in components were all equal to 1.0, but across components reinforcer-magnitude ratios varied from 1:7 through 7:1. Three different overall reinforcer rates were arranged across conditions. In Experiment 2, the reinforcer-rate ratios varied across components from 27:1 to 1:27, and the reinforcer-magnitude ratios for each alternative were changed across conditions from 1:7 to 7:1. The results of Experiment 1 replicated the results for changing reinforcer-rate ratios across components reported by Davison and Baum (2000, 2002): Sensitivity to reinforcer-magnitude ratios increased with increasing numbers of reinforcers in components. Sensitivity to magnitude ratio, however, fell short of sensitivity to reinforcer-rate ratio. The degree of carryover from component to component depended on the reinforcer rate. Larger reinforcers produced larger and longer postreinforcer preference pulses than did smaller reinforcers. Similar results were found in Experiment 2, except that sensitivity to reinforcer magnitude was considerably higher and was greater for magnitudes that differed more from one another. Visit durations following reinforcers measured either as number of responses emitted or time spent responding before a changeover were longer following larger than following smaller reinforcers, and were longer following sequences of same reinforcers than following other sequences. The results add to the growing body of research that informs model building at local levels.     

Effect of reinforcer devaluation on discriminative control of instrumental behavior

Two experiments examined the effect of reinforcer devaluation on the ability of a discriminative stimulus (Sd) to control instrumental behavior in Sprague-Dawley rats. In Experiment 1 reinforcer devaluation reduced, but did not eliminate, the ability of the Sd to control performance of the original response and to transfer its control to a new response trained with the same reinforcer. The effect of devaluation was more complete in Experiment 2, in which the reinforcer was delivered directly into the oral cavity. However, retraining the response with a different reinforcer partially restored the ability of the Sd to control performance of that response. These results suggest that an Sd may not augment its trained responses when the reinforcer has been completely devalued but may promote responses with which it shares a reinforcer, as long as those responses are associated with some reinforcer that retains its value. The implications of these results for the way that discriminative stimuli control instrumental behavior are discussed.     

Sensitivity to relative reinforcer rate in concurrent schedules: independence from relative and absolute reinforcer duration

Twelve pigeons responded on two keys under concurrent variable-interval (VI) schedules. Over several series of conditions, relative and absolute magnitudes of reinforcement were varied. Within each series, relative rate of reinforcement was varied and sensitivity of behavior ratios to reinforcer-rate ratios was assessed. When responding at both alternatives was maintained by equal-sized small reinforcers, sensitivity to variation in reinforcer-rate ratios was the same as when large reinforcers were used. This result was observed when the overall rate of reinforcement was constant over conditions, and also in another series of concurrent schedules in which one schedule was kept constant at VI ached 120 s. Similarly, reinforcer magnitude did not affect the rate at which response allocation approached asymptote within a condition. When reinforcer magnitudes differred between the two responses and reinforcer-rate ratios were varied, sensitivity of behavior allocation was unaffected although response bias favored the schedule that arranged the larger reinforcers. Analysis of absolute response rates ratio sensitivity to reinforcement occurrred on the two keys showed that this invariance of response despite changes in reinforcement interaction that were observed in absolute response rates on the constant VI 120-s schedule. Response rate on the constant VI 120-s schedule was inversely related to reinforcer rate on the varied key and the strength of this relation depended on the relative magnitude of reinforcers arranged on varied key. Independence of sensitivity to reinforcer-rate ratios from relative and absolute reinforcer magnitude is consistent with the relativity and independence assumtions of the matching law.     

Effects of reinforcer magnitude on responding under differential-reinforcement-of-low-rate schedules of rats and pigeons

Experiment I investigated the effects of reinforcer magnitude on differential-reinforcement-of-low-rate (DRL) schedule performance in three phases. In Phase 1, two groups of rats (n = 6 and 5) responded under a DRI. 72-s schedule with reinforcer magnitudes of either 30 or 300 microl of water. After acquisition, the water amounts were reversed for each rat. In Phase 2, the effects of the same reinforcer magnitudes on DRL 18-s schedule performance were examined across conditions. In Phase 3, each rat responded unider a DR1. 18-s schedule in which the water amotnts alternated between 30 and 300 microl daily. Throughout each phase of Experiment 1, the larger reinforcer magnitude resulted in higher response rates and lower reinforcement rates. The peak of the interresponse-time distributions was at a lower value tinder the larger reinforcer magnitude. In Experiment 2, 3 pigeons responded under a DRL 20-s schedule in which reinforcer magnitude (1-s or 6-s access to grain) varied iron session to session. Higher response rates and lower reinforcement rates occurred tinder the longer hopper duration. These results demonstrate that larger reinforcer magnitudes engender less efficient DRL schedule performance in both rats and pigeons, and when reinforcer magnitude was held constant between sessions or was varied daily. The present results are consistent with previous research demonstrating a decrease in efficiency as a function of increased reinforcer magnituide tinder procedures that require a period of time without a specified response. These findings also support the claim that DRI. schedule performance is not governed solely by a timing process.     

Behavioral control by the response–reinforcer correlation

Using a discrete‐trials procedure, two experiments examined the effects of response–reinforcer correlations on responding while controlling molecular variables that operated at the moment of reinforcer delivery (e.g., response–reinforcer temporal contiguity, interresponse times preceding reinforcement). Each trial consisted of three successive components: Response, Timeout, and Reinforcement, with the duration of each component held constant. The correlation between the number of responses in the Response component and reinforcer deliveries in the Reinforcement component was varied. In the Positive‐correlation condition, a larger number of responses in the Response component programmed a higher reinforcement rate (Experiment 1) or a shorter time to reinforcement (Experiment 2) in the Reinforcement component. Although programmed in this way, the actual reinforcer delivery was dependent on, and occurred immediately after, a response in the Reinforcement component. In the Zero‐correlation condition, the programmed rates of reinforcement (Experiment 1) or the times to reinforcement (Experiment 2) in the Reinforcement component of each trial were yoked to those in the preceding Positive‐correlation condition. Responding in the Response component was higher in the Positive‐ than in the Zero‐correlation condition, without systematic changes in molecular variables. The results suggest that the response–reinforcer correlation can be a controlling variable of behavior.