Nonstable concurrent choice in pigeons

Six pigeons were trained on concurrent variable-interval schedules in which the arranged reinforcer ratios changed from session to session according to a 31-step pseudorandom binary sequence. This procedure allows a quantitative analysis of the degree to which performance in an experimental session is affected by conditions in previous sessions. Two experiments were carried out. In each, the size of the reinforcer ratios arranged between the two concurrent schedules was varied between 31-step conditions. In Experiment 1, the concurrent schedules were arranged independently, and in Experiment 2 they were arranged nonindependently. An extended form of the generalized matching law described the relative contribution of past and present events to present-session behavior. Total performance in sessions was mostly determined by the reinforcer ratio in that session and partially by reinforcers that had been obtained in previous sessions. However, the initial exposure to the random sequence produced a lower sensitivity to current-session reinforcers but no difference in overall sensitivity to reinforcement. There was no evidence that the size of the reinforcer ratios available on the concurrent schedules affected either overall sensitivity to reinforcement or the sensitivity to reinforcement in the current session. There was also no evidence of any different performance between independent and nonindependent scheduling. Because of these invariances, this experiment validates the use of the pseudorandom sequence for the fast determination of sensitivity to reinforcement.     

Rat choice in rapidly changing concurrent schedules

In two experiments, experimentally naïve rats were trained in concurrent variable‐interval schedules in which the reinforcer ratios changed daily according to a pseudorandom binary sequence. In Experiment 1, relative response rates showed clear sensitivity to current‐session reinforcer ratios, but not to previous sessions' reinforcer ratios. Within sessions, sensitivity to the current session's reinforcement rates increased steadily, and by session end, response ratios approached matching to the current‐session reinforcer ratios. Across sessions, sensitivity to the current session's reinforcer ratio decreased with continued exposure to the pseudorandom binary sequence, contrary to expectations based on previous studies demonstrating learning sets. Using a second group of naïve rats, Experiment 2 replicated the main results from Experiment 1 and showed that although there were increases over sessions in both changeover rate and response rate during the changeover delay, neither could explain the accompanying reductions in sensitivity. We consider the role of reinforcement history, showing that our results can be simulated using two separate representations, one local and one nonlocal, but a more complex approach will be needed to bring together these results and other history effects such as learning sets and spontaneous recovery.     

Determination of a behavioral transfer function: White-noise analysis of session-to-session response-ratio dynamics on concurrent VI VI schedules

Six pigeons were exposed to concurrent variable-interval schedules in which the programmed reinforcer ratios changed from session to session according to a pseudorandom binary sequence. This procedure corresponded to the stochastic identification paradigm (“white-noise experiment”) of systems theory and enabled the relation between log response ratios in the current session and log reinforcer ratios in all previous sessions to be determined. Such dynamic relations are called linear transfer functions. Both nonparametric and parametric representations of these, in the form of “impulse-response functions,” were determined for each bird. The session-to-session response ratios resulting from the session-to-session pseudorandom binary variations in reinforcer ratios were well predicted by the impulse-response functions identified for each pigeon. The impulse-response functions were well fitted by a second-order dynamic model involving only two parameters: a time constant and a gain. The mean time constant was 0.67 sessions, implying that the effects of abrupt changes in log reinforcer ratios should be 96% complete within about five sessions. The mean gain was 0.53, which was surprisingly low inasmuch as it should equal the sensitivity to reinforcement ratio observed under steady-state conditions. The same six pigeons were subjected to a similar experiment 10 months following the first. Despite individual differences in impulse-response functions between birds within each experiment, the impulse-response functions determined from the two experiments were essentially the same.     

Does overall reinforcer rate affect discrimination of time‐based contingencies?

Overall reinforcer rate appears to affect choice. The mechanism for such an effect is uncertain, but may relate to reinforcer rate changing the discrimination of the relation between stimuli and reinforcers. We assessed whether a quantitative model based on a stimulus‐control approach could be used to account for the effects of overall reinforcer rate on choice under changing time‐based contingencies. On a two‐key concurrent schedule, the likely availability of a reinforcer reversed when a fixed time had elapsed since the last reinforcer, and the overall reinforcer rate was varied across conditions. Changes in the overall reinforcer rate produced a change in response bias, and some indication of a change in discrimination. These changes in bias and discrimination always occurred quickly, usually within the first session of a condition. The stimulus‐control approach provided an excellent account of the data, suggesting that changes in overall reinforcer rate affect choice because they alter the frequency of reinforcers obtained at different times, or in different stimulus contexts, and thus change the discriminated relation between stimuli and reinforcers. These findings support the notion that temporal and spatial discriminations can be understood in terms of discrimination of reinforcers across time and space.     

SENSITIVITY OF CONDITIONAL-DISCRIMINATION PERFORMANCE TO WITHIN-SESSION VARIATION OF REINFORCER FREQUENCY

The present experiment developed a methodology for assessing sensitivity of conditional-discrimination performance to within-session variation of reinforcer frequency. Four pigeons responded under a multiple schedule of matching-to-sample components in which the ratio of reinforcers for correct S₁ and S₂ responses was varied across components within session. Initially, five components, each arranging a different reinforcer-frequency ratio (from 19 to 91), were presented randomly within a session. Under this condition, sensitivity to reinforcer frequency was low. Sensitivity failed to improve after extended exposure to this condition, and under a condition in which only three reinforcer-frequency ratios were varied within session. In a later condition, three reinforcer-frequency ratios were varied within session, but the reinforcer-frequency ratio in effect was differentially signaled within each component. Under this condition, values of sensitivity were similar to those traditionally obtained when reinforcer-frequency ratios for correct responses are varied across conditions. The effects of signaled vs. unsignaled reinforcer-frequency ratios were replicated in two subsequent conditions. The present procedure could provide a practical alternative to parametric variation of reinforcer frequency across conditions and may be useful in characterizing the effects of a variety of manipulations on steady-state sensitivity to reinforcer frequency.     

Reinforcement magnitude and pausing on progressive-ratio schedules

Rats responded under progressive-ratio schedules for sweetened milk reinforcers; each session ended when responding ceased for 10 min. Experiment 1 varied the concentration of milk and the duration of postreinforcement timeouts. Postreinforcement pausing increased as a positively accelerated function of the size of the ratio, and the rate of increase was reduced as a function of concentration and by timeouts of 10 s or longer. Experiment 2 varied reinforcement magnitude within sessions (number of dipper operations per reinforcer) in conjunction with stimuli correlated with the upcoming magnitude. In the absence of discriminative stimuli, pausing was longer following a large reinforcer than following a small one. Pauses were reduced by a stimulus signaling a large upcoming reinforcer, particularly at the highest ratios, and the animals tended to quit responding when the past reinforcer was large and the stimulus signaled that the next one would be small. Results of both experiments revealed parallels between responding under progressive-ratio schedules and other schedules containing ratio contingencies. Relationships between pausing and magnitude suggest that ratio pausing is under the joint control of inhibitory properties of the past reinforcer and excitatory properties of stimuli correlated with the upcoming reinforcer, rather than under the exclusive control of either factor alone.     

Rapid acquisition of choice and timing and the provenance of the terminal-link effect

Eight pigeons responded in a concurrent-chains procedure in which terminal-link schedules changed pseudorandomly across sessions. Pairs of terminal-link delays either summed to 15 s or to 45 s. Across sessions, the location of the shorter terminal link changed according to a pseudorandom binary sequence. On some terminal links, food was withheld to obtain start and stop times, measures of temporal control. Log initial-link response ratios stabilized within the first half of each session. Log response ratio was a monotonically-increasing but nonlinear function of programmed log terminal-link immediacy ratio. There was an effect of absolute terminal-link duration on log response ratio: For most subjects, preference for the relatively shorter terminal-link delay was stronger when absolute delays were long than when absolute delays were short. Polynomial regressions and model comparison showed that differences in degree of nonlinearity, not in sensitivity to log immediacy ratio, produced this effect. Temporal control of stop times was timescale invariant with scalar variability, but temporal control of start times was not consistent across subjects or terminal-link durations.     

Effects of relative reinforcer frequency on complex color detection

Pigeons were trained under a discrete-trials detection procedure in which one of a set of color stimuli was presented on the center key and a single response turned off the stimulus and illuminated two side keys. Single responses to one or the other side key produced occasional reinforcers depending on the value of the color stimulus. In Experiment 1, one color-stimulus set comprised 559, 564, 569, and 574 nm, and right-key pecks were occasionally reinforced following presentations of members of this set. The other stimulus set comprised 579, 584, 589, and 594 nm, and left-key pecks were occasionally reinforced following presentations of members of this set. Across seven experimental conditions, the left/(left + right) relative reinforcer frequency was varied from .1 to .9. In Experiment 2, one stimulus set contained only one member, 574 nm, and right-key responses were occasionally reinforced following its presentation. Over 12 experimental conditions, two manipulations were carried out. First, the number of stimuli comprising the other stimulus set was increased from one (579 nm) to two (579 and 584 nm) to three (579, 584, and 589 nm) and to four (579, 584, 589, and 594 nm), and left-key responses were reinforced occasionally following center-key presentations of members of this set. Second, for each stimulus combination, the left/(left + right) relative reinforcer frequency was varied from .1 to .5 to .9 across three experimental conditions. The principal finding of Experiments 1 and 2 was that reinforcers and stimuli interacted in their effects on behavior. In Experiment 3, pairs of adjacent stimuli (5 nm apart) in the range 559 to 594 nm were presented in each experimental condition, and the left/(left + right) relative reinforcer frequency was held constant at .5. The data from all three experiments were analyzed according to a detection model describing performance in multiple-stimulus two-response procedures. This model provided independent measures of stimulus discriminability, contingency discriminability, and bias. The analysis showed that (a) consistent with the color-naming function, pigeons were better able to discriminate between higher nanometer values than lower nanometer values; (b) their ability to discriminate between the stimuli was independent of the number of wavelengths comprising each stimulus set; (c) they allocated delivered reinforcers very accurately to the previously emitted response; and (d) no consistent biases emerged.     

The effects of reinforcer choice on rates of challenging behavior and free operant responding in individuals with severe disabilities

A multielement design was used to evaluate the effects of three reinforcement conditions on the free-operant and challenging responses of two individuals with severe disabilities. Preference assessments identified three stimuli that could function as reinforcers for each participant. Reinforcers were delivered on a variable ratio schedule for a free-operant response. In the participant-selected condition, each reinforcer delivery consisted of placing an array of three different stimuli in front of the participant, who was allowed to select one. In the constant condition, each reinforcer delivery consisted of placing an array of three identical stimuli in front of the participant. Within each session the three stimuli were held constant, but varied across sessions. In the experimenter-selected condition, each reinforcer delivery consisted of an array of three identical stimuli being placed in front of the participant. Within each session, the three stimuli were presented in an order predetermined by the experimenter such that, on average, each stimulus was available every third reinforcer delivery. For both individuals, the participant-selected condition resulted in lower average rates of challenging responses and slightly higher average free-operant response rates. No differences in responding were noted between the constant condition and the experimenter-selected condition. © 1998 John Wiley & Sons, Ltd.     

Fixed-ratio pausing: Joint effects of past reinforcer magnitude and stimuli correlated with upcoming magnitude

Pigeons responded on fixed-ratio schedules ending in small or large reinforcers (grain presentations of different duration) interspersed within each session. In mixed-schedule conditions, the response key was lit with a single color throughout the session, and pausing was directly related to the past reinforcer (longer pauses after large reinforcers than after small ones). In multiple-schedule conditions, different colors accompanied the ratios ending in small and large reinforcers, and pausing was affected by the upcoming reinforcer as well as the past one. Pauses were shorter before large reinforcers than before small ones, but they continued to be longer after large reinforcers than after small ones. The influence of the past reinforcer was modulated by the magnitude of the upcoming reinforcer; in the presence of the stimulus before the small reinforcer, the effect of the past reinforcer was enhanced relative to its effect in the stimulus before the large reinforcer. These results show that pausing between ratios is jointly determined by two competing factors: past conditions of reinforcement and stimuli correlated with upcoming conditions.     

Rapid acquisition of preference in concurrent chains when alternatives differ on multiple dimensions of reinforcement

Pigeons responded in a concurrent-chains procedure in which terminal-link reinforcer variables were changed unpredictably across sessions. In Experiment 1, the terminal-link schedules were fixed-interval (FI) 8 s and FI 16 s, and the reinforcer magnitudes were 2 s and 4 s. In Experiment 2 the probability of reinforcement (100% or 50%) was varied with immediacy and magnitude. Multiple-regression analyses showed that pigeons' initial-link response allocation was determined by current-session reinforcer variables, similar to previous studies which have varied only immediacy (Grace, Bragason, & McLean, 2003). Sensitivity coefficients were positive and statistically significant for all reinforcer variables in both experiments. Analyses of responding within individual sessions showed that final levels of preference for dominated sessions, in which all reinforcer variables favored the same terminal link, were more extreme than for tradeoff sessions in which at least one reinforcer variable favored each alternative. This result implies that response allocation was determined by multiple reinforcer variables within individual sessions, consistent with the concatenated matching law. However, in Experiment 2, there was a nonlinear (sigmoidal) relationship between response allocation and relative value, which suggests the possibility that reinforcer variables may interact during acquisition, contrary to the matching law.     

Effects of reinforcer magnitude on responding under differential-reinforcement-of-low-rate schedules of rats and pigeons

Experiment I investigated the effects of reinforcer magnitude on differential-reinforcement-of-low-rate (DRL) schedule performance in three phases. In Phase 1, two groups of rats (n = 6 and 5) responded under a DRI. 72-s schedule with reinforcer magnitudes of either 30 or 300 microl of water. After acquisition, the water amounts were reversed for each rat. In Phase 2, the effects of the same reinforcer magnitudes on DRL 18-s schedule performance were examined across conditions. In Phase 3, each rat responded unider a DR1. 18-s schedule in which the water amotnts alternated between 30 and 300 microl daily. Throughout each phase of Experiment 1, the larger reinforcer magnitude resulted in higher response rates and lower reinforcement rates. The peak of the interresponse-time distributions was at a lower value tinder the larger reinforcer magnitude. In Experiment 2, 3 pigeons responded under a DRL 20-s schedule in which reinforcer magnitude (1-s or 6-s access to grain) varied iron session to session. Higher response rates and lower reinforcement rates occurred tinder the longer hopper duration. These results demonstrate that larger reinforcer magnitudes engender less efficient DRL schedule performance in both rats and pigeons, and when reinforcer magnitude was held constant between sessions or was varied daily. The present results are consistent with previous research demonstrating a decrease in efficiency as a function of increased reinforcer magnituide tinder procedures that require a period of time without a specified response. These findings also support the claim that DRI. schedule performance is not governed solely by a timing process.     

EFFECTS OF POINT‐LOSS PUNISHERS ON HUMAN SIGNAL‐DETECTION PERFORMANCE

Three experiments using human participants varied the distribution of point‐gain reinforcers or point‐loss punishers in two‐alternative signal‐detection procedures. Experiment 1 varied the distribution of point‐gain reinforcers for correct responses (Group A) and point‐loss punishers for errors (Group B) across conditions. Response bias varied systematically as a function of the relative reinforcer or punisher frequencies. Experiment 2 arranged two conditions — one where an unequal ratio of reinforcement (5:1 or 1:5) was presented without punishment (R‐only), and another where the same reinforcer ratio was presented with an equal distribution of point‐loss punishers (R+P). Response bias was significantly greater in the R‐only condition than the R+P condition, supporting a subtractive model of punishment. Experiment 3 varied the distribution of point‐gain reinforcers for correct responses across four unequal reinforcer ratios (5:1, 2:1, 1:2, 1:5) both without (R‐only) and with (R+P) an equal distribution of point‐loss punishers for errors. Response bias varied systematically with changes in relative reinforcer frequency for both R‐only and R+P conditions, with 5 out of 8 participants showing increases in sensitivity estimates from R‐only to R+P conditions. Overall, the results indicated that punishers have similar but opposite effects to reinforcers in detection procedures and that combined reinforcer and punisher effects might be better modeled by a subtractive punishment model than an additive punishment model, consistent with research using concurrent‐schedule choice procedures.     

Response-reinforcer relations and the maintenance of behavior

The effects on pigeons' key pecking of unsignaled delays of reinforcement and response-independent reinforcement were compared after either variable-interval or differential-reinforcement-of-low-rate baseline schedules. One 30-min session arranging delayed reinforcement and one 30-min session arranging response-independent reinforcement were conducted daily, 6 hr apart. A within-subject yoked-control procedure equated reinforcer frequency and distribution across the two sessions. Response rates usually were reduced more by response-independent than by delayed but response-contingent delivery of reinforcers. Under both schedules, response rates were lower when obtained delays were greater. These results bear upon methodological and conceptual issues regarding comparisons of contingencies that change the temporal response–reinforcer relations.     

Resistance To Change As A Function Of Stimulus-reinforcer And Location-reinforcer Contingencies

Pigeons responded on two keys in each component of a multiple concurrent schedule. In one series of conditions the distribution of reinforcers between keys within one component was varied so as to produce changes in ratios of reinforcer totals for key locations when summed across components. In a second series, reinforcer allocation between components was varied so as to produce changes in ratios of reinforcer totals for components, summed across key locations. In each condition, resistance to change was assessed by presenting response-independent reinforcers during intercomponent blackouts and (for the first series) by extinction of responding on both keys in both components. Resistance to change for response totals within a component was always greater for the component with the larger total reinforcer rate. However, resistance to change for response totals at a key location was not a positive function of total reinforcement for pecking that key; indeed, relative resistance to extinction for the two locations showed a weak negative relation to ratios of reinforcer totals for key location. These results confirm the determination of resistance to change by stimulus—reinforcer contingencies.     

A Preliminary Examination of Motivating Operation and Reinforcer Class Interaction

The interaction effects between motivating operations and reinforcer classes were evaluated using a superordinate multielement design. Two individuals with developmental disabilities participated. Participants were exposed to a motivating operation condition—pre‐session access until rejection or pre‐session restriction for 24 h—followed by a reinforcer assessment, in which the reinforcer response requirements progressively increased. Reinforcers, stimuli representative of primary, conditioned, and token reinforcers, were available on alternating sessions. The results showed that pre‐session access and restriction produced reliable evocative and abative effects across reinforcer classes. For one of the participants, the effectiveness of the motivating operation appeared to be influenced by the reinforcer class. The interaction effects between motivating operations and reinforcer classes are discussed. Copyright © 2015 John Wiley & Sons, Ltd.     

Choice in a variable environment: effects of unequal reinforcer distributions

Six pigeons were trained in a procedure in which sessions included seven unsignaled components, each offering two pecking keys, and each providing a potentially different reinforcer ratio between the two keys. Across conditions, various combinations of reinforcer ratios and reinforcer-magnitude ratios were used to create unequal reinforcer distributions between the two alternatives when averaged across a session. The results extended previous research using the same basic procedure that had included only reinforcer distributions symmetrical around 1:1. Data analyses suggested that the variables controlling choice operated at a number of levels: First, individual reinforcers had local effects on choice; second, sequences of successive reinforcers obtained at the same alternative (continuations) had cumulative effects; and, third, when these sequences themselves occurred with greater frequency, their effects further cumulated. A reinforcer obtained at the other alternative following a sequence of continuations (a discontinuation) had a large effect and apparently reset choice to levels approximating the sessional reinforcer ratio.     

Contrast and undermatching with regular or irregular alternation of components

Behavioral contrast and response-ratio sensitivity to reinforcement were compared in multiple schedules in which components alternated strictly or according to a pseudorandom sequence. Average component durations in the two regimes were always 60 s, and order of presentation of component alternation regimes was counterbalanced across subjects. In Part 1, the reinforcer rate in one component was reduced from 60 per hour to zero, while that in the other component was unchanged. Positive behavioral contrast occurred in the constant component in that response rates increased, but neither the reliability nor the magnitude of contrast was affected by the manner in which components alternated. Part 2 was similar, except that a number of different reinforcer rates were used in the varied component. Neither contrast nor sensitivity of response ratios to changes in reinforcer ratios depended on the regime of component alternation. Thus, the predictability in time of future reinforcement conditions, which is a feature of regular multiple scheduling, does not appear to be a determinant of multiple-schedule performance.     

Sensitivity to relative reinforcer rate in concurrent schedules: independence from relative and absolute reinforcer duration

Twelve pigeons responded on two keys under concurrent variable-interval (VI) schedules. Over several series of conditions, relative and absolute magnitudes of reinforcement were varied. Within each series, relative rate of reinforcement was varied and sensitivity of behavior ratios to reinforcer-rate ratios was assessed. When responding at both alternatives was maintained by equal-sized small reinforcers, sensitivity to variation in reinforcer-rate ratios was the same as when large reinforcers were used. This result was observed when the overall rate of reinforcement was constant over conditions, and also in another series of concurrent schedules in which one schedule was kept constant at VI ached 120 s. Similarly, reinforcer magnitude did not affect the rate at which response allocation approached asymptote within a condition. When reinforcer magnitudes differred between the two responses and reinforcer-rate ratios were varied, sensitivity of behavior allocation was unaffected although response bias favored the schedule that arranged the larger reinforcers. Analysis of absolute response rates ratio sensitivity to reinforcement occurrred on the two keys showed that this invariance of response despite changes in reinforcement interaction that were observed in absolute response rates on the constant VI 120-s schedule. Response rate on the constant VI 120-s schedule was inversely related to reinforcer rate on the varied key and the strength of this relation depended on the relative magnitude of reinforcers arranged on varied key. Independence of sensitivity to reinforcer-rate ratios from relative and absolute reinforcer magnitude is consistent with the relativity and independence assumtions of the matching law.