Typical delay determines waiting time on periodic-food schedules: Static and dynamic tests

Pigeons and other animals soon learn to wait (pause) after food delivery on periodic-food schedules before resuming the food-rewarded response. Under most conditions the steady-state duration of the average waiting time, t, is a linear function of the typical interfood interval. We describe three experiments designed to explore the limits of this process. In all experiments, t was associated with one key color and the subsequent food delay, T, with another. In the first experiment, we compared the relation between t (waiting time) and T (food delay) under two conditions: when T was held constant, and when T was an inverse function of t. The pigeons could maximize the rate of food delivery under the first condition by setting t to a consistently short value; optimal behavior under the second condition required a linear relation with unit slope between t and T. Despite this difference in optimal policy, the pigeons in both cases showed the same linear relation, with slope less than one, between t and T. This result was confirmed in a second parametric experiment that added a third condition, in which T + t was held constant. Linear waiting appears to be an obligatory rule for pigeons. In a third experiment we arranged for a multiplicative relation between t and T (positive feedback), and produced either very short or very long waiting times as predicted by a quasi-dynamic model in which waiting time is strongly determined by the just-preceding food delay.     

Time-place learning by pigeons, Columba livia

In each of two experiments, 2 pigeons received discrimination training in which food reinforcement for key pecking was conditional upon both spatial and temporal cues. In Experiment 1, food was available for periods of 30 s at each of three locations (pecking keys) during trials that lasted 90 s. In Experiment 2, food was available for periods of 15 min at each of four locations (pecking keys) during a 60-min trial. In both experiments, pigeons' key pecking was jointly controlled by the spatial and temporal cues. These data, and other recent experiments, suggest that animals learn relationships between temporal and spatial cues that predict stable patterns of food availability.     

The interaction of stimulus and reinforcer control in complex temporal discrimination.

Six pigeons were trained in a discrete-trials signal-detection procedure to discriminate between a fixed-duration stimulus (5 s or 20 s) and a set of variable durations ranging from 2.5 s to 57.5 s in steps of 5 s. For each fixed-duration stimulus, the ratio of reinforcer frequencies contingent upon reporting the fixed versus the variable stimulus was systematically manipulated. Detection performance was well controlled by both the stimulus value and the reinforcer ratio. Both the discriminability between the fixed duration and the set of variable durations, and the discriminability between the fixed duration and each of the variable durations, were independent of the reinforcer-frequency ratio when discriminability was measured as log d. The sensitivity of response bias to reinforcement-ratio changes was independent of the value of the fixed duration, but was not independent of the discriminability of the variable durations from the fixed durations. Under current models, discriminability measures in complex temporal discrimination may be independent of biasing manipulations, but bias measures are not independent of stimulus values.     

Delayed temporal discrimination in pigeons: A comparison of two procedures

A within-subjects comparison was made of pigeons' performance on two temporal discrimination procedures that were signaled by differently colored keylight samples. During stimulus trials, a peck on the key displaying a slanted line was reinforced following short keylight samples, and a peck on the key displaying a horizontal line was reinforced following long keylight samples, regardless of the location of the stimuli on those two choice keys. During position trials, a peck on the left key was reinforced following short keylight samples and a peck on the right key was reinforced following long keylight samples, regardless of which line stimulus appeared on the correct key. Thus, on stimulus trials, the correct choice key could not be discriminated prior to the presentation of the test stimuli, whereas on position trials, the correct choice key could be discriminated during the presentation of the sample stimulus. During Phase 1, with a 0-s delay between sample and choice stimuli, discrimination learning was faster on position trials than on stimulus trials for all 4 birds. During Phase 2, 0-, 0.5-, and 1.0-s delays produced differential loss of stimulus control under the two tasks for 2 birds. Response patterns during the delay intervals provided some evidence for differential mediation of the two delayed discriminations. These between-task differences suggest that the same processes may not mediate performance in each.     

Behavioral contrast in fixed-interval components: effects of extinction-component duration.

Seven albino rats were exposed to a multiple schedule of reinforcement in which the two components (fixed interval and extinction) alternated such that a presentation of the extinction component followed each fixed-interval reinforcement. In baseline sessions, the duration of the extinction component was constant and always one-third of the fixed-interval value. Probe sessions contained a probe segment in which the duration of the extinction component was increased; the response rate in fixed-interval components during the probe segment was compared with the response rate in the segments preceding and following the probe. The effect of increasing the duration of the extinction component was studied under three values of fixed interval: 30 s, 120 s, and 18 s, in three successive conditions. Response rate within fixed intervals was a direct function of duration of the extinction component. Pausing at the beginning of the fixed interval decreased as extinction duration increased. These effects were larger and more consistent for the shorter fixed-interval values (18 s and 30 s). These results indicate a functional relation between relative component duration and responding. For the component providing more frequent reinforcement, this could be stated as an inverse relationship between relative component duration and response rate. This relation is similar to findings regarding the ratio of trial and intertrial duration in Pavlovian conditioning procedures, and suggests that behavioral contrast may be related to Pavlovian contingencies underlying the multiple schedule.     

Temporal control by signals of interval duration within variable-interval schedules

Rats' lever pressing produced sucrose reinforcers on a variable-interval schedule where, in different conditions, the duration of a stimulus presented immediately after reinforcement was either correlated or uncorrelated with the duration of the current interreinforcement interval. Under the baseline schedule, in which no stimulus was presented, the minimum interreinforcement interval was 8 s and the mean postreinforcement pause of each subject approximated this value. Response rates increased slowly over the first 10 to 15 s and then remained roughly constant throughout the remainder of the interval. In both the correlated and uncorrelated conditions, the added stimulus resulted in the postreinforcement pauses lengthening to values in excess of the duration of the preceding stimulus. This resulted in a poststimulus pause which was, in most cases, roughly constant irrespective of the duration of the preceding stimulus, or of the reinforcement contingencies prevailing immediately after stimulus offset. Local response-rate patterns in the uncorrelated conditions were similar to those obtained under the baseline schedule in which no stimulus was presented. However, in the correlated condition local response rates increased across the remainder of the interreinforcer interval. Further, the rate of acceleration was inversely related to the duration of the preceding stimulus. These results show that a correlation between stimulus duration and the ensuing time to reinforcement can control behavior—a type of temporal control not previously reported.     

Reinforcer pathology,probabilistic choice,and medication adherence in patients with multiple sclerosis

The reinforcer pathology model posits that core behavioral economic mechanisms, including delay discounting and behavioral economic demand, underlie adverse health decisions and related clinical disorders. Extensions beyond substance use disorder and obesity, however, are limited. Using a reinforcer pathology framework, this study evaluates medical adherence decisions in patients with multiple sclerosis. Participants completed behavioral economic measures, including delay discounting, probability discounting, and a medication purchase task. A medical decision-making task was also used to evaluate how sensitivity to mild side effect risk and efficacy contributed to the likelihood of taking a hypothetical disease-modifying therapy. Less steep delay discounting and more intense (greater) medication demand were independently associated with greater adherence to the medication decision-making procedure. More generally, the pattern of interrelations between the medication-specific and general behavioral economic metrics was consistent with and contributes to the reinforcer pathology model. Additional research is warranted to expand these models to different populations and health behaviors, including those of a positive health orientation (i.e., medication adherence).     

The role of effort type and intensity in children's decisions about effort-based outcomes

Despite the importance of persistence in early learning, we know little about how children reason about outcomes that result from their efforts. Here we examined the role of effort type (i.e., physical vs. cognitive) and intensity (i.e., high vs. low effort) in shaping children's decision making about effort-based rewards. Five- to 7-year-olds (N = 133) were assigned to one of four conditions (High Physical Effort, Low Physical Effort, High Cognitive Effort, Low Cognitive Effort) and completed a series of tasks to construct a toy. Tasks varied in the type (physical/cognitive) and intensity (high/low) of effort required to complete them. After constructing their toy, children completed a series of tasks to probe how much they valued that toy. Across conditions, children preferred their toy and gave it a higher monetary value, relative to a stranger's. However, when choosing their toy came at a cost, children no longer preferred it. Only children who built their toy through either cognitive or low effort were willing to incur a cost for their toy. Older children, across conditions, were also more likely to incur a cost for their toy. These findings demonstrate that by age five, children are sensitive to variations in effort type and intensity, and these factors shape how they evaluate effort-based rewards.     

Reciprocal relationships between personality traits and psychological well-being

This study used an American sample collected over a period of approximately 2 decades (at 3 time points) to examine the temporal relationships between psychological well-being and personality traits (i.e., neuroticism, extraversion, agreeableness, conscientiousness, and openness to experience). The random-intercept cross-lagged panel model was used to separate between-person and within-person sources of variation. Between-person correlations were comparable to those of previous studies. New insights were gained at the within-person level. There were reciprocal relationships between psychological well-being and openness and extraversion, suggesting the joint development of plasticity-related traits and well-being over time. The relationships between psychological well-being and conscientiousness and agreeableness were unidirectional, with psychological well-being preceding these traits. Despite a strong between-person association between neuroticism and psychological well-being, the two were not related at the within-person level.     

TEMPORAL CONTROL ON INTERVAL SCHEDULES: WHAT DETERMINES THE POSTREINFORCEMENT PAUSE?

On fixed-interval or response-initiated delay schedules of reinforcement, the average pause following food presentation is proportional to the interfood interval. Moreover, when a number of intervals of different durations occur in a programmed cyclic series, postreinforcement pauses track the changes in interval value. What controls the duration of postreinforcement pauses under these conditions? Staddon, Wynne, and Higa (1991), in their linear waiting model, propose control by the preceding interfood interval. Another possibility is that delay to reinforcement, signaled by a key peck and/or stimulus change, determines the subsequent pause. The experiments reported here examined the role of these two possible time markers by studying the performance of pigeons under a chained cyclic fixed-interval procedure. The data support the linear waiting model, but suggest that more than the immediately preceding interfood interval plays a role in temporal control.