Signal detection and matching: analyzing choice on concurrent variable-interval schedules.

Pigeons' pecks on a red key and a green key were followed by access to grain according to pairs of concurrent independent variable-interval schedules in a combined signal detection/matching law paradigm. Pecks on the red key were reinforced by the richer variable-interval schedule if a short-duration tone had been presented; pecks on the green key were reinforced by the richer variable-interval schedule if a long-duration tone had been presented. Pecks on the green key given a short-duration tone, or on the red key given a long-duration tone, were reinforced by the leaner variable-interval schedule. The data were analyzed according to both signal detection's and the matching law's separate measures of, first, the discrimination of the choices and, second, the bias to make one response or another. Increasing the difficulty of the tone-duration discrimination decreased both methods' measures of the discrimination of the choices and did not change both methods' measures of the bias to make one response or another. Changing the leaner variable-interval schedule so that it approached the richer variable-interval schedule decreased signal detection's measure of discrimination but left its measure of response bias and the matching law measures unchanged. Data collected only until a subject's first changeover response following presentation of a long or a short tone showed higher values for both methods' measures of discrimination, no change in signal detection's measure of response bias, and lower values for the matching law's measure of response bias. Relationships between the matching law's and signal detection's methods of analyzing choice are discussed. It is concluded that a signal detection analysis is more efficient for examining changes in the difficulty of a discrimination, whereas a matching law analysis is more effective for examining the effects of changes in relative reinforcer frequency.     

Determinants of pigeons' waiting time: Effects of interreinforcement interval and food delay

Four pigeons performed on three types of schedules at short (i.e., 10, 30, or 60 s) interreinforcement intervals: (a) a delay-dependent schedule where interreinforcement interval was held constant (i.e., increases in waiting time decreased food delay), (b) an interreinforcement-interval-dependent schedule where food delay was held constant (i.e., increases in waiting time increased interreinforcement interval), and (c) a both-dependent schedule where increases in waiting time produced increases in interreinforcement interval but decreases in food delay. Waiting times were typically longer under the delay-dependent schedules than under the interreinforcement-interval-dependent schedules. Those under both-dependent schedules for 1 subject were intermediate between those under the other two schedule types, whereas for the other subjects waiting times under the both-dependent procedure were similar either to those under the delay-dependent schedule or to those under the interreinforcement-interval-dependent schedule, depending both on the subject and the interreinforcement interval. These results indicate that neither the interreinforcement interval nor food delay is the primary variable controlling waiting time, but rather that the two interact in a complex manner to determine waiting times.     

Second-order schedules: discrimination of components

Pigeons were exposed to a series of second-order schedules in which the completion of a fixed number of fixed-interval components produced food. In Experiment 1, brief (2 sec) stimulus presentations occurred as each fixed-interval component was completed. During the brief-stimulus presentation terminating the last fixed-interval component, a response was required on a second key, the brief-stimulus key, to produce food. Responses on the brief-stimulus key before the last brief-stimulus presentation had no scheduled consequences, but served as a measure of the extent to which the final component was discriminated from preceding components. Whether there were one, two, four, or eight fixed-interval components, responses on the brief-stimulus key occurred during virtually every brief-stimulus presentation. In Experiment 2, an attempt was made to punish unnecessary responses on the brief-stimulus key, i.e., responses on the brief-stimulus key that occurred before the last component. None of the pigeons learned to withhold these responses, even though they produced a 15-sec timeout and loss of primary reinforcement. In Experiment 3, different key colors were associated with each component of a second-order schedule (a chain schedule). In contrast to Experiment 1, brief-stimulus key responses were confined to the last component. It was concluded that pigeons do not discriminate well between components of second-order schedules unless a unique exteroceptive cue is provided for each component. The relative discriminability of the components may account for the observed differences in initial-component response rates between comparable brief-stimulus, tandem, and chain schedules.     

Concurrent schedules: Effects of time- and response-allocation constraints

Five pigeons were trained on concurrent variable-interval schedules arranged on two keys. In Part 1 of the experiment, the subjects responded under no constraints, and the ratios of reinforcers obtainable were varied over five levels. In Part 2, the conditions of the experiment were changed such that the time spent responding on the left key before a subsequent changeover to the right key determined the minimum time that must be spent responding on the right key before a changeover to the left key could occur. When the left key provided a higher reinforcer rate than the right key, this procedure ensured that the time allocated to the two keys was approximately equal. The data showed that such a time-allocation constraint only marginally constrained response allocation. In Part 3, the numbers of responses emitted on the left key before a changeover to the right key determined the minimum number of responses that had to be emitted on the right key before a changeover to the left key could occur. This response constraint completely constrained time allocation. These data are consistent with the view that response allocation is a fundamental process (and time allocation a derivative process), or that response and time allocation are independently controlled, in concurrent-schedule performance.     

Performances on ratio and interval schedules of reinforcement: Data and theory

Two differences between ratio and interval performance are well known: (a) Higher rates occur on ratio schedules, and (b) ratio schedules are unable to maintain responding at low rates of reinforcement (ratio “strain”). A third phenomenon, a downturn in response rate at the highest rates of reinforcement, is well documented for ratio schedules and is predicted for interval schedules. Pigeons were exposed to multiple variable-ratio variable-interval schedules in which the intervals generated in the variable-ratio component were programmed in the variable-interval component, thereby “yoking” or approximately matching reinforcement in the two components. The full range of ratio performances was studied, from strained to continuous reinforcement. In addition to the expected phenomena, a new phenomenon was observed: an upturn in variable-interval response rate in the midrange of rates of reinforcement that brought response rates on the two schedules to equality before the downturn at the highest rates of reinforcement. When the average response rate was corrected by eliminating pausing after reinforcement, the downturn in response rate vanished, leaving a strictly monotonic performance curve. This apparent functional independence of the postreinforcement pause and the qualitative shift in response implied by the upturn in variable-interval response rate suggest that theoretical accounts will require thinking of behavior as partitioned among at least three categories, and probably four: postreinforcement activity, other unprogrammed activity, ratio-typical operant behavior, and interval-typical operant behavior.     

Stochastic choice models: A comparison between Bush-Mosteller and a source-independent reward-following model

Horner and Staddon (1987) argued that a class of reward-following processes defined by a property they termed ratio invariance is a better model for the probabilistic choice performance of pigeons than competing molecular accounts such as momentary maximizing, melioration, and the Bush-Mosteller model. The critical data were provided by choice distributions-distributions of a variable S, the proportion of Right choices, defined on a moving window typically 32 choices long-obtained under a frequency-dependent schedule. The schedule prescribed equal payoff probabilities, p(S), for both choices. p(S) was a maximum when S = 0.5 and declined linearly for S values above and below 0.5. Pigeons showed generally bimodal choice distributions with the modes at equal p(S) values. These data do not follow easily from melioration or momentary maximizing and are inconsistent with molar maximizing, but they may be consistent with Bush-Mosteller. We present here the results of computer simulations showing that the ratio-invariance model studied yields, as expected, choice modes at equal p(S) values, but that Bush-Mosteller, although capable of generating bimodal choice distributions, does not have choice modes at equal p(S) values.     

More on concurrent interval-ratio schedules: a replication and review.

It has been suggested that the failure to maximize reinforcement on concurrent variable-interval, variable-ratio schedules may be misleading. Inasmuch as response costs are not directly measured, it is possible that subjects are optimally balancing the benefits of reinforcement against the costs of responding. To evaluate this hypothesis, pigeons were tested in a procedure in which interval and ratio schedules had equal response costs. On a concurrent variable time (VT), variable ratio-time (VRT) schedule, the VT schedule runs throughout the session and the VRT schedule is controlled by responses to a changeover key that switches from one schedule to the other. Reinforcement is presented independent of response. This schedule retains the essential features of concurrent VI VR, but eliminates differential response costs for the two alternatives. It therefore also eliminates at least one significant ambiguity about the reinforcement maximizing performance. Pigeons did not maximize rate of reinforcement on this procedure. Instead, their times spent on the alternative schedules matched the relative rates of reinforcement, even when schedule parameters were such that matching earned the lowest possible overall rate of reinforcement. It was further shown that the observed matching was not a procedural artifact arising from the constraints built into the schedule.     

The effects of morphine on the production and discrimination of interresponse times

Recent experiments suggest that the effects of drugs of abuse on the discrimination of the passage of time may differ for experimenter-imposed and subject-produced events. The current experiment examined this suggestion by determining the effects of morphine on the discrimination of interresponse times (IRTs). Pigeons pecked a center key on a random-interval 20-s schedule of matching-to-sample trials. Once the interval had timed out, a choice trial randomly followed either a short (2- to 3-s) or long (6- to 9-s) IRT on the center key. Pecking the side key lit one color produced food after a short IRT, and pecking the side key lit the other color produced food after a long IRT. Two experimental phases differed in the functional role of the different key colors. Under control conditions, the IRT distributions had two modes, one at the lower bound of the short category and a smaller one at the lower bound of the long category. Pigeons accurately categorized the duration of the IRTs: One key color was pecked following short IRTs and the other key color was pecked following long IRTs. Morphine flattened the IRT distribution and reduced the accuracy of categorizing IRTs. Categorization of long IRTs was particularly disrupted. Morphine did not produce overestimation of time as assessed by the production or categorization of IRTs. These results are similar to those obtained previously for the effects of morphine on the discrimination of the duration of experimenter-imposed events.     

Dynamics of time discrimination: II. The effects of multiple impulses.

According to a diffusion generalization model, time discrimination is determined by the frequency and recency of preceding intervals of time. A procedure for studying rapid timing was used to investigate whether pigeons' wait-time responses were sensitive to these factors. In Experiment 1 the number (two or eight) and spacing (consecutive or far apart) of 5-s interfood intervals (called impulses) intercalated in a series of 15-s interfood intervals (nonimpulses) were studied. Experiment 2 was identical to the first but the interfood intervals were increased by a factor of three. Overall, impulses shortened wait times in the next interfood interval. However, several impulses occurring in succession extended the localized effect of an impulse: Wait times following a set of eight-close impulses were slow to recover to preimpulse levels. The results show that linear waiting is only an approximation to the dynamic process, and a process that is sensitive to events in an animal's remote past, such as the diffusion generalization model, provides a better account of rapid timing effects.     

Reinforcement and punishment effects in concurrent schedules: A test of two models

The joint effects of punishment and reinforcement on the pigeon's key-peck response were examined in three choice experiments conducted to compare predictions of Farley and Fantino's (1978) subtractive model with those made by Deluty's (1976) and Deluty and Church's (1978) model of punishment. In Experiment 1, the addition of equal punishment schedules to both alternatives of a concurrent reinforcement schedule enhanced the preference exhibited for the more frequent reinforcement alternative. Experiment 2 demonstrated decreases in the absolute response rate for each member of a concurrent reinforcement schedule when increasing frequencies of punishment were added to each alternative. Experiment 3 found that preference for the denser of two reinforcement schedules diminished when the absolute frequencies of reinforcement were increased by a constant factor and conditions of punishment for both alternatives were held constant. Diminished preferences were obtained regardless of whether the frequency of punishment associated with the denser reinforcement schedule was greater or less than that associated with the lean reinforcement alternative. The results from all three experiments uniquely supported Farley and Fantino's (1978) subtractive model of punishment and reinforcement.     

The effect of foreperiod duration on reaction time and its relation to interval schedules of reinforcement

Two groups of pigeons were exposed to a simple reaction-time procedure in which mean foreperiod duration was 5, 10, or 20 seconds. For one group, the foreperiods had an arithmetic, or rectangular, distribution; for the second group, they had a constant-probability, or Bernoulli, distribution. Under both distributions, mean response latency was an increasing, negatively accelerated function of mean foreperiod duration. On a given trial, response latency was a function of its associated foreperiod duration: latency was a decreasing function of foreperiod duration in the arithmetic distribution, and an increasing function of foreperiod duration in the constant-probability distribution. Examination of the distribution of latencies revealed a harmonic structure reminiscent of distributions of interresponse times under variable-interval schedules of reinforcement. Taken together, the results confirm and extend previous findings with human subjects, and also suggest numerous similarities to behavior maintained by variable-interval schedules.     

Concurrent schedules: Effects of reinforcement rate and changeover delay on time allocation in a three-alternative procedure

Pigeons were exposed to a continuous choice procedure where three alternatives alternated in a fixed, recycling order (ABCABC, etc.). Responses were reinforced according to independent variable-interval schedules. For three birds, the reinforcement rate for responses on alternative C was varied. For three other birds, the duration of the changeover delay after the changeover to C was varied. For both groups, the reinforcement rates and changeover delay durations associated with A and B were constant throughout the experiment. The time proportion at A relative to B increased as a function of the reinforcement rate for responses on C and decreased as a function of the duration of the changeover delay during C. The results show that the proportion of time spent at a variable-interval alternative of a continuous choice procedure is not completely determined by the reinforcement rates provided by the alternatives. The results support the assumption that time allocation is governed by delayed reinforcement of changeover behaviour.     

The effects of morphine on fixed-interval patterning and temporal discrimination

Changes produced by drugs in response patterns under fixed-interval schedules of reinforcement have been interpreted to result from changes in temporal discrimination. To examine this possibility, this experiment determined the effects of morphine on the response patterning of 4 pigeons during a fixed-interval 1-min schedule of food delivery with interpolated temporal discrimination trials. Twenty of the 50 total intervals were interrupted by choice trials. Pecks to one key color produced food if the interval was interrupted after a short time (after 2 or 4.64 s). Pecks to another key color produced food if the interval was interrupted after a long time (after 24.99 or 58 s). Morphine (1.0 to 10.0 mg/kg) decreased the index of curvature (a measure of response patterning) during fixed intervals and accuracy during temporal discrimination trials. Accuracy was equally disrupted following short and long sample durations. Although morphine disrupted temporal discrimination in the context of a fixed-interval schedule, these effects are inconsistent with interpretations of the disruption of response patterning as a selective overestimation of elapsed time. The effects of morphine may be related to the effects of more conventional external stimuli on response patterning.     

Temporal constraint on choice: Sensitivity and bias in multiple schedules

In multiple schedules of reinforcement, ratios of responses in successive components are relatively insensitive to ratios of obtained reinforcers. An analysis is proposed that attributes changes in absolute response rates to concurrent interactions between programmed reinforcement and extraneous reinforcement in other components. The analysis predicts that ratios of responses in successive components vary with reinforcer ratios, qualified by a term describing the reinforcement context, that is, programmed and extraneous reinforcers. Two main predictions from the analysis were confirmed in an experiment in which pigeons' responses were reinforced in the components of a multiple schedule and analog extraneous reinforcement was scheduled for an alternative response in each component. Sensitivity of response and time ratios to reinforcer ratios in the multiple schedules varied as a function of the rate of extraneous reinforcers. Bias towards responding in one component of the multiple schedule varied as an inverse function of the ratios of extraneous reinforcer rate in the two components. The data from this and previous studies of multiple-concurrent performance were accurately predicted by our analysis and supported our contention that the allocation of behavior in multiple-schedule components depends on the relative values of concurrently-available reinforcers within each component.     

Travel time and concurrent-schedule choice: retrospective versus prospective control

Six pigeons were trained on concurrent variable-interval schedules in which two different travel times between alternatives, 4.5 and 0.5 s, were randomly arranged. In Part 1, the next travel time was signaled while the subjects were responding on each alternative. Generalized matching analyses of performance in the presence of the two travel-time signals showed significantly higher response and time sensitivity when the longer travel time was signaled compared to when the shorter time was signaled. When the data were analyzed as a function of the previous travel time, there were no differences in sensitivity. Dwell times on the alternatives were consistently longer in the presence of the stimulus that signaled the longer travel time than they were in the presence of the stimulus that signaled the shorter travel time. These results are in accord with a recent quantitative account of the effects of travel time. In Part 2, no signals indicating the next travel time were given. When these data were analyzed as a function of the previous travel time, time-allocation sensitivity after the 4.5-s travel time was significantly greater than that after the 0.5-s travel time, but no such difference was found for response allocation. Dwell times were also longer when the previous travel time had been longer.     

Why pigeons say what they do: reinforcer magnitude and response requirement effects on say responding in say-do correspondence

The effects of reinforcer magnitude and response requirement on pigeons' say choices in an experimental homologue of human say-do correspondence were assessed in two experiments. The procedure was similar to a conditional discrimination procedure except the pigeons chose both a sample stimulus (the say component) and a comparison stimulus that corresponded to it (the do component). Correspondence was trained on red, green, and white key colors before the duration of food presentations following correspondence on each key color (Experiment 1) and the number of key pecks required as the say response on each key color (Experiment 2) were manipulated in an attempt to influence the initial say response. The frequency of say responses on each key color coincided with programmed changes in the duration of food presentations and the key-peck requirements assigned to each key color. Correspondence accuracy remained stable in all conditions, even those in which the say responding occurred primarily on two of the three key colors. Implications for human behavior are discussed.     

Being there on time: Reinforcer effects on timing and locating

In research on timing, reinforcers often are assumed to influence discrimination of elapsed time. We asked whether changes in choice used to measure timing arise because of joint control by elapsed time and reinforcers, rather than from the direct modification of control by elapsed time by reinforcers. Pigeons worked on a concurrent-choice task in which 1 response was 9 times more likely to produce a reinforcer, reversing between locations when 19 s had elapsed since the marker event. Across conditions, we manipulated the percentage of reinforcers arranged before the probability reversal from 5 to 95%. These changes in reinforcer percentages altered control by location-based elements of the contingency, but not by time-based elements. Choice was well described by a model that assumes that control by the contingency is weakened by generalization across the time and location of reinforcers, and that these generalizations become more likely at later times since a marker. These findings add to a growing body of research that suggests that reinforcers share the same function as other environmental events in determining how the environment controls behavior.     

About Skinner and time: behavior-analytic contributions to research on animal timing

The article discusses two important influences of B. F. Skinner, and later workers in the behavior-analytic tradition, on the study of animal timing. The first influence is methodological, and is traced from the invention of schedules imposing temporal constraints or periodicities on animals in The Behavior of Organisms, through the rate differentiation procedures of Schedules of Reinforcement, to modern temporal psychophysics in animals. The second influence has been the development of accounts of animal timing that have tried to avoid reference to internal processes of a cognitive sort, in particular internal clock mechanisms. Skinner's early discussion of temporal control is first reviewed, and then three recent theories-Killeen & Fetterman's (1988) Behavioral Theory of Timing; Machado's (1997) Learning to Time; and Dragoi, Staddon, Palmer, & Buhusi's (2003) Adaptive Timer Model-are discussed and evaluated.