Choice in a variable environment: effects of blackout duration and extinction between components

Pigeons were trained in a procedure in which sessions included seven four- or 10-reinforcer components, each providing a different reinforcer ratio that ranged from 27:1 to 1:27. The components were arranged in random order, and no signals differentiated the component reinforcer ratios. Each condition lasted 50 sessions, and the data from the last 35 sessions were analyzed. Previous results using 10-s blackouts between components showed some carryover of preference from one component to the next, and this effect was investigated in Experiment 1 by varying blackout duration from 1 s to 120 s. The amount of carryover decreased monotonically as the blackout duration was lengthened. Preference also decreased between reinforcers within components, suggesting that preference change during blackout might follow the same function as preference change between reinforcers. Experiment 2 was designed to measure preference change between components more directly and to relate this to preference change during blackout. In two conditions a 60-s blackout occurred between components, and in two other conditions a 60-s period of unsignaled extinction occurred between components. Preference during the extinction period progressively fell toward indifference, and the level of preference following extinction was much the same as that following blackout. Although these results are consistent with Davison and Baum's (2000) theory of the effects of reinforcers on local preference, other findings suggest that theory is incomplete: After a sequence of reinforcers from one alternative, some residual preference remained after 60 s of extinction or blackout, indicating the possibility of an additional longer term accumulation of reinforcer effects than originally suggested.     

Drug discrimination under concurrent variable-ratio variable-ratio schedules

Pigeons were trained to discriminate 5 mg/kg pentobarbital from saline under concurrent variable-ratio (VR) VR schedules, in which responses on the pentobarbital-biased lever were reinforced under the VR schedule with the smaller response requirements when pentobarbital was given before the session, and responses on the saline-biased key were reinforced under the VR schedule with the larger response requirements. When saline was administered before the session, the reinforcement contingencies associated with the two response keys were reversed. When responding stabilized under concurrent VR 20 VR 30, concurrent VR 10 VR 40, or concurrent VR 5 VR 50 schedules, pigeons responded almost exclusively on the key on which fewer responses were required to produce the reinforcer. When other doses of pentobarbital and other drugs were substituted for the training dose, low doses of all drugs produced responding on the saline-biased key. Higher doses of pentobarbital and chlordiazepoxide produced responding only on the pentobarbital-biased key, whereas higher doses of ethanol and phencyclidine produced responding only on this key less often. d-Amphetamine produced responding primarily on the saline-biased key. When drugs generalized to pentobarbital, the shape of the generalization curve under concurrent VR VR schedules was more often graded than quantal in shape. Thus, drug discrimination can be established under concurrent VR VR schedules, but the shapes of drug-discrimination dose-response curves under concurrent VR VR schedules more closely resemble those seen under interval schedules than those seen under fixed-ratio schedules. Graded dose-response curves under concurrent VR VR schedules may relate to probability matching and difficulty in discriminating differences in reinforcement frequency.     

A tuned-trace theory of interval-timing dynamics

Animals on interval schedules of reinforcement can rapidly adjust a temporal dependent variable, such as wait time, to changes in the prevailing interreinforcement interval. We describe data on the effects of impulse, step, sine-cyclic, and variable-interval schedules and show that they can be explained by a tuned-trace timing model with a one-back threshold-setting rule. The model can also explain steady-state timing properties such as proportional and Weber law timing and the effects of reinforcement magnitude. The model assumes that food reinforcers and other time markers have a decaying effect (trace) with properties that can be derived from the rate-sensitive property of habituation (the multiple-time-scale model). In timing experiments, response threshold is determined by the trace value at the time of the most recent reinforcement. The model provides a partial account for the learning of multiple intervals, but does not account for scalloping and other postpause features of responding on interval schedules and has some problems with square-wave schedules.     

Interval length and time-use by children with AD/HD: a comparison of four models

Predictions made by 4 competing models of time use by children with AD/HD were tested using a computerized version of the Matching Familiar Figures Test in 2 studies. In Study 1 teacher-identified AD/HD children (N = 10) and non-AD/HD controls (N = 10) completed the task under 3 different trial duration conditions (5, 10, and 15 s). In Study 2 the same task was completed by a group of children with a diagnosis of Hyperkinetic Disorder (N = 12) and controls (N = 12). In both studies AD/HD children performed poorly on trials of both 5- and 15-s duration but as well as controls on the 10-s trials. This quadratic function provided support for the State Regulation Deficit model of time use in AD/HD. The value of tailoring the temporal features of learning contexts to the conceptual style of AD/HD children is discussed.     

Preference and resistance to change with constant-duration schedule components

Previous research on preference between variable-interval terminal links in concurrent chains has most often used variable-duration terminal links ending with a single reinforcer. By contrast, most research on resistance to change in multiple schedules has used constant-duration components that include variable numbers of reinforcers in each presentation. Grace and Nevin (1997) examined both preference and resistance in variable-duration components; here, preference and resistance were examined in constant-duration components. Reinforcer rates were varied across eight conditions, and a generalized-matching-law analysis showed that initial-link preference strongly over-matched terminal-link reinforcer ratios. In multiple schedules, baseline response rates were unaffected by reinforcer rates, but resistance to intercomponent food, to extinction, and to intercomponent food plus extinction was greater in the richer component. The between-component difference in resistance to change exhibited additive effects for the three resistance tests, and was systematically related to reinforcer ratios. However, resistance was less sensitive to reinforcer ratios than was preference. Resistance to intercomponent food and to intercomponent food plus extinction was more sensitive to reinforcer ratios in the present study than in Grace and Nevin (1997). Thus, relative to variable-duration components, constant-duration components increased the sensitivity of both preference and relative resistance, supporting the proposition that these are independent and convergent measures of the effects of a history of reinforcement.     

Timeout postponement without increased reinforcement frequency

Three experiments were conducted to examine pigeons' postponement of signaled extinction periods (timeouts) from a schedule of food reinforcement when such responding neither decreased overall timeout frequency nor increased the overall frequency of food reinforcement. A discrete-trial procedure was used in which a response during the first 5 s of a trial postponed an otherwise immediate 60-s timeout to a later part of that same trial but had no effect on whether the timeout occurred. During time-in periods, responses on a second key produced food according to a random-interval 20-s schedule. In Experiment 1, the response-timeout interval was 45 s under postponement conditions and 0 s under extinction conditions (responses were ineffective in postponing timeouts). The percentage of trials with a response was consistently high when the timeout-postponement contingency was in effect and decreased to low levels when it was discontinued under extinction conditions. In Experiment 2, the response-timeout interval was also 45 s but postponement responses increased the duration of the timeout, which varied from 60 s to 105 s across conditions. Postponement responding was maintained, generally at high levels, at all timeout durations, despite sometimes large decreases in the overall frequency of food reinforcement. In Experiment 3, timeout duration was held constant at 60 s while the response-timeout interval was varied systematically across conditions from 0 s to 45 s. Postponement responding was maintained under all conditions in which the response-timeout interval exceeded 0 s (the timeout interval in the absence of a response). In some conditions of Experiment 3, which were designed to control for the immediacy of food reinforcement and food-correlated (time-in) stimuli, responding postponed timeout but the timeout was delayed whether a response occurred or not. Responding was maintained for 2 of 3 subjects, suggesting that behavior was negatively reinforced by timeout postponement rather than positively reinforced by the more immediate presentation of food or food-correlated (time-in) stimuli.     

Use of a lag differential reinforcement contingency to increase varied selections of classroom activities

The present study evaluated the effects of a lag differential reinforcement contingency on 2 students' activity selections using reversal designs. Results showed that the lag contingency was responsible for promoting increased novel selections, engagement in diverse activities, and greater progress with respect to programmed academic activities.     

Concurrent second-order schedules: some effects of variations in response number and duration

To examine the effects on concurrent performance of independent manipulations of response-unit duration and number, 6 hens were exposed to concurrent second-order schedules of reinforcement. Each first-order operant unit required completion of a fixed-ratio schedule within the time specified by a fixed-interval schedule, with one further response completing the fixed-interval schedule. The fixed-ratio and fixed-interval requirements comprising the first-order operant units were systematically and independently varied under three pairs of concurrent variable-interval schedules to produce differences in the first-order response and duration requirements (response and duration differentials). These manipulations produced consistent changes in response, time, and operant-unit biases. A 1:4 response differential biased the time and operant-unit measures towards the smaller fixed ratio, but to a degree less than the imposed response differential. The response-based biases favored the larger fixed ratio. Duration differentials of 4:1 and 8:1 biased the response and operant-unit measures towards the shorter fixed interval, again less than the imposed duration differential, but the time biases remained close to zero. Both sorts of differentials acted to bias operant-unit completions more systematically than the other measures, but undermatching to the differentials occurred. The undermatching appears to have arisen from a pattern of fix and sample (in which visits to the less preferred alternative involved only a single completed operant unit) under combinations of unequal operant-unit requirements and reinforcer rates. The response and time bias measures appeared to arise as by-products of the changes in operant-unit completions.     

Variation, repetition, and choice

Experiment 1 investigated the controlling properties of variability contingencies on choice between repeated and variable responding. Pigeons were exposed to concurrent-chains schedules with two alternatives. In the REPEAT alternative, reinforcers in the terminal link depended on a single sequence of four responses. In the VARY alternative, a response sequence in the terminal link was reinforced only if it differed from the n previous sequences (lag criterion). The REPEAT contingency generated low, constant levels of sequence variation whereas the VARY contingency produced levels of sequence variation that increased with the lag criterion. Preference for the REPEAT alternative tended to increase directly with the degree of variation required for reinforcement. Experiment 2 examined the potential confounding effects in Experiment 1 of immediacy of reinforcement by yoking the interreinforcer intervals in the REPEAT alternative to those in the VARY alternative. Again, preference for REPEAT was a function of the lag criterion. Choice between varying and repeating behavior is discussed with respect to obtained behavioral variability, probability of reinforcement, delay of reinforcement, and switching within a sequence.