Children's choice: Sensitivity to changes in reinforcer density

Two experiments were carried out in which children's sensitivity to changes in reinforcer density (number of reinforcers per session) was measured in a choice paradigm. In Experiment 1, 24 girls (ages 6, 9, and 12 years) performed on concurrent-chain schedules of reinforcement. The initial links were variable-interval 10-s schedules. One terminal link always gave three tokens after 30 s, but the parameters associated with the other were varied. Independent manipulations of reinforcer size (two tokens or four tokens) and prereinforcement delay (25 s or 65 s) led to equal changes in the relative density of tokens that could be earned on the schedules. Subjects at all ages were sensitive to changes in reinforcer density brought about by changes in reinforcer size, whereas only 3 12-year-olds showed sensitivity to the changes brought about by manipulation of prereinforcer delay. In Experiment 2, titration procedures were used to test the extent of this insensitivity to delay in 32 6- and 12-year-old children. In these procedures, a repeated choice of the large reinforcer increased the delay to its delivery, and a repeated choice of the small reinforcer reduced the delay to the delivery of the large reinforcer. Whereas 6-year-old boys and girls tended to maintain a strong preference for the large reinforcer, so increasing the delay to its delivery, 12-year-olds tended to distribute their responses to both alternatives, thus producing a stable level of delay to the large reinforcer. The results from the two experiments support the idea of two stages in the development of adaptive intertemporal choice.     

A quantitative analysis of chain-schedule performance

Six pigeons were trained with a chain variable-interval variable-interval schedule on the left key and with reinforcers available on the right key on a single variable-interval schedule arranged concurrently with both links of the chain. All three schedules were separately and systematically varied over a wide range of mean intervals. During these manipulations, the obtained reinforcer rates on constant arranged schedules also frequently changed systematically. Increasing reinforcer rates in Link 2 of the chain increased response rates in both links and decreased response rates in the variable-interval schedule concurrently available with Link 2. Increasing Link-1 reinforcer rates increased Link-1 response rates and decreased Link-2 response rates. Increasing reinforcer rates on the right-key schedule decreased response rates in Link 1 of the chain but did not affect the rate in Link 2. The results extend and amplify previous analyses of chain-schedule performance and help define the effects that a quantitative model must describe. However, the complexity of the results, and the fact that constant arranged reinforcer schedules did not necessarily lead to constant obtained reinforcer rates, precluded a quantitative analysis.     

Connectionist models of conditioning: A tutorial

Models containing networks of neuron-like units have become increasingly prominent in the study of both cognitive psychology and artificial intelligence. This article describes the basic features of connectionist models and provides an illustrative application to compound-stimulus effects in respondent conditioning. Connectionist models designed specifically for operant conditioning are not yet widely available, but some current learning algorithms for machine learning indicate that such models are feasible. Conversely, designers for machine learning appear to have recognized the value of behavioral principles in producing adaptive behavior in their creations.     

Probability and delay of reinforcement as factors in discrete-trial choice.

Pigeons chose between two alternatives that differed in the probability of reinforcement and the delay to reinforcement. A peck on the red key always produced a delay of 5 s and then a possible reinforcer. The probability of reinforcement for responding on this key varied from .05 to 1.0 in different conditions. A response on the green key produced a delay of adjustable duration and then a possible reinforcer, with the probability of reinforcement ranging from .25 to 1.0 in different conditions. The green-key delay was increased or decreased many times per session, depending on a subject's previous choices. The purpose of these adjustments was to estimate an indifference point, or a delay that resulted in a subject's choosing each alternative about equally often. In conditions where the probability of reinforcement was five times higher on the green key, the green-key delay averaged about 12 s at the indifference point. In conditions where the probability of reinforcement was twice as high on the green key, the green-key delay at the indifference point was about 8 s with high probabilities and about 6 s with low probabilities. An analysis based on these results and those from studies on delay of reinforcement suggests that pigeons' choices are relatively insensitive to variations in the probability of reinforcement between .2 and 1.0, but quite sensitive to variations in probability between .2 and 0.     

Determination of a behavioral transfer function: White-noise analysis of session-to-session response-ratio dynamics on concurrent VI VI schedules

Six pigeons were exposed to concurrent variable-interval schedules in which the programmed reinforcer ratios changed from session to session according to a pseudorandom binary sequence. This procedure corresponded to the stochastic identification paradigm (“white-noise experiment”) of systems theory and enabled the relation between log response ratios in the current session and log reinforcer ratios in all previous sessions to be determined. Such dynamic relations are called linear transfer functions. Both nonparametric and parametric representations of these, in the form of “impulse-response functions,” were determined for each bird. The session-to-session response ratios resulting from the session-to-session pseudorandom binary variations in reinforcer ratios were well predicted by the impulse-response functions identified for each pigeon. The impulse-response functions were well fitted by a second-order dynamic model involving only two parameters: a time constant and a gain. The mean time constant was 0.67 sessions, implying that the effects of abrupt changes in log reinforcer ratios should be 96% complete within about five sessions. The mean gain was 0.53, which was surprisingly low inasmuch as it should equal the sensitivity to reinforcement ratio observed under steady-state conditions. The same six pigeons were subjected to a similar experiment 10 months following the first. Despite individual differences in impulse-response functions between birds within each experiment, the impulse-response functions determined from the two experiments were essentially the same.     

Effects of ratio reinforcement schedules on discrimination performance by Japanese monkeys

In Experiment 1, Japanese monkeys were trained on three conditional position-discrimination problems with colors as the conditional cues. Within each session, each problem was presented for two blocks of ten reinforcements; correct responses were reinforced under continuous-reinforcement, fixed-ratio 5, and variable-ratio 5 schedules, each assigned to one of the three problems. The assignment of schedules to problems was rotated a total of three times (15 sessions per assignment) after 30 sessions of acquisition training. Accuracy of discrimination increased to a moderate level with fewer trials under CRF than under ratio schedules. In contrast, the two ratio schedules, fixed and variable, were more effective in maintaining accurate discrimination than was CRF. With further training, as asymptotes were reached, accuracy was less affected by the schedule differences. These results demonstrated an interaction between the effects of reinforcement schedules and the level of acquisition. In Experiment 2, ratio sizes were gradually increased to 30. Discrimination accuracy was maintained until the ratio reached 20; ratio 30 strained the performance. Under FR conditions, accuracy increased as correct choice responses cumulated after reinforcement.     

Handling time and choice in pigeons

According to optimal foraging theory, animals should prefer food items with the highest ratios of energy intake to handling time. When single items have negligible handling times, one large item should be preferred to a collection of small ones of equivalent total weight. However, when pigeons were offered such a choice on equal concurrent variable-interval schedules in a shuttlebox, they preferred the side offering many small items per reinforcement to that offering one or a few relatively large items. This preference was still evident on concurrent fixed-cumulative-duration schedules in which choosing the alternative with longer handling time substantially lowered the rate of food intake.     

Temporal proximity and reinforcement sensitivity in multiple schedules

Distributions of reinforcers between two components of multiple variable-interval schedules were varied over a number of conditions. Sensitivity to reinforcement, measured by the exponent of the power function relating ratios of responses in the two components to ratios of reinforcers obtained in the components, did not differ between conditions with 15-s or 60-s component durations. The failure to demonstrate the “short-component effect,” where sensitivity is high for short components, was consistent with reanalysis of previous data. With 60-s components, sensitivity to reinforcement decreased systematically with time since component alternation, and was higher in the first 15-s subinterval of the 60-s component than for the component whose total duration was 15 s. Varying component duration and sampling behavior at different times since component transition may not be equivalent ways of examining the effects of average temporal distance between components.     

Concurrent access to two concentrations of orally delivered phencyclidine: effects of feeding conditions.

Two experiments addressed the effects of food satiation and deprivation on oral self-administration of two concurrently available phencyclidine concentrations. In the first experiment, 8 rhesus monkeys self-administered either of two concentrations of phencyclidine ("PCP, angel dust") and water under concurrent fixed-ratio 16 schedules. One concentration was always held constant (0.25 mg/mL) while a series of other phencyclidine concentrations, ranging from 0 (water) to 1.0 mg/mL, was presented in a nonsystematic order. Initially the monkeys were tested while food satiated, and the procedure was then repeated during food deprivation. The monkeys usually selected the higher concentration within the first few minutes of the session, indicating that taste and/or other immediate postingestional effects were important factors. Contrary to a number of previous reports, there were no consistent differences across subjects in the mean number of liquid deliveries or mean drug intake (mg/kg) during food satiation and deprivation. However, for all monkeys the within-session time course of responding during food satiation consistently differed from that during deprivation. A second experiment assessed whether the failure to find consistent differences in drug intake during food satiation and deprivation had been due to the history of concurrent access to different phencyclidine concentrations or to the extended experience with phencyclidine under food-satiation conditions. Six additional monkeys (Group 2) were exposed to the phencyclidine self-administration procedure (during food satiation and deprivation) for the same length of time as the monkeys in Experiment 1 (Group 1), except they received only concurrent access to phencyclidine (0.25 mg/mL) and water. Both groups then received concurrent access to phencyclidine and water during five repeated cycles of food deprivation and satiation. There were also marked individual differences in Group 2: During food satiation, 2 of the monkeys' responding increased, 1 showed no change, and 3 decreased. Examination of a number of historical variables indicated that the greater the percentage of total sessions spent during food satiation with phencyclidine available (before these experiments began), the greater the amounts of phencyclidine consumed during food satiation and the smaller the differences in phencyclidine intake when the two feeding conditions were compared.