Optimal choice

We present a classification and theoretical analysis of discrete-trial and free-operant choice procedures in which reinforcement is assigned to one alternative only, or independently to both, is either always available or conditionally available, and is either "held" or not from trial to trial. Momentary-maximizing and (globally) optimal choice sequences are defined in terms of initializing and marker events. Free-operant choice is analyzed in terms of a clock space whose axes are the times since the last A and B choices. The analysis shows that most molar matching data are derivable from momentary maximizing, and that the momentary-maximizing hypothesis has not been adequately tested in either discrete-trial or free-operant situations.     

Eccentric stimuli on multiple fixed-interval schedules

The effects of presenting a different (“eccentric”) stimulus for one interval during either or both components of a cyclic multiple fixed-interval fixed-interval schedule, with 12 short and four long intervals per cycle, were studied in three experiments. Eccentric stimuli in the short-interval component reliably produced a persistent, substantial elevation in key-peck rate. The effect appears to depend on schedule context and an initial “disinhibiting” effect of the eccentric stimulus.     

Dynamics of waiting in pigeons

Two experiments used response-initiated delay schedules to test the idea that when food reinforcement is available at regular intervals, the time an animal waits before its first operant response (waiting time) is proportional to the immediately preceding interfood interval (linear waiting; Wynne & Staddon, 1988). In Experiment 1 the interfood intervals varied from cycle to cycle according to one of four sinusoidal sequences with different amounts of added noise. Waiting times tracked the input cycle in a way which showed that they were affected by interfood intervals earlier than the immediately preceding one. In Experiment 2 different patterns of long and short interfood intervals were presented, and the results implied that waiting times are disproportionately influenced by the shortest of recent interfood intervals. A model based on this idea is shown to account for a wide range of results on the dynamics of timing behavior.     

Stochastic choice models: A comparison between Bush-Mosteller and a source-independent reward-following model

Horner and Staddon (1987) argued that a class of reward-following processes defined by a property they termed ratio invariance is a better model for the probabilistic choice performance of pigeons than competing molecular accounts such as momentary maximizing, melioration, and the Bush-Mosteller model. The critical data were provided by choice distributions-distributions of a variable S, the proportion of Right choices, defined on a moving window typically 32 choices long-obtained under a frequency-dependent schedule. The schedule prescribed equal payoff probabilities, p(S), for both choices. p(S) was a maximum when S = 0.5 and declined linearly for S values above and below 0.5. Pigeons showed generally bimodal choice distributions with the modes at equal p(S) values. These data do not follow easily from melioration or momentary maximizing and are inconsistent with molar maximizing, but they may be consistent with Bush-Mosteller. We present here the results of computer simulations showing that the ratio-invariance model studied yields, as expected, choice modes at equal p(S) values, but that Bush-Mosteller, although capable of generating bimodal choice distributions, does not have choice modes at equal p(S) values.     

Quasi-dynamic choice models: Melioration and ratio invariance

There is continuing controversy about the behavioral process or processes that underlie the major regularities of free-operant choice such as molar matching and systematic deviations therefrom. A recent interchange between Vaughan and Silberberg and Ziriax concerned the relative merits of melioration, and a computer simulation of molecular maximizing. There are difficulties in evaluating theories expressed as computer programs because many arbitrary decisions must often be made in order to get the programs to operate. I therefore propose an alternative form of model that I term quasi-dynamic as a useful intermediate form of theory appropriate to our current state of knowledge about free-operant choice. Quasi-dynamic models resemble the game-theoretic analyses now commonplace in biology in that they can predict stable and unstable equilibria but not dynamic properties such as learning curves. It is possible to interpret melioration as a quasi-dynamic model. An alternative quasi-dynamic model for probabilistic choice, ratio invariance, has been proposed by Horner and Staddon. The present paper compares the predictions of melioration and ratio invariance for five experimental situations: concurrent variable-interval variable-interval schedules, concurrent variable-interval variable-ratio schedules, the two-armed bandit (concurrent random-ratio schedules), and two types of frequency-dependent schedule. Neither approach easily explains all the data, but ratio invariance seems to provide a better picture of pigeons' response to probabilistic choice procedures. Ratio invariance is also more adaptive (less susceptible to “traps”) and closer to the original expression of the law of effect than pure hill-climbing processes such as momentary maximizing and melioration, although such processes may come in to play on more complex procedures that provide opportunities for temporal discrimination.     

A note on behavioural contrast and frustration

The suggestion has been made that positive contrast, the increase in response rate in one successively-presented stimulus following a change in conditions that decreases responding in the other, may not depend on differential stimulus control. Evidence is reviewed suggesting that contrast effects on most multiple interval schedules do depend upon discrimination, but that effects similar to contrast that result from the omission of reinforcement on fixed-interval schedules (the frustration/omission effect) reflect other factors. The link between contrast and the omission effect is the discrimination mechanism which allows animals to respond only at times or places correlated with food delivery. Contrast is a direct result of this mechanism; but the omission effect depends on the difference between the inhibitory discriminative after-effects acquired by food on fixed-interval schedules, and those acquired by a “neutral” stimulus presented in its place.     

Some properties of spaced responding in pigeons

Pigeons exposed to a schedule which reinforces interresponse times (IRTs) longer than a given value (DRL schedule) eventually reach a stable pattern of responding which is shown to be a function both of the DRL value and of previous experience with other DRL values. On any given DRL schedule, the stable performance of most pigeons which have been previously exposed to a variety of such schedules, shows an IRT distribution with median equal to the DRL value. For DRL values longer than about 30 sec, however, the median IRT falls short of the DRL value; this failure of adjustment to longer values appears to be a species characteristic of pigeons. The function relating reinforcement rate to 1/DRL value is also shown to be approximately linear over the same range, with variable slope (less than 45°) and a downturn in the vicinity of DRL 30.     

SCHEDULE-INDUCED DRINKING: ELICITATION,ANTICIPATION, OR BEHAVIORAL INTERACTION?

We carried out five experiments with rats on fixed-time schedules in order to define the relation between drinking and individual food-pellet presentations. In Experiment 1, unsignaled extra food occurred at the end of occasional fixed intervals, and we compared subsequent drinking patterns with drinking before the extra food presentation. In Experiment 2 we presented signaled and unsignaled extra food and measured elicited and anticipatory drinking patterns. In Experiment 3, we observed the persistence of modified drinking patterns when several consecutive intervals ended with extra pellets. In Experiments 4 and 5, we varied the magnitude of food delivery across (rather than within) sessions to replicate published findings. Results show that schedule-induced drinking is neither elicited by food presentations nor induced by stimuli associated with a high food rate. All subjects seemed to follow a simple rule: during any stimulus signaling an increase in the local probability of food delivery within a session, engage in food-related behavior to the exclusion of drinking. Schedule-induced drinking appears to be the result of dynamic interactions among food-related behavior, drinking, and other motivated behavior, rather than a direct effect of the contingencies of food reinforcement.