Purchasing decisions,partial knowledge,and economic search experimental and simulation evidence

Several experimental studies have discovered high-performing heuristic strategies in the search for the most economic price of a good in situations where a searcher has both full prior knowledge and no prior knowledge of the underlying price distribution. This paper describes a new experiment set in a more realistic ‘partial-knowledge’ framework. Several new search heuristics are suggested which match well with observed behaviour, and which, over a large number of simulated trials, perform well in comparison with the optimal model and better than previously reported ‘no-knowledge’ rules.     

EXTINCTION OF SELF-INJURIOUS ESCAPE BEHAVIOR WITH AND WITHOUT INSTRUCTIONAL FADING

Three individuals with developmental disabilities participated in a study of the treatment of self-injurious behavior (SIB) maintained by negative reinforcement (escape from educational tasks). Treatment was implemented in a multiple baseline design across subjects, in which two treatments were compared in a multielement format. Both treatment conditions included an escape-extinction component in which SIB no longer produced escape. One of the conditions also included a fading component in which the frequency of instructions was initially reduced to zero and then was gradually faded back in across sessions until the instructional rate matched that of the original baseline. Results indicated that extinction alone reduced SIB to the end-of-treatment criterion in fewer sessions than did extinction plus fading for all 3 subjects. For 2 of the 3 subjects, however, there was an initial increase in the frequency of SIB at the outset of treatment with extinction (an extinction burst) that was not observed when extinction was combined with the fading component.     

ANALYSIS OF THE REINFORCEMENT AND EXTINCTION COMPONENTS IN DRO CONTINGENCIES WITH SELF-INJURY

Previous research has shown that self-injurious behavior (SIB) maintained by positive reinforcement may be reduced under differential-reinforcement-of-other-behavior (DRO) contingencies. In this study, we conducted an analysis of the reinforcement and extinction components of DRO while treating the self-injury of 3 women with developmental disabilities. A functional analysis revealed that each subject's SIB was maintained by positive reinforcement in the form of attention. Subsequent reinforcer assessments identified preferred and nonpreferred stimuli for later use in conjunction with DRO. Results showed high rates of SIB for all 3 subjects during baseline, which persisted when DRO was implemented without the relevant extinction component (withholding of attention for SIB) for 2 of the subjects. Low rates of SIB were observed for all subjects when DRO plus extinction was implemented or when extinction was implemented alone, suggesting that extinction may be a critical component of DRO schedules.     

A selectionist approach to reinforcement.

We describe a principle of reinforcement that draws upon experimental analyses of both behavior and the neurosciences. Some of the implications of this principle for the interpretation of behavior are explored using computer simulations of adaptive neural networks. The simulations indicate that a single reinforcement principle, implemented in a biologically plausible neural network, is competent to produce as its cumulative product networks that can mediate a substantial number of the phenomena generated by respondent and operant contingencies. These include acquisition, extinction, reacquisition, conditioned reinforcement, and stimulus-control phenomena such as blocking and stimulus discrimination. The characteristics of the environment-behavior relations selected by the action of reinforcement on the connectivity of the network are consistent with behavior-analytic formulations: Operants are not elicited but, instead, the network permits them to be guided by the environment. Moreover, the guidance of behavior is context dependent, with the pathways activated by a stimulus determined in part by what other stimuli are acting on the network at that moment. In keeping with a selectionist approach to complexity, the cumulative effects of relatively simple reinforcement processes give promise of simulating the complex behavior of living organisms when acting upon adaptive neural networks.     

A cellular analogue of operant conditioning.

Using the hippocampal-slice preparation, we attempted to model operant conditioning in vitro by reinforcing pyramidal cell bursting responses with local micropressure applications of transmitters and drugs. The same injections were administered independently of bursting to provide a "noncontingent" control for direct pharmacological stimulation or facilitation of firing. The results suggested that the bursting responses of individual CA1 pyramidal neurons may be reinforced in a dose-related manner by response-contingent (but not noncontingent) injections of dopamine and the selective dopamine D2 agonist, N-0923. N-0924, a stereoisomer of N-0923 that is largely devoid of D2-agonist activity, failed to reinforce CA1 bursting. Burst-contingent injections of the excitatory neurotransmitter glutamate also failed to reinforce CA1 bursting; indeed, the glutamate applications (whether contingent or random) reduced the likelihood of bursts while increasing the frequency of solitary spikes. Reinforcement delays exceeding 200 ms largely eliminated the reinforcing efficacy of the D2 agonist N-0437 in CA1 operant conditioning. The results are consistent with the suggestion that the behaviorally reinforcing effects of dopaminergic agents can be modeled in vitro in the hippocampal-slice preparation.     

Irrelevance of sample stimuli and directed forgetting in pigeons

A successive matching procedure was used to investigate which aspect of the test-omission procedure is responsible for establishing a postsample stimulus as a cue to forget in pigeons. It was found that a postsample stimulus that reliably followed a sample that was irrelevant to performance functioned as a cue to forget. This result was obtained regardless of whether termination of that postsample stimulus was followed by reinforcement or by the presentation of sample-independent discriminative stimuli. It was also found that a postsample stimulus that functioned as a cue to forget at the beginning of training lost that function when it was repeatedly presented on trials in which the sample was relevant to performance. These findings reveal that (a) neither a reduction in reinforcement rate nor the omission of the opportunity for discriminated responding is necessary to establish an effective cue to forget and (b) irrelevance of the sample to performance is a sufficient condition to establish a cue to forget. These results suggest that a postsample stimulus that is presented on trials in which remembering the sample is not reinforced differentially will come to set the occasion for not remembering the sample.     

The effects of diazepam and triazolam on repeated acquisition and performance of response sequences with an observing response.

Drugs often disrupt the acquisition of new response sequences at doses that fail to disrupt the performance of a previously acquired response sequence. This selective drug effect may result from differences in the control exerted by the stimuli presented after each response in the acquisition and performance sequences. To examine the function of these stimuli, an observing procedure was incorporated into a multiple schedule of repeated acquisition and performance of response sequences, in which stimulus presentations were contingent upon an observing response. Three experiments were conducted with humans. Experiment 1 compared responding with and without the observing contingency. No difference was found in the overall percentage of errors across the two conditions. Within the observing condition, observing behaviour was maintained in the acquisition component as long as errors occurred, but was not maintained in the performance component. Experiment 2 examined whether a contingency that increased errors also would increase observing in both the acquisition and performance components. Specifically, reinforcer delivery in each component was contingent upon emitting 10 correct responses and one, two, or four errors. Observing responses increased in the acquisition component as the error requirement increased, whereas observing responses in the performance component increased only when the error requirement was four. Experiment 3 assessed the effects of diazepam (0, 7.5, 15, and 30 mg/70 kg, p.o.) and triazolam (0, 0.375, and 0.75 mg/70 kg, p.o.) on repeated acquisition and performance baselines with the observing contingency. Selective drug effects were obtained in this modified procedure; that is, the percentage of errors in the acquisition component increased at doses that failed to affect the percentage of errors in the performance components. Importantly, drug effects were selective, even though observing responses were not emitted in the performance component and, hence, the stimulus presentations did not occur in that component. These findings suggest that alternative explanations for these differential effects are needed; in that regard, a response-unit account of the selective drug effects is discussed.     

Matching, delay-reduction, and maximizing models for choice in concurrent-chains schedules.

Models of choice in concurrent-chains schedules are derived from melioration, generalized matching, and optimization. The resulting models are compared with those based on Fantino's (1969, 1981) delay-reduction hypothesis. It is found that all models involve the delay reduction factors (T - t2L) and (T - t2R), where T is the expected time to primary reinforcement and t2L, t2R are the durations of the terminal links. In particular, in the case of equal initial links, the model derived from melioration coincides with Fantino's original model for full (reliable) reinforcement and with the model proposed by Spetch and Dunn (1987) for percentage (unreliable) reinforcement. In the general case of unequal initial links, the model derived from melioration differs from the revised model advanced by Squires and Fantino (1971) only in the factors affecting the delay-reduction terms (T - t2L) and (T - t2R). The models of choice obtained by minimizing the expected time to reinforcement depend on the type of feedback functions used. In particular, if power feedback functions are used, the optimization model coincides with that obtained from melioration.     

Suboptimal choice in a percentage-reinforcement procedure: effects of signal condition and terminal-link length.

Pigeons' choice between reliable (100%) and unreliable (50%) reinforcement was studied using a concurrent-chains procedure. Initial links were fixed-ratio 1 schedules, and terminal links were equal fixed-time schedules. The duration of the terminal links was varied across conditions. The terminal link on the reliable side always ended in food; the terminal link on the unreliable side ended with food 50% of the time and otherwise with blackout. Different stimuli present during the 50% terminal links signaled food or blackout outcomes under signaled conditions but were uncorrelated with outcomes under unsignaled conditions. In signaled conditions, most pigeons displayed a nearly exclusive preference for the 100% alternative when terminal links were short (5 or 10 s), but with terminal links of 30 s or longer, preference for the 100% alternative was sharply reduced (often to below .5). In unsignaled conditions, most pigeons showed extreme preference for the 100% alternative with either short (5 s) or longer (30 s) terminal links. Thus, pigeons' choice between reliable and unreliable reinforcement is influenced by both the signal conditions on the unreliable alternative and the duration of the terminal-link delay. With a long delay and signaled outcomes, many pigeons display a suboptimal tendency to choose the unreliable side.     

“DO I HAVE TO BE GOOD ALL DAY?” THE TIMING OF DELAYED REINFORCEMENT AS A FACTOR IN GENERALIZATION

Delayed reinforcement, sometimes delivered just after the setting in which the critical behavior had occurred (Early), and sometimes delivered only after several further settings had been encountered (Late), was used to improve a variety of behaviors in seven preschool children, and to control their generalization. Performance of those behaviors was measured in two classroom settings: the Contingent setting, within which performance of the specified behavior determined the later (Early or Late) reinforcement, and the Generalization setting, in which there were no experimental contingencies, immediate or delayed, for the performance of the same behavior. Performances by all children in the Contingent setting were controlled by delayed reinforcement, whether Early or Late. All children showed consistent generalization from the Contingent setting to the Generalization setting during the Late condition, when reinforcement was delivered at the end of the school day. Generalization did not occur during the Early condition, when reinforcement was provided immediately after the Contingent setting (prior to the Generalization setting), unless that condition had been preceded by a Late condition (as it was for S₆ and S₇). The results suggest that the Late timing of delayed reinforcement was an effective and efficient generalization-promotion technique for performances that did not generalize spontaneously.