Suboptimal choice in a percentage-reinforcement procedure: effects of signal condition and terminal-link length.

Pigeons' choice between reliable (100%) and unreliable (50%) reinforcement was studied using a concurrent-chains procedure. Initial links were fixed-ratio 1 schedules, and terminal links were equal fixed-time schedules. The duration of the terminal links was varied across conditions. The terminal link on the reliable side always ended in food; the terminal link on the unreliable side ended with food 50% of the time and otherwise with blackout. Different stimuli present during the 50% terminal links signaled food or blackout outcomes under signaled conditions but were uncorrelated with outcomes under unsignaled conditions. In signaled conditions, most pigeons displayed a nearly exclusive preference for the 100% alternative when terminal links were short (5 or 10 s), but with terminal links of 30 s or longer, preference for the 100% alternative was sharply reduced (often to below .5). In unsignaled conditions, most pigeons showed extreme preference for the 100% alternative with either short (5 s) or longer (30 s) terminal links. Thus, pigeons' choice between reliable and unreliable reinforcement is influenced by both the signal conditions on the unreliable alternative and the duration of the terminal-link delay. With a long delay and signaled outcomes, many pigeons display a suboptimal tendency to choose the unreliable side.     

Theories of probabilistic reinforcement.

In three experiments, pigeons chose between two alternatives that differed in the probability of reinforcement and the delay to reinforcement. A peck at a red key led to a delay of 5 s and then a possible reinforcer. A peck at a green key led to an adjusting delay and then a certain reinforcer. This delay was adjusted over trials so as to estimate an indifference point, or a duration at which the two alternatives were chosen about equally often. In Experiments 1 and 2, the intertrial interval was varied across conditions, and these variations had no systematic effects on choice. In Experiment 3, the stimuli that followed a choice of the red key differed across conditions. In some conditions, a red houselight was presented for 5 s after each choice of the red key. In other conditions, the red houselight was present on reinforced trials but not on nonreinforced trials. Subjects exhibited greater preference for the red key in the latter case. The results were used to evaluate four different theories of probabilistic reinforcement. The results were most consistent with the view that the value or effectiveness of a probabilistic reinforcer is determined by the total time per reinforcer spent in the presence of stimuli associated with the probabilistic alternative. According to this view, probabilistic reinforcers are analogous to reinforcers that are delivered after variable delays.     

Contiguity and conditioned reinforcement in probabilistic choice

In a baseline condition, pigeons chose between an alternative that always provided food following a 30-s delay (100% reinforcement) and an alternative that provided food half of the time and blackout half of the time following 30-s delays (50% reinforcement). The different outcomes were signaled by different-colored keylights. On average, each alternative was chosen approximately equally often, replicating the finding of suboptimal choice in probabilistic reinforcement procedures. The efficacy of the delay stimuli (keylights) as conditioned reinforcers was assessed in other conditions by interposing a 5-s gap (keylights darkened) between the choice response and one or more of the delay stimuli. The strength of conditioned reinforcement was measured by the decrease in choice of an alternative when the alternative contained a gap. Preference for the 50% alternative decreased in conditions in which the gap preceded either all delay stimuli, both delay stimuli for the 50% alternative, or the food stimulus for the 50% alternative, but preference was not consistently affected in conditions in which the gap preceded only the 100% delay stimulus or the blackout stimulus for the 50% alternative. These results support the notion that conditioned reinforcement underlies the finding of suboptimal preference in probabilistic reinforcement procedures, and that the signal for food on the 50% reinforcement alternative functions as a stronger conditioned reinforcer than the signal for food on the 100% reinforcement alternative. In addition, the results fail to provide evidence that the signal for blackout functions as a conditioned punisher.     

Entorhinal but Not Hippocampal or Subicular Lesions Disrupt Latent Inhibition in Rats

Latent inhibition (LI) is the deficit of conditioning resulting from repeated nonreinforced preexposure to a conditioned stimulus before its pairing with an unconditioned stimulus. There are cumulative data showing that large lesions of the hippocampal formation disrupt LI. However, the effects of selective lesions of the different components of the hippocampal formation have never been directly addressed in the same study and conditioning paradigm. The first experiment of the present study aimed at investigating the effects of excitotoxic lesions of the hippocampus, subiculum, or entorhinal cortex on LI in an "off-baseline"-conditioned emotional response procedure. Hippocampus or subiculum lesions had no effect on either LI or conditioning. In contrast, entorhinal cortex lesions disrupted LI without modifying conditioning. In Experiment 2, locomotor activity in a novel environment was assessed in the same rats. Whereas lesions of hippocampus increased locomotor activity, lesions of the subiculum or the entorhinal cortex were devoid of effect. Although both LI and habituation to novel environmental cues are thought to involve interactions between the hippocampal formation and the mesolimbic pathway, these results indicate a functional dissociation between the hippocampus and the entorhinal cortex.     

The conditioned reinforcement of repeated acquisition

Three monkeys were trained to emit a chain of three responses on three separate levers in a set of six levers to obtain food. The chain producing food (correct chain) was changed each day. During a trial, a press on any lever produced a feedback stimulus; a press on a correct lever produced an additional distinctive stimulus; the third correct press produced a food pellet. Test sessions in which either the food or the distinctive stimuli were removed were interspersed with baseline sessions. In tests without food presentations, the subjects acquired the correct chain rapidly, with a level of accuracy comparable to baseline. Removing the distintive stimuli for either the first or second member of the correct chain greatly retarded acquisition of that member of the chain. Removing all distinctive stimuli often reduced accuracy throughout the chain to chance level, even though food was presented following each correct chain. These results were interpreted as evidence that the distinctive stimuli presented after correct responses functioned as conditioned reinforcers. Reductions in accuracy following an omitted distinctive stimulus indicated that they were also discriminative stimuli for correct responding in their presence.     

A test of the reinforcing properties of stimuli correlated with nonreinforcement

The information hypothesis of conditioned reinforcement predicts that a stimulus that “reduces uncertainty” about the outcome of a trial will acquire reinforcing properties, even when the stimulus reliably predicts nonreinforcement. Four pigeons' key pecks produced one of two 5-sec stimuli with 0.50 probability according to a discriminated variable-interval schedule. One stimulus was followed by reinforcement; a second stimulus was followed by blackout. To the same extent, therefore, both stimuli reduced uncertainty about the possibility that food would arrive at the termination of the schedule interval. When a second key in the chamber was lighted, each peck on it could produce the stimulus preceding reinforcement, the stimulus preceding nonreinforcement, a novel stimulus, or no stimulus, across separate conditions. The stimulus preceding food maintained responding at substantial levels on the second, stimulus-producing, key. Such responding was not maintained by other stimuli. These data, replicated when the stimuli were reversed on the variable-interval schedule, do not support the prediction that uncertainty-reducing stimuli are necessarily conditioned reinforcers.     

The effects of d-amphetamine on the temporal control of operant responding in rats during a preshock stimulus.

The operant behavior of six rats was maintained by a random-interval schedule of reinforcement. Three-minute periods of noise were superimposed on this behavior, each period ending with the delivery of an unavoidable shock. Overall rates of responding were generally lower during the periods of noise than in its absence (conditioned suppression). These suppressed response rates also exhibited temporal patterning, with responding becoming less frequent as each noise period progressed. The effects of d-amphetamine on this behavioral baseline were then assessed. In four animals the relative response rates during the noise and in its absence suggested that the drug produced a dose-related decrease in the amount of conditioned suppression. However, this effect was often due to a decrease in the rates of responding in the absence of the preshock stimulus, rather than to an increase in response rates during the stimulus. Temporal patterning in response rates during the preshock stimulus was abolished, an effect that was interpreted in terms of rate-dependent effect of d-amphetamine. This study thus extends rate-dependent analyses of the effects of amphetamines to the patterns of operant behavior that occur during a preshock stimulus, and which have been discussed in terms of the disrupting effects of anxiety on operant behavior.     

Human observing: Maintained by stimuli correlated with reinforcement but not extinction

College students received points exchangeable for money (reinforcement) on a variable-time 60-second schedule that alternated randomly with an extinction component. Subjects were informed that responding would not influence either the rate or distribution of reinforcement. Instead, presses on either of two levers (“observing responses”) produced stimuli. In each of four experiments, stimuli positively correlated with reinforcement and/or stimuli uncorrelated with reinforcement were each chosen over stimuli correlated with extinction. These results are consistent with prior results from pigeons in supporting the conditioned-reinforcement hypothesis of observing and in not supporting the uncertainty-reduction hypothesis.     

The effects of diazepam and triazolam on repeated acquisition and performance of response sequences with an observing response.

Drugs often disrupt the acquisition of new response sequences at doses that fail to disrupt the performance of a previously acquired response sequence. This selective drug effect may result from differences in the control exerted by the stimuli presented after each response in the acquisition and performance sequences. To examine the function of these stimuli, an observing procedure was incorporated into a multiple schedule of repeated acquisition and performance of response sequences, in which stimulus presentations were contingent upon an observing response. Three experiments were conducted with humans. Experiment 1 compared responding with and without the observing contingency. No difference was found in the overall percentage of errors across the two conditions. Within the observing condition, observing behaviour was maintained in the acquisition component as long as errors occurred, but was not maintained in the performance component. Experiment 2 examined whether a contingency that increased errors also would increase observing in both the acquisition and performance components. Specifically, reinforcer delivery in each component was contingent upon emitting 10 correct responses and one, two, or four errors. Observing responses increased in the acquisition component as the error requirement increased, whereas observing responses in the performance component increased only when the error requirement was four. Experiment 3 assessed the effects of diazepam (0, 7.5, 15, and 30 mg/70 kg, p.o.) and triazolam (0, 0.375, and 0.75 mg/70 kg, p.o.) on repeated acquisition and performance baselines with the observing contingency. Selective drug effects were obtained in this modified procedure; that is, the percentage of errors in the acquisition component increased at doses that failed to affect the percentage of errors in the performance components. Importantly, drug effects were selective, even though observing responses were not emitted in the performance component and, hence, the stimulus presentations did not occur in that component. These findings suggest that alternative explanations for these differential effects are needed; in that regard, a response-unit account of the selective drug effects is discussed.