Overshadowing of a stimulus-reinforcer association by an instrumental response

In two experiments, rats were first exposed to pairings of a clicker and food; they were subsequently, in order to measure the effectiveness of the clicker as a conditioned reinforcer, given the opportunity to press a lever which turned the clicker on. For one group of animals the food originally delivered in the presence of the clicker had been contingent on their performance of an instrumental response (running in a running wheel); for a second the contingency between clicker and food had been purely classical. Although the actual correlation between clicker and food was identical for the two groups, the clicker was a less effective conditioned reinforcer for the first group than for the second. In a third experiment, all animals were initially required to run to obtain food in the presence of the clicker, but one group received additional trials on which food was delivered contingent on running in the absence of the clicker. This group showed less tendency to lever press for the clicker than a second group that had received free food on trials when the clicker was not presented. The results of all three experiments suggest that conditioning to the clicker could be overshadowed if the occurrence of food was more reliably predicted by the execution of an instrumental running response; they thus support the view that instrumental conditioning depends on the establishment of an association between response and reinforcer similar to the association between stimulus and reinforcer underlying classical conditioning.     

Dissociating uncertainty responses and reinforcement signals in the comparative study of uncertainty monitoring

Although researchers are exploring animals' capacity for monitoring their states of uncertainty, the use of some paradigms allows the criticism that animals map avoidance responses to error-causing stimuli not because of uncertainty monitored but because of feedback signals and stimulus aversion. The authors addressed this criticism with an uncertainty-monitoring task in which participants completed blocks of trials with feedback deferred so that they could not associate reinforcement signals to particular stimuli or stimulus-response pairs. Humans and 1 of 2 monkeys were able to make cognitive, decisional uncertainty responses that were independent of feedback or reinforcement history within a task. This finding unifies the comparative literature on uncertainty monitoring. The dissociation of performance from reinforcement has theoretical implications, and the deferred-feedback technique has many applications.     

Uncertainty and the orientation reaction

In Experiment I, blurred pictures evoked longer desynchronization than clear pictures but not more intense GSRs. Experiment 2 confirmed that the EEG effect depended on subjective uncertainty by showing that it did not occur when a blurred picture was immediately preceded by a clear version of the same picture.     

Enhancement of conditioned reinforcement by uncertainty

Pigeons were trained in three conditions. In the baseline condition, the birds responded on a fixed-interval schedule with the response key white. When the interval was completed, the key turned either red or green for a delay interval that was terminated by a grain presentation dependent on no key pecks during the final 2 sec. In the uncertainty condition, no grain was presented at the end of the delay periods when the key was red. In the certainty condition, the white light appeared only on occasions when pecking would turn the key green and produce food. Otherwise, the key was illuminated red throughout the total time period. The highest response rate in white occurred in the uncertainty condition, the next highest in the certainty condition, and the lowest in baseline. The results suggest that uncertainty facilitated responding, although uncertainty is not a necessary condition for conditioned reinforcement.     

The effect of exercise on the selective performance of an instrumental response

Dogs were used to condition two instrumental reactions—bending of the left or of the right forepaw respectively. In the experimental situation a dish containing food was presented to the dog at a distance but could be drawn close and retained by lifting the forepaw to which (left or right) a lever had been attached. If both paws were simultaneously attached to levers, the dog had to choose between two reactions. Preliminary trials showed a right paw preference in spite of the equal effectiveness (food reinforcement) of the left paw. The aim of this experiment was to alter this right paw preference by a paradigm of differential practice with left paw dominance. The plan was to have ten training sessions followed by one test with choice. According to the probabilities, P_left = 0.8, P_right = 0.2, a random table was used to determine the order of the right-left paw attachments to the levers during training. In all, 1,000 training sessions and 100 tests with choice were carried out in each of three dogs. According to the training program, in every 100 training sessions the left paw had to perform the reaction a total of 80 times and 20 times by the right paw. An absolute alteration in paw preference occurred P_lelt = 1.0) even though both forepaws were equally effective in obtaining food reinforcement on the test trials. The present results indicate that not only the Law of Effect but Exercise is also important in instrumental conditioning.     

Prediction and uncertainty in human Pavlovian to instrumental transfer

Attentional capture and behavioral control by conditioned stimuli have been dissociated in animals. The current study assessed this dissociation in humans. Participants were trained on a Pavlovian schedule in which 3 visual stimuli, A, B, and C, predicted the occurrence of an aversive noise with 90%, 50%, or 10% probability, respectively. Participants then went on to separate instrumental training in which a key-press response canceled the aversive noise with a .5 probability on a variable interval schedule. Finally, in the transfer phase, the 3 Pavlovian stimuli were presented in this instrumental schedule and were no longer differentially predictive of the outcome. Observing times and gaze dwell time indexed attention to these stimuli in both training and transfer. Aware participants acquired veridical outcome expectancies in training--that is, A > B > C, and these expectancies persisted into transfer. Most important, the transfer effect accorded with these expectancies, A > B > C. By contrast, observing times accorded with uncertainty--that is, they showed B > A = C during training, and B < A = C in the transfer phase. Dwell time bias supported this association between attention and uncertainty, although these data showed a slightly more complicated pattern. Overall, the study suggests that transfer is linked to outcome prediction and is dissociated from attention to conditioned stimuli, which is linked to outcome uncertainty.