Observing responses and uncertainty reduction. II The effect of varying the probability of reinforcement

The observing response paradigm was used to assess the reinforcing properties of discriminative stimuli by allowing animals either to work for food in the presence of a neutral stimulus or to first make an observing response by pressing a lever. On a progressive ratio schedule this resulted in the appearance of easily discriminable stimuli which marked positive and negative trials. The uncertainty associated with the imposed neutral stimulus was varied by manipulating the proportion of positive and negative trials in the session. The animals switched most often when the probability of reinforcement was low, less often when it was intermediate, and least often when it was high. This was not consistent with an uncertainty reduction hypothesis but could be explained if the uncertainty had to do with response rather than outcome.     

Observing responses and informative stimuli

Pigeons were trained on a trial procedure. A trial began with the illumination of a pecking key by a white light. After a fixed interval, a key peck could turn the key to one of two equi-probable colors and produce a delayed trial outcome—an equi-probable occurrence of either reinforcement or nonreinforcement. After a trial, the key turned dark and the trial ended. The response could be made into an observing response by correlating the key colors with the outcomes. Response rates in the fixed interval then increased to a level greater than when the colors and outcomes were uncorrelated. In another phase, the response produced only the colors. The trial outcomes occurred some seconds after the fixed interval without a response being required. Correlating the colors with the outcomes again increased response rates. In a second experiment, a further condition was added in which reinforcement was the outcome on every trial. Response rates were lower than when there were equi-probable reinforcement and nonreinforcement outcomes with correlated colors, and about the same as when there were equi-probable outcomes with uncorrelated colors. The results suggest that stimuli providing information about the probability of reinforcement are themselves reinforcing.     

Observing responses and discrimination learning

An operant response in the pigeon, whose performance results in exposure to the discriminative stimuli, is described and suggested as an experimental analogue for “observing.” Such an operant response is then used to explore the relationship between observing responses and discrimination learning in a variety of discrimination situations, of progressively increasing complexity. In general, the results support the contention that the development and maintainence of observing responses is closely related to the degree of differential behaviour manifested toward the discriminative stimuli. Certain modifications are suggested in the theoretical formulation underlying the concept of “observing responses.”     

Observing responses in pigeons

Pigeons were trained on an observing-response procedure in which periods of VR 100 and EXT alternated unpredictably during a white light (mixed stimulus). During VR 100, responses on a food-producing key (the first key) were intermittently reinforced. Responses on the observing key (the second key) produced a green light (positive stimulus) when VR 100 was in effect, and a red light (negative stimulus) for EXT. The birds did not respond on either key during the negative stimulus, but they responded on the food-producing key when the positive stimulus appeared. When observing responses produced the positive or negative stimulus on FR, observing responses were maintained until the FR reached a maximum; beyond this, only food-producing responses occurred. When observing responses did not produce either stimulus, the observing-response rates fell to zero. With prolonged exposure to an FR 20 schedule of observing, observing-response rates during EXT were higher than during VR 100. Chlorpromazine hydrochloride decreased the total response output but markedly increased observing-response rates except when it was administered before sessions of observing response extinction.     

Observing responses maintained by conditional discriminative stimuli.

In a conditional discrimination procedure, pigeons' observing responses were analyzed to examine whether two color stimuli (blue or red), conditionally related to whether each of two line stimuli (vertical or horizontal) accompanied reinforcement or nonreinforcement, functioned as conditioned reinforcers. If a variable-interval (VI) 10-s requirement was fulfilled, an observing response produced onset of a color stimulus. A little later, a line stimulus was presented independently of responding, added to the color stimulus to form a compound stimulus. If 55 s elapsed with a response not having occurred either through 55 s or after the variable-interval 10-s had timed out, one of the color-line compound stimuli was presented independently of responding. To control for sensory reinforcement effects and for earlier entrance to the later link, a simple discrimination procedure also was conducted in which reinforcement was not correlated with the color stimuli but with the line stimuli only. As in the conditional discrimination, the observing response also could produce earlier presentation of blue or red. The observing response occurred more frequently during the conditional discrimination than during the simple discrimination. The results were related to different theoretical accounts of conditioned reinforcement, particularly the information hypothesis.     

Dissociating uncertainty responses and reinforcement signals in the comparative study of uncertainty monitoring

Although researchers are exploring animals' capacity for monitoring their states of uncertainty, the use of some paradigms allows the criticism that animals map avoidance responses to error-causing stimuli not because of uncertainty monitored but because of feedback signals and stimulus aversion. The authors addressed this criticism with an uncertainty-monitoring task in which participants completed blocks of trials with feedback deferred so that they could not associate reinforcement signals to particular stimuli or stimulus-response pairs. Humans and 1 of 2 monkeys were able to make cognitive, decisional uncertainty responses that were independent of feedback or reinforcement history within a task. This finding unifies the comparative literature on uncertainty monitoring. The dissociation of performance from reinforcement has theoretical implications, and the deferred-feedback technique has many applications.     

Decision deadlines and uncertainty monitoring: the effect of time constraints on uncertainty and perceptual responses

The behavioral uncertainty response has grounded the study of animal metacognition and influenced the study of human psychophysics. However, the interpretation of this response is debated—especially whether it is a behavioral index of metacognition. The authors advanced this interpretation using the dissociative technique of response deadlines. Uncertainty responding, if it is higher level or metacognitive, should depend on a slower, more controlled decisional process and be more vulnerable to time constraints. Humans performed sparse–uncertain–dense or sparse–middle–dense discriminations in which, respectively, they could decline difficult trials or positively identify middle stimuli. Uncertainty responses were sharply and selectively reduced under a decision deadline, as compared to primary perceptual responses (i.e., “sparse,” “middle,” and “dense” responses). This dissociation suggests that the uncertainty response does reflect a higher-level, decisional response. It grants the uncertainty response a distinctive psychological role in its task and encourages an interpretation of this response as an elemental behavioral index of uncertainty that deserves continuing research.     

Observing responses in pigeons: effects of schedule component duration and schedule value

Pigeons were exposed to a procedure under which five pecks on one response key (the observing key) changed the schedule on a second key (the food key) from a mixed schedule to a multiple schedule for 25 sec. In Experiment I, a random-ratio 50 schedule alternated with extinction. The duration of the random-ratio 50 schedule component was varied between 1.25 and 320 sec, and extinction was scheduled for a varying time, ranging from the duration of the random-ratio 50 to four times that value. Each set of values was scheduled for a block of sessions. Before observing-key pecks were allowed at each set of parameter values, the pigeons were exposed to a condition where the mixed and multiple schedule alternated every 10 min, and observing-key pecks were not permitted. Rates of pecking on the observing key were high for all values of random-ratio component durations except 1.25 sec. Experiment II was conducted with the random-ratio component duration equal to 40 sec, and the random-ratio schedule was varied from random-ratio 50 to 100, 200, and 400. Observing-key pecking rates were high for all values of the random-ratio schedule except random-ratio 400. In both experiments, observing response rates were relatively little affected, suggesting that neither schedule component duration nor schedule value is a strong determinant of observing responses.     

Observing responses in rats: Support for the secondary reinforcement hypothesis

Rats' lever presses on a retractable lever earned brief presentations of discriminative stimuli signalling periods in which responding on an alternative lever was either non-reinforced (extinction) or reinforced on a random ratio schedule. The predictions of two theoretical accounts of this behaviour were tested by studying the effects of omitting either the stimulus signalling the reinforced or that signalling the non-reinforced schedule component. Under these conditions rats' behaviour is determined by the conditioned, affective properties of the stimuli rather than by their purely informational properties.     

Predictive accuracy and not reduction in uncertainty influences responding during serial autoshaping

Two experiments with pigeons employed a serial autoshaping design in which food was presented on every trial, and the initial element was of variable duration. When the first element was followed randomly by one stimulus of fixed short duration and another stimulus of fixed long duration, then it elicited a relatively high rate of responding. But when the second elements were of equal mean durations, whether fixed or variable, then responding during the first element was relatively slow. Two explanations for these findings were considered. One stipulates that the rate of responding during a stimulus is related to its predictive accuracy, the other stipulates that responding is influenced by the reduction in uncertainty that follows the offset of a stimulus. It is argued that the evidence favours the first of these alternatives.     

Type A behavior and emotional responses to uncertainty: A test of the self-appraisal model

The self-appraisal model of Type A behavior holds that Type A's desire an accurate appraisal of abilities, and respond to uncertainty about abilities with exaggerated attempts to generate diagnostic information. The model also predicts that the major emotional response to uncertainty for Type A's is anxiety. In this study, Type A's and B's were induced to succeed or fail on a task where they were either very certain or very uncertain of their subsequent ability levels. Task failure produced greater depression, anger, and frustration for all subjects. Consistent with the self-appraisal model's prediction, uncertainty (regardless of success or failure) led to greater anxiety for Type A's than for Type B's. In fact, greater uncertainty produced lower anxiety for Type B's, indicating that Type A's and B's differ dramatically in their emotional responses to uncertainty about abilities.