Net Amount of Food Affects Autoshaped Response Rate, Response Latency, and Gape Amplitude in Pigeons

The contribution of net amount of food to conditioned response strength and topography was assessed in four pigeons under autoshaping contingencies. In each session, under one baseline phase and three replication phases, three trial types were presented sequentially: One conditioned stimulus (keylight) signaled one small pellet, another signaled one large pellet, and a third signaled seven small pellets which were weight-matched to one large pellet. Five dependent variables were response rates and latencies based on the occurrence of both keyswitch closures and gapes (beak openings) and gape amplitudes. In result, net amount of food, not pellet diameter or number, affected all dependent variables. Notably, gape amplitudes elicited by the seven-small pellet keylights were larger than the gape amplutides elicited by the one-small pellet keylights even though the gape amplitudes elicited by both unconditioned stimuli (one or seven pellets) were equally small. This mismatch between conditioned and unconditioned responses is incompatible with stimulus substitution accounts but is compatible with an associative strength account. Furthermore, the changes in the dependent variables were most likely determined by Pavlovian and not by inadvertent operant contingencies. The findings demonstrate that an analysis of classical conditioning benefits from the inclusion of topographical measures.     

The form of the auto-shaped response with food or water reinforcers

The relation between the form of auto-shaped responses to the lighting of a key and the consummatory responses of pecking grain and drinking water was examined in pigeons. Responses on the key were analyzed by means of high-speed photography, recordings of the force of contact, and judges' ratings of response-form based on film and videotape recordings. The first experiment showed that food-deprived birds presented grain as a reinforcer responded on the key with a grain-pecking movement, while water-deprived birds presented water as a reinforcer responded with drinking-like movements. The second and third experiments showed that the resemblance between auto-shaped and consummatory responses does not require the dominance of the deprivational state appropriate to the reinforcer. Changing the dominant state of deprivation did not immediately change the form of the key response, and in subjects simultaneously deprived of food and water, the form of response depended on the reinforcer. In the fourth and fifth experiments, subjects simultaneously deprived of food and water received one stimulus signalling food and another signalling water in a random series. In most subjects, the response to each stimulus resembled the consummatory response to the particular reinforcer that was signalled by the stimulus. This result demonstrates the role of association between a stimulus and a reinforcer in producing a resemblance of the auto-shaped response to the consummatory response.     

Autoshaped key pecking maintained by access to a social space.

When four experimentally naive pigeons were exposed to occasional forward pairings of a keylight followed by a doorlight (that signaled access to a large social space), all subjects began to peck the lit key. In a second experiment, where the keylight either preceded the presentation of the doorlight or was presented independently of it, key pecking was maintained only in the former circumstance. The unconditioned stimulus in these experiments--arrival in the social space--did not elicit pecking. Hence, the conditioned response of key pecking and the unconditioned response of entering the social space differed. This demonstration of autoshaping with a social-space unconditioned stimulus argues against a stimulus-substitution account of the findings.     

The associative relation underlying autoshaping in the pigeon

Fifteen pigeons were exposed to either response-independent or response-dependent schedules of water reinforcement, whereby water was injected directly into the unrestrained pigeons' mandibles. Key-contact responses were released by a lighted key correlated with water, but not by a lighted key uncorrelated with water. A negative response-reinforcer contingency suppressed autoshaped key-contact responses, resulting in responding directed away from the lighted key. In all pigeons, water injected directly into the mandibles elicited a consummatory fixed-action pattern of “mumbling” and swallowing. The lighted key correlated with water released a broader set of both appetitive and consummatory responses: approach to the lighted key, “bowing”, “rooting”, “mumbling”, and swallowing. Key-contact responses were “rooting” and “mumbling” motions of the beak on the surface of the key. Views of autoshaping based on stimulus substitution or stimulus surrogation do not fully explain the origin of autoshaped responses not previously elicited by the reinforcer. The present findings are consonant with views of conditioning that emphasize the large degree of biological pre-organization in conditioned response patterns, and the importance of associative factors in the control of such patterns.     

Visual reinforcement in fighting cocks

Fighting cocks were conditioned to emit a key-pecking response on a fixed ratio reinforcement schedule leading to the visual image of another fighting cock. In addition, the relative reinforcing properties of the visual reinforcer were compared with food and water reinforcers in a three-choice, non-reversible option situation. The relative reinforcing effects of mirror presentation and another rooster visually presented through a window, were compared. The mirror maintained a relatively lower response output.     

Control of behavior by an establishing stimulus

Seventeen pigeons were exposed to a three-key discrete-trial procedure in which a peck on the lit center key produced food if, and only if, the left keylight was lit. The center key was illuminated by a peck on the lit right key. Of interest was whether subjects pecked the right key before or after the response-independent onset of the left keylight. Pecks on the right key after left-keylight onset suggest control of behavior by the left keylight—an establishing stimulus. In three experiments, the strength of center-keylight onset as conditioned reinforcer for a response on the right key was manipulated by altering the size of the reduction in time to food delivery correlated with its onset. Control of pigeons' key pecks by onset of the left keylight occurred on more trials per session when the center keylight was a relatively weak conditioned reinforcer and on fewer trials per session when the center keylight was a relatively strong condtioned reinforcer. Differences across conditions in the degree of control by onset of the establishing stimulus were greatest when changes in conditioned reinforcer strength occurred relatively frequently and were signaled. The results provide evidence of the function of an establishing stimulus.     

Goal-tracking behavior in the pond snail, Lymnaea stagnalis

The occurrence of goal-tracking, an unconditioned stimulus (US)-directed autoshaping behavior, was studied in open-field tests with control and classically conditioned pond snails, Lymnaea stagnalis. In an appetitive classical conditioning paradigm with a tactile stimulus as conditioned stimulus (CS) and a localized food stimulus as US a conditioned feeding response built up in the experimental but not in the control animals. In the post-training open-field tests the experimental group alone showed an enhanced attraction toward the source of water current in the environment which previously signalled the arrival of the US but did not act as CS in the classical conditioning procedure. We suggest that this stimulus-directed goal-tracking behavior in Lymnaea is the result of a classical-operant interaction, described so far only in vertebrate animals, and that neurophysiological analysis of this behavior is possible in this snail.     

Discrimination and differentiation of response number in stimulus directed pecking of pigeons.

In Experiment 1, autoshaping trials terminated with food only if pigeons emitted more than a target number of responses during a trial in one condition and fewer than a target number in another. The median number of responses per trial shifted in accordance wtih the requirements. The responding of yoked-control birds that received response-independent reinforcers did not vary with the response requirements. In Experiment 2, the number of responses in autoshaping trial became the discriminative stimulus for reinforcement in the second component of a chained schedule. In one condition, responding was reinforced only if the number of responses in the first component was above a target value; in the other condition, responding was reinforced only if the number was below the target value. The distribution of the first-component response numbers did not shift systematically between discrimination conditions, but response rates in the second component indicated that the number of responses in the autoshaping trial was a discriminable property behavior.     

Context blocking in rat autoshaping: Sign-tracking versus goal-tracking

Prior experience of unsignaled food can interfere with subsequent acquisition by birds of autoshaped key-pecking at a signal light. This has been understood to indicate that unsignaled food results in context conditioning, which blocks subsequent learning about the keylight-food relationship. In the present experiment with rats lever insertion as the conditioned stimulus (CS) preceded sucrose delivery as the unconditioned stimulus (US). Half the rats initially received unsignaled USs in the conditioning context, while the remainder did not. Both lever-presses (sign-tracking) and magazine-entries (goal-tracking) were recorded. Under immediate reinforcement conditions, prior unsignaled US interfered with sign-tracking, but had no effect on goal-tracking. In two further groups, a trace condition prevented development of sign-tracking. In this case, prior context conditioning interfered with goal-tracking. These results suggest that interference with sign-tracking may reflect response competition, while interference with goal-tracking under trace conditions may reflect failure to acquire a CS-US association.     

The representation of the reinforcer and the force of the pigeon's keypeck in first and second-order conditioning

The aim of this study was to determine whether reinforcer-specific conditioned responding would occur in a situation in which responding was not thought to be mediated by a representation that encodes information about the specific properties of the reinforcer. The force of the pigeon's keypeck was monitored during first and second-order conditioning with either food or water as the unconditioned stimulus (US). Each pigeon was trained with four different stimuli: a first-order cue predicting that responding would be reinforced with grain (S1f), a first-order cue predicting water as the reinforcer (S1w), a second-order stimulus predicting the S1f (S2f), and a second-order cue predicting the S1w (S2w). Following conditioning, the pigeons were selectively satiated with one of the two reinforcers and presented with the first and second-order cues in an extinction test. At the end of training, the pigeon's keypecks were less forceful to the S1w than to the S1f. There was not, however, a reliable difference between the force of the pecks to the S2f and the S2w. These force differences are consistent with the conclusion that the topography of the keypecks was systematically related to the nature of the primary reinforcer during first but not during second-order conditioning. The results from the selective satiation test are difficult to interpret. There was no evidence to indicate that second-order responding was mediated by a detailed representation of the primary US, but a detailed representation of the reinforcer may have been mediating first-order responding. Taken together, these findings are consistent with the view that a representation of the reinforcer is an important determinant of the topography of conditioned responding.     

A comparison of pecking generated by serial, delay, and trace autoshaping procedures

Pigeons were exposed to serial, delay, and trace autoshaping procedures. In Experiment I, all conditioned stimuli (CSs) were changes in illumination of the response key. The number of trials to acquisition of the keypeck increased from serial, to 4-sec delay, 8-sec delay, and 8-sec trace procedures, in that order. In Experiment II, which used a longer intertrial interval, trials to criterion increased from 8-sec delay, to 28-sec delay, 8-sec trace, and 28-sec trace procedures, in that order. In Experiment III, two groups received serial procedures in which the first CS was either a tone or a houselight, and the second was a keylight. The tone group acquired the key peck more rapidly than the houselight group. Early in conditioning in these experiments, and when the conditioned stimulus was a change in the keylight, there was a short latency to the onset of pecking and pecking was directed at the CS. After extensive conditioning, or when the CS was relatively diffuse, pecking still occurred, but had a longer latency and was not reliably directed toward the conditioned stimulus.     

Contributions of elicitation to measures of self-control

Pigeons' not pecking or pecking constituted choice between a delayed, large reinforcer and an immediate, small reinforcer (self-control) and at other times between a delayed reinforcer and no reinforcer (omission). Both a tone and a keylight were tested as choice signals, and the delayed reinforcer was either response independent or response dependent. Pigeons pecked during the choice signals on over 95% of the trials in the self-control procedure, and pecked during the choice signals on over 75% of the trials in the omission procedure. Consistent pecking was observed with either the tone or the keylight as a choice signal, with the exception that a tone paired with a response-independent delayed reinforcer did not maintain pecking in the omission procedure. Pigeons pecked during more choice signals when delayed reinforcers were response dependent than when the delayed reinforcers were response independent. These results indicate that Pavlovian conditioning influences self-control experiments, especially in single-key procedures.     

Blocking, unblocking, and overexpectation in autoshaping with pigeons

Three experiments used pigeons in an autoshaping procedure and a single-subject design to examine compound stimulus control in classical conditioning. Experiment 1 examined the blocking effect, and Experiment 2 examined the unblocking effect. In both experiments, response-independent food was first delivered intermittently in the presence of one distinctively colored houselight but not another. Then, conventional autoshaping trials were carried out in the presence of each houselight. In Experiment 1, the keylight readily elicited responding in the presence of the houselight that had been negatively correlated with food, but not in the presence of the houselight that had been positively correlated with food. In Experiment 2, the keylight readily elicited responding in the presence of the houselight positively correlated with food, but only when the amount of food used on the autoshaping trials was either greater or less than that previously delivered in the presence of the houselight. Experiment 3 examined the overexpectation effect. Conventional autoshaping trials were first carried out by presenting each of two keylights individually. Then, additional autoshaping trials were carried out by presenting the two keylights as a compound, with either the same amount of food or a greater amount of food per trial. Finally, the keylights were retested by again presenting them individually. The number of responses per trial elicited by the keylights decreased when the amount of food used in compound trials was the same as that used in individual trials. However, the number of responses per trial remained approximately the same when the amount of food used in compound trials was greater than that used in individual trials. Taken together, the results of the three experiments demonstrate (a) the generality of the blocking, unblocking, and overexpectation effects by virtue of their extension to appetitive unconditioned stimuli; (b) the suitability of pigeons as subjects and autoshaping as a procedure for studying classical conditioning; and (c) the appropriateness of single-subject designs.     

Aspects of the reinforcer learned in second-order Pavlovian conditioning

Four experiments used an autoshaping procedure in pigeons to explore learning about the reinforcer in a second-order conditioning paradigm. Experiment 1 conditioned two visual second-order stimuli (S2), using as reinforcers two visual first-order stimuli (S1), each of which had previously been paired with food. Animals for which the S2 stimuli were each consistently paired with one particular S1 developed second-order responding more rapidly than did animals for which the identity of S1 varied from trial to trial. Moreover, following consistent pairings, extinction of an S1 had a depressive effect upon second-order responding which was peculiar to the S2 with which it had been paired. Both results suggest that in this preparation the organism identifies a particular S1 as the reinforcer for each S2. The remaining experiments examined the details of that identification. A compound S1, itself composed of two separable elements, was used to reinforce an S2. Subsequent extinction of either element of S1 led to a depression in the responding to S2, which indicates that both elements were involved in the second-order conditioning. Moreover, the use of several complex discriminations, which produced different behavior to S1 and to its elements, suggested that the organism had associated the S2 with the compound S1 rather than with its separate elements. However, even complete extinction of the response to S1 left some residual behavior to S2, which indicates that a portion of the second-order conditioning is independent of the current state of the reinforcer. These results demonstrate that in some situations the organism associates a conditioned stimulus with a rich representation of the reinforcer.     

Superconditioning from a reduced reinforcer

In four autoshaping experiments pigeons received conditioned inhibition training of the form A++, AB-, where ++ is a strong reinforcer and - is nonreinforcement. Subsequent AB+ training, in which + is a moderate reinforcer, resulted in enhanced conditioning of A, relative to an A stimulus receiving no treatment, one receiving A+ treatment, and one receiving A++ treatment. This enhancement of conditioning to Aconstitutes a demonstration of “superconditioning”. The presence of the inhibitory B sufficiently enhanced the reinforcing power of the moderate reinforcer (+) that it was able to further increase the excitation controlled by A. This occurred even though A had previously been paired with a stronger reinforcer (++). Superconditioning was also observed when A and B were extinguished prior to treatment with + or were originally neutral stimuli followed by + in the presence of an inhibitor trained elsewhere.     

Positive conditioned suppression: conditioned suppression using positive reinforcers as the unconditioned stimuli

Research has revealed the phenomenon of conditioned suppression in which the rate of responding is reduced during a stimulus that is paired with noncontingent shock. The present study replicated this procedure, but used noncontingent positive reinforcers instead of the aversive shock. The lever-pressing responses of rats were reinforced with food or water. While the rats were responding, a stimulus was occasionally presented and paired with the delivery of a noncontingent positive reinforcer, which was either food, water, or brain stimulation for different rats. The result was a reduction in the rate of responding during the conditioned stimulus. This finding shows that conditioned suppression occurs during a signal for reinforcing as well as aversive stimuli.     

THE “WHERE IS IT?” REFLEX: AUTOSHAPING THE ORIENTING RESPONSE

The goal of this review is to compare two divergent lines of research on signal-centered behavior: the orienting reflex (OR) and autoshaping. A review of conditioning experiments in animals and humans suggests that the novelty hypothesis of the OR is no longer tenable. Only stimuli that represent biological “relevance” elicit ORs. A stimulus may be relevant a priori (i.e., unconditioned) or as a result of conditioning. Exposure to a conditioned stimulus (CS) that predicts a positive reinforcer causes the animal to orient to it throughout conditioning. Within the CS-US interval, the initial CS-directed orienting response is followed by US-directed tendencies. Experimental evidence is shown that the development and maintenance of the conditioned OR occur in a similar fashion both in response-independent (classical) and response-dependent (instrumental) paradigms. It is proposed that the conditioned OR and the signal-directed autoshaped response are identical. Signals predicting aversive events repel the subject from the source of the CS. It is suggested that the function of the CS is not only to signal the probability of US occurrence, but also to serve as a spatial cue to guide the animal in the environment.     

A comparison of neuronal reactions during classical and instrumental conditioning under similar conditions

During the elaboration of an instrumental reflex, it is not obligatory to use a conditioned stimulus, which signals the necessity to generate an instrumental reaction in order to receive reinforcement. However, the presence of a conditioned stimulus simplifies analysis of instrumental reaction, which in this case is the response to the conditioned stimulus. On the other hand, it is necessary to distinguish between instrumental and classical conditioning, since in both cases the response to a conditioned stimulus increases. We studied neuronal analogs of classical and instrumental conditioning in the identified neurons responsible for the defensive closure of the pneumostome in the Helix mollusk under the same conditions. During classical conditioning, a mollusk received punishment after a tactile stimulus. During instrumental conditioning, a mollusk received punishment when an identified neuron did not generate an action potential in response to a tactile stimulus. The appearance of a painful stimulus did not depend on the generation or failure of a spike in the related control neuron. Another tactile stimulus, which was never paired with an unconditioned stimulus, was used as a discriminated stimulus. We also compared the behavior of such identified neurons during pseudoconditioning. The experiments were carried out in a semi-intact preparation. We examined how responses to the tactile and painful stimuli changed during different forms of training. It was shown that the dynamics of neuronal responses to a conditioned tactile stimulus were much more complex during instrumental conditioning and consisted of several phases. Throughout a learning session, neural system consecutively acquired information as to which kind of learning was presented, whether a reaction of the neural system must be generated or inhibited and which instrumental reaction is correct. We have demonstrated that response to a painful stimulus during classical conditioning decreases after short-term initial increase. However, during instrumental learning, the neurons controlling instrumental action remained highly sensitive to the unconditioned stimulus. Meanwhile, foreign neurons decreased their responses to the unconditioned stimulus. We may tentatively conclude that classical and instrumental paradigms are fundamentally different at the cellular level.     

An analysis of second-order autoshaping

Three mechanisms can explain second-order conditioning: (1) The second-order conditioned stimulus (CS2) could activate a representation of the first-order conditioned stimulus (CS1), thereby provoking the conditioned response (CR); The CS2 could enter into an excitatory association with either (2) the representation governing the CR, or (3) with a representation of the reinforcer evoked by the CS1. A series of experiments using second-order autoshaping with birds was performed to examine these possibilities. Following second-order autoshaping, birds’ responding to the CS2 was found to be unaffected by extinction of the CS1, a result interpreted as showing mechanism (1) to be unimportant and implicating either one or both of the other mechanisms. The CS2 was also shown to provide the birds with specific sensory information about the reinforcer, a pattern of results uniquely predicted by mechanism (3). Implications for the understanding of second-order conditioning and for the SOP model (Wagner, 1981) of associative learning are discussed.     

Key-peck probability and topography in a concurrent variable-interval variable-interval schedule with food and water reinforcers.

The relation between variables that modulate the probability and the topography of key pecks was examined using a concurrent variable-interval variable-interval schedule with food and water reinforcers. Measures of response probability (response rates, time allocation) and topography (peck duration, gape amplitude) were obtained in 5 water- and food-deprived pigeons. Key color signaled reinforcer type. During baseline, response rates and time allocations were greater to the food key than to the water key, and food-key pecks had larger gapes and shorter durations. Relative probability measures (for the food key) were increased by prewatering and decreased by prefeeding. Deprivation effects upon topography measures were apparent only when food- and water-key pecks were analyzed separately. Food-key gape amplitudes increased with prewatering and decreased with prefeeding. The clearest effect occurred with prewatering. There were no consistent effects upon water-key gapes. The key color-reinforcer relation was reversed for 3 pigeons to determine how response topography was modulated during the transition from food- to water-key pecks. Reacquisition was faster for the probability than for the topography measures. Analysis of gape-amplitude distributions during reversal indicated that response-form modulation proceeded through the generation of intermediate gape sizes.