Acquisition and extinction in autoshaping

C. R. Gallistel and J. Gibbon (2000) presented quantitative data on the speed with which animals acquire behavioral responses during autoshaping, together with a statistical model of learning intended to account for them. Although this model captures the form of the dependencies among critical variables, its detailed predictions are substantially at variance with the data. In the present article, further key data on the speed of acquisition are used to motivate an alternative model of learning, in which animals can be interpreted as paying different amounts of attention to stimuli according to estimates of their differential reliabilities as predictors.     

Higher order autoshaping

A series of experiments is reported on appetitive higher order conditioning in the pigeon. Experiment I showed that second order autoshaping can be produced by pairing a neutral keylight with a keylight of another colour, previously paired with food. Experiment II employed an omission procedure to show that second order autoshaping is a consequence of the contingency between first and second order stimuli. In Experiment III, extinction of responding to the first order stimulus was shown to reduce responding to the second order stimulus. Experiments IV and V showed firstly that this reduction is not due to generalization of extinction, and secondly that second order key pecks may be produced in the absence of any pecking to the first order stimulus. The results suggest that second order autoshaping is based largely on a direct association between the first and second order stimuli.     

Contextual specificity of extinction of delay but not trace eyeblink conditioning in humans

Renewal of an extinguished conditioned response has been demonstrated in humans and in animals using various types of procedures, except renewal of motor learning such as eyeblink conditioning. We tested renewal of delay and trace eyeblink conditioning in a virtual environment in an ABA design. Following acquisition in one context (A, e.g., an airport) and extinction in a different context (B, e.g., a city), tests for renewal took place in the acquisition (A) and extinction context (B), in a counterbalanced order. Results showed renewal of the extinguished conditioned response in the delay but not trace condition.     

Temporal specificity of perceptual learning in an auditory discrimination task

Although temporal processing is used in a wide range of sensory and motor tasks, there is little evidence as to whether a single centralized clock or a distributed system underlies timing in the range of tens to hundreds of milliseconds. We investigated this question by studying whether learning on an auditory interval discrimination task generalizes across stimulus types, intervals, and frequencies. The degree to which improvements in timing carry over to different stimulus features constrains the neural mechanisms underlying timing. Human subjects trained on a 100- or 200-msec interval discrimination task showed an improvement in temporal resolution. This learning generalized to a perceptually distinct duration stimulus, as well as to the trained interval presented with tones at untrained spectral frequencies. The improvement in performance did not generalize to untrained intervals. To determine if spectral generalization was dependent on the importance of frequency information in the task, subjects were simultaneously trained on two different intervals identified by frequency. As a whole, our results indicate that the brain uses circuits that are dedicated to specific time spans, and that each circuit processes stimuli across nontemporal stimulus features. The patterns of generalization additionally indicate that temporal learning does not rely on changes in early, subcortical processing, because the nontemporal features are encoded by different channels at early stages.     

Behavioural contrast and autoshaping

This experiment was designed to investigate the importance of autoshaping to a signal for reinforcement in the production of behavioural contrast. Two groups of pigeons were given discrimination training on a mult VI-EXT schedule: the stimuli present in the two schedule components shared common attributes, but were distinguished by the presence or absence of a visual feature. For one group (the feature positive group) the feature signalled the availability of reinforcement. For the other group (the feature negative group) the feature signalled nonrein-forcement, and for this group there was no stimulus element which unambiguously signalled reinforcement. The feature positive group showed a higher response rate during the VI component of the mult VI-EXT schedule than the feature negative group. This finding was interpreted as support for the autoshaping explanation of behavioural contrast. The results differed from those of Jenkins and Sainsbury (1969, 1970) in that both the feature positive and the feature negative groups showed discrimination learning.     

The effects of ITI fillers in autoshaping

The present experiments used pigeons in an autoshaping procedure to examine the effects of a nonreinforced, nontarget stimulus presented during the intertrial interval on responding to a target CS. Experiments 1 and 2 found that a filler stimulus presented during a substantial portion of the ITI retarded responding to the target CS relative to a group not exposed to the filler. This group difference in performance was subsequently abolished by continued training and omitting the filler in the former group (Experiment 1) or adding a filler for the latter group (Experiment 2). Experiment 3 found that when the “filler” occupied most of the interreinforcement interval, CS-US pairings embedded within the “filler” stimulus yielded superior autoshaping relative to a group that received CS-US pairings embedded in static apparatus cues in the absence of the filler. The results are discussed with reference to the ways that a nontarget stimulus during the ITI can influence contextual modulation of responding to a discrete CS and the necessity for comparator theories to incorporate a “local context” view of cycle time to explain the present findings.     

An analysis of second-order autoshaping

Three mechanisms can explain second-order conditioning: (1) The second-order conditioned stimulus (CS2) could activate a representation of the first-order conditioned stimulus (CS1), thereby provoking the conditioned response (CR); The CS2 could enter into an excitatory association with either (2) the representation governing the CR, or (3) with a representation of the reinforcer evoked by the CS1. A series of experiments using second-order autoshaping with birds was performed to examine these possibilities. Following second-order autoshaping, birds’ responding to the CS2 was found to be unaffected by extinction of the CS1, a result interpreted as showing mechanism (1) to be unimportant and implicating either one or both of the other mechanisms. The CS2 was also shown to provide the birds with specific sensory information about the reinforcer, a pattern of results uniquely predicted by mechanism (3). Implications for the understanding of second-order conditioning and for the SOP model (Wagner, 1981) of associative learning are discussed.     

The role of local context in autoshaping

Three experiments used an autoshaping procedure with pigeons to examine the effects of nonreinforced, nontarget stimuli (ITI-fillers) during the intertrial interval on responding to a reinforced target CS. Experiment 1 replicated a previous demonstration that an ITI-filler that occupied a substantial portion of the ITI attenuated responding to the target CS relative to a group trained with a similar ITI but lacking the ITI-filler. Experiment 2 found that the superiority of responding typically found with a long ITI relative to a short ITI, that is, the trial-spacing effect, can be reversed by imposing a filler stimulus during the ITI in the former condition. Experiment 3, using a within-subject design, found that when one background stimulus condition occupied a majority of the interreinforcement interval, pairings of one target CS with reinforcement that were embedded within this background condition, that is, a long duration local context occurred for this CS, yielded better performance that a second, reinforced, target CS paired with reinforcement in the shorter duration background condition. These results confirmed predictions derived from a local context view of cycle time. Comparator theories of classical conditioning require incorporation of this notion into their conceptualization of effective cycle time in order to explain the present findings.     

Temporal specificity in cross-modal transfer of the rabbit nictitating membrane response

Two experiments examine cross-modal transfer of response features specific to the interstimulus interval (ISI) between a conditioned stimulus (CS) and an unconditioned stimulus. Rabbits were given initial training with a stimulus (CSA) in one modality (e.g., tone) at a designated ISI (e.g., 600 ms). Training was then shifted to a new stimulus (CSB) in another modality (e.g., light) at a new ISI (e.g., 400 ms). The timing of early conditioned responses (CRs) to CSB reflected the ISI of CSA. Ultimately, CRs to CSB shifted to a temporal location conforming to the ISI of CSB. When the ISI of CSB was shorter than that of CSA, CRs to CSA also shifted to a locus conforming to the ISI of CSB. The present results confirmed previous findings that training in one CS modality accelerates CR acquisition to a CS in another modality. The findings are compared with the transfer of response patterns in instrumental learning sets and are discussed regarding their implications for theories of cross-modal transfer.     

Simultaneous and sequential conditioned inhibition in autoshaping

Two experiments examined sequential and simultaneous conditioned inhibition in an autoshaping preparation. In Experiment 1, a 5-sec coloured keylight (A) was reinforced when presented alone. However, reinforcement was withheld from trials on which A was preceded by one grid pattern or presented simultaneously with another pattern. A third (control) pattern was presented separately without reinforcement. Inhibition developed most strongly to the grid pattern presented simultaneously with A. However, a transfer test with another excitatory coloured keylight found both inhibitors to be equally strong whether they were tested in simultaneous or in sequential compound. Experiment 2 compared grid patterns that consistently had sequential or simultaneous relations to the excitor with one that was sometimes presented in sequential and sometimes in simultaneous compound. The mixed treatment pattern showed levels of inhibition like the simultaneous or sequential pattern, depending on the temporal relation it bore to A on a particular trial. These results suggest that sequential and simultaneous relations between inhibitors and excitors yield similar amounts of inhibition. However, that inhibition is better exhibited when tested in simultaneous compound with the excitor.     

Effect of signaling intertrial unconditioned stimuli in autoshaping

Context-unconditioned-stimulus (US) associations have been suggested as the mediator of the response decrement that occurs when extra USs are added to the intertrial intervals (ITIs) of an otherwise standard Pavlovian conditioning situation. The present autoshaping experiments were concerned with the effect of signaling those extra USs, since such signaling might be expected to lessen their ability to condition the context. Experiments 1 and 2 showed that signaling the ITI USs did reduce their detrimental effects on responding to the conditioned stimulus (CS). To determine whether that reduction was due to an impact of signaling on the target-CS-US association or on performance to the target-CS, Experiment 3 examined responding to differentially trained CSs in a common context, as well as responding to identically trained CSs in differentially trained contexts. Whether the CS was tested in a context of relatively high or low associative strength, more responding occurred to the CS trained with signaled, as compared with unsignaled, ITI USs; further, there was more responding to that CS in the more highly valued context. The pattern of results suggests that contextual value does interact with CS-US learning and may also affect performance to the CS.     

The effect of intertrial conditioned stimuli in autoshaping

Three autoshaping experiments, using pigeon subjects, examined the effect of presenting an S+ or an S- on acquisition to a different conditioned stimulus (CS). Experiment 1 found that signalling intertrial food with an S+ or an S- allowed acquisition of responding to the target-CS; however, that acquisition was slower when the signal was the S+. Experiment 2 found that even when the S+ or S- were presented without food in the intertrial interval, acquisition was slower in the group receiving the S+. Experiment 3 found that a similar, but weaker effect occurred when the S+ or S- stimuli were presented in the session prior to target-CS training, rather than intermixed. These results have implications for the interpretation of experiments that use a signalling manipulation to assess the interaction of CS and context in Pavlovian conditioning; they suggest that signalling unconditioned stimuli (USs) may have consequences other than that of modulating context-US learning.     

The associative relation underlying autoshaping in the pigeon

Fifteen pigeons were exposed to either response-independent or response-dependent schedules of water reinforcement, whereby water was injected directly into the unrestrained pigeons' mandibles. Key-contact responses were released by a lighted key correlated with water, but not by a lighted key uncorrelated with water. A negative response-reinforcer contingency suppressed autoshaped key-contact responses, resulting in responding directed away from the lighted key. In all pigeons, water injected directly into the mandibles elicited a consummatory fixed-action pattern of “mumbling” and swallowing. The lighted key correlated with water released a broader set of both appetitive and consummatory responses: approach to the lighted key, “bowing”, “rooting”, “mumbling”, and swallowing. Key-contact responses were “rooting” and “mumbling” motions of the beak on the surface of the key. Views of autoshaping based on stimulus substitution or stimulus surrogation do not fully explain the origin of autoshaped responses not previously elicited by the reinforcer. The present findings are consonant with views of conditioning that emphasize the large degree of biological pre-organization in conditioned response patterns, and the importance of associative factors in the control of such patterns.     

Errorless discrimination established by differential autoshaping

In Experiment I, pigeons exposed to a differential autoshaping procedure pecked a key in the presence of the stimulus associated with reinforcement but did not peck, or pecked infrequently, in the presence of the stimulus associated with nonreinforcement. In Experiment II, pigeons were exposed to a differential autoshaping procedure in which one stimulus was associated with reinforcement and two stimuli were associated with nonreinforcement. The birds initially responded in the presence of one stimulus associated with nonreinforcement but never responded in the presence of the second stimulus associated with nonreinforcement. They were subsequently exposed to an autoshaping procedure in which reinforcement followed both these stimuli. The number of stimulus-reinforcement pairings required to establish pecking in the presence of the stimulus during which responses had not previously occurred suggested that such stimuli are inhibitory. These findings have implications for autoshaping, errorless discrimination, inhibition, and theories of discrimination byproducts.     

Context blocking in rat autoshaping: Sign-tracking versus goal-tracking

Prior experience of unsignaled food can interfere with subsequent acquisition by birds of autoshaped key-pecking at a signal light. This has been understood to indicate that unsignaled food results in context conditioning, which blocks subsequent learning about the keylight-food relationship. In the present experiment with rats lever insertion as the conditioned stimulus (CS) preceded sucrose delivery as the unconditioned stimulus (US). Half the rats initially received unsignaled USs in the conditioning context, while the remainder did not. Both lever-presses (sign-tracking) and magazine-entries (goal-tracking) were recorded. Under immediate reinforcement conditions, prior unsignaled US interfered with sign-tracking, but had no effect on goal-tracking. In two further groups, a trace condition prevented development of sign-tracking. In this case, prior context conditioning interfered with goal-tracking. These results suggest that interference with sign-tracking may reflect response competition, while interference with goal-tracking under trace conditions may reflect failure to acquire a CS-US association.     

The role of context in intertrial interval effects in autoshaping

Two experiments evaluated the role of differential conditioning of the context in mediating the effect of intertrial interval (ITI) in autoshaping. In Experiment 1 pigeons were given acquisition with two keylights, each presented in a particular context. A given keylight/context combination had associated with it either a short (10-sec) or a long (2-min) ITI. Acquisition was more rapid with the long ITI. Tests with those keylights in a common third context indicated that the longer ITI had resulted in greater conditioning. On the other hand, pigeons trained on keylights with mixed ITIs in a third context evoked more responding when they were tested in the short ITI context compared with the long ITI context. That suggests that a context with a history of a short ITI enhances performance. In Experiment 2, two keylights were initially conditioned with mixed ITIs and then extinguished in different contexts under different ITI lengths. Extinction was more rapid for the keylight presented with a short ITI. That difference persisted when the keylights were tested with mixed ITIs in a common third context, suggesting a difference in associative strength of the keylights. The results are interpreted in terms of differential context conditioning resulting in differences in learning about the keylight.     

Generalization gradients following differential intradimensional autoshaping.

Three pigeons were trained on a differential, intradimensional autoshaped discrimination. A 45 degrees line tilt was always paired with food whereas a 15 degrees line tilt was never paired with food. All subjects learned the discrimination within 17 sessions. The pigeons were then given generalization tests in extinction over seven line tilts (0 degrees, 15 degrees, 30 degrees, 45 degrees, 60 degrees, 75 degrees and 90 degrees). The subjects yielded generalization gradients with maxima at 45 degrees and minima at 15 degrees. An area shift, but no peak shift, was found for each subject.     

The role of autoshaping in cooperative two-player games between starlings

We report a study of the behavior of starlings in laboratory situations inspired by the “prisoner's dilemma.” Our purpose is to investigate some possible mechanisms for the maintenance of cooperation by reciprocity and to investigate the process of autoshaping at a trial-by-trial level. In Experiment 1, pairs of starlings housed in adjacent cages played a discrete-trial “game” in which food could be obtained only by “cooperation.” In this game, pecking at a response key eliminated the opportunity to obtain food but produced food for the partner. If neither bird pecked, neither had the opportunity to obtain food in that trial. Some level of cooperation persisted for several sessions whether the birds had been pretrained for a high or low probability of pecking at the key. The probability of a cooperative response was higher after trials in which the partner responded (and a reward was obtained) than after trials in which neither bird responded (and no reward was obtained), but the probability of a response was even higher after trials in which the same bird had responded, even though no reward was obtained by the actor in these trials. This behavior did not require visual presence of another player, because similar results were obtained in Experiment 2 (a replicate of Experiment 1 in which the members of the pair could not see each other) and in Experiment 3, a game in which each starling played with a computer responding with “tit for tat.” Using an omission schedule, in which food was given in all trials in which the bird did not peck, Experiment 4 showed that pecking could be maintained by autoshaping. In this experiment, overall probability of pecking decreased with experience, due to a drop in the tendency to peck in consecutive trials. The probability of pecking in trials following a reinforced trial did not decrease with experience. An implementation of the Rescorla–Wagner model for this situation was capable of reproducing molar, but not molecular, aspects of our results. The results violate the predictions of several game-theoretical models for the evolution of cooperation, including tit for tat, generous tit for tat, and the superior win-stay-lose-shift.     

The autoshaping procedure as a residual block clock

In the first experiment, 4 pigeons were each presented with a recurring sequence of four key colors followed by the delivery of grain (block clock). Once the rate of pecking had stabilized, three of the colors were replaced, during different series of sessions, by a darkening of the key. The rate of pecking was reduced within those segments of the interval between deliveries of food during which the key was dark; when the key was dark during the final portion of the interval, rates were reduced throughout the entire interval. In the second experiment, 3 new pigeons were exposed to a different sequence of colors, and the final stimulus was replaced in successive conditions by a novel color, a darkened key, and a restoration of the original color. The data indicated that darkening the key had a more severe, more extensive, and more persistent effect than did a mere change in color. These results suggest that it may be fruitful to conceptualize the autoshaping procedure as a special version of the block clock in which pecking is suppressed throughout the greater part of the interval by darkening the key. In the final condition, the same stimulus appeared in each of the last three portions of the interval. The rate of pecking was lower during the last two portions than when distinctive colors were presented, with the peak rate now appearing in the fifth of seven equal temporal components.