Control of responding by sounds of different quality: an evolutionary analysis.

Two experiments investigated the acquisition of discriminations between two acoustic stimuli of different quality (noise bursts vs. a 2-kHz pulsed signal) when features of the everyday environment were incorporated into the experiments. In Experiment 1, rats were trained, using food, to press a lever. Throughout all sessions, 5-s trials of noise bursts (the random stimulus) were presented, after variable intertrial intervals, through a remote speaker mounted outside the experimental enclosure. The noise burst occurred randomly with respect to reinforcement of lever pressing and had no programmed relationship to the animal's behavior. When lever pressing was established, the 2-kHz signal was presented through a speaker adjacent to the response lever according to a different set of variable intertrial intervals. A response in the presence of the 2-kHz signal terminated the trial and was reinforced. The 2-kHz signal acquired control of responding within the first few trials, whereas the random stimulus exerted no control of responding. In Experiment 2, rats were trained to press the lever in the presence of the 2-kHz signal presented through the adjacent speaker on a variable intertrial interval. After 14 sessions, 5-s trials of noise bursts (random stimulus) were presented through the remote speaker on the second variable intertrial interval. The random stimulus initially elicited exploratory behavior, which then rapidly declined. Subsequently, the random stimulus exerted no or weak control of responding. The introduction of the random stimulus had no effect on responding in the presence of the adjacent stimulus. In Experiments 3 and 4 the random stimulus was presented through the adjacent speaker, and the stimulus correlated with reinforcement was presented through the remote speaker. In both experiments, there was persistent control of responding by the random stimulus and slow development of control by the stimulus correlated with reinforcement. In Experiment 5, both stimuli were presented through the adjacent speaker. There was persistent control of responding by the random stimulus.     

Blocking between occasion setters and contextual stimuli

Two appetitive conditioning experiments with rats used a blocking procedure to compare the mechanisms through which contexts and positive occasion setters control responding to conditioned stimuli (CSs) in discrimination learning. In Experiment 1, a light (L) initially set the occasion for food reinforcement of a tone CS (T). Then a compound of L and a novel context signaled reinforcement of T. Previous learning about L blocked contextual control of responding to T. Blocking was not due to simple excitation conditioned to L during discrimination training; comparable excitation to L in a control group did not result in blocking. Conversely, in Experiment 2, initial learning about the context blocked the acquisition of occasion setting to L. Excitation conditioned to the context could not account for the blocking. In general, the results suggest that contexts and occasion setters may control responding to CSs through similar mechanisms.     

Overexpectation: response loss during sustained stimulus compounding in the rabbit nictitating membrane preparation

Rabbits were given reinforced training of the nictitating membrane (NM) response using separate conditioned stimuli (CSs), which were a tone, light, and/or tactile vibration. Then, two CSs were compounded and given further pairings with the unconditioned stimulus (US). Evidence of both overexpectation and summation effects appeared. That is, responding to the individual CSs declined despite their continued pairing with the US on compound trials (overexpectation), and responding on the compound trials was greater than responding to the individual CSs (summation). The response loss appeared regardless of the testing regime, that is, whether the test presentations of the individual CSs were themselves reinforced (Experiment 2), not reinforced (Experiment 1), or deferred until the end of compound training (Experiment 2). The results are discussed with respect to the roles of excitatory versus inhibitory processes, elemental versus configural processes, and the possible roles of cerebellar and hippocampal pathways.     

Duration of signals for intertrial reinforcement and nonreinforcement in random control procedures

In the random control procedure, responding to a conditioned stimulus (target CS) is prevented when the probability of unsignaled, unconditioned stimuli (USs) in the intertrial interval (ITI) is equal to the probability of the US in the presence of the target CS. Three experiments used an autoshaping procedure with White Carneaux pigeons to examine the effects of the temporal duration of signals for the ITI USs (cover CSs) and for concomitant periods of nonreinforcement. In Experiment 1, a short duration cover, but not a long duration cover, resulted in responding to the target CS. In Experiment 2, an explicit CS- cue during periods of nonreinforcement did not affect target acquisition. In Experiment 3, a long CS-, but not a short cover CS, was a sufficient condition for the acquisition of responding to the target CS. These results imply that the acquisition of responding to a target CS requires a discriminable period of nonreinforcement that is long relative to the target CS duration.     

Effects of contextual conditioning and unconditional stimulus presentation on performance in appetitive conditioning

Four experiments with rat subjects examined the effects of contextual conditioning on conditioned appetitive performance. Experiment 1 compared the effects of contextual conditioning on performance to conditioned stimuli (CSs) with different conditioning histories. Contextual conditioning enhanced performance to the CS if the CS had first been conditioned and then extinguished, but had no effect on performance when the CS had been merely paired or unpaired with food. Experiments 2 and 3 then asked whether the effect on the extinguished CS was due to contextual conditioning acting as a cue for conditioning. In Experiment 2, extinction procedures in which extra unconditioned stimuli (USs) were presented during the intertrial intervals were found to reduce the CS's sensitivity to enhancement by contextual conditioning, but had no effect on spontaneous recovery. In Experiment 3, USs added to conditioning or extinction acquired the ability to cue the corresponding performance. Under some conditions, USs added to conditioning could suppress performance (Experiment 4). The results suggest that contextual conditioning has complex effects that can be better understood by recognizing that contextual conditioning, as well as the USs that create it,Mayacquire discriminative control over conditioned responding.     

Responding to a positive stimulus by “satiated” pigeons

Skinner (1938) found that rats given discrimination training (Phase I) and then reinforced to “satiation” for responses in the presence of the negative stimulus (S?) (Phase II), began to respond again when the positive stimulus (S+) was reintroduced (Phase III). Experiment I replicated Skinner's finding with pigeons, alternating S+ and S? presentations during Phase III. In Experiment II, Phase II was extended, and Phase III results were similar to those of Experiment I, demonstrating that the recovery of S+ responding could not be attributed to a lax Phase II satiation criterion. In Experiment III, a uniform schedule of reinforcement was maintained throughout the three phases, and results similar to those of Experiment I were found, indicating that renewed S+ responding was not due to the shift in schedule between phases. In Experiment IV, Phase I consisted of discrimination training with two positive stimuli (S+_s and S+_n), and Phase II consisted of reinforcement for responses in the presence of S? and S+_s. During Phase III, significantly more responding was found to S+_s and S+_n than to S?, but no difference in responding was found between the two positive stimuli. In Experiment V, Phase I consisted of simple discrimination training, and during Phase II, responses in the presence of both S? and a novel stimulus (So) were reinforced. During Phase III, significantly more responding was found to S+ than to either S? or So, with no difference found between S? and So responding. Renewed responding to S+ during Phase III in the present experiments is best explained by behavioral contrast developed during Phase I.     

The impact of arbitrarily applicable relational responding on evaluative learning about hypothetical money and shock outcomes

Evaluative learning comprises changes in preferences after co-occurrences between conditioned stimuli (CSs) and an unconditioned stimulus (US) of affective value. Co-occurrences may involve relational responding. Two experiments examined the impact of arbitrary relational responding on evaluative preferences for hypothetical money and shock outcomes. In Experiment 1, participants were trained to make arbitrary relational responses by placing CSs of the same size but different colours into boxes and were then instructed that these CSs represented different intensities of hypothetical USs (money or shock). Liking ratings of the CSs were altered in accordance with the underlying bigger/smaller than relations. A reversal of preference was also observed: the CS associated with the smallest hypothetical shock was rated more positively than the CS associated with the smallest amount of hypothetical money. In Experiment 2, procedures from Relational Frame Theory (RFT) established a relational network of more than/less than relations consisting of five CSs (A-B-C-D-E). Overall, evaluative preferences were altered, but not reversed, depending on (a) how stimuli had been related to one another during the learning phase and (b) whether those stimuli referred to money or shocks. The contribution of RFT to evaluative learning research is discussed.     

The effect of intertrial conditioned stimuli in autoshaping

Three autoshaping experiments, using pigeon subjects, examined the effect of presenting an S+ or an S- on acquisition to a different conditioned stimulus (CS). Experiment 1 found that signalling intertrial food with an S+ or an S- allowed acquisition of responding to the target-CS; however, that acquisition was slower when the signal was the S+. Experiment 2 found that even when the S+ or S- were presented without food in the intertrial interval, acquisition was slower in the group receiving the S+. Experiment 3 found that a similar, but weaker effect occurred when the S+ or S- stimuli were presented in the session prior to target-CS training, rather than intermixed. These results have implications for the interpretation of experiments that use a signalling manipulation to assess the interaction of CS and context in Pavlovian conditioning; they suggest that signalling unconditioned stimuli (USs) may have consequences other than that of modulating context-US learning.     

Effect of signaling intertrial unconditioned stimuli in autoshaping

Context-unconditioned-stimulus (US) associations have been suggested as the mediator of the response decrement that occurs when extra USs are added to the intertrial intervals (ITIs) of an otherwise standard Pavlovian conditioning situation. The present autoshaping experiments were concerned with the effect of signaling those extra USs, since such signaling might be expected to lessen their ability to condition the context. Experiments 1 and 2 showed that signaling the ITI USs did reduce their detrimental effects on responding to the conditioned stimulus (CS). To determine whether that reduction was due to an impact of signaling on the target-CS-US association or on performance to the target-CS, Experiment 3 examined responding to differentially trained CSs in a common context, as well as responding to identically trained CSs in differentially trained contexts. Whether the CS was tested in a context of relatively high or low associative strength, more responding occurred to the CS trained with signaled, as compared with unsignaled, ITI USs; further, there was more responding to that CS in the more highly valued context. The pattern of results suggests that contextual value does interact with CS-US learning and may also affect performance to the CS.     

RESISTANCE TO CHANGE AND FREQUENCY OF RESPONSE‐DEPENDENT STIMULI UNCORRELATED WITH REINFORCEMENT

Stimuli uncorrelated with reinforcement have been shown to enhance response rates and resistance to disruption; however, the effects of different rates of stimulus presentations have not been assessed. In two experiments, we assessed the effects of adding different rates of response‐dependent brief stimuli uncorrelated with primary reinforcement on relative response rates and resistance to change. In both experiments, pigeons responded on variable‐interval 60‐s schedules of food reinforcement in two components of a multiple schedule, and brief response‐dependent keylight‐color changes were added to one or both components. Although relative response rates were not systematically affected in either experiment, relative resistance to presession feeding and extinction were. In Experiment 1, adding stimuli on a variable‐interval schedule to one component of a multiple schedule either at a low rate (1 per min) for one group or at a high rate (4 per min) for another group similarly increased resistance to disruption in the components with added stimuli. When high and low rates of stimuli were presented across components (i.e., within subjects) in Experiment 2, however, relative resistance to disruption was greater in the component presenting stimuli at a lower rate. These results suggest that stimuli uncorrelated with food reinforcement do not strengthen responding in the same way as primary reinforcers.     

Acquisition with partial and continuous reinforcement in rat magazine approach

Partial reinforcement, compared with continuous reinforcement, is widely considered detrimental to Pavlovian conditioned responding. However, time-accumulation models predict an invariance in acquisition when learning is assessed as a function of number of reinforcements, not trials, and intertrial interval is held constant. Three experiments examined this prediction in a rat magazine-approach procedure. All experiments showed superior responding with continuous reinforcement. Experiments 1 and 3 used common tests in between- and within-subject designs, respectively. Experiment 2 showed the same pattern in a discrimination. Earlier results are reanalyzed informally and in a meta-analysis. Contrary to previous summaries of the literature, evidence points to superior conditioned responding with continuous reinforcement in a number of procedures. Results are generally consistent with traditional associative models of learning.     

Timeout postponement without increased reinforcement frequency

Three experiments were conducted to examine pigeons' postponement of signaled extinction periods (timeouts) from a schedule of food reinforcement when such responding neither decreased overall timeout frequency nor increased the overall frequency of food reinforcement. A discrete-trial procedure was used in which a response during the first 5 s of a trial postponed an otherwise immediate 60-s timeout to a later part of that same trial but had no effect on whether the timeout occurred. During time-in periods, responses on a second key produced food according to a random-interval 20-s schedule. In Experiment 1, the response-timeout interval was 45 s under postponement conditions and 0 s under extinction conditions (responses were ineffective in postponing timeouts). The percentage of trials with a response was consistently high when the timeout-postponement contingency was in effect and decreased to low levels when it was discontinued under extinction conditions. In Experiment 2, the response-timeout interval was also 45 s but postponement responses increased the duration of the timeout, which varied from 60 s to 105 s across conditions. Postponement responding was maintained, generally at high levels, at all timeout durations, despite sometimes large decreases in the overall frequency of food reinforcement. In Experiment 3, timeout duration was held constant at 60 s while the response-timeout interval was varied systematically across conditions from 0 s to 45 s. Postponement responding was maintained under all conditions in which the response-timeout interval exceeded 0 s (the timeout interval in the absence of a response). In some conditions of Experiment 3, which were designed to control for the immediacy of food reinforcement and food-correlated (time-in) stimuli, responding postponed timeout but the timeout was delayed whether a response occurred or not. Responding was maintained for 2 of 3 subjects, suggesting that behavior was negatively reinforced by timeout postponement rather than positively reinforced by the more immediate presentation of food or food-correlated (time-in) stimuli.     

Resistance to change and relapse of observing

Four experiments examined relapse of extinguished observing behavior of pigeons using a two-component multiple schedule of observing-response procedures. In both components, unsignaled periods of variable-interval (VI) food reinforcement alternated with extinction and observing responses produced stimuli associated with the availability of the VI schedule (i.e., S+). The components differed in the rate of food arranged (Rich = VI 30 s; Lean = VI 120 s). In Experiment 1, following baseline training, extinction of observing involved removal of both food and S+ deliveries, and reinstatement was examined by presenting either response-independent food or S+ deliveries. In Experiment 2, extinction involved removal of only food deliveries while observing responses continued to produce S+. Reinstatement was examined by delivering food contingent upon the first two food-key responses occurring in the presence of the S+. Experiment 3 assessed ABA renewal of observing by extinguishing food-key and observing responses in the presence of one contextual stimulus (i.e., B) and then returning to the original training context (i.e., A) during continued extinction. Experiment 4 examined resurgence by introducing food reinforcement for an alternative response during extinction, and subsequently removing that alternative source of food. Across experiments, relative resistance to extinction and relapse of observing tended to be greater in the component previously associated with the higher rate of primary reinforcement. Relapse of observing or attending to stimuli associated with primary reinforcement appears to be impacted by frequency of primary reinforcement in a manner similar to responding maintained directly by primary reinforcement.     

Examining stimuli paired with alternative reinforcement to mitigate resurgence in children diagnosed with autism spectrum disorder and pigeons

In two laboratory experiments, we examined whether stimuli paired with alternative reinforcers could mitigate resurgence of a previously reinforced target response with pigeons (Experiment 1) and children diagnosed with Autism Spectrum Disorder (Experiment 2). In Phase 1, we arranged food reinforcement according to a variable-ratio schedule for engaging in a target response. In Phase 2, we arranged extinction for target responding and differentially reinforced alternative responding according to a fixed-ratio schedule, with every alternative-reinforcer delivery paired with a change in keylight color (Experiment 1) or automated verbal (praise) statement (Experiment 2). In Phase 3, we assessed resurgence during extinction of target and alternative responding in the presence versus absence of continued presentation of the paired stimulus. Despite variation across sessions, resurgence on average was lower when continuing to present the paired stimuli in all pigeons and children while maintenance of alternative responding did not differ between assessments. These findings indicate that stimuli paired with alternative reinforcement can modestly decrease resurgence, but further examination of their efficacy and a better understanding of the underlying processes are necessary before they can be recommended for clinical use in reducing resurgence of clinically relevant problem behavior.     

Emotional conditioning to masked stimuli and modulation of visuospatial attention

Two studies investigated the effects of conditioning to masked stimuli on visuospatial attention. During the conditioning phase, masked snakes and spiders were paired with a burst of white noise, or paired with an innocuous tone, in the conditioned stimulus (CS)+ and CS- conditions, respectively. Attentional allocation to the CSs was then assessed with a visual probe task, in which the CSs were presented unmasked (Experiment 1) or both unmasked and masked (Experiment 2), together with fear-irrelevant control stimuli (flowers and mushrooms). In Experiment 1, participants preferentially allocated attention to CS+ relative to control stimuli. Experiment 2 suggested that this attentional bias depended on the perceived aversiveness of the unconditioned stimulus and did not require conscious recognition of the CSs during both acquisition and expression.     

The role of local context in autoshaping

Three experiments used an autoshaping procedure with pigeons to examine the effects of nonreinforced, nontarget stimuli (ITI-fillers) during the intertrial interval on responding to a reinforced target CS. Experiment 1 replicated a previous demonstration that an ITI-filler that occupied a substantial portion of the ITI attenuated responding to the target CS relative to a group trained with a similar ITI but lacking the ITI-filler. Experiment 2 found that the superiority of responding typically found with a long ITI relative to a short ITI, that is, the trial-spacing effect, can be reversed by imposing a filler stimulus during the ITI in the former condition. Experiment 3, using a within-subject design, found that when one background stimulus condition occupied a majority of the interreinforcement interval, pairings of one target CS with reinforcement that were embedded within this background condition, that is, a long duration local context occurred for this CS, yielded better performance that a second, reinforced, target CS paired with reinforcement in the shorter duration background condition. These results confirmed predictions derived from a local context view of cycle time. Comparator theories of classical conditioning require incorporation of this notion into their conceptualization of effective cycle time in order to explain the present findings.     

Positive behavioral contrast in 3-month-old infants on multiple conjugate reinforcement schedules.

Positive behavioral contrast was assessed in two experiments with young infants using multiple conjugate reinforcement schedules. Reinforcement was produced by footkicks which activated the objects of an overhead crib mobile in a manner proportional to the vigor and rate of responding. Distinctive color/pattern cues on the sides of the objects served as discriminative stimuli for components of the multiple schedule. In Experiment 1, infants were trained with one cue (S+) only before insertion of S+ into a multiple schedule with an extinction component. A control group received S+ throughout all sessions. In Experiment 2, a multiple schedule was introduced at the outset, and responses in both components were reinforced before the introduction of extinction in the second component. In a final phase, reinforcement was reintroduced into the second component. Positive behavioral contrast occurred in both experiments. Response reduction in the extinction component was seen only in individual relative response curves. In both experiments, negative emotional behaviors accompanied the extinction component, and in Experiment 1, cooing accompanied presentations of S+.     

Summation of reinforcement rates when conditioned stimuli are presented in compound

Three experiments used delay conditioning of magazine approach in rats to examine the summation of responding when two conditioned stimuli (CSs) are presented together as a compound. The duration of each CS varied randomly from trial-to-trial around a mean that differed between the CSs. This meant that the rats' response rate to each CS was systematically related to the reinforcement rate of that CS, but remained steady as time elapsed during the CS (Harris & Carpenter, 2011; Harris, Gharaei, & Pincham, 2011). When the rats were presented with a compound of two CSs that had been conditioned separately, they responded more during the compound than during either of the CSs individually. More significantly, however, in all three experiments, the rats responded to the compound at the same rate as they responded to a third CS that had been reinforced at a rate equal to the sum of the reinforcement rates of the two CSs in compound. We discuss the implications of this finding for associative models (e.g., Rescorla & Wagner, 1972) and rate-based models (Gallistel & Gibbon, 2000) of conditioning.     

Blocking of occasion setting in feature-positive discriminations

In two experiments rats received feature-positive discrimination training in which brief conditioned stimuli (CSs) were paired with food during presentations of an extended feature stimulus, and non-reinforced in its absence. In Experiment 1 a novel feature was trained in compound with a second, pretrained feature. Acquisition of control over responding to the CS by the novel feature was blocked if the pretrained feature had also been trained in a feature-positive discrimination, compared to a group for whom the pretrained feature had been accompanied by uncorrelated presentations of CSs and food. Experiment 2 employed a within-subjects design. It demonstrated that the feature from a feature-positive discrimination with a particular CS, x, blocked acquisition of control by an added, novel feature over responding to x, compared to the control acquired by the same novel feature over a novel, CS y.     

Transfer across unconditioned stimuli in serial feature discrimination training

The transfer of conditioned modulation across conditioned stimuli (CS) and unconditioned stimuli (US) was examined in 3 experiments that used Pavlovian appetitive training procedures with rats. In Experiment 1, after training in a positive patterning discrimination (X-->A+/X-/A-), X increased conditioned responding elicited by another trained-then-extinguished CS as long as that CS had been trained with the same US as was used in discrimination training. In Experiment 2, after training with a feature-negative discrimination (X-->A-/A+), X inhibited conditioned responding elicited by another trained-then-extinguished CS as long as that CS had been trained with the same US. Experiments 1 and 2 used a between-groups design, whereas Experiment 3 used a more powerful within-groups design. In Experiment 3, rats were trained in a feature-positive discrimination (X-->A+/A-). In transfer tests, X increased conditioned responding elicited by another CS trained then extinguished with the same US from training. This increase was greater than the X increased conditioned responding elicited by another CS trained then extinguished with a different US from training. The results supported the suggestion that features trained in serial discrimination tasks influence behavior indirectly by transiently raising or lowering the threshold for activation of the US representations by its target stimuli and by any other stimuli that may be associated with that US. Other interpretations of the findings were also considered.