Partial reinforcement and context switch effects in human predictive learning

Human participants were trained in a trial-by-trial contingency judgements task in which they had to predict the probability of an outcome (diarrhoea) following different cues (food names) in different contexts (restaurants). Cue P was paired with the outcome on half of the trials (partial reinforcement), while cue C was paired with the outcome on all the trials (continuous reinforcement), both cues in Context A. Test was conducted in both Context A and a different but equally familiar context (B). Context change decreased judgements to C, but not to P (Experiment 1). This effect was found only in the cue trained in the context where a different cue was partially reinforced (Experiment 2). Context switch effects disappeared when different cues received partial reinforcement in both contexts of training (Experiment 3). The implications of these results for an explanation of context switch effects in terms of ambiguity in the meaning of the cues prompting attention to the context (e.g., Bouton, 1997) are discussed.     

Context-outcome associations mediate context-switch effects in a human predictive learning task

Four experiments explored the role of contexts in information retrieval after different levels of acquisition training in human predictive learning. Participants were trained where cue (X) was followed by an outcome in context A while a different cue (Y) was followed by the absence of the outcome in context B. When 4 training trials with each cue were conducted, testing the stimuli in the alternative contexts decreased predictive judgments to X and increased predictive judgments to Y. These effects disappeared both when training was increased up to 18 trials (Experiments 1a and 1b), and when the outcome was presented in both contexts A and B (Experiments 2 and 4). When the outcome was presented in both contexts, the nonreinforced cue Y, trained in the presumably excitatory context B, became a conditioned inhibitor (Experiment 3). Additional experience with one of the contexts, but not with both, made the context-switch effect reappear (Experiment 4). These results suggest that irrelevant contexts may enter into direct associations with the outcome before prolonged training leads participants to discard them as predictors.     

Re-examination of the role of within-compound associations in the retrospective revaluation of causal judgements

We investigated blocking and retrospective revaluation of causal judgements using a scenario in which food cues acted as potential causes of an allergic reaction as the outcome. In the blocking contingency,the treatment cues were either paired or unpaired with the outcome prior to a second stage in which sequential compounds of treatment and target cues were paired with the outcome. The order of this compound and treatment training was reversed in retrospective revaluation contingencies. When the interstimulus interval between the treatment and target cues was unfilled on compound trials (Experiments 1 and 3), both blocking and retrospective revaluation were observed in that the target cue trained in compound with the paired treatment cue attracted lower causal ratings than the target cue trained in compound with the unpaired treatment cue. By contrast, performing a mental arithmetic task using numerals presented during the interstimulus interval had no effect on the magnitude of blocking but rendered retrospective revaluation unreliable (Experiments 2 and 3). These results provide further support for accounts of revaluation based upon within-compound associations.     

Partial Reinforcement in Appetitive Pavlovian Conditioning with Rats

Four experiments examined the effects of a partial reinforcement schedule on extinction using appetitive Pavlovian conditioning. Extinction was slower after partial than after continuous reinforcement when the schedules were administered to different groups (Experiment 1). The opposite result was found in Experiments 2 and 3 when both schedules were presented to the same group in the same context. When the schedules were presented to the same group in different contexts, then extinction was again slower after partial reinforcement (Experiment 3). Experiment 4 demonstrated that a change of context facilitates extinction to a greater extent after conditioning with a partial reinforcement schedule than with a continuous one. The results are explained by assuming that the nonreinforced trials of a partial reinforcement schedule create an internal state that serves as a contextual cue.     

Contextual control of appetitive conditioning: Influence of a contextual stimulus generated by a partial reinforcement procedure

Three experiments with rat subjects examined the effects of a context switch after conditioning treatments in which the conditioned stimulus (CS) was either paired with food on every presentation (continuous reinforcement) or on some of its presentations (partial reinforcement). In each experiment, a target CS was given one of these treatments in Context A, and another CS was given a treatment during sessions that were intermixed in Context B. Final tests of the target CS in Context A and Context B often revealed no loss of responding with the switch to B. However, a loss was observed when partial reinforcement had been associated with Context A and continuous reinforcement had been associated with Context B. Those conditions caused equal decrements in responding to partially reinforced and continuously reinforced targets. The results suggest that under the present conditions partial reinforcement can generate a contextual stimulus that becomes associated with the physical context and controls responding to the CS.     

Spatial separation of target and competitor cues enhances blocking of human causality judgements

Three experiments were carried out. Each required subjects to make judgements about the causal status of cues following a two-stage blocking procedure. In Stage 1 a competitor cue was consistently paired with an outcome, and in Stage 2 the competitor continued to be paired with the outcome but was accompanied by a target cue. It was predicted that causal judgements for the target would be reduced by the presence of the competitor. In Experiments 1 and 2 the blocking procedure was implemented as a computer simulation of a card game during which subjects had to learn which cards produced the best payouts. The cues that subjects used to make their judgement were colours and symbols that appeared on the backs of the cards. When the target and competitor cues appeared on the same card blocking effects did not emerge, but when they appeared as part of different cards blocking effects were found. Thus, spatial separation of target and competitor cues appeared to facilitate blocking. Experiment 3 replicated the blocking result using spatially separated target and competitor cues.     

Recovery from blocking between outcomes

Contemporary associative learning research largely focuses on cue competition phenomena that occur when 2 cues are paired with a common outcome. Little research has been conducted to investigate similar phenomena occurring when a single cue is trained with 2 outcomes. Three conditioned lick suppression experiments with rats assessed whether treatments known to alleviate blocking between cues would also attenuate blocking between outcomes. In Experiment 1, conditioned responding recovered from blocking between outcomes when a long retention interval was interposed between training and testing. Experiment 2 obtained recovery from blocking between outcomes when the blocking outcome was extinguished after the blocking treatment. In Experiment 3, a recovery from blocking between outcomes occurred when a reminder stimulus was presented in a novel context prior to testing. Collectively, these studies demonstrate that blocking of outcomes, like blocking of cues, appears to be caused by a deficit in the expression of an acquired association.     

Between-subject PREE and within-subject reversed PREE in spaced-trial extinction with pigeons

Three experiments explored the partial reinforcement extinction effect (PREE; greater resistance to extinction after partial, rather than continuous, reinforcement training), in a spaced-trial situation with pigeons. Experiments 1 and 2 report conventional PREEs with 24-h intertrial intervals and between-subject designs. The corresponding outcome (food reinforcement or nonreinforcement) was delivered after satisfaction of a fixed-ratio 10 (Experiment 1) or a fixed-ratio 1 (Experiment 2). Experiment 3 reports a reversed PREE in a within-subject design with a fixed-ratio 10 requirement. Extinction occurred faster for the response paired with 50% partial reinforcement than for the response paired with continuous reinforcement. A third response paired with a small reinforcer (1 pellet/trial) in 100% of the trials extinguished faster than a response paired with a large reinforcer (15 pellets/trial). These results are discussed in the context of spaced-trial instrumental performance (key pecking and running), in pigeons.     

Forward blocking in human learning sometimes reflects the failure to encode a cue–outcome relationship

In two “allergist” causal judgement experiments, participants were trained with a blocking design (A?+?|AB+). The procedure allowed different food cues to be paired with different fictitious allergic reactions. On test, participants were asked to rate the causal efficacy of the target cues and to recall the particular allergic reaction (outcome) that had followed each cue during training. Forward blocking was observed on the causal judgement measure and on the outcome recall measure in both Experiment 1 and Experiment 2. A backward blocking contingency was also trained in Experiment 2 (AB?+?|A+). Backward blocking was not observed either on the causal judgement or on the outcome recall measure. The evidence from the recall measure suggests that forward blocking in this task results from a failure to encode the B–outcome relationship during training. Associative and nonassociative mechanisms of forward blocking are discussed.     

Forward blocking in human learning sometimes reflects the failure to encode a cue-outcome relationship

In two “allergist” causal judgement experiments, participants were trained with a blocking design (A + |AB+). The procedure allowed different food cues to be paired with different fictitious allergic reactions. On test, participants were asked to rate the causal efficacy of the target cues and to recall the particular allergic reaction (outcome) that had followed each cue during training. Forward blocking was observed on the causal judgement measure and on the outcome recall measure in both Experiment 1 and Experiment 2. A backward blocking contingency was also trained in Experiment 2 (AB + |A+). Backward blocking was not observed either on the causal judgement or on the outcome recall measure. The evidence from the recall measure suggests that forward blocking in this task results from a failure to encode the B-outcome relationship during training. Associative and nonassociative mechanisms of forward blocking are discussed.     

Partial reinforcement and breadth of learning

Two experiments on partial reinforcement were undertaken to test predictions made by a two process model of discrimination learning. In the first experiment rats were trained on a discrimination involving two relevant cues: one group (C) was trained on a 100: o schedule, the other (P) on a 50:0 schedule. Both groups were then given transfer tests with the two cues presented individually; finally all animals were extinguished on the original training stimuli and on the single cue stimuli. During extinction there was a negative correlation between the number of correct responses made by individual subjects of Group C to each single cue; whereas the correlation was positive for subjects of Group P. The second experiment employed basically the same design, but subjects were trained with seven relevant cues. The results of transfer tests showed that subjects of Group P learned to attach the correct response to many more cues than subjects of Group C. This suggests that the breadth of learning is greater under partial than under consistent reinforcement. The results were predicted by the model of discrimination learning under test.     

Context switch effects on acquisition and extinction in human predictive learning

Four experiments tested context switch effects on acquisition and extinction in human predictive learning. A context switch impaired probability judgments about a cue-outcome relationship when the cue was trained in a context in which a different cue underwent extinction. The context switch also impaired judgments about a cue trained in a context different from the extinction context, whenever this training was concurrent with extinction of another cue. After extinction, new cue-outcome relationships learned, even in a different task, became context specific. Moreover, renewal was consistently observed. It is suggested that context switch effects result from a process by which ambiguity leads participants to attend to the contexts.     

Context effects in evaluative conditioning of implicit evaluations

Propositional models of evaluative conditioning postulate that the impact of stimulus pairings on liking should depend not on the pairings themselves but on what the pairings imply about the relation between stimuli. Hence, context manipulations that change the implications of stimulus pairings should moderate evaluative conditioning. We manipulated context by varying the way in which context cues were paired with affective outcomes while keeping the pairings between target cues and affective outcomes constant. All participants saw one target cue compound that was followed by a positive outcome (XF+) and another target cue compound that was followed by a negative outcome (YG?). In condition Same, each context cue was consistently paired with a positive or negative outcome, regardless of whether it was presented alone or in compound with another cue (A+, B+, AB+; C?, D?, CD?). In condition Opposite, however, a context cue was paired with a certain outcome when presented alone and with an outcome of the opposite valence when presented in a compound with another cue (A+, B+, AB?; C?, D?, CD+). Employing several implicit measures, we assessed the implicit evaluations of the target cues X and Y. In all three studies, the outcome of the measurement procedure differed between conditions. In condition Same, the positively paired cue X was evaluated more positively than the negatively paired cue Y. In condition Opposite, however, this preference was not present. This pattern of results suggests that EC is determined not only by the objective pairings but also by the context in which these pairings occur. Implications for models of evaluative conditioning are discussed.     

Proactive interference between cues trained with a common outcome in first-order Pavlovian conditioning

The nature of interference between cues (X, A) trained apart with a common outcome (O; an unconditioned stimulus) was explored by assessing proactive interference in first-order Pavlovian conditioning (i.e., A-O, X-O, resulting in attenuated responding to X). Three lick-suppression studies were conducted with water-deprived rats. Posttraining extinction of the interfering cue (A) attenuated proactive interference (Experiment 1), which mirrors the observation that extinction of the competing cue can reduce competition between cues trained together (e.g., recovery from overshadowing). Proactive interference was also attenuated with manipulations known to attenuate interference between outcomes trained apart (e.g., counterconditioning), namely reminder cues (Experiment 2) and renewal (Experiment 3). The findings suggest that similar processes underlie interference between cues trained apart, between cues trained together, and between outcomes trained apart.     

Contextual control of fluid consumption: The effects of context extinction

Rats were trained on a conditional discrimination task in which saccharin was paired with LiCl in one context but paired with saline in another context. Rats drank less saccharin in the danger context than in the safe context, and consumption in the home cage was intermediate to consumption in the two training contexts. Rats also avoided the danger context on a choice test. After discrimination training, rats were given extinction trials with the danger context alone (Experiment 1 and 2) or with the danger context + water (Experiment 2). Extinction trials with the context + water abolished contextual control over saccharin consumption but not the avoidance of the danger context on the choice test. Extinction trials with the context alone abolished avoidance of the danger context but not contextual control over saccharin consumption. These data suggest that occasion setting, not simple context conditioning, is the mechanism by which contextual cues modulate fluid consumption.     

Backward blocking: The role of within-compound associations and interference between cues trained apart

Most theoretical accounts of backward blocking place heavy stress on the necessity of the target cue having been trained in compound with the competing cue to produce a decrement in responding. Yet, other evidence suggests that a similar reduction in responding to the target cue can be observed when the outcome is later paired with a novel cue never trained in compound with the target cue (interference between cues trained apart). The present experiment shows that pairing another nonassociated cue with the same outcome may be sufficient to produce a decremental effect on the target cue, but the presence of a within-compound association between the target and the competing cue adds to this effect. Thus, both interference between cues trained apart and within-compound associations independently contribute to backward blocking.     

Backward blocking: the role of within-compound associations and interference between cues trained apart

Most theoretical accounts of backward blocking place heavy stress on the necessity of the target cue having been trained in compound with the competing cue to produce a decrement in responding. Yet, other evidence suggests that a similar reduction in responding to the target cue can be observed when the outcome is later paired with a novel cue never trained in compound with the target cue (interference between cues trained apart). The present experiment shows that pairing another nonassociated cue with the same outcome may be sufficient to produce a decremental effect on the target cue, but the presence of a within-compound association between the target and the competing cue adds to this effect. Thus, both interference between cues trained apart and within-compound associations independently contribute to backward blocking.     

Verbal representation in task order control: An examination with transition and task cues in random task switching

Recent task-switching studies in which a predictable task sequence has been used have indicated that verbal representation contributes to the control of task order information. The present study focused on the role of verbal representation in sequential task decisions, which are an important part of task order control, and examined the effects of articulatory suppression in a random-task-cuing paradigm with two different types of cues presented just before the presentation of a stimulus: a transition cue and a task cue. The former cue provided information only about switching or repeating the task, whereas the latter was associated directly with the identity of the task (i.e., indicating a parity or a magnitude task). In Experiment 1, in which transition cues guided task sequences, articulatory suppression impaired performance in both repetition and switch trials, thereby increasing the mixing costs. In Experiment 2, in which a task cue, rather than a transition cue, was presented to examine the influence of a cue-decoding process, articulatory suppression had no specific effect on task performance. Experiment 3, in which the transition cue and the task cue were randomly presented in the same block to equalize the memory load and task strategy for the two types of cues, confirmed that articulatory suppression significantly increased the mixing costs only in transition cue trials. The results from the three experiments indicated that the use of verbal representation is effective in sequential task decision—that is, in selecting a task set on the basis of transient task order information in both repetition and switch trials.     

Similarity in Spatial Origin of Information Facilitates Cue Competition and Interference

Two lick suppression studies were conducted with water-deprived rats to investigate the influence of spatial similarity in cue interaction. Experiment 1 assessed the influence of similarity of the spatial origin of competing cues in a blocking procedure. Greater blocking was observed in the condition in which the auditory blocking cue and the auditory blocked cue originated at the same spatial location. Recent investigations have demonstrated that manipulations that impact competition between cues trained together have similar effects on interference between cues trained apart. Therefore, Experiment 2 investigated the influence of similarity of the spatial origin in proactive interference of Pavlovian conditioning by separately pairing two auditory cues with a common outcome, originating at the same spatial location or different spatial locations. Greater proactive interference was observed in the condition in which the interfering cue and target cue originated at the same spatial location. The results are considered in light of the possibility that a similar mechanism may underlie interference between cues trained apart and cue competition between cues trained together.     

Context cue focality influences strategic prospective memory monitoring

Monitoring the environment for the occurrence of prospective memory (PM) targets is a resource-demanding process that produces cost (e.g., slower responding) to ongoing activities. However, research suggests that individuals are able to monitor strategically by using contextual cues to reduce monitoring in contexts in which PM targets are not expected to occur. In the current study, we investigated the processes supporting context identification (i.e., determining whether or not the context is appropriate for monitoring) by testing the context cue focality hypothesis. This hypothesis predicts that the ability to monitor strategically depends on whether the ongoing task orients attention to the contextual cues that are available to guide monitoring. In Experiment 1, participants performed an ongoing lexical decision task and were told that PM targets (TOR syllable) would only occur in word trials (focal context cue condition) or in items starting with consonants (nonfocal context cue condition). In Experiment 2, participants performed an ongoing first letter judgment (consonant/vowel) task and were told that PM targets would only occur in items starting with consonants (focal context cue condition) or in word trials (nonfocal context cue condition). Consistent with the context cue focality hypothesis, strategic monitoring was only observed during focal context cue conditions in which the type of ongoing task processing automatically oriented attention to the relevant features of the contextual cue. These findings suggest that strategic monitoring is dependent on limited-capacity processing resources and may be relatively limited when the attentional demands of context identification are sufficiently high.