Pavlovian processes in the motivational control of instrumental performance

Hungry rats were rewarded for pressing a lever on a multiple schedule. During one component the reward was a sucrose solution, whereas food pellets acted as the reward during another component. Lever pressing was never rewarded during the third component. When the drive state was switched from hunger to thirst and the animals tested in extinction, they pressed more in the presence of the component stimulus that had been associated with the sucrose reward during training. A similar effect was observed during the extinction test of a second study in which the component stimuli had signalled non-contingent presentations of either the sucrose or pellet rewards in the absence of the lever. This suggests that the instrumental irrelevant incentive effect observed in the first experiment was due, at least in part, to the Pavlovian relationship between the component stimuli and the reinforcers during training. In fact, when the size of the effects controlled by purely Pavlovian and supposedly instrumental contingencies was compared directly in the final study, no difference could be detected.     

Incentive learning and the motivational control of instrumental performance

Hungry rats were trained to press a lever and pull a chain concurrently, with one action being reinforced with a sucrose solution and the other with food pellets. In addition, in the first two experiments all animals experienced non-contingent presentations of the two incentives in the absence of the operant manipulanda while either thirsty or hungry and either before (Experiment 1A) or after (Experiment 1B) the instrumental training. When lever pressing was assessed subsequently in extinction under thirst, the animals pressed at a relatively high rate only if (1) this action had been reinforced with the sucrose solution rather than the food pellets during training and (2) they had received the non-contingent presentations of the sucrose solution and food pellets on days on which they were thirsty rather than hungry. A third experiment demonstrated that non-contingent exposure to the sucrose solution alone, but not to water under thirst was sufficient to bring about this type of motivational control of instrumental performance.     

Motivational control of instrumental performance: The role of prior experience of the reinforcer

In four experiments we investigated the conditions that are necessary for instrumental performance to adjust appropriately to a change in drive state. Rats were trained to press a lever and pull a chain concurrently, with one action being reinforced by sucrose solution and the other by food pellets. In Experiment 1 for animals that were hungry throughout training the rate of lever pressing in an extinction test under thirst was unaffected by the type of reinforcer produced by this action during training, even though the sucrose solution would maintain a higher rate than the food pellets during training under thirst. In contrast, animals that experienced the instrumental contingencies arranged by the concurrent schedule while thirsty at some point during training pressed the lever more during the extinction test under thirst when this action had been reinforced with the sucrose solution rather than the food pellets. The remaining three experiments demonstrated that for this incentive effect to occur it is sufficient that the sucrose solution be delivered non-contingently under thirst at some stage of training. Thus, it would appear that performance mediated by an instrumental contingency adjusts appropriately to reinforcer revaluation brought about by a drive shift only if the animals have had prior experience with the incentive under the test drive state. This observation was interpreted in terms of Tolman's “cathexis” theory of motivation.     

Re-examination of the role of the instrumental contingency in the sodium-appetite irrelevant incentive effect

Thirsty rats were trained to perform one action—either lever pressing or chain pulling—for a sodium chloride solution and the other for water on a concurrent random interval schedule. When the experimental groups were tested in extinction under a sodium appetite, the instrumental action trained with the sodium solution was not performed at a significantly higher rate than that reinforced by water relative to control groups tested while thirsty. In a second experiment all animals were tested in extinction under a sodium appetite after training with either a sodium solution and water in the experimental condition or a potassium chloride solution and water in the control condition. The experimental animals chain-pulled more rapidly on test than the control rats, irrespective of whether this action was reinforced with the sodium solution or with water. These results confirm the findings of Dickinson and Nicholas (1983a) that the instrumental contingency plays little role in the irrelevant incentive effect based upon a sodium appetite.     

Contiguity and contingency in instrumental conditioning

There have been several attempts to construct molar theories of conditioning on the empirical basis that response-reinforcement contingency is important in determining conditioning. Such theories claim to explain behavior in terms of molar level processes which transcend the molecular level and, therefore, are not reducible to molecular processes or their interaction. These molar theories are critically examined and found to be seriously flawed because they fail to account for the effects of delaying reinforcement (degree of contiguity). Observed molar level relations, such as the matching law, do not require a molar theory, but may be more usefully considered as a product of molecular processes.     

The effect of the instrumental training contingency on susceptibility to reinforcer devaluation

Two experiments investigated performance of instrumental lever pressing by rats following post-conditioning devaluation of the sucrose reinforcer produced by establishing an aversion to it. In Experiment I rats responded less in an extinction test after being averted from the sucrose following training on a ratio schedule, but not following an equivalent amount of training on an interval schedule. This was true even though the devalued sucrose would not act as an effective reinforcer on either the ratio or interval schedule. Experiment II provided a further investigation of the insensitivity of interval responding to reinforcer devaluation by comparing test performance under simple extinction with responding when the devalued reinforcer was presented on either a response-contingent or non-contingent schedule during the test. Once again simple extinction performance was unaffected by prior reinforcer devaluation. Furthermore, neither non-contingent nor contingent presentations of the devalued reinforcer significantly depressed responding below the level seen in the extinction condition. Ratio, but not interval performance appears to be controlled by knowledge about the instrumental contingency that encodes specific properties of the training reinforcer.     

The role of contingency in classical conditioning

The assumption that classical conditioning depends on a contingent relation between the conditioned stimulus (CS) and the unconditioned stimulus (US), which was proposed some decades ago as an alternative to the traditional contiguity assumption, still is widely accepted as an empirical generalization, if no longer as a theoretical postulate. The first support for the contingency assumption was provided by experiments in which occasional CS-US pairings produced no response to the CS in random training--i.e., training in which the probability of the US was the same in the presence and absence of the CS. Those early experiments, the results of which too often are taken at face value, are reconsidered along with various later experiments that show conditioning, both of the CS and its context, in random training. The evidence suggests that CS-US contingency is neither necessary nor sufficient for conditioning and that the concept has long outlived any usefulness it may once have had in the analysis of conditioning.     

When negative affect drives attentional control: The role of motivational orientation

According to recent theorizing, cognitive control and affective processing are closely linked. Specifically, it has been suggested that negative affect acts as a driving force for attentional adjustment. Empirical support comes from a study by van Steenbergen et al. (Psychol Sci 20:1473–1477, 2009) demonstrating that negative stimuli increase the congruence sequence effect (CSE), a behavioural marker that indexes attentional adjustment. In this research, we provide a theoretical analysis that identifies motivational orientation (approach vs. avoidance) as a potential moderator. To test this, we replicated the study by van Steenbergen et al. (2009) and additionally manipulated the motivational orientation of participants. Participants either gained extra money after task completion (gain-focused group) or lost money after task completion (loss-focused group). Results showed that negative stimuli boost attentional adjustment, but only in the loss-focused group. This finding highlights the role of motivation for recent theorizing on emotion-driven control.

     

The role of the response-reinforcer contingency in negative automaintenance

When a response key is briefly illuminated before a grain reinforcer is presented, key pecking is reliably developed and maintained in pigeons, even if pecking prevents reinforcement (negative automaintenance). This experiment demonstrated that pigeons are sensitive to a negative response-reinforcer contingency, even though it does not eliminate responding. Within individual pigeons, two kinds of trials were compared: red key trials, in which reinforcement was negatively contingent on responding, and white key trials, in which reinforcement was unrelated to responding. Reinforcement frequency in non-contingent trials was yoked to the obtained reinforcement frequency in negatively contingent trials. All eight pigeons pecked substantially more on the non-contingent key than on the negative key, and preferred the non-contingent key to the negative key on occasional “choice” trials where both were presented together. When the stimuli correlated with the two conditions were reversed, the pigeons' behavior also shifted. These response differences are taken as evidence that pigeons are sensitive to the negative response-reinforcer contingency.     

The role of the peck-food contingency on fixed-interval schedules

Pigeons were trained to peck on a fixed-interval schedule of food reinforcement and then exposed to three schedules in which there was either no, or an indirect, relation between pecking and food delivery: (a) a conjunctive schedule in which food was delivered at fixed intervals, providing at least one peck was emitted in the interval; (b) a recycling version of the conjunctive schedule that essentially eliminated occasional peck-food contiguities (recycling conjunctive); (c) delivery of food at fixed intervals independently of the birds' behavior (fixed time). The rates and patterns of pecking sustained by these procedures depended on interfood interval and relative proximity of pecks to food.     

The role of CS-US contingency in human evaluative conditioning

Recent studies suggest that in Pavlovian conditioning, two different processes may be operative: signal learning and evaluative learning, resulting in two qualitatively different associative structures. Signal-learning is hypothesized to be responsible for providing us with genuine predictors (CS) for significant events (US). This proposition logically entails that the statistical-correlational relation, i.e. the contingency between CS and US should be a crucial determinant of signal learning. Evaluative conditioning, on the other hand, refers to the observation that the mere pairing of neutral with (dis)liked stimuli changes the valence of the originally neutral stimuli in a (negative) positive direction. As argued elsewhere, evaluative conditioning is probably based on the CS acquiring a mere referential value to the US, without any genuine CS-US expectancy being involved. From this, it was hypothesized that evaluative conditioning might not be dependent on CS-US contingency. Using the standard evaluative conditioning paradigm, four different levels of CS-US contingency were created on a between-subject base. The overall effect of evaluative conditioning was strongly significant, and was not mediated by awareness of the CS-US relation. Of crucial importance, this conditioning effect did not interact with the level of contingency, supporting the hypothesis that CS-US contingency is not a crucial determinant of evaluative conditioning. Moreover, this effect was obtained in a situation in which Ss simultaneously evidenced to have consciously registered quite accurately the different levels of CS-US contingency.     

Stimulus control in a simple instrumental task: The role of internal and external stimuli

Two experiments using rats in a straight alley runway task tested the hypothesis that standard stimuli such as tones and lights (“external” stimuli) and schedule-generated aftereffect stimuli (“internal” stimuli) operate similarly and are similarly subject to the compounding rules specified by R. A. Rescorla and A. R. Wagner (In A. Black & W. Prokasy (Eds.), Classical conditioning II: Current theory and research. New York: Appleton-Century-Crofts, 1972). The experimental task was an instrumental analog of the classic blocking experiment (L. J. Kamin, In M. R. Jones (Ed.). In Experiment 1, blocking of an external discriminative stimulus by a different asymptotic discrimination was accomplished. In Experiment 2, blocking of an internal stimulus discrimination by an asymptotic external stimulus discrimination was obtained. The present results support the view that internal and external stimuli are indeed similar and function in the same manner with regard to blocking of stimulus control.     

The role of the magazine-response contingency on signal-directed responding in pigeons

Three experiments used a procedure for directly delivering water into the thirsty pigeon's mouth to explore the role that the instrumental magazine contingency plays in autoshaped responding. Magazine training was accomplished by requiring birds to contact a “magazine” key when it was illuminated, in order to obtain intraoral injections of water. Other subjects received water injections independent of a magazine-response contingency. Subsequently, magazine-trained subjects showed a transient enhancement in responding to the illumination of another “signal” key, whether that stimulus was presented alone or paired with water delivery. The overall level of maintained autoshaped responding was little influenced by the instrumental magazine contingency, although the within-trial time course of signal-directed responding was affected. When illumination of the signal key preceded access to water only when the signal was not contacted, pigeons directed many responses toward the signal key. However, there was no evidence that the instrumental magazine contingency enhanced responding on this omission schedule of reinforcement. These results thus confirm a small, but measurable contribution of the magazine-response contingency to signal-directed responding; they fail to support the conclusion that the instrumental magazine contingency greatly affects the outcomes of autoshaping studies.     

Performance and motivational deficits of helplessness: the role of motivational orientations

The present study examined the hypothesis that the effect of failure feedback in producing learned helplessness would depend on the motivational orientation of a child. Extrinsically motivated children were predicted to exhibit performance decrement following a failure experience, whereas the opposite was predicted for intrinsically motivated children. In addition, success feedback was predicted to enhance subsequent performance only for the intrinsic group. Following success, failure, or no feedback on an activity reflecting spatial skills (an incomplete picture task), subjects' performance on an activity tapping different skills (i.e., anagrams) provided by a second experimenter served as the primary measure of helplessness. Subjects' intrinsic motivation in performing the incomplete picture task, a similar task (embedded figures) and a dissimilar task (dots-to-dots) was also examined. The results supported the predictions on both performance and intrinsic motivation measures. The results are discussed in terms of implications for learned helplessness as well as cognitive evaluation theories.     

The role of age and prior beliefs in contingency judgment

This experiment investigated how prior beliefs affect young and older adults' ability to detect differences in objective contingency. Participants received new evidence that the objective contingency between two events was positive, negative, or zero when they believed that there was a positive or negative relationship between events, when they believed that the events were unrelated, and when they had no knowledge of the relationship between the events. They were then asked to estimate the objective contingency and recall the contingency evidence. Beliefs that events were or could be related improved young adults' contingency discrimination. Moreover, these beliefs did not produce biases in young adults' memory for the contingency evidence, but rather affected how they weighted this evidence at judgment. In contrast, these same beliefs did not improve older adults' contingency discrimination, but did produce biases in their memory for the evidence that were similar to those seen in their judgment. These findings are discussed in terms of age-related changes in working memory executive processes that impair older adults' ability to fully evaluate both belief-confirming and disconfirming contingency evidence and update their beliefs with this information.     

Stimulus control and the response-reinforcement contingency

Pigeons were trained under a schedule in which reinforcement was made available at varying periods of time after a prior reinforcement. The first key peck after a reinforcer was available began a timer and a second key peck, which exceeded a specified minimal time interval, produced the reinforcer. It was shown that a contingency which contains a minimal interresponse time does not necessarily weaken stimulus control by an exteroceptive stimulus.     

Motivational control of instrumental performance following a shift from thirst to hunger

Thirsty rats were trained to press a lever for either a sucrose solution or saline before performance was tested in extinction while the animals were either hungry alone or experiencing both hunger and a sodium appetite. Reinforcer-specific motivational control was observed in that the animals trained with the sucrose solution pressed more than those trained with the saline when they were tested hungry, but not when they were tested under combined hunger and sodium appetite. In order to assess the role of a Pavlovian incentive process in this effect, thirsty animals received non-contingent pairings of one stimulus with the sucrose solution and another with saline in the second experiment. In an extinction test the sucrose stimulus augmented lever pressing relative to the saline stimulus when the animals were hungry, but not when they were thirsty. In the subsequent experiments the contribution of the Pavlovian process was equated by giving concurrent training with both incentives. Lever pressing and chain pulling were reinforced concurrently, one with the sucrose solution and the other with saline, while the animals were thirsty. Once again, the animals pressed more in extinction if this action had been trained with the sucrose solution rather than the saline, but only if they were hungry rather than thirsty. Thus, instrumental performance across a thirst-to-hunger shift can also be controlled by an instrumental incentive process. The direct engagement of the instrumental process by this motivational shift contrasts to the absence of such control following a hunger-to-thirst transition (Dickinson & Dawson, 1987a), a fact attributed to the asymmetrical motivational interactions produced by water and food deprivation.     

Helplessness deficits in students: The role of motivational orientation

This paper includes research comprised of field studies and laboratory experiments to examine our diathesis-stress model that students with an extrinsic motivational orientation given evaluative/controlling directives are at risk for maladaptive achievement patterns, including performance, cognitive, and emotional deficits of helplessness. The findings, using our multimethod approach, confirmed our assumptions and indicated that motivational orientation is a more reliable predictor of helplessness than either attributions or perceptions of competence. The pattern of data obtained across studies is discussed in light of other theoretical approaches to understanding achievement patterns in students.The research reported here was supported, in part, by Grant No. MN45566 from NIMH. Address all correspondence to Ann K. Boggiano, Department of Psychology, Campus Box 345, University of Colorado, Boulder, Colorado 80309-0345.