Motivational control of instrumental performance: The role of prior experience of the reinforcer

In four experiments we investigated the conditions that are necessary for instrumental performance to adjust appropriately to a change in drive state. Rats were trained to press a lever and pull a chain concurrently, with one action being reinforced by sucrose solution and the other by food pellets. In Experiment 1 for animals that were hungry throughout training the rate of lever pressing in an extinction test under thirst was unaffected by the type of reinforcer produced by this action during training, even though the sucrose solution would maintain a higher rate than the food pellets during training under thirst. In contrast, animals that experienced the instrumental contingencies arranged by the concurrent schedule while thirsty at some point during training pressed the lever more during the extinction test under thirst when this action had been reinforced with the sucrose solution rather than the food pellets. The remaining three experiments demonstrated that for this incentive effect to occur it is sufficient that the sucrose solution be delivered non-contingently under thirst at some stage of training. Thus, it would appear that performance mediated by an instrumental contingency adjusts appropriately to reinforcer revaluation brought about by a drive shift only if the animals have had prior experience with the incentive under the test drive state. This observation was interpreted in terms of Tolman's “cathexis” theory of motivation.     

Place learning in scopolamine-treated rats: the roles of distal cues and catecholaminergic mediation

Experiments 1 and 2 tested the hypothesis that cholinergic receptor antagonists impair place learning in a water maze by interfering with the processing of distal, visual cues. Extramaze cues were offered to rats in the form of geometrical patterns arranged on the inner circumference of a curtain surrounding the water maze. In Experiment 1 the animals were offered both the distal cues and proximal cues in the form of pingpong balls in fixed positions on the surface of the water while only distal cues were present in Experiment 2. Animals were injected with either scopolamine (0.5 mg/kg body wt) or saline 20 min prior to the daily place learning sessions. Upon reaching criterion level performance the animals were tested on "rotation" sessions on which the distal cues were displaced. The outcome of such "rotations" demonstrated that-regardless of the presence or absence of proximal cues-scopolamine-treated rats relied at least as much as normal animals on the distal cues. The acquisition phase of both Experiments 1 and 2 demonstrated an almost complete lack of scopolamine-associated impairment in acquisition and performance of the place learning task. In Experiment 3 (when scopolamine was no longer administered) the subjects of Experiment 2 were exposed to a series of pharmacological "challenges" of their place learning performance and eventually to surgical ablation of the anteromedial prefrontal cortex. The outcome of the pharmacological challenges and the postoperative test of task performance demonstrated that the place learning performance of animals which had acquired the task under scopolamine was mediated by a neural substrate dissimilar to the substrate of task performance in normal animals. Rats acquiring the task while deprived of the cholinergic system demonstrated above-normal contributions to task mediation from catecholaminergic-probably dopaminergic-mechanisms and tentative results pointed to a "shift" toward prefrontal task mediation.     

Galef’s Transfer-of-Information Effect Occurs in a Free-Foraging Situation

Galef and his colleagues have repeatedly shown that one rat may transfer information regarding the type of food it has consumed to other conspecifics. Such experiments typically have been conducted in wire-mesh cages or a wooden maze. The present experiments sought to extend this paradigm to the open-field foraging situation having six food patches to choose from. Following interaction with a demonstrator that had consumed either a cocoa or a cinnamon diet, single observers (Experiment 1) were tested in the foraging situation. Food-consumption scores indicated that observers consumed significantly more of their specific demonstrator’s diet than a second diet that was available also. Experiment 2 involved the simultaneous testing of two observers in the foraging laboratory. In Experiment 3 two observers were once again tested, but each had been provided a different food-type message prior to foraging. Positive results, mirroring those of Experiment 1, were obtained in both Experiments 2 and 3. The results of these three experiments underscore the robustness of this phenomenon and its generalizability to other testing conditions.

     

Memories and anticipations control responding by rats (<Emphasis Type="Italic">Rattus norvegicus</Emphasis>) in a Pavlovian procedure

In Experiment 1 each rat received two different fixed series of three trials each. The unconditioned stimulus occurred on Trial 1 of one series and on Trial 3 of the other series, all other trials being nonreinforced. Previous Pavlovian investigations have shown that rats can remember the immediately prior reward outcome and anticipate the immediately subsequent reward outcome. Experiment 1 demonstrated that rats could remember and anticipate even more remote reward outcomes. In Experiment 2 two groups received a series of two nonrewarded trials followed by a rewarded trial. It was demonstrated that a change in the conditioned stimulus (CS) from Trial 2 to Trial 3, which occurred in one group, produced weaker responding than in the other group that did not experience such CS change. On the basis of these findings it was suggested that the rats organized the trials of a series into a unit or chunk. This was concluded for two reasons. First, remembering and anticipating remote reward outcomes strongly suggests that responding is being controlled by events extending beyond the current trial. Secondly, the experimental manipulations employed in the Pavlovian situation here are similar to those used in prior human learning and animal instrumental learning investigations concerned with chunking. Thus, it would appear that chunking is a ubiquitous phenomenon appearing in human serial learning (e.g., Bower and Winzenz 1969; Crowder 1976), in animal instrumental learning (e.g., Capaldi 1992; Hulse and Dorsky 1977; Terrace 1987), and now in Pavlovian learning.     

Interference with judgements of control and learning as a result of prior exposure to controllable and uncontrollable feedback during concept-learning tasks

Two experiments examined whether exposure to an uncontrollable relationship between an action and its outcome during a nonaversive pretreatment phase would attenuate subsequent ratings of control given to actions emitted by subjects. In Experiment 1, such an interference effect was demonstrated relative to a group that received prior training with a controllable action- outcome relationship, and relative to a group not exposed to any prior relationship. In Experiment 2, these effects were replicated, and interference was also found to occur when learning a maze task. Thus, the effects of helplessness were shown to be quite general, to be produced by a nonaversive induction procedure, and to occur most readily when the current contingency between action and outcome was weakest.     

Asymmetrical interactions between thirst and hunger in pavlovian-instrumental transfer

Pavlovian-instrumental transfer experiments have demonstrated that a stimulus paired with a sucrose solution under hunger will increase instrumental performance under thirst relative to a stimulus previously paired with food pellets. In Experiment 1 it was demonstrated that this difference is, in part, produced by suppression induced by the pellet stimulus, which, it was found, acted to reduce instrumental performance under thirst. In Experiment 2, the reverse shift was examined, comparing the effects of stimuli paired with either a saline solution or a sucrose solution under thirst on instrumental performance under hunger. Although the sucrose stimulus was found to elevate performance when hungry, the saline stimulus was found to be without effect. This asymmetry in the interaction between hunger and thirst is discussed in terms of the way motivational states control the interaction between sensory and affective components of the reinforcer.     

Sex differences in performance attribution and contingency judgment

In two studies, male and female subjects were given attribution measures before and after performance on a novel finger maze. Neither study revealed any sex differences in expectancy and anticipated attributions prior to maze performance. In Experiment 1, no sex differences in attributions were obtained regardless of whether the outcome was success or failure. In Experiment 2, where the outcome was made completely noncontingent on behavior, females had greater illusion of control as well as higher luck attribution. This paradoxical finding was interpreted as reflecting females' tendency to depend on external and internal attributions simultaneously.The data presented here are based on a larger research project on reinforcement contingencies and performance attributions supported by a research grant from the Social Sciences and Humanities Research Council of Canada.     

Signalling and incentive processes in instrumental reinforcer devaluation

We have previously reported that conditioning an aversion to the reinforcer using an isotonic lithium chloride (LiCl) solution following instrumental training reduces performance in a subsequent extinction test only if animals are re-exposed to the reinforcer prior to the test. Rescorla (1992), in contrast, reported an immediate devaluation effect using a hypertonic LiCl solution that did not depend upon re-exposure. In two experiments we examined the effect of using a hypertonic LiCl solution to condition the aversion to the reinforcer on subsequent instrumental performance in extinction, with and without re-exposure. In Experiment 1 thirsty rats were trained to press a lever for a sucrose solution before being injected with 0.6 M LiCl either immediately or after a delay. Half of the immediate and delay groups were then re-exposed to the sucrose in the absence of the lever, with the remainder being exposed to water. Contrary to the previously reported effects of isotonic LiCl, a hypertonic solution induced a reinforcer devaluation effect in all the immediately poisoned animals, which did not depend upon re-exposure to the reinforcer. In Experiment 2 the possibility that this devaluation effect was induced by the discomfort associated with the hypertonicity of the solution was assessed by replicating Experiment 1 but, in addition, using two immediately poisoned groups given the LiCl injection under anaesthesia. In the absence of anaesthesia, the devaluation effect observed without re-exposure to the reinforcer in Experiment 1 was replicated. When the injection was given under anaesthesia, however, a reinforcer devaluation effect was observed only in animals that were re-exposed to the reinforcer prior to the extinction test. These results were interpreted as evidence that a reinforcer devaluation effect induced by pairing the reinforcer with illness depends upon a process of incentive learning, whereas a devaluation effect mediated by learning a signalling relationship between the reinforcer and somatic discomfort does not.     

Working harder for less: Effect of incentive value on force of instrumental response in humans

The present article reports evidence from human subjects that high payoff for instrumental performance may cause reduced vigour of response compared to the vigour observed for lower payoffs. Vigour of response, as measured by force of pressing (Experiment 1) and number of fortune wheel turns (Experiment 2), was inversely related to payoffs, with higher vigour in the low payoff conditions. Experiment 2 further demonstrated this effect, regardless of whether or not payoff depended on performance. These findings are related to parallel animal data, and an analysis in terms of a learned incentive-effort hypothesis is offered. Experiment 3 presented support for this hypothesis. The results are argued to be of importance to the understanding of how reinforcers affect human instrumental performance.     

Music interferes with learning from television during infancy

Infants are frequently exposed to music during daily activities, including free play, and while viewing infant‐directed videotapes that contain instrumental music soundtracks. In Experiment 1, an instrumental music soundtrack was played during a live or televised demonstration to examine its effects on deferred imitation by 6‐, 12‐, and 18‐month‐old infants. Transfer of information was indexed via deferred imitation of the target actions following a 24‐h delay. For half the infants, the music context was also reinstated at the time of test. Performance by experimental groups was compared to that of a baseline control group that participated in the test session without prior exposure to the demonstration. Imitation performance was above baseline for the live groups but not for the video groups regardless of age or the music context at test. In Experiment 2, we added matched sound effects to the video demonstration and infants performed above baseline. We conclude that the music track creates additional cognitive load, disrupts selective attention to the target actions and inhibits transfer of learning from television of the imitation task. Music may impair an infant's ability to translate information from a two‐dimensional to three‐dimensional world even if the auditory context remains the same. Copyright © 2010 John Wiley & Sons, Ltd.     

Contingency effects with maintained instrumental reinforcement

In three experiments we investigated the effect on the performance of thirsty rats of varying the instrumental contingency between lever pressing and the delivery of a saccharin reinforcer. In Experiment 1, the subjects performed more slowly in a non-contingent condition, in which the momentary probability of reinforcement was unaffected by whether or not the animals pressed, than in a contingent condition in which the reinforcer was never presented except following a lever press. This was true of performance under both random ratio and interval schedules in which the function determining the probability of reinforcement following a lever press remained the same across the contingent and non-contingent conditions. Experiment 2 demonstrated that instrumental performance was less affected when the contingency was degraded by the introduction of free reinforcers if these reinforcers were signalled. In Experiment 3, lever pressing was reinstated to some degree after non-contingent training by giving non-reinforced exposure to the operant chamber in the absence of the lever. These results suggest that free reinforcers depress instrumental behaviour through a performance mechanism engaged by their ability to support conditioning of the contextual cues.     

Discrimination between outcomes in instrumental learning: Effects of preexposure to the reinforcers

In two experiments rats received instrumental training with two response levers, one response being reinforced by sucrose solution and the other by sucrose pellets. Prior to a test session, on which both levers were made available in the absence of reinforcement, the rats were given free access to one of the reinforcers, a procedure known to reduce its value. It was found that the rats responded at a lower rate on the lever that had produced the now-devalued reinforcer, but that this effect was substantial only in rats that had received preexposure to the two reinforcers before instrumental training was begun (Experiment 1). Experiment 2 demonstrated that this effect was obtained only when presentations of the two reinforcers were presented according to an inter-mixed schedule during preexposure. It is suggested that this result constitutes an instance of the perceptual learning effect in which intermixed preexposure to similar events enhances their discriminability.     

Alleviating forgetting of active shock-avoidance by apparently unrelated food presentations

A series of four experiments investigated why forgetting of an active shock-avoidance response was reduced when an apparently unrelated appetitive experience occurred both prior to avoidance training and during the retention interval between training and testing. Experiments 1 and 2 demonstrated the reliability of the phenomenon and found that a sufficient condition for the effect was the presentation of food from a pellet dispenser (not an operant response per se). Experiment 3 demonstrated that the effect was not the result of enhanced activity. Experiment 4 showed that forgetting was still reduced when the appetitive experience occurred only prior to active avoidance or only during the retention interval. The results are discussed in regard to the reinstatement of the reinforcer representation.     

Memory of self-performed tasks: Self-performing during recognition

Two experiments focused on whether performing actions described by to-be-remembered phrases during recognition enhances recognition compared with results of a standard verbal recognition test. The enhancement was predicted when the actions described by the phrases had been performed during study, but not when the phrases were verbally encoded by simply listening to and memorizing the material. Both experiments showed that enactment prior to recognition improved memory performance, but only when subjects had encoded by enactment. Experiment 1 also demonstrated that this test-procedure effect was independent of a bizarreness effect, which was observed only with the verbal encoding task. Experiment 2 showed that the effect of enactment during recognition was reduced when subjects used different hands for performing the actions during study and recognition- The findings support the assumption that some kind of motor memory record underlies the enactment effect that occurs when actions are performed during recognition.     

The prediction of group motor performance from individual member abilities

Two experiments were conducted to investigate the prediction of group performance on the motor-maze task from individual member abilities. In both experiments subjects performed 20 individual trials on the maze task in an initial session prior to the actual experiment. On the basis of these individual performance scores, two-person groups were formed so that the groups represented a range of average ability levels and a range of discrepancies in abilities between partners. In Experiment 1, all groups performed a cooperative and a noncooperative group task in separate sessions both involving competition with another group. Only the cooperative group task was used in Experiment 2, but all groups performed under both competitive and noncompetitive conditions. Multiple regression analyses yielded a moderate, positive relationship between member abilities and group performance in both experiments; group performance on the highly cooperative task was dominated by the lower-ability partner.     

The role of R-S1 expectancy in discrimination and discrimination reversal learning

Four experiments assessed the role of response-reinforcement (R-S¹) expectancy as rats either learned a discrimination or a discrimination reversal. Experiment 1 showed that the higher the percentage of reinforcement during nondiscriminative pretraining the more quickly a subsequent discrimination is learned. Experiment 2 demonstrated that the overlearning reversal effect (ORE) can be obtained even though the overtraining phase consists of responses to a single neutral card. In Experiment 3 subjects were again overtrained on a single neutral stimulus under conditions of varying reinforcement. ORE was apparent under reinforcement of a high percentage of responses but was not apparent under reinforcement of a low percentage. Experiment 4 revealed that the fewer errors a subject makes prior to reversal the better its reversal performance is. Results of these studies were discussed in terms of generalized attention theories and the expectancy notions of Bolles (Psychological Review, 1972, 79(5), 394–409.     

Incentive learning and the motivational control of instrumental performance

Hungry rats were trained to press a lever and pull a chain concurrently, with one action being reinforced with a sucrose solution and the other with food pellets. In addition, in the first two experiments all animals experienced non-contingent presentations of the two incentives in the absence of the operant manipulanda while either thirsty or hungry and either before (Experiment 1A) or after (Experiment 1B) the instrumental training. When lever pressing was assessed subsequently in extinction under thirst, the animals pressed at a relatively high rate only if (1) this action had been reinforced with the sucrose solution rather than the food pellets during training and (2) they had received the non-contingent presentations of the sucrose solution and food pellets on days on which they were thirsty rather than hungry. A third experiment demonstrated that non-contingent exposure to the sucrose solution alone, but not to water under thirst was sufficient to bring about this type of motivational control of instrumental performance.     

The effect of the instrumental training contingency on susceptibility to reinforcer devaluation

Two experiments investigated performance of instrumental lever pressing by rats following post-conditioning devaluation of the sucrose reinforcer produced by establishing an aversion to it. In Experiment I rats responded less in an extinction test after being averted from the sucrose following training on a ratio schedule, but not following an equivalent amount of training on an interval schedule. This was true even though the devalued sucrose would not act as an effective reinforcer on either the ratio or interval schedule. Experiment II provided a further investigation of the insensitivity of interval responding to reinforcer devaluation by comparing test performance under simple extinction with responding when the devalued reinforcer was presented on either a response-contingent or non-contingent schedule during the test. Once again simple extinction performance was unaffected by prior reinforcer devaluation. Furthermore, neither non-contingent nor contingent presentations of the devalued reinforcer significantly depressed responding below the level seen in the extinction condition. Ratio, but not interval performance appears to be controlled by knowledge about the instrumental contingency that encodes specific properties of the training reinforcer.     

Does a cognitive map guide choices in the radial-arm maze?

Fifteen rats performed in a standard radial-arm maze task (Experiment 1) and in a modified task with a set of forced choices and a 15-min retention interval prior to completion of the maze (Experiment 2). In addition to the standard measure of choice in the radial-arm maze, orientation toward arms was measured and considered to constitute go-no-go "microchoice" decisions. Rats investigated but rejected many arms. A model of choice was developed in which it was assumed that choice decisions about arms were made independently and that microchoices were not selectively guided toward baited arms. The model performed nearly as well as the rats. These results place important limitations on the theory that choice behavior in the radial-arm maze is guided by a cognitive map.     

The effects of a target–stimulus reminder on performance in a novel object recognition task

The effects of original training-stimulus pre-test reminders were examined in a novel object recognition (NOR) task. NOR is a task that examines memory for complex stimuli, and is driven by the rats’ tendency to spend significantly more time exploring novel objects over those previously experienced. In this task, a delay is imposed between a training experience during which the animal is allowed to investigate a set of identical objects, and a later test exposure where the animal encounters one of the original objects and a novel object with which it has had no previous experience. Experiment 1 demonstrated that performance at 24 h is significantly worse than at an immediate delay (1 min). In the second experiment, it was demonstrated that neither a 10-s nor a 30-s reminder treatment, in the absence of training, resulted in a level of preference for novelty, a measure of memory for the original object, that was significantly greater than chance. Experiment 3 illustrated significant performance effects of a 30-s training stimulus reminder administered 15 min prior to test with a 24-h retention interval. The final experiment illustrated that the additional 30-s of object exposure is effective in enhancing performance only if it occurs shortly prior to test. Animals receiving the additional 30-s immediately following training did not experience such beneficial effects. It was concluded, based upon these results, that pre-test training-stimulus reminders in this task produce effects similar to those seen in more traditional tasks of learning and memory.