Ordinal Position Learning and Remote Anticipation

Rats were runway trained on each of two 3-trial series of reward outcomes. The series are labeled XNY and ZNN, for which X represents a trial that was rewarded with Noyes pellets and N represents a trial that ended with no reward. Units of distinctively flavored breakfast cereals served as reward on trials labeled Y and Z. One group (Floor) had each series occur with a correlated runway floor, either smooth and black or rough and white. For a second group (Memory), the floor cue was uncorrelated with series. Animals in both groups learned to approach the goal rapidly on the 1st trials of the 2 series and slowly on the 2nd trials, but only Group Floor learned to differentiate the 3rd trials of the series. These results recommend a view of serial learning that emphasizes the role played by information about the ordinal position of series items.     

Higher Order Chunking in Serial Pattern Learning by Rats in the T-Maze

Rats were repeatedly exposed to two- and three-trial series consisting of different reinforced (R) and nonreinforced (N) trials in a fixed order on extended visually distinct runways in a T-maze. When initially presented with the same sequence, RR or RN in Experiment 1 and RRN or RNR in Experiment 2, on each series in a session (Separate Presentations), rats developed slower running speeds or bypassed the runway in returning to the start chamber on N more than on R trials. These serial response patterns occurred whether rats experienced each sequence on a specific runway (Associated group) or equally on different runways over sessions (Nonassociated group). When sequences were intermixed within each session, only the Associated group maintained its serial pattern responses to both sequences while the Nonassociated group only did so within the RRN sequence. When allowed to choose between runways over trials, the Associated group tended to select the runway associated with the RR sequence on the first two free-choice trials while rats in the Nonassociated group tended to alternate their choices of runways. Switching the relevancy of the runway context in the second experiment caused rats in each group to react like rats in the other group on forced-choice trials. In terms of Capaldi's (1992) model of levels of chunking in serial pattern learning, our findings indicate that rats learned to anticipate at least second trial outcomes by using higher order list or contextually cued chunking and also to anticipate third trial outcomes by using lower order series chunking.     

Organized responding in instrumental learning: Chunks and superchunks

Two investigations attempted to determine if rats could learn that the second series of runway trials, the test series, was the same as the first series, the study series. Series were constructed from runway responses which terminated either in food reward (R) or nonreward (N). In the series RNR, for example, three successive responses terminated in R, N, and R, respectively. Rats manifested mastery of a series by running fast to R and slow to N. In Experiment 1 the test series (either RNR or RNN) occurred in a black runway about 15 s after the study series presented in a white runway. In Experiment 2 the test series (either RN or RRN) occurred in a gray runway about 15 min after the study series, also in a gray runway. In both experiments rats learned that the study series was the same as the test series. A hierarchical interpretation was suggested in which runway trials are organized into series, series being organized into lists.     

Integration and representation in rats' serial pattern learning in the T-maze

Rats were exposed to three-trial series consisting of reinforced (R) trials and one nonreinforced (N) trial in a fixed order, RRN and RNR (Experiments 1 and 2) or NRR and RRN (Experiment 3), on extended visually distinct runways in a T-maze. When initially presented with the same sequence on each series in a session (separate presentations) with the same runway on all trials within a series (Experiments 1 and 3), all the rats developed slower running speeds on N than on R trials. When a runway was sometimes changed between the first and next two trials during separate presentations training (Experiment 2) or both sequences were later intermixed within each session in each experiment, only rats exposed to each sequence on a specific runway maintained these serial running patterns. Rats displayed serial running patterns on a test RNN sequence similar to that on the RNR sequence (Experiment 2), as would be predicted by an intertrial association model of serial pattern learning (Capaldi & Molina, 1979), but responded on test RRR and NRN sequences (Experiment 3) as would be predicted by an ordinal-trial-tag/intratrial association model (Burns, Wiley, & Payne, 1986). Results from test series of free-choice trials in Experiments 1 and 2 failed to support a prediction of the intratrial association model that these rats would integrate RRN and RNR sequences. Rather than always selecting a baited runway on both the second and the third free-choice trials, the rats only selected a baited runway on the third trial on the basis of their choice on the second trial, as would be predicted by the intertrial association model. Only after experiencing all possible outcome sequences during forced-choice training in Experiment 3 did these rats predominantly select a baited runway on every free-choice trial.     

Rats’ anticipation of current and future trial outcomes in the ordered RNR/RNN serial pattern task

In the ordered RNR/RNN serial pattern task, rats often reduce their running speeds on trial 2 less within the RNR than within the RNN series. Initially, investigators (Capaldi, 1985; Capaldi et al., 1983) considered this trial 2 differential speed effect evidence for rats’ anticipation of inter-trial outcomes within each series. Later findings, however, suggest that this effect reflects some generalization of the ordinal position of trial 3 (Burns et al., 1986) or its similar runway cues during trial 2 (Capaldi et al., 1999). To test these generalization hypotheses, we made trial 2 more distinct from trial 3 in each series by forcing rats to alternate runways in a T-maze only on the last trial rather than on trial 2 in each series in Experiment 1, or by forcing rats to alternate runways between trials rather than to run down the same runway on all trials within each series in Experiment 2. Although enhancing the distinctiveness between these trials reduced the trial 2 differential speed effect, extensive training failed to eliminate it. Therefore, this residual difference between trial 2 speeds could reflect rats’ anticipation of trial 3 outcomes during trial 2 as originally proposed by Capaldi (1985) Experiment 3 was designed to determine whether we could enhance rats’ final trial outcome expectancies during trial 2 by making different trial 2 choices distinctive cues for each trial 3 outcome. The trial 2 speed effect was greater when rats were forced to alternate over all trials only within one of the series than when they were sometimes forced to do so in either series. Post-training probe tests revealed that both series position and the relevant within-series runway events contributed to this enhanced anticipation of trial 3 outcomes.     

Grouping, chunking, memory, and learning

An example of grouping is dividing a long series of digits into two or more smaller units by, for example, pausing for a short time between items within groups but for a longer time between groups. Grouping, virtually neglected in animal learning, was examined in each of five rat investigations reported here in which a series of two or more runway trials was either grouped together with or apart from a terminal large reward trial. It was found that running speed on early small reward trials in the series was greater when prior trials were grouped together with rather than apart from the terminal large reward trial and this when the intertrial interval was short, 20 sec, or long, 20 to 40 min. Three possible explanations of the present findings were examined: a reward schedule view, a rule-learning view, and an anticipation view based on memory. The most feasible of these explanations, it was suggested, was the anticipation view. According to this view, when prior trials are grouped together with a terminal large reward, there is a tendency for the rat to anticipate the terminal large reward well before its scheduled occurrence, elevating running speed on earlier small reward trials. Such anticipation occurs, it was suggested, because when trials are grouped together the memory of each reward event in the series is retrieved on each subsequent trial, including the remote terminal large reward trial, where it becomes a signal for large reward. Thus the present results implicate not merely adjacent associations—associations between adjacent events—but also the highly controversial remote associations—associations between two events separated not only in time but by one or more intervening events as well.     

Response persistence in the rat following intermittent punishment and partial reward: Effects of trial sequence

Thirty-six rats were given 16 days of partial reward training in a runway. During the final 12 days each of the animals received one foot-shock experience each day. One group received shock on an N trial preceding an R trial (P-R), a second group was shocked on N trials not followed by an R trial (R-P), and the third group received shock after completing all daily trials (Control). Following acquisition the rats were split within each group (one half received 24 trials of unpunished extinction and one half continued to receive partial reward but were punished on every trial). During consistent punishment the P-R animals were more persistent than the R-P or Control rats and during unpunished extinction the P-R and Control animals were equal in persistence but both were superior to the R-P animals. The results were discussed in terms of Capaldi's sequential trial theory.     

Numerical and sensory categorization of serial information by animals

Rats were trained in a straight runway on a series consisting of 4 elements, each element being defined in terms of the number of .045-gm. Noyes pellets used as reward on 4 runway trials separated by about 20 sec. The series elements were 2, 6, 12, and 0 (2-6-12-0). The animals were trained each day on 3 such series, the series themselves being separated within a day by about 12 min. for 28 days before introducing a new transfer series to assess the relevance of the order of the series elements. In previous experiments with the simpler series, 2-12-0, alterations in order of elements eliminated the animals' tendencies to run much slower to the terminal nonreward element of the series, but transfer was clearly shown here when we shifted from 2-6-12-0 to 6-12-2-0. That is, the animals continued on the transfer test to approach the terminal element slowly, a result which we interpret as supporting the view that the rats either counted the rewarded trials and used that count to predict terminal nonreward, or they behaved on the basis of cues associated with the ordinal position of the trial.     

A test for order relevance in a three-element serial learning task

Rats were trained in a straight runway on a simple three-element series of differing reward quantities. The first trial of the series ended with a two-pellet reward, and the second and third trials ended with a 12-pellet reward and a nonreward, respectively (2-12-0). All animals developed accurate anticipation of the terminal nonreward in the 2-12-0 series before it was rearranged for two test days during which the elements appeared in the order 12-2-0. The rats' anticipation of the terminal nonreward did not transfer to the reordered series, a result taken to mean that anticipation had been based upon interitem associations among memorial representations of the differing reward quantities. A second transfer test to 0-0-0, given after anticipation was reestablished to the 2-12-0 series, gave evidence that the ordinal position of the series elements was also a source of anticipation; the animals continued to run relatively slowly on the third trial in extinction. In neither transfer test was there evidence that the rats employed the strategy of enumerating rewarded trials, because counting is an order-irrelevant process.     

Memories and anticipations control responding by rats (<Emphasis Type="Italic">Rattus norvegicus</Emphasis>) in a Pavlovian procedure

In Experiment 1 each rat received two different fixed series of three trials each. The unconditioned stimulus occurred on Trial 1 of one series and on Trial 3 of the other series, all other trials being nonreinforced. Previous Pavlovian investigations have shown that rats can remember the immediately prior reward outcome and anticipate the immediately subsequent reward outcome. Experiment 1 demonstrated that rats could remember and anticipate even more remote reward outcomes. In Experiment 2 two groups received a series of two nonrewarded trials followed by a rewarded trial. It was demonstrated that a change in the conditioned stimulus (CS) from Trial 2 to Trial 3, which occurred in one group, produced weaker responding than in the other group that did not experience such CS change. On the basis of these findings it was suggested that the rats organized the trials of a series into a unit or chunk. This was concluded for two reasons. First, remembering and anticipating remote reward outcomes strongly suggests that responding is being controlled by events extending beyond the current trial. Secondly, the experimental manipulations employed in the Pavlovian situation here are similar to those used in prior human learning and animal instrumental learning investigations concerned with chunking. Thus, it would appear that chunking is a ubiquitous phenomenon appearing in human serial learning (e.g., Bower and Winzenz 1969; Crowder 1976), in animal instrumental learning (e.g., Capaldi 1992; Hulse and Dorsky 1977; Terrace 1987), and now in Pavlovian learning.     

Schedules of reward shift in the double runway

Eighty food deprived rats received 62 trials in a double runway. On Trials 1-30, reward in the first goal box (GB1) was either always two food pellets or always zero pellets. All subjects received two pellets in the second goal box (GB2). On Trials 31-62 subjects in each preshift group (GB1 reward or GB1 nonreward) were shifted to the opposite GB1 reward level on 0, 25, 50, 75 or 100% of occasions. GB2 reward remained unaltered in all cases. For subjects experiencing reward decrease, second runway (A2) run and goal speeds after nonreward were generally enhanced, both within-group and in comparison with never rewarded controls. No such effect was evident on A2 start speed, nor was there any evidence to suggest that A2 performance after decreased reward was a function of the schedule of decrease. Increased GB1 reward resulted in general within-group impairment of A2 start and run speeds, with no effect on A2 goal performance. However, comparisons of speeds after increased reward with those of always rewarded controls revealed no difference on A2 start or run but indicated impairment of A2 goal performance. With the 50% schedule of reward increase, A2 run speeds after nonreward (the training level) exceeded those of never rewarded controls. Results are discussed with reference to McHose's contrast account of double runway phenomena and Amsel's frustration theory.     

Effect of schedule and magnitude of reinforcement on instrumental learning in the toad, Bufo arenarum

Extinction after training with continuous (CR) or 50% partial (PR) reinforcement, and with different magnitudes of reward, was studied in the amphibian Bufo arenarum, in a runway situation. In Experiment 1, a group of toads received massed-trial, CR training with access to water as the reward. Performance improved during acquisition, including an improvement on the first trial of each session. Extinction was rapid and there was evidence for spontaneous recovery of the running response. In Experiment 2, groups of toads received PR or CR training at a rate of one trial per day. PR impaired acquisition and resulted in poor responding during extinction, compared to CR. Experiment 3 factorially studied the effects of schedule (PR vs CR) and distribution of practice (15 s vs 300 s intertrial interval). Acquisition was impaired by PR training but had little effect on extinction performance. Different magnitudes of water reinforcement were used in Experiment 4 in a one-trial-per-day situation. Terminal acquisition performance was a monotonic function of reward magnitude, but there were no differences in extinction performance across groups. The results are discussed in relation to comparative and developmental data on the paradoxical effects of reward.     

The temporal course of the frustration effect

General activity subsequent to reward (R) and nonreward (N) was monitored at 5-sec intervals with a stabilimeter in the runway goal box. Activity of never-rewarded control Ss was also measured. In Expt 1 it was found that the frustration effect (difference between N- and R-trial activities) disappeared after about 40 sec of goal box confinement. This disappearance of the frustration effect was due to activity increase on R trials rather than activity decrease on N trials as a function of time. Comparison of N-trial activity with control group activity indicated that frustration does not dissipate within 60 sec. Expt 2 investigated activity following reward and nonreward as a function of reward magnitude. Evidence from these experiments suggests that the late R-trial activity increase results from frustration, possibly conditioned to apparatus cues on N trials.     

The operation of an acquired drive in satiated rats

A series of four short experiments indicates that the behaviour of satiated rats in a runway, at the end of which they have previously been rewarded, differs significantly from the behaviour of satiated rats without previous reward in the experimental situation. The former group reach the end box more quickly after having been put in the starting box of the runway and if provided with food in the end box proceed to eat it, although they have just refused similar pieces of food in their home cages. This is shown not to be due to defective satiation or the operation of fear in the control group. When runs and feeding in the end box are separated during the training period, the previously rewarded group still shows a more vigorous response on satiated trials, indicating that it is a reward expectancy about the goal box rather than a running habit which has become “functionally autonomous”, acting as a situation-specific drive.     

The interaction of hunger and thirst in the rat

Two groups of rats were given a series of trials in an enclosed runway with a food reward at the end, one group being run hungry, the other hungry plus thirsty. Then each group was split into three sub-groups: one run hungry, the second thirsty and the other hungry plus thirsty, in each case without food reward.

It was found that, whereas on the rewarded runs the extra, “irrelevant,” thirst increased running speed, on unrewarded runs it had the opposite effect and slowed up performance. Thus on unrewarded runs the two sub-groups running thirsty, and hungry plus thirsty, ran as slowly as those running hungry. Differences were found not to depend on whether the animals had been hungry or hungry plus thirsty on previous rewarded runs.

The interaction of primary needs therefore depends on the external situation. This can be accounted for in terms of the Pavlovian theories of mutual induction and conditioning, but not in terms of Hull's theory of “irrelevant drives.”     

Single landmark learning in rats: Sex differences in a navigation task

In Experiments 1 and 2, rats were trained in a Morris pool to find a hidden platform located some distance away from a single landmark. Males learned to swim to the platform faster than females, but on test trials without the platform, males, unlike females, spent less time in the platform quadrant of the pool in the second half of each test trial than in the first. They also showed greater persistence in searching in the platform quadrant over a series of extinction trials. In Experiments 3a and 3b, the problem was made easier by locating the platform closer to the solitary landmark. Now males and females learned to swim to the platform equally rapidly, and both stopped searching in the platform quadrant in the second half of each test trial. Experiment 4 ruled out the possibility that males´ shorter latencies to find the platform in Experiment 2 were due to their swimming faster than females.     

Resistance to extinction and punishment following training with shock and non-reinforcement: Failure to obtain cross-tolerance

Two experiments investigated the phenomenon of cross-tolerance between the partial reinforcement extinction effect (PREE) and the partial punishment effect (PPE). Three groups of rats were trained in acquisition to run in a straight alley. The continuously reinforced (CRF) group received a reward on every trial. The partially reinforced (PRF) group was rewarded on a quasi-random 50% schedule. The partially punished (PP) group received food reward on every trial but, in addition, received foot shocks of gradually increasing intensity in the goal box on a random 50% of the trials. In the test stage, half of the animals in each training condition were tested in extinction, where no reward was given on any of the trials, and the other half were tested in punishment, with both food and shock presented on each trial. Experiment 1 used a 1-trial/day procedure; Experiment 2 used a multi-trial procedure. In both procedures, clear PREE and PPE were obtained. In the 1-trial/day procedure, no cross-tolerance was evident: animals trained on a PRF or PP schedule did not show increased resistance to punishment and extinction, respectively. In the multi-trial procedure, only weak cross-tolerance was obtained in animals trained on partial reinforcement and tested in punishment.     

Positive contrast following gradual and abrupt increases in reward magnitude using delay of reinforcement

In Experiment 1, rats were given a 1-pellet reward for 48 preshift trials. During a subsequent 20-trial postshift phase, one group was shifted to a 12-pellet reward on Trial 1, a second was shifted on Trial 11, and a third was given 1 more pellet each trial and then 12 pellets for the last 10 trials. The speeds of all three groups increased to a level above that of a control group given a 12-pellet food reward throughout training (positive contrast). In experiment 2, rats were shifted from 1 to 12 pellets either gradually or abruptly following either abbreviated training (9 trials) or extended training (20 trials). One group of control subjects received 12 pellets throughout training. The results revealed a positive contrast effect for gradually shifted subjects following extended training but not following abbreviated training. The abrupt shift procedure produced positive contrast following abbreviated training but only a marginal effect following extended training. These results indicate that, contingent upon the amount of preshift training, either gradual or abrupt reward increases may produce positive contrast.     

Retention interval and intertrial interval in a serial learning or delayed discrimination task

In each of four experiments, rats were provided with the same three-event decreasing series (18-1-0) of 0.045-g food pellets in a runway. Tracking, running fast to 18 pellets and running slow to 1 and 0 pellets, was investigated as a function of the temporal interval elapsing between the events of the series (the retention interval), shifts in retention interval, and number of trials each day (or the intertrial interval), a trial being defined as presentation of each of the three events of the series. Neither retention interval, which varied from 15 s to 30 min in various investigations, nor shifts in retention interval affected tracking when only one trial was given each day. But when more than one daily trial was given, tracking was acquired more slowly and was disrupted by a shift in retention interval from 15 s to 5 min. Tracking was also disrupted by a shift from one to two trials each day. These results indicate that when given one 18-1-0 trial each day, the rat partitions events on a first-event/subsequent-event basis; that little forgetting occurs even at long retention intervals; that somewhat different memories signal events when one or more than one 18-1-0 trial occurs each day; and that retention interval deficits can arise owing to the same or similar memories' signaling different events. The results described limit the generality of three hypotheses suggested in two recent investigations: that as retention interval increases, rats find it increasingly difficult to remember and utilize serial position cues; that tracking in serial tasks is not influenced by number of trials each day; and that there are specific stimuli associated with each retention interval which, when changed, necessarily disrupt performance.     

Reward schedule effects in extinction: Intertrial interval, memory, and memory retrieval

Rats were trained in a runway such that partial reward occurred on Trial 1 of the day and consistent reward on subsequent massed trials (Group PRT1), or consistent reward occurred on Trial 1 of the day and partial reward on subsequent massed trials (Group PRTM). Under spaced (24-hr) extinction, Group PRT1 was more resistant to extinction than Group PRTM and under massed (1-min) extinction, Group PRTM was more resistant to extinction than Group PRT1. These findings suggest that (a) distinctive stimuli are associated with Trial 1 of the day and with subsequent massed trials, (b) these distinctive stimuli function as retrieval cues for memories, memory retrieval being independent of intertrial interval, and (c) behavior in extinction is controlled by a stimulus compound consisting of the memory of nonreward plus stimuli which accompany the memory of nonreward on rewarded acquisition trials.