What is learned in sequential learning? An associative model of reward magnitude serial-pattern learning

A computational model of sequence learning is described that is based on pairwise associations and generalization. Simulations by the model predicted that rats should learn a long monotonic pattern of food quantities better than a nonmonotonic pattern, as predicted by rule-learning theory, and that they should learn a short nonmonotonic pattern with highly discriminable elements better than 1 with less discriminable elements, as predicted by interitem association theory. In 2 other studies, the model also simulated behavioral "rule generalization," "extrapolation," and associative transfer data motivated by both rule-learning and associative perspectives. Although these simulations do not rule out the possibility that rats can use rule induction to learn serial patterns, they show that a simple associative model can account for the classical behavioral studies implicating rule learning in reward magnitude serial-pattern learning.     

A test for order relevance in a three-element serial learning task

Rats were trained in a straight runway on a simple three-element series of differing reward quantities. The first trial of the series ended with a two-pellet reward, and the second and third trials ended with a 12-pellet reward and a nonreward, respectively (2-12-0). All animals developed accurate anticipation of the terminal nonreward in the 2-12-0 series before it was rearranged for two test days during which the elements appeared in the order 12-2-0. The rats' anticipation of the terminal nonreward did not transfer to the reordered series, a result taken to mean that anticipation had been based upon interitem associations among memorial representations of the differing reward quantities. A second transfer test to 0-0-0, given after anticipation was reestablished to the 2-12-0 series, gave evidence that the ordinal position of the series elements was also a source of anticipation; the animals continued to run relatively slowly on the third trial in extinction. In neither transfer test was there evidence that the rats employed the strategy of enumerating rewarded trials, because counting is an order-irrelevant process.     

Reversal learning by fornix-transected monkeys

Experiment 1 examined visual reversal learning and in Experiment 2 monkeys were trained to criterion in a serial reversal set between “FR” and “DRO” response requirements. In both cases impairments were observed in fornix-transected monkeys. These results are discussed in connection with previous findings that in serial reversals damage to the hippocampal system in monkeys causes a deficit in spatial but not in visual learning. A unified account is proposed.     

Verbal rehearsal and selective attention in children with learning disabilities: A developmental lag

To investigate the development of verbal rehearsal strategies and selective attention in learning disabled children, Hagen's Central-Incidental task was administered to younger learning disabled (M CA = 8.68 years) and normal (M CA = 8.62 years) boys in Experiment 1 and to intermediate (M CA = 10.18 years) and older (M CA = 13.48 years) learning disabled boys in Experiment 2. Also, in Experiment 2, an experimentally induced verbal rehearsal condition was included to determine its effects on serial recall and selective attention performance. In Experiment 1, the serial postion curve of the normals revealed both a primacy and a recency effect, whereas that of the learning disabled revealed a recency effect only. In Experiment 2, both the intermediate and the older learning disabled exhibited both primacy and recency effects under both standard and rehearsal conditions. A developmental analysis of central recall for the three learning disabled groups revealed constant age-related increases in overall central recall and in primacy recall. That the normals recalled more central, but not more incidental, information than the learning disabled in Experiment 1 suggests that the learning disabled are deficient in selective attention. Correlational findings suggest that the selective attention of the learning disabled improves with age. The results were interpreted as support for the hypothesis of a developmental lag in the learning disabled population.     

Concurrent Cognitive Processes in Rat Serial Pattern Learning: Item Memory, Serial Position, and Pattern Structure

Three experiments examined the processes mediating rat serial pattern learning for rule-consistent versus rule-violating pattern elements (“violation elements”). In all three experiments, rats were trained to press retractable levers in a circular array in a specific sequence for brain-stimulation reward (BSR). Experiment 1 examined the role of lever location (L) and element serial position (SP) cues in rats’ ability to learn to anticipate a violation element positioned at the end of a 24-element serial pattern. Rats with L cues either alone or in combination with SP cues learned to anticipate the violation element, whereas those with SP cues alone did not. Rats in groups L and L + SP underwent a series of transfers designed to remove various cues that might have controlled their performance on the violation element. Results indicated that intra-chamber lever location cues mediated performance on the violation element whereas performance on rule-consistent elements within pattern chunks was mediated by an internal mnemonic representation that was insensitive to changes in lever location cues. Experiment 2 examined whether rats could learn to use SP cues alone to anticipate a violation element if it was positioned earlier in a serial pattern. Rats learned to anticipate the violation element based on SP cues alone when it was located in SP6 in a 24-element pattern, but not when it was in SP12. Experiment 3 examined whether or not rats spontaneously encode information about chunk length and the serial position of phrasing cues in serial patterns. Rats were trained to a high criterion on the serial pattern used in Experiment 1, then were challenged with three probe patterns that manipulated both chunk length and overall pattern length. Results indicated that rats spontaneously encoded information regarding the serial position of phrasing cues in relation to chunk length. Thus, rats appear to use at least three cognitive processes concurrently in serial pattern learning tasks, namely, item memory involving external discriminative cues, counting- or timing-like processes for encoding serial position, and rule abstraction for encoding an internal representation of pattern structure.     

Varying response-reinforcer contiguity in a recycling conjunctive schedule

Three experiments describe the effects of manipulating the frequency of response-reinforcer contiguity in a recycling conjunctive schedule. The schedule arranged that a reinforcer was delivered after 30 s provided at least one response had occurred; otherwise the next cycle started immediately. In Experiment 1, this schedule produced the typical pause–respond–pause pattern, with most responses at mid-interval; and, when a limited number of contiguities between responses and food delivery were added, the pattern became more like the monotonic scallop seen on fixed-interval schedules. In Experiment 2, the schedule was initially presented with an additional contingency that allowed contiguity on every trial. Fixed-interval-like behavior occurred and tended to persist as contiguities were gradually eliminated. In Experiment 3, the recycling conjunctive schedule alternated with a condition in which a large number of contiguities occurred. The pause–respond–pause pattern and fixed-interval-like performance occurred with few or many contiguities, respectively. The results of all three experiments illustrate how contiguity interacts with a small number of other variables to determine performance on interval schedules and illuminate previous findings with fixed-interval and fixed-time schedules.     

Amygdala and “emotional” modulation of the relative use of multiple memory systems

The basolateral amygdala modulates the cognitive and habit memory processes mediated by the hippocampus and caudate nucleus, respectively. The present experiments used a plus-maze task that can be acquired using either hippocampus-dependent “place” learning or caudate-dependent “response” learning to examine whether peripheral or intra-basolateral amygdala injection of anxiogenic drugs would bias rats towards the use of a particular memory system. In Experiment 1, adult male Long–Evans rats were trained to swim from the same start point to an escape platform located in a consistent goal arm, and received pre-training peripheral injections of the α₂-adrenoceptor antagonists yohimbine (2.5 or 5.0 mg/kg), RS 79948-197 (0.05, 0.1, or 0.2 mg/kg), or vehicle. On a drug-free probe trial from a novel start point administered 24 h following acquisition, vehicle treated rats predominantly displayed hippocampus-dependent place learning, whereas rats previously treated with yohimbine (2.5, 5.0 mg/kg) or RS 79948-197 (0.1 mg/kg) predominantly displayed caudate-dependent response learning. In Experiment 2, rats receiving pre-training intra-basolateral amygdala infusions of RS 79948-197 (0.1 μg/0.5 μl) also predominantly displayed response learning on a drug-free probe trial. The findings indicate (1) peripheral injections of anxiogenic drugs can influence the relative use of multiple memory systems in a manner that favors caudate-dependent habit learning over hippocampus-dependent cognitive learning, and (2) intra-basolateral amygdala infusion of anxiogenic drugs is sufficient to produce this modulatory influence of emotional state on the use of multiple memory systems.     

Superordinate categorization via learned stimulus equivalence: quantity of reinforcement, hedonic value, and the nature of the mediator

Three experiments examined superordinate categorization via stimulus equivalence training in pigeons. Experiment 1 established superordinate categories by association with a common number of food pellet reinforcers, plus it established generalization to novel photographic stimuli. Experiment 2 documented generalization of choice responding from stimuli signaling different numbers of food pellets to stimuli signaling different delays to food reinforcement. Experiment 3 indicated that different numbers of food pellets did not substitute as discriminative stimuli for the photographic stimuli with which the food pellets had been paired. The collective results suggest that the effective mediator of superordinate categories that are established via learned stimulus equivalence is not likely to be an accurate representation of the reinforcer, neither is it likely to be a distinctive response that is made to the discriminative stimulus. Motivational or emotional mediation is a more likely account.     

Combining CSs Associated with the Same or Different USs

In three experiments, hungry rats received appetitive training with four stimuli, A, B, X, and Y. In each Experiment, A and B were paired with one unconditioned stimulus (US; e.g. food pellets) whereas X and Y were paired with a second US (e.g. sucrose). Subsequently, rats responded more vigorously to combinations of stimuli associated with different USs (A-Y & X-B) than to combinations of stimuli associated with the same US (A-B &X-Y; Experiments 1, 2, & 3). This effect was observed when the stimuli were presented simultaneously and during the second elements of serial compounds (Experiments 2 & 3). Moreover, combining CSs associated with different USs resulted in a more marked CR than combining CSs that had each been paired with both US1 and US2 (Experiment 3). These results suggest that the sensory properties of appetitive reinforcers have an important influence on performance.     

Sequential dependencies in recall of sequences: Filling in the blanks

Sequential dependencies can provide valuable information about the processes supporting memory, particularly memory for serial order. Earlier analyses have suggested that anticipation errors—reporting items ahead of their correct position in the sequence—tend to be followed by recall of the displaced item, consistent with primacy gradient models of serial recall. However, a more recent analysis instead suggests that anticipation errors are followed by further anticipation errors, consistent with chaining models. We report analyses of 21 conditions from published serial recall data sets, in which we observed a systematic pattern whereby anticipations tended to be followed by the “filling in” of displaced items. We note that cases where a different pattern held tended to apply to recall of longer lists under serial learning conditions or to conditions where participants were free to skip over items. Although the different patterns that can be observed might imply a dissociation (e.g., between short- and long-term memory), we show that these different patterns are naturally predicted by Farrell’s (Psychological Review 119:223–271, 2012) model of short-term and episodic memory and relate to whether or not spontaneously formed groups of items can be skipped over during recall.     

Resistance to satiation: Reinforcing effects of food and eating under satiation

It has been shown previously that rats which have learned a response when hungry will continue to make that response when tested satiated, a phenomenon labeled resistance to satiation. Here we showed that rats which were previously trained hungry will learn a new response for the opportunity to consume pellets in a new situation when tested satiated. In four experiments various groups received each of the components of the training given when rats learn an instrumental response when hungry. Rats were placed in the goalbox of a straight alley and given food pellets when hungry or were hungry only in their home cages prior to running a straight alley in the satiated test in Experiment 1. In Experiments 2, 3, and 4 learning of a differential conditioning problem for pellets in S+ (nonreward in S?) was measured in the satiated test. Groups given pellets in their home cages when hungry with or without alley exposure learned to run more rapidly in S+ than in S? in the satiated test phase. The tendency to eat pellets in the apparatus and the reinforcing effect of eating the pellets was larger for rats which ate the pellets when hungry in their home cage than for rats which ate the pellets when satiated in their home cage. Being hungry in the home cage with no pellets was not sufficient to produce eating or running for pellets in the satiated test, indicating that any inherent reinforcing effect of the pellets is not sufficient to produce eating or running, and that incomplete satiation cannot account for the learning. These data indicate that a reinforcing effect of eating pellets under satiation is an important determiner of resistance to satiation.     

Motivational properties of spaced food presentation in hungry rats: Long-lasting effects on drinking and eating

Hungry rats were given 60 food pellets either at spaced intervals or massed in one or two presentations, or no food, followed by a consumption test in the home cage. Drinking was higher after spaced food sessions than after massed food or no food sessions (Experiments 1, 2, 3, and 5) and higher after the presentation of standard food pellets than after quinine-flavored pellets (Experiment 2). An effect of spacing was found even with a delay of 1 hr between the end of the food session and the drinking test (Experiment 3). Spaced food presentation also produced an increase in the consumption of quinine-flavored food pellets after the end of the session (Experiment 4). These results suggest that the enhancement of behavior caused by intermittent reinforcement cannot be attributed entirely to the short-term eliciting effects of the reinforcer or to its thirst-inducing properties.     

Serial learning in rats: A test of three hypotheses

Findings obtained by providing rats with a single fixed series of events, A-B-C-…, often are equally compatible with three alternative serial learning interpretations: that the signal for items is (A) their position in the series (position view), (B) the prior item of the series (chaining view), and (C) one, two, or more prior items of the series (sequential view). By employing a novel procedure of supplying rats with two different series, rather than a single series, it was possible to choose between the three alternatives. Employing 10 and 0, 0.045 g, Noyes pellets as items, rats in both runway experiments reported here received a three trial series, either a 10-0-10 series or a 0-0-10 series. In Experiment 1, the other series was a single 0 pellet trial (along with 10-0-10) or a single 10 pellet trial (along with 0-0-10). In Experiment 2, the other series was either 0-0 (along with 10-0-10) or 10-0 (along with 0-0-10). Considering both experiments, findings were consistent with the sequential view but not with either the position view or the chaining view. The possibility was suggested that, under some other experimental conditions, particularly those not employing reward events as items, greater control over discriminative responding might be exercised by position cues than by item cues.     

Domain effects in lexical development: learning words for foods and toys

We investigated 3-year-olds’ and adults’ use of domain cues in learning words for solid and nonsolid material entities. In Experiments 1 and 2, participants heard a novel neutral noun (e.g., “my X”) for a standard solid or nonsolid entity described as either a toy or a food. They then were asked to extend the word to one of two other entities. Both options matched the standard in solidity; but one differed from it in an object-relevant property (shape) and the other in a substance-relevant property (color, texture, or smell). Both children and adults were more likely to select the same-shaped entity if the standard was (1) solid than if it was nonsolid, and (2) described as a toy than if it was described as a food. Their interpretations of novel words for material entities were thus affected not only by perceptual information (about solidity) but also by conceptual information (about domain). In Experiment 3, the novel noun was presented in a syntactic context that suggested the solid entity should be interpreted as an object (e.g., “an X”) and that the nonsolid entity should be interpreted as a substance (e.g., “some X”). For adults, these changes largely eliminated the effect of the entity’s domain (toy, food) on interpretation. We interpret these findings in terms of the proposal that domain cues, like solidity cues, furnish information about whether an entity’s structure should be thought of as arbitrary or nonarbitrary and, hence, about whether a word should be interpreted as naming an object or a substance construal.     

Serial learning: A multilevel access analysis

In Experiment I the lists were 36 and 48 unrelated words. Each was divided into successive groups of four words and learned to a perfect criterion. In Experiment II the lists were made up of six categorical groups of five exemplars each. Degree of learning was varied. In both experiments serial anticipation learning was followed by ordinary free recall and free recall under speed stress. Analyses of acquisition and of both recall tests indicate that group access is a marked function of serial position but that within-group retrieval given group access is constant over serial position. It is argued that serial learning proceeds simultaneously at different levels of representation.     

Consistent and Variable Practice Conditions: Effects on Relative and Absolute Timing

The authors conducted the present experiments to resolve the discrepancy between studies in which relative-timing learning has been found to be enhanced by consistent practice conditions and contextual interference experiments in which relative-timing learning has been found to be enhanced more by random practice than by blocked practice. There were 40 participants in Experiment 1 and 48 in Experiment 2. The results of Experiment 1 extended previous findings: The learning of the relative-timing pattern was systematically enhanced by the degree to which the practice conditions promoted movement consistency (constant > blocked > serial > random). Experiment 2 provided evidence that the discrepancy between the relative-timing effects in the 2 groups of studies was a product of the way in which relative-timing goals and feedback were presented. When the feedback was presented as segment times, random practice resulted in generally more stable relative-timing patterns during acquisition than blocked practice did. Thus, in both experiments, the learning of the relative-timing pattern was enhanced by more stable relative-timing conditions during acquisition. Absolute-timing learning, as indexed by the transfer tests, was enhanced by serial or random practice as compared with constant or blocked practice, and was relatively unaffected by feedback conditions directed at the relative-timing pattern. In terms of motor programming theory, those findings are taken as additional evidence for the disassociation of memories supporting generalized motor program (GMP) performance, as indexed by relative timing, and parameter performance, as indexed by absolute timing.     

Serial rote-learning: Incremental or all-or-none?

If the associative connections in a serial list are acquired in an all-or-none fashion, rather than gradually with every trial adding an increment of associative strength, then changing the serial order of the middle items in the list during the course of practice should have no effect on the rate of learning the list as a whole or even of the particular items that have been interchanged. Thirty subjects learned a serial list by the anticipation method. The middle items of the list were reversed in serial order approximately half-way through the number of trials required for mastery. The subjects took no longer to learn the list and made no more errors than did 30 control subjects for whom there was no change in serial order. The serial-position curves of the two groups were almost identical. It was also shown that the learning “curves” of single items in the series, when plotted for individual subjects do not reveal a gradually increasing probability of the correct response, but show instead a sudden jump on one trial from the chance guessing level to a level close to 100 per cent, correct responses. The results are consistent with a non-incremental theory of serial learning.     

Food delivery as a conditional stimulus: Feature-learning and memory in pigeons

Three experiments investigated the learning and memory of discriminations based on presence versus absence of a pre-trial food delivery. In Experiment 1 half the illuminations of a response key were followed by food regardless of the subject's behavior. In one group an extra food delivery preceded only reinforced trials (feature-positive condition), whereas in a second group it preceded only nonreinforced trials (feature-negative condition). Key pecks and approaches revealed more rapid and superior discrimination learning in the first group. Experiment 2 replicated the results of Experiment 1 but yielded no evidence that greater “unexpectedness” of pretrial food conditions facilitates discriminative performance. In Experiment 3, individual pigeons trained on a conditional discrimination exhibited a within-subject feature-positive superiority. Delay between pretrial and trial stimuli interacted with feature-positive versus feature-negative training in both the between-group (Experiment 2) and within-subject (Experiment 3) procedures: performance was decremented at both short and long delays in the feature-positive condition but was decremented only at longer delays in the feature-negative condition. The feature-positive superiority obtained here is incompatible with explanations based on either the general concept of “perceptual organization” or on the conditional nature of feature-negative discriminations.     

Consistent and variable practice conditions: effects on relative and absolute timing

The authors conducted the present experiments to resolve the discrepancy between studies in which relative-timing learning has been found to be enhanced by consistent practice conditions and contextual interference experiments in which relative-timing learning has been found to be enhanced more by random practice than by blocked practice. There were 40 participants in Experiment 1 and 48 in Experiment 2. The results of Experiment 1 extended previous findings: The learning of the relative-timing pattern was systematically enhanced by the degree to which the practice conditions promoted movement consistency (constant > blocked > serial > random). Experiment 2 provided evidence that the discrepancy between the relative-timing effects in the 2 groups of studies was a product of the way in which relative-timing goals and feedback were presented. When the feedback was presented as segment times, random practice resulted in generally more stable relative-timing patterns during acquisition than blocked practice did. Thus, in both experiments, the learning of the relative-timing pattern was enhanced by more stable relative-timing conditions during acquisition. Absolute-timing learning, as indexed by the transfer tests, was enhanced by serial or random practice as compared with constant or blocked practice, and was relatively unaffected by feedback conditions directed at the relative-timing pattern. In terms of motor programming theory, those findings are taken as additional evidence for the disassociation of memories supporting generalized motor program (GMP) performance, as indexed by relative timing, and parameter performance, as indexed by absolute timing.     

Motivational control of instrumental performance: The role of prior experience of the reinforcer

In four experiments we investigated the conditions that are necessary for instrumental performance to adjust appropriately to a change in drive state. Rats were trained to press a lever and pull a chain concurrently, with one action being reinforced by sucrose solution and the other by food pellets. In Experiment 1 for animals that were hungry throughout training the rate of lever pressing in an extinction test under thirst was unaffected by the type of reinforcer produced by this action during training, even though the sucrose solution would maintain a higher rate than the food pellets during training under thirst. In contrast, animals that experienced the instrumental contingencies arranged by the concurrent schedule while thirsty at some point during training pressed the lever more during the extinction test under thirst when this action had been reinforced with the sucrose solution rather than the food pellets. The remaining three experiments demonstrated that for this incentive effect to occur it is sufficient that the sucrose solution be delivered non-contingently under thirst at some stage of training. Thus, it would appear that performance mediated by an instrumental contingency adjusts appropriately to reinforcer revaluation brought about by a drive shift only if the animals have had prior experience with the incentive under the test drive state. This observation was interpreted in terms of Tolman's “cathexis” theory of motivation.