Intertrial interval effects on internal stimulus control of behavior in brightness differential conditioning

In Experiment 1, groups were given a trial sequence in differential conditioning in which all S+ trials preceded all S? trials (+? schedule) or one in which some S+ trials followed S? trials (+?+ schedule) and either a 1- or a 30-min intertrial interval (ITI). ITI affected discrimination learning only in the +? schedule condition; schedule affected discrimination only at massed trials. In Experiment.2, all groups received a +?+ schedule. In two groups, given a 1- or 15-min ITI between all trials, discrimination learning was independent of ITI. Discrimination learning was facilitated in two other groups given a 1-min ITI between all trials except between S? and the subsequent S+ trial, when the ITI was either 15 or 60 min. The results were discussed in terms of their implications for internal reward-related stimulus control of behavior in differential conditioning.     

Blocking in successive differential conditioning: Prior acquisition of control by internal cues blocks the acquisition of control by brightness

In each of four experiments the acquisition of discriminative stimulus control by brightness cues (black vs white runway) in a successive go/no-go instrumental discrimination was blocked in groups given prior discrimination training with internal (reward-produced and intertrial interval-related) cues as relevant discriminanda and brightness cues irrelevant. The blocking effects obtained here in instrumental conditioning were substantial and in most cases complete. Blocking occurred whether brightness in Phase I varied across trials but was uncorrelated with reinforcement (varied-irrelevant, V-I, condition) or whether all Phase 1 trials occurred to a single value on the brightness dimension (constant-irrelevant, C-I, condition) which then served as a redundant S+ cue in Phase 2 while the previously unexperienced brightness cue was added to the S- stimulus compound, or vice versa. Blocking in the V-I condition was shown not to be due simply to nondifferential reinforcement of brightness in Phase 1, but to depend on the prior acquisition of discriminative stimulus control by internal cues. As in Pavlovian conditioning, blocking here was an increasing function of amount of prior conditioning to the blocking stimulus. The results encourage the prospect that the procedures used here can be developed into a viable instrumental conditioning companion to the Pavlovian procedures now used almost exclusively to study blocking.     

The differential reinforcement of reward-produced and response-produced stimuli

In two experiments, groups received successive large-reward trials on odd-numbered days and successive small-reward trials on even-numbered days in the same gray alleyway. This produced a discrimination problem in which the memory of large reward (S^L) was reliably discriminative of large reward and the memory of small reward (S^S) was reliably discriminative of small reward. Intertrial interval (ITI) was varied both between and within groups. In the within-groups manipulation, ITI separating S+ trials differed from that separating S- trials. Experimental groups learned the discrimination, running slower to S^S (S- cue) than to S^L (S+ cue), and showed a negative contrast effect, running slower to S- than a small-reward control group. Discrimination was somewhat faster at massed than at spaced trials. The within-groups manipulation of ITI suggested that the effects of ITI were mediated by time-dependent changes in internal cues produced by reward events and by the instrumental response. The control exercised by internal cues was shown to be associative rather than nonassociative, e.g., motivational. Similarities with, and implications for, conventional brightness differential conditioning were discussed.     

Short-term retention of response outcome as a determinant of serial reversal learning

The correlation between short-term retention of the outcome of the preceding response and overall learning proficiency was investigated for serial reversal learning. Pigeons were trained to asymptote on a serial reversal problem and then were presented a percentage reinforcement schedule where only some correct trials were rewarded. Nonrewarded correct trials were treated exactly as incorrect trials. The difference in error probability following the two types of correct trials was then used as a measure of short-term retention. When intertrial intervals (ITI) were short (6 sec), substantial differences occurred. When the ITI's were increased, the difference in accuracy declined regularly to no difference at an ITI of 60 sec. This demonstration of a short-term retention gradient, coupled with the finding that overall reversal learning was much better with the shorter ITI's, suggests that a primary mechanism of improvement in serial reversal learning is the acquisition of a conditional discrimination based on the outcome of the preceding response.     

Dissociations among ABA, ABC, and AAB recovery effects

In a human predictive learning experiment, the strengths of ABA, ABC, and AAB recovery effects after discrimination reversal learning were compared. Initially, a discrimination between two stimuli (X+, Y−) was trained in Context A. During Phase 2, participants received discrimination reversal training (X−, Y+) either in Context A (Group AAB) or in Context B (Group ABA, Group ABC). Testing occurred in the reversal context and in a neutral context (Group AAB, Group ABC) or in the reversal context and in the acquisition context (Group ABA). Recovery effects were similar in ABA and ABC procedures, whereas no AAB recovery effect occurred. Furthermore, the rate of discrimination reversal learning during Phase 2 was unaffected by whether training was conducted in the same or in a different context as initial learning. Results indicate that contextual stimuli are processed before interfering information is presented. These representations, however, do not acquire the ability to control performance.     

How transitions from nonrewarded to rewarded trials regulate responding in Pavlovian and instrumental learning following extensive acquisition training

In both discrimination learning and partial reinforcement, transitions may occur from nonrewarded to rewarded trials (NR transition). In discrimination learning, NR transitions may occur in two different stimulus alternatives (NR different transitions). In partial reward, NR transitions may occur in a single stimulus alternative (NR same transitions). Available instrumental learning data indicate that resistance to extinction is increased by both types of NR transitions following limited acquisition training. Following more extensive acquisition training, resistance to extinction appears to be increased by NR same transitions but not by NR different transitions. In Experiment 1, it was shown for the first time that following extensive acquisition training the effects of the two types of transitions are the same in the Pavlovian situation as in the instrumental situation. This finding indicates that on the current trial the rat remembered the conditioned stimulus (CS) and the unconditioned stimulus (US) from the prior trial and those memories along with the CS on the current trial become the signal for the current US. Experiments 2 and 3, which tested hypotheses about instrumental learning, identified why NR different transitions lose their capacity to promote vigorous responding following extensive acquisition training. This is because cues occurring on the rewarded trials of NR different transitions more validly signal reward than other situational cues and thus overshadow them. Finally, some implications of the present findings for understanding the role of NR different transitions in discrimination learning situations were discussed.     

Dominance as a function of cue similarity: A test of the intrinsic cue dominance model

A nonselective model postulating intrinsic cue dominance was tested in simultaneous discrimination tasks involving reversal on one dimension. In this procedure two dimensions are relevant throughout training; however, following initial discrimination training the reward contingency is reversed for one dimension but maintained for the other. Cue dominance was assessed following acquisition of reversal by the use of opposed-cues test trials, and was defined as a greater number of choices of the test compound containing the positive cue of the reversed dimension than of the test compound containing the positive cue of the maintained dimension. In Experiment I, brightness cues dominated orientation cues. In Experiment II, which employed two different sets of relevant cues, more disparate brightness cues dominated the orientation cues for one set and orientation cues dominated less disparate brightness cues for the other. From this, it was concluded that dominance is a function of relative cue similarity.     

Stimulus control of responding in the early trials of differential conditioning

In Experiment 1 two groups of rats were given 12 differential conditioning trials, seven to the rewarded alley (S+) and five to the nonrewarded alley (S?), prior to being extinguished in both alleys. Group S?S+ received S+ trials, following S? trials in acquisition, while Group S+S? did not receive S+ trials following S? trials in acquisition. In extinction S+ and S? trials were presented according to a quasi-random sequence for both groups. Running on the last 3 trials of acquisition was found to be faster following S+ than following S? trials. Group S?S+ showed greater resistance to extinction and less discriminative responding in extinction than Group S+S?. These results suggest that responding in differential conditioning is controlled not merely by S+ and S? but by the memories of reward (S^R) and of nonreward (S^N) as well. When the joint effects of both classes of cues were considered, e.g., S^R+S+, responding in the early trials of differential conditioning was shown to be highly orderly. Experiment 2 was highly similar to Experiment 1 except that Groups S?S+ and S+S? were equated along dimensions not equated in Experiment 1. The results obtained in Experiment 2 were highly similar to those obtained in Experiment 1.     

Memory, acquisition, and category clustering in learning-disabled children.

In Experiment 1, single trial, immediate-free recall of learning disabled and nondisabled children was compared. The primacy effect in learning-disabled children was lower, suggesting that rehearsal or other types of elaborative encoding may be deficient in these children. In Experiment 2, acquisition of randomly presented categorical lists in a multitrial-free recall task was compared in learning disabled and nondisabled children. One-half of each group was required to learn the same number of words (34 per list), whereas list length for the other half exceeded the primacy effect of each child in immediate-free recall to the same degree. When the same number of items was learned, acquisition was slower in learning disabled than nondisabled children. When the number of items varied according to the primacy effect of each child, acquisition of both groups was similar. Clustering was lower in learning disabled than nondisabled children. In Experiment 3, multitrial-free recall acquisition of categorical lists was examined in a subject-paced task. When the number of words learned exceeded the primacy effect of each child to the same degree, trials to criterion were similar in both groups but, when the children learned the same number of items, learning-disabled children required more trials to criterion. Presentation rates were faster in learning-disabled children. Presentation rates were negatively correlated with trials to criterion and positively correlated with clustering and primacy in immediate-free recall, suggesting that study time may be taken up by clustering, rehearsal, and/or other encoding strategies. Deficient elaborative encoding may be responsible for the slower acquisition of learning-disabled children.     

The effect of training procedures on the overlearning reversal effect in young children

The performance of 80 preschool children on a reversal problem was studied as a function of amount of training and type of training procedure used during acquisition and reversal. In the extinction phase of reversal learning, subjects given a correction procedure during the reversal problem made fewer perseverative errors than subjects given noncorrection. In the reversal midplateau phase of reversal learning, overtraining facilitated reversal learning for subjects receiving noncorrection during the acquistion problem, but not for subjects receiving correction. A shift in training procedure between acquisition and reversal increased the number of subjects who reached criterion immediately after perseveration. Since these results are difficult to explain in terms of traditional learning theories, an alternative response-switching strategy explanation was proposed.     

Contextual control in discrimination reversal learning

In 3 human predictive learning experiments, the authors examined contextual control of responding in discrimination reversal learning. In Phase 1, a discrimination between 2 stimuli (A+, B-) was trained in Context 1. During Phase 2, participants received discrimination reversal training (A-, B+) in Context 2. Testing occurred in Context 1 and Context 2 (Experiments 1A and 1B) or in Context 1 and Context 3 (Experiment 2). During the test phase, performance in Context 1 and Context 2 reflected the contingencies trained during Phase 1 and Phase 2, respectively. When testing occurred in Context 3, there was no discriminative responding between A and B. In addition, the experiments demonstrated that discriminating stimuli with a consistent reinforcement history were also affected by contextual manipulations. Results indicate that each training context acquires the ability to control performance. Unique-cue and configural approaches account for a major part of the results.     

Effects of error and errorless discrimination acquisition on reversal learning

The effectiveness of trial-and-error, graded-choice, and verbal-instruction procedures on the acquisition and maintenance of a two-choice simultaneous color discrimination in an intradimensional double-reversal learning situation was studied using 18 first-grade children. After acquiring a red-green discrimination during one 70-trial session, the discriminative roles of the stimuli were reversed for 30 trials, followed by a second reversal for 30 trials. Children in the graded-choice and verbal-instruction groups acquired and maintained the discriminations with fewer errors than children who learned by trial and error. The importance of the results in terms of two-stage discrimination learning theories is pointed out and similarities between errorless learning and overtraining are discussed.     

Responding to a positive stimulus by “satiated” pigeons

Skinner (1938) found that rats given discrimination training (Phase I) and then reinforced to “satiation” for responses in the presence of the negative stimulus (S?) (Phase II), began to respond again when the positive stimulus (S+) was reintroduced (Phase III). Experiment I replicated Skinner's finding with pigeons, alternating S+ and S? presentations during Phase III. In Experiment II, Phase II was extended, and Phase III results were similar to those of Experiment I, demonstrating that the recovery of S+ responding could not be attributed to a lax Phase II satiation criterion. In Experiment III, a uniform schedule of reinforcement was maintained throughout the three phases, and results similar to those of Experiment I were found, indicating that renewed S+ responding was not due to the shift in schedule between phases. In Experiment IV, Phase I consisted of discrimination training with two positive stimuli (S+_s and S+_n), and Phase II consisted of reinforcement for responses in the presence of S? and S+_s. During Phase III, significantly more responding was found to S+_s and S+_n than to S?, but no difference in responding was found between the two positive stimuli. In Experiment V, Phase I consisted of simple discrimination training, and during Phase II, responses in the presence of both S? and a novel stimulus (So) were reinforced. During Phase III, significantly more responding was found to S+ than to either S? or So, with no difference found between S? and So responding. Renewed responding to S+ during Phase III in the present experiments is best explained by behavioral contrast developed during Phase I.     

Influence of two types of information trials on the formation of a discrimination-reversal learning set in Japanese monkeys

Abstract: Three groups of male adult Japanese monkeys were trained in 30 discrimination-reversal learning-set (LS) problems using the Wisconsin General Test Apparatus. A control group ( N = 3) was trained in the standard procedure: acquisition of up to 12 consecutive correct responses, reversal to the same criterion, and then shift to a new problem. To the other two groups, three information trials signaling the stimulus value in reversal were given after they reached the criterion in acquisition. Subjects in the positive-stimulus information (PSI) group ( N = 4) were exposed only to the reversal positive stimulus, and a response to it was rewarded. Subjects in the negative-stimulus information (NSI) group ( N = 4) were exposed only to the reversal negative stimulus, and any response to it was not rewarded. Information trials reduced reversal errors and NSI trials were more effective than PSI trials in error reduction ( p < .05). Information trials also aided in acquisition: the NSI and PSI groups learned the acquisition phase of the problems with significantly fewer errors than control group ( p < .01) in the last stage of training. The implications of these results are discussed in relation to strategy learning and attention learning in LS formation.     

Transfer of a response controlling stimulus duration across discrimination problems

In the acquisition phase, pigeons learned to peck at a changeover key to shorten the duration of S? but not of S+ presented on the food key in a discrimination problem. In the transfer phase, the significance of S+ and S? was changed through extinction of both, equal reinforcement, or discrimination reversal, while the changeover key was not available. Transfer tests then showed appropriate modification of the changeover response. Similar transfer was demonstrated across orthogonal stimulus dimensions. Further analytic studies showed that this transfer of the changeover response did not depend upon mediation due to differential response rates to the food key. This research strategy enriches the study of the “second learning process” by providing an indicator of stimulus control in all phases of the procedure. Direct transfer between different problems also indicates that discriminative stimuli, although physically dissimilar, have the same “psychological value” for the subject.     

Stimulus function in simultaneous discrimination

In discrimination learning, the negativity of the stimulus correlated with nonreinforcement (S-) declines after 100 training trials while the stimulus correlated with reinforcement (S+) is paradoxically more positive with lesser amounts of discrimination training. Training subjects on two simultaneous discrimination tasks revealed a within-subjects overlearning reversal effect, where a more-frequently presented discrimination problem was better learned in reversal than was a discrimination problem presented less frequently during training.     

The influence of NMDA receptors in the dorsomedial striatum on response reversal learning

In mammals, the dorsomedial striatum is one brain area shown to be critical for the flexible shifting of response patterns. At present, the neurochemical mechanisms that underlie learning during a shift in response patterns are unknown. The present study examined the effects of NMDA competitive antagonist, DL-2-amino-5-phosphonopentanoic acid (AP-5), injected into the dorsomedial striatum on the acquisition and reversal of a response discrimination. Male Long-Evans rats were tested across two consecutive days in a modified cross-maze. Rats received an infusion of either saline or AP-5 (5 or 25 nmol) 5 min prior to each test session. In the acquisition phase rats learned to turn in one direction (right or left) to receive a cereal reinforcement. In the reversal learning phase rats learned to turn in the opposite direction as in the acquisition phase. In both phases, criterion was achieved when a rat made 10 consecutive correct trials. Infusions of AP-5 did not impair acquisition, but impaired reversal learning of a response discrimination in a dose-dependent fashion. The reversal learning deficit induced by AP-5 resulted from reversions back to the originally learned response pattern following the initial shift. These results suggest that activation of NMDA receptors in the dorsomedial striatum are critical for the flexible shifting of response patterns by enhancing the reliable execution of a new response pattern under changing task contingencies.     

Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals

Recent work indicates that both orbitofrontal cortex (OFC) and the basolateral complex of the amygdala (ABL) are involved in processes by which cues are associated with predicted outcomes. To examine the respective roles of these structures in discrimination learning, rats with bilateral sham or neurotoxic lesions of either OFC or ABL were trained on a series of four 2-odor discrimination problems in a thirst-motivated go, no-go task. After acquisition of the series of odor problems, the rats were trained on serial reversals of the final odor problem. Performance on each problem was assessed by monitoring accuracy of choice behavior, and also by measuring latency to respond for fluid outcomes after odor sampling. During discrimination learning, rats in both lesioned groups had similar deficits, failing to show normal changes in response latency during learning, while at the same time exhibiting normal choice behavior relative to controls. Choice behavior was affected only during the reversal phase of training, in which OFC and ABL lesions produced distinctive deficits. Rats with ABL lesions were impaired on the first reversal (S1−/S2+), but were unimpaired at acquiring a reversal back to the original odor-outcome contigencies (S1+/S2−), whereas rats with OFC lesions were impaired on both types of reversals. These findings suggest that OFC and ABL serve partially overlapping roles in the use of incentive information that supports normal discrimination performance.     

Errorless learning of a conditional temporal discrimination

In the present study we extended errorless learning to a conditional temporal discrimination. Pigeons' responses to a left-red key after a 2-s sample and to a right-green key after a 10-s sample were reinforced. There were two groups: One learned the discrimination through trial and error and the other through an errorless learning procedure. Then, both groups were presented with three types of tests. First, they were exposed to intermediate durations between 2 s and 10 s, and given a choice between both keys (stimulus generalization test). Second, a delay from 1 s to 16 s was included between the offset of the sample and the onset of the choice keys (delay test). Finally, pigeons learned a new discrimination in which the stimuli were switched (reversal test). Results showed that pigeons from the Errorless group made significantly fewer errors than those in the Trial-and-Error group. Both groups performed similarly during the stimulus generalization test and the reversal test, but results of the delay test suggested that, on long stimulus trials, responding in the errorless training group was less disrupted by delays.     

Transfer after pretraining with the negative stimulus

Two groups of rats trained in the jumping stand acquired a horizontal stripes vs. vertical stripes discrimination and then learned its reversal. The groups differed in their pretraining. One group had previously learned a black vs. white discrimination; the other a discrimination in which the S- was the same as that used in the horizontal against vertical problem. S- pretraining facilitated not only acquisition but also the reversal of the orientation discrimination thereby confirming a prediction that reversal learning will be rapid when the original problem is learned chiefly in terms of the S-. It is suggested that overtraining may facilitate reversal learning, in some cases at least, by inducing animals to concentrate on the S-.