On the effects of signaling reinforcer probability and magnitude in delayed matching to sample

Two experiments examined whether postsample signals of reinforcer probability or magnitude affected the accuracy of delayed matching to sample in pigeons. On each trial, red or green choice responses that matched red or green stimuli seen shortly before a variable retention interval were reinforced with wheat access. In Experiment 1, the reinforcer probability was either 0.2 or 1.0 for both red and green responses. Reinforcer probability was signaled by line or cross symbols that appeared after the sample had been presented. In Experiment 2, all correct responses were reinforced, and the signaled reinforcer durations were 1.0 s and 4.5 s. Matching was more accurate when larger or more probable reinforcers were signaled, independently of retention interval duration. Because signals were presented postsample, the effects were not the result of differential attention to the sample.     

The effects of reinforcer magnitude in the preceding and upcoming ratios on between‐ratio pausing in multiple,mixed, and single fixed‐ratio schedules

Hens responded under multiple fixed‐ratio schedules with equal response requirements and either a 1‐s or a 6‐s reinforcer. Upcoming reinforcer size was indicated by key color. Components were presented in a quasirandom series so that all four component transitions occurred. Postreinforcement pauses were affected by the upcoming and preceding reinforcer size, with longer pauses after large reinforcers followed by small reinforcers than when followed by large ones, and longer pauses after small reinforcers that were followed by small reinforcers rather than large ones. Pauses increased with fixed‐ratio size and the effects of reinforcer size were larger the larger the ratio. When reinforcer size was not signaled—mixed fixed‐ratio schedules—pauses were shorter after small than after large reinforcers. Signalling the upcoming reinforcer attenuated the effect of the previous reinforcer size on pause duration when small was followed by small and when either small or large by large, but enhanced the effect when large was followed by small. There was no effect of reinforcer size on pause duration when single fixed‐ratio schedules were arranged. The effects of reinforcer size on pauses depends on the size and range of the fixed ratios as well as the exact procedures used in the study.     

The effect of cue-familiarity, cue-distinctiveness, and retention interval on prospective remembering

Five experiments investigated the effects of cue familiarity, cue distinctiveness, and retention interval on prospective remembering. Results showed that (1) performance in a prospective memory task is facilitated when the cue is unfamiliar and/or distinctive; and (2) it is impaired by 3-minutes' delay between the instructions and the task (Experiment 1). A beneficial effect of distinctiveness was also found when perceptual rather than semantic distinctiveness was tested (Experiment 2). Experiments 3 and 4 ruled out the hypotheses that “unfulfilled expectancy” of an event (i.e. non-appearance of the cue during training) (Experiment 3), or some sort of “habituation” in the target context (Experiment 4), may have caused the low performance observed in the delayed conditions. Finally, results from Experiment 5 showed that delay negatively affected prospective remembering when it was filled with either a demanding interpolated activity (practice in a STM task) or an undemanding motoric activity (repetitive hands movements). Unfilled delay and an undemanding verbal activity (counting) were found not to affect prospective memory. Implications for the mechanisms underlying prospective remembering are discussed.     

Reversing the course of forgetting

Forgetting functions were generated for pigeons in a delayed matching-to-sample task, in which accuracy decreased with increasing retention-interval duration. In baseline training with dark retention intervals, accuracy was high overall. Illumination of the experimental chamber by a houselight during the retention interval impaired performance accuracy by increasing the rate of forgetting. In novel conditions, the houselight was lit at the beginning of a retention interval and then turned off partway through the retention interval. Accuracy was low at the beginning of the retention interval and then increased later in the interval. Thus the course of forgetting was reversed. Such a dissociation of forgetting from the passage of time is consistent with an interference account in which attention or stimulus control switches between the remembering task and extraneous events.     

Delayed matching-to-sample performance: Effects of relative reinforcer frequency and of signaled versus unsignaled reinforcer magnitudes

Six pigeons were trained on a delayed red-green matching-to-sample task that arranged four delays within sessions. Matching responses intermittently produced either 1.5-s access to food or 4.5-s access to food, and nonmatching responses produced either 1.5-s or 4.5-s blackout. Two phases were conducted: a signaled phase in which the reinforcer magnitudes (small and large) were signaled by houselights (positioned either on the left or right of the chamber), and an unsignaled phase in which there was no correlation between reinforcer magnitude and houselight position. In both phases, the relative frequency with which red and green matching responses produced food was varied across five values. Both matching accuracy and the sensitivity of performance to the distribution of reinforcers for matching responses decreased with increasing delays in both phases. In addition, accuracy and reinforcer sensitivity were significantly lower on signaled small-reinforcer trials compared with accuracy and sensitivity values on signaled large-reinforcer trials and on both types of unsignaled trials. These results are discussed in the context of research on both nonhuman animal and human memory.     

Successive incrementing non-matching-to-samples in rats: An automated version of the odor span task

The odor span task is a procedure frequently used to study remembering of multiple stimuli in rodents. A large arena is used and odor stimuli are presented using scented cups. Selection of each odor is reinforced when first presented, but not on subsequent presentations; correct selections depend on remembering which stimuli were previously presented. The use of an arena setting with manual stimulus presentation makes the odor span task labor-intensive and limits experimental control; thus, an automated version of the task would be of value. The present study used an operant chamber equipped with an olfactometer and trained rats using successive conditional discrimination procedures under an incrementing non-matching-to-samples contingency. High rates of responding developed to odor stimuli when they were session-novel with low rates of responding to subsequent presentations of that odor. Additional experiments assessed variations of the procedure to determine the role of the frequency of odor presentation and the retention interval separating sample and comparison. Discrimination was impaired with long retention intervals suggesting the importance of this variable. These findings confirmed that rats differentiate between stimuli that are session-novel and those previously encountered and support the use of an automated procedure as an alternative to the odor span task.     

Choice As A Function Of Reinforcement Ratios In Delayed Matching-to-sample

Pigeons were studied in two experiments using a delayed matching-to-sample task. In Experiment 1, 4 subjects were exposed to a task in which the proportion of reinforcement associated with matching and nonmatching, and the overall proportion of reinforcement associated with selecting each choice, regardless of the sample stimulus, were varied. Choice was sensitive to both proportions. A least squares regression analysis showed that Wixted's (1989) proportions of reinforcement model closely fit the data from Experiment 1; however, the model failed to make accurate qualitative predictions for some test conditions. In Experiment 2, 4 subjects were exposed to a delayed matching-to-sample task in which the retention intervals and the reduction in delay to reinforcement signaled by the onset of the sample stimulus were independently varied. When the retention interval was short and when the delay-reduction value of the sample stimulus was high, the sample exerted greater control over choice; the control by the overall proportion of reinforcements for selecting each choice stimulus was correspondingly low. Conversely, when the retention interval was long and the delay-reduction value of the sample stimulus was low, the sample exerted relatively less control over choice; control by the overall proportion of reinforcements obtained for selecting each choice stimulus was correspondingly high. A signal detection analysis found that sensitivity to reinforcement varied directly with retention interval. Data were also consistent with misallocation models. No evidence was found to suggest that pigeons ignore the rate at which selecting individual choice stimuli is reinforced, as has been reported in studies with human subjects.     

Sample-duration effects on pigeons' delayed matching as a function of predictability of duration

Three experiments assessed the impact of sample duration on pigeons' delayed matching as a function of whether or not the samples themselves signaled how long they would remain on. When duration was uncorrelated with the sample appearing on each matching trial, the typical effect of duration was observed: Choice accuracy was higher with long (15-s) than with short (5-s) durations. By contrast, this difference either disappeared or reversed when the 5- and 15-s durations were correlated with the sample stimuli. Sample duration itself cued comparison choice by some birds in the latter (predictable) condition when duration was also correlated with the reinforced choice alternatives. However, even when duration could not provide a cue for choice, pigeons matched predictably short-duration samples as accurately as, or more accurately than, predictably long-duration samples. Moreover, this result was observed independently of whether the contextual conditions of the retention interval were the same as, or different from, those of the intertrial interval. These results strongly support the view that conditional stimulus control by the samples is partly a function of their conditioned reinforcing properties, as determined by the relative reduction in overall delay to reinforcement that they signal.     

Reinforcement: Food Signals the Time and Location of Future Food

It has long been understood that food deliveries may act as signals of future food location, and not only as strengtheners of prefood responding as the law of effect suggests. Recent research has taken this idea further—the main effect of food deliveries, or other “reinforcers”, may be signaling rather than strengthening. The present experiment investigated the ability of food deliveries to signal food contingencies across time after food. In Phase 1, the next food delivery was always equally likely to be arranged for a left‐ or a right‐key response. Conditions were arranged such that the next food delivery was likely to occur either sooner on the left (or right) key, or sooner on the just‐productive (or not‐just‐productive) key. In Phase 2, similar contingencies were arranged, but the last‐food location was signaled by a red keylight. Preference, measured in 2‐s bins across interfood intervals, was jointly controlled by the likely time and location of the next food delivery. In Phase 1, when any food delivery signaled a likely sooner next food delivery on a particular key, postfood preference was strongly toward that key, and moved toward the other key across the interreinforcer interval. In other conditions in which food delivery on the two keys signaled different subsequent contingencies, postfood preference was less extreme, and quickly moved toward indifference. In Phase 2, in all three conditions, initial preference was strongly toward the likely‐sooner food key, and moved to the other key across the interfood interval. In both phases, at a more extended level of analysis, sequences of same‐key food deliveries caused a small increase in preference for the just‐productive key, suggesting the presence of a “reinforcement effect”, albeit one that was very small.     

Disruption of short-term memory for reinforcement by ambient illumination

Pigeons were trained on a delayed conditional discrimination (DCD) in which choice of one of two simultaneously presented stimuli was reinforced if the trial had been initiated by presentation of a food sample. On trials in which no sample was presented, choice of the other colour was reinforced. Illumination of the houselights during the retention interval was provided in an attempt to interfere with retention of information about the food sample which served as a conditional cue. In two experiments, retention interval illumination produced a greater disruption of DCD performance on no sample trials than on food sample trials. The finding that retention interval illumination disrupts DCD performance on no sample trials suggests that this manipulation does not affect memorial processes since there was good evidence that performance on no sample trials did not depend on remembering what happened at the outset of the trial. Furthermore, the magnitude of the disruption was larger if the illumination immediately preceded the choice stimuli than if it followed presentation of the sample stimuli. These results support the hypothesis that retention interval illumination disrupts DCD performance by interfering with discriminative control of the choice response rather than with memorial processes. In neither study did retention interval illumination impair discriminative autoshaping to keylight stimuli that immediately preceded the food sample and no sample DCD trials.     

Directed forgetting of elements in compound samples.

Pigeons were trained in a delayed matching-to-sample procedure in which the sample stimuli consisted of a compound of color (red or green) and spatial location (left or right). A postsample cue (houselight on or off) signaled whether color matching or location matching would be required following the delay. In Experiment 1, the reduction in performance on probe trials (in which the houselight condition was reversed relative to that on regular trials) was greater for location matching than for color matching. The birds showed overt mediational behavior during the delays on location-matching trials. On color-matching trials, the birds exhibited behavior during delays that might have interfered with that mediational behavior. In Experiment 2, the houselight condition was changed shortly before presentation of the comparison stimuli on probe trials. Accuracy of location matching was reduced when the cue initially signaled color matching and was then changed to signal location matching, whereas matching accuracy was not reduced by a change in the opposite direction. Accuracy of color matching was reduced by a change in illumination level from dark to light, regardless of type of the relevant dimension signaled by houselight illumination. Discussion of these findings focuses on the variables critical to establishment of an effective cue to forget.     

A theory of attending, remembering, and reinforcement in delayed matching to sample

A theory of attending and reinforcement in conditional discriminations is extended to working memory in delayed matching to sample by adding terms for disruption of attending during the retention interval. Like its predecessor, the theory assumes that reinforcers and disruptors affect the independent probabilities of attending to sample and comparison stimuli in the same way as the rate of overt free-operant responding as suggested by Nevin and Grace, and that attending is translated into discriminative performance by the model of Davison and Nevin. The theory accounts for the effects of sample-stimulus discriminability and retention-interval disruption on the levels and slopes of forgetting functions, and for the diverse relations between accuracy and sensitivity to reinforcement reported in the literature. It also accounts for the effects of reinforcer probability in multiple schedules on the levels and resistance to change of forgetting functions; for the effects of reinforcer probabilities signaled within delayed-matching trials; and for the effects of reinforcer delay, sample duration, and intertrial-interval duration. The model accounts for some data that have been problematic for previous theories, and makes testably different predictions of the effects of reinforcer probabilities and disruptors on forgetting functions in multiple schedules and signaled trials.     

Direct remembering,mediated remembering,and atypical forgetting functions

Atypical forgetting functions have been demonstrated in several recent studies of delayed matching to sample, in which experimental conditions are altered partway through the retention interval. The forgetting functions are atypical in that accuracy or discriminability is not always a negatively accelerated monotonic function of increasing retention interval duration, but may increase at later times in the retention interval. Atypical forgetting functions reflect changes in levels of discrimination. A switch from a lower level to a higher level of discrimination, or vice versa, can occur at any time in the retention interval. The behavioral theories of remembering proposed by Nevin, Davison, Odum, and Shahan (2007), and White and Brown (2014), offer quantitative predictions of forgetting functions that differ in intercept or slope. Both theories are able to account for atypical forgetting functions, by assuming time‐independent changes in the mediating effect of attending to sample and comparison stimuli (in Nevin et al.'s model) or in the direct effect of the context of reinforcement of the conditional discrimination (in White & Brown's model). Despite differences in their main assumptions, the theories have an edge over any theory that assumes that forgetting is time‐dependent.     

Stimulus control of information processing in pigeon short-term memory

Six pigeons were trained initially on a delayed successive matching-to-sample task using red and green fields as sample and test stimuli. Following acquisition, each sample was followed either by a vertical line (“remember” cue), which indicated that sample memory would be tested, or by a horizontal line (“forget” cue), which indicated that sample memory would not be tested. During the experiments, sample memory on forget trials was tested occasionally. A series of five experiments revealed: (a) better retention on remember trials than on forget trials, (b) increased effectiveness of a forget cue when it followed closely sample offset, (c) more rapid forgetting over a retention interval ranging from 3 to 6 sec on forget trials than on remember trials, (d) a “cancellation” effect in which a remember cue which followed immediately the offset of a forget cue attenuated markedly the effectiveness of the forget cue, and (e) an “insulation” effect in which the effectiveness of a forget cue was reduced considerably when presented after a remember cue. It was concluded that pigeons actively process or rehearse the sample memory during the retention interval.     

Effects of reinforcer consumption and magnitude on response rates during noncontingent reinforcement

Results of previous research on the effects of noncontingent reinforcement (NCR) have been inconsistent when magnitude of reinforcement was manipulated. We attempted to clarify the influence of NCR magnitude by including additional controls. In Study 1, we examined the effects of reinforcer consumption time by comparing the same magnitude of NCR when session time was and was not corrected to account for reinforcer consumption. Lower response rates were observed when session time was not corrected, indicating that reinforcer consumption can suppress response rates. In Study 2, we first selected varying reinforcer magnitudes (small, medium, and large) on the basis of corrected response rates observed during a contingent reinforcement condition and then compared the effects of these magnitudes during NCR. One participant exhibited lower response rates when large-magnitude reinforcers were delivered; the other ceased responding altogether even when small-magnitude reinforcers were delivered. We also compared the effects of the same NCR magnitude (medium) during 10-min and 30-min sessions. Lower response rates were observed during 30-min sessions, indicating that the number of reinforcers consumed across a session can have the same effect as the number consumed per reinforcer delivery. These findings indicate that, even when response rate is corrected to account for reinforcer consumption, larger magnitudes of NCR (defined on either a per-delivery or per-session basis) result in lower response rates than do smaller magnitudes.     

Reinforcement magnitude and pausing on progressive-ratio schedules

Rats responded under progressive-ratio schedules for sweetened milk reinforcers; each session ended when responding ceased for 10 min. Experiment 1 varied the concentration of milk and the duration of postreinforcement timeouts. Postreinforcement pausing increased as a positively accelerated function of the size of the ratio, and the rate of increase was reduced as a function of concentration and by timeouts of 10 s or longer. Experiment 2 varied reinforcement magnitude within sessions (number of dipper operations per reinforcer) in conjunction with stimuli correlated with the upcoming magnitude. In the absence of discriminative stimuli, pausing was longer following a large reinforcer than following a small one. Pauses were reduced by a stimulus signaling a large upcoming reinforcer, particularly at the highest ratios, and the animals tended to quit responding when the past reinforcer was large and the stimulus signaled that the next one would be small. Results of both experiments revealed parallels between responding under progressive-ratio schedules and other schedules containing ratio contingencies. Relationships between pausing and magnitude suggest that ratio pausing is under the joint control of inhibitory properties of the past reinforcer and excitatory properties of stimuli correlated with the upcoming reinforcer, rather than under the exclusive control of either factor alone.     

Effects of shifts in food reinforcement context on rats’ consumption of concurrently available water or sucrose solution

The purpose of this study was to investigate the effects of signaled transitions from relatively rich to lean conditions of food reinforcement on drinking concurrently available water or sucrose‐sweetened water in rats. Past research demonstrated that these negative incentive shifts produce behavioral disruption in the form of extended pausing on fixed‐ratio schedules. Four male Long‐Evans rats operated on a two‐component multiple fixed‐ratio fixed‐ratio schedule. In one manipulation, the ratio was held constant and the components arranged either a large six‐pellet reinforcer (rich) or small one‐pellet reinforcer (lean). In a second manipulation, the components both produced a one‐pellet reinforcer but differed in terms of the ratio requirement, with the rich and lean conditions corresponding to relatively small and large ratios. In both manipulations, components were pseudorandomly presented to arrange four transitions signaled by retractable levers: lean‐to‐lean, lean‐to‐rich, rich‐to‐rich, and rich‐to‐lean (the negative incentive shift). During experimental conditions, a bottle with lickometer was inserted in the chamber, providing concurrent access either to tap water or a 10% sucrose solution. The negative incentive shift produced considerably more drinking than the other transitions in all rats during both manipulations. The level of drinking was not polydipsic; rather, it appears that the negative incentive shift enhanced the value of concurrently available reinforcers relative to food reinforcement.     

延迟学习判断的时间加工进程

学习判断是指人们在学习之后对自己学习效果所做的一种预测,是元记忆监测性判断的一种形式。学习判断按其发生时间的不同可以分为即时学习判断和延迟学习判断。与即时学习判断相比,延迟学习判断的相对准确性较高,被称为延迟学习判断效应。实验借助事件相关电位技术探究延迟学习判断的时间加工进程,发现高等级学习判断比低等级学习判断诱发了更大的时间窗为400-600ms的早期新旧效应和800-1200ms的晚期右侧额区新旧效应。上述结果表明,延迟学习判断包含不止一个加工过程,早期新旧效应反映了线索再认的过程,晚期右侧额区效应则反映了对提取之后的评估过程,该结果进一步验证了两阶段模型解释延迟学习判断效应的有效性。     

Fixed-ratio pausing: Joint effects of past reinforcer magnitude and stimuli correlated with upcoming magnitude

Pigeons responded on fixed-ratio schedules ending in small or large reinforcers (grain presentations of different duration) interspersed within each session. In mixed-schedule conditions, the response key was lit with a single color throughout the session, and pausing was directly related to the past reinforcer (longer pauses after large reinforcers than after small ones). In multiple-schedule conditions, different colors accompanied the ratios ending in small and large reinforcers, and pausing was affected by the upcoming reinforcer as well as the past one. Pauses were shorter before large reinforcers than before small ones, but they continued to be longer after large reinforcers than after small ones. The influence of the past reinforcer was modulated by the magnitude of the upcoming reinforcer; in the presence of the stimulus before the small reinforcer, the effect of the past reinforcer was enhanced relative to its effect in the stimulus before the large reinforcer. These results show that pausing between ratios is jointly determined by two competing factors: past conditions of reinforcement and stimuli correlated with upcoming conditions.     

Response–reinforcer dependency and resistance to change

The effects of the response–reinforcer dependency on resistance to change were studied in three experiments with rats. In Experiment 1, lever pressing produced reinforcers at similar rates after variable interreinforcer intervals in each component of a two‐component multiple schedule. Across conditions, in the fixed component, all reinforcers were response‐dependent; in the alternative component, the percentage of response‐dependent reinforcers was 100, 50 (i.e., 50% response‐dependent and 50% response‐independent) or 10% (i.e., 10% response‐dependent and 90% response‐independent). Resistance to extinction was greater in the alternative than in the fixed component when the dependency in the former was 10%, but was similar between components when this dependency was 100 or 50%. In Experiment 2, a three‐component multiple schedule was used. The dependency was 100% in one component and 10% in the other two. The 10% components differed on how reinforcers were programmed. In one component, as in Experiment 1, a reinforcer had to be collected before the scheduling of other response‐dependent or independent reinforcers. In the other component, response‐dependent and ‐independent reinforcers were programmed by superimposing a variable‐time schedule on an independent variable‐interval schedule. Regardless of the procedure used to program the dependency, resistance to extinction was greater in the 10% components than in the 100% component. These results were replicated in Experiment 3 in which, instead of extinction, VT schedules replaced the baseline schedules in each multiple‐schedule component during the test. We argue that the relative change in dependency from Baseline to Test, which is greater when baseline dependencies are high rather than low, could account for the differential resistance to change in the present experiments. The inconsistencies in results across the present and previous experiments suggest that the effects of dependency on resistance to change are not well understood. Additional systematic analyses are important to further understand the effects of the response–reinforcer relation on resistance to change and to the development of a more comprehensive theory of behavioral persistence.