Immediate reinforcement in delayed reward learning in pigeons

Pigeons were trained on simultaneous red-green discrimination procedures with delayed reward and sequences of stimuli during the delay. In Experiment 1, three stimuli appeared during the 60-second intervals between the correct responses and reward, and the incorrect responses and nonreward. The stimulus that immediately followed a correct response also preceded nonreward, and the stimulus that followed an incorrect response preceded reward. These stimuli were 10 or .33 second in duration for different groups. Stimuli during the remainder of the delay interval differed following correct and incorrect responses. Group 10 initially persisted in the nonrewarded choice, but shifted to a preponderance of rewarded responses after further training. Group .33 rapidly acquired the correct response. Similar results were obtained in Experiment 2 where delay intervals consisted of opposite sequences of two stimuli of equal duration and total delays were 6, 20, or 60 seconds. Early in training, generalization of differential conditioned-reinforcing properties from the conditions preceding reward and nonreward to postchoice conditions had a greater effect relative to backchaining than it did later. It was concluded that delayed-reward learning is best analyzed in terms of the conditioned-reinforcing value of the patterns of cues that follow immediately after rewarded and nonrewarded responses.     

Stimulus control and response bias in an analogue prey-detection procedure

The present study compared the performance of 6 pigeons trained to detect luminance differences in two different signal-detection procedures. Exposed to a three-key array, the pigeons were trained to peck the left key when the brighter of two light intensities had been presented on the center key and to peck the right key when the dimmer of two light intensities had been presented on the center key. Procedure A was a standard signal-detection procedure in which left/bright and right/dim responses produced food reinforcement and left/dim and right/bright responses produced periods of timeout. Procedure B was designed to simulate some of the contingencies operating in a prey-detection situation. Left-key responses produced reinforcement following the brighter center-key stimulus and a period of timeout following the dimmer center-key stimulus. Right-key responses always produced a short period of timeout irrespective of the stimulus. Within each procedure, the duration of timeout arranged for false alarms (left/dim responses) was varied between 3 s and 120 s. Measures of accuracy and response bias were compared between the two procedures. The timeout manipulation produced systematic, but relatively small, changes in these measures when right/dim responses (i.e., correct rejections) produced reinforcement (Procedure A). Arranging timeout for right/dim responses in Procedure B produced greater variability in accuracy and response bias than did arranging reinforcement, but this variability was not related to timeout duration. Overall, discrimination accuracy was considerably higher when right/dim responses produced timeout than when they resulted in reinforcement, and accuracy was accompanied by a large bias toward the response associated with reinforcement. These results are consistent with a recently proposed model of signal detection.     

Conjoint control of performance in conditional discriminations by successive and simultaneous stimuli.

In a conditional discrimination, reinforcement of pigeons' responses to pairs of simultaneously presented wavelength stimuli depended on the orientation of white lines superimposed on the wavelengths. Over different conditions in Experiment 1, three wavelength differences were combined with two differences between successively presented line orientations. Measures of stimulus discriminability increased with increases in the difference between both orientation and wavelength stimuli. Conditional-discrimination performance was thus conjointly determined by stimulus disparity in the successive and simultaneous discriminations. In Experiment 2, ratios of rates of reinforcement contingent upon the two categories of correct responses were varied over several conditions for difficult and easy discriminations. Ratios of responses to wavelength pairs were sensitive to variations in the reinforcement ratio to a greater extent for the more difficult orientation discrimination than for the easier orientation discrimination. Performance in the conditional discrimination was therefore determined by the interacting effects of stimulus disparity and the relative rates of reinforcement contingent upon the two correct choices. It was concluded that the effect of temporally distant reinforcement on behavior in a prevailing schedule component is attenuated to an extent that depends on similarity of stimuli that delineate the successive components.     

Working Memory for Temporal and Nontemporal Events in Monkeys

This is the first report that introduces appropriate behavioral tasks for monkeys for investigations of working memory for temporal and nontemporal events. Using several behavioral tests, the study also shows how temporal information is coded during retention intervals in the tasks. Each of three monkeys was trained with two working memory tasks: delayed matching-to-sample of stimulus duration (DMS-D) and delayed matching-to-sample of stimulus color (DMS-C). The two tasks employed an identical apparatus and responses and differed only in the temporal and nontemporal attribute of the stimuli to be retained for correct performance. When a retention interval between the sample and comparison stimuli was prolonged, the monkeys made more incorrect responses to short samples in the DMS-C task, suggesting “trace decay” of memory for short stimuli. However, the same monkeys showed no such increase in incorrect responses to short samples in the DMS-D task, suggesting active coding of temporal information, that is, the length of stimulus duration, during the retention interval. When variable lengths of samples were presented with a fixed retention interval, the monkeys made more incorrect responses when length differences between short and long samples were small in the DMS-D task, but not in the DMS-C task. This suggests that the codes of working memory retained in the DMS-D task were not absolute (analogical) but rather were relative (categorical) and related to differences in the duration of the samples.     

Duration discrimination: effects of probability of stimulus presentation

Monkeys initiated a stimulus by pressing on the center of three levers and the stimulus terminated independently of behavior 60, 80, 90, or 100 sec later. Presses on the right lever were reinforced with food following the three briefer durations, and presses on the left lever, following the 100-sec duration. Incorrect responses produced a 10-sec timeout. Probability of presenting the 100-sec duration was manipulated in the range from 0.25 to 0.75, with the probabilities of the briefer durations remaining equal and summing to one minus the probability of the 100-sec duration. Percentage of responses on either side lever was functionally related to both the probability of presenting the 100-sec stimulus and to stimulus duration. An analysis of the data based on the theory of signal detection resulted in operating characteristics that were linear when plotted on normal-normal coordinates. The percentage of responses on either lever approximated the optimal values for maximizing reinforcement probability in each condition of the experiment.     

Tests of a two-stage model of visual matching

Predictions from a model of visual matching were tested in two experiments. The model consists of a wholistic comparison process followed by an element-by-element comparison process. All stimuli are processed by the first stage but only those that permit a decision based on a wholistic comparison produce responses. When discrimination is difficult and a decision cannot be reached by a wholistic comparison, the second stage of processing is initiated. Degree of discriminability and stimulus duration (100 and 1000 msec.) were varied in both experiments. In Exp. 1, the stimulus elements were arranged in a square configuration to facilitate a wholistic comparison. As predicted, the hard-different stimuli took longer to match than the same or easy-different stimuli. The hard-different stimuli presented for 1000 msec. took longer to match than those presented for 100 msec. There was no difference in accuracy between responses to hard-different pairs at the two durations. In Exp. 2, the stimulus elements were arranged in a horizontal row and placed one above the other to facilitate element-by-element comparison. As predicted, these stimuli produced slower and more accurate responses for same and hard-different stimulus pairs only when they were exposed for 1000 msec. Responses to easy-different stimulus pairs were made quickly and accurately.     

Relative durations of conditioned stimulus and intertrial interval in conditioned suppression.

The effects of the relative durations of the conditional stimulus and the intertrial interval on bar pressing during a conditioned-suppression procedure were examined as a function of two additional variables--type of operant baseline schedule and rate of shock presentation. In Experiment 1, response suppression was compared across components of a multiple fixed-ratio, random-ratio, fixed-interval, random-interval schedule, at relative conditioned-stimulus/intertrial-interval durations of 1/1, 1/4, and 1/9. In Experiment 2, relative conditioned-stimulus/intertrial-interval duration (1/5, 3/3, or 5/1) was manipulated across groups, while shock frequency (2, 6, or 10 shocks/hr) was manipulated within groups. In both experiments, suppression during the signal was virtually complete at all relative durations. Responding was also suppressed during the intertrial interval, but that suppression varied as a function of experimental manipulations. In Experiment 1, intertrial-interval response rates were higher when relative signal duration was 1/9 than when it was 1/1, although both relative signal duration and shock frequency, which covaried, could have contributed to the difference. In Experiment 2, the patterning of response rates between successive shocks was affected by relative duration, absolute rates during the intertrial interval varied as a function of shock frequency, and differences between suppression during the signal and suppression during the intertrial interval were affected by both relative duration and shock frequency. The data support an analysis based upon relationships between shock-correlated and intertrial-interval stimuli and, as assessed by the relative-delay-to-reinforcement metric, are comparable to results that have been reported from experiments using similar manipulations under the autoshaping paradigm.     

A real-time criterion theory of duration discrimination

The common idea that a measure is taken of a duration stimulus over its temporal extent, and that the decision as to whether the stimulus is relatively long or short is based upon such a measure, is shown to be incorrect. Two experiments, which require speeded responding in duration discrimination and consider response latencies as well as response probabilities, demonstrate that the response that is made is determined by the outcome of a race between an internally timed interval, the criterion, and the presented stimulus. The onset of the stimulus triggers the criterion; if the criterion ends first, the response “long” is elicited. Duration discrimination is a matter of temporal order discrimination, and in the limit, “short” responses are simple reactions while “long” responses are time estimation responses. A specific model of the real-time criterion hypothesis is tested, and these initial tests generally confirm it. From this, it is concluded that errors in duration discrimination are due entirely to variability of the criterion and that afferent latencies are not necessarily variable. This adds additional evidence for the existence of deterministic afferent latencies.     

Control of responding by stimulus duration

Pigeons were trained on a procedure in which the key was white for 30 sec, alternating with periods of darkness, or timeout. In a nondifferential training procedure, timeout duration was held constant at either 9 or 21 sec for different animals, and pecks on the white key were reinforced on a variable-interval 36-sec schedule. After 30 sessions an extinction generalization test was conducted where the duration of the timeout was varied from 3 to 27 sec. This test showed no differences in responding following timeouts of different durations. In a differential training procedure, timeout durations of either 9 or 21 sec were randomly scheduled for each animal. The variable-internal schedule was in effect following the same timeout duration as in the prior nondifferential procedure. No pecks were reinforced after the other timeout duration. In 40 sessions, differences in response rates following the two durations gradually developed. A maintained generalization procedure was then imposed in which timeout durations were varied from 3 to 27 sec, with the variable-interval schedule in effect following only the same duration as in the previous procedures. The first maintained generalization session showed that the prior differential training had established control of the animals' behavior by the timeout duration. In continued training on the maintained generalization procedure, control by the timeout duration decreased.     

Context specificity of conditioned-reinforcement effects on discrimination acquisition

Pigeons were trained on a series of reversals of a simultaneous form discrimination in which the trial outcomes were separated from the choice responses by an 8-s delay interval. Different conditions were defined by the stimuli occurring during the two halves of the delay interval. Discrimination learning was greatly facilitated by having differential stimuli during the delay following correct versus incorrect choices. When the differential stimuli appeared only at the midpoint of the delay, some facilitation occurred relative to when no different stimuli occurred, but there was substantially less facilitation than when the differential stimuli occurred immediately contingent on choice. A reversed-stimulus condition, in which the stimulus at the onset of the delay following a correct choice was the same as that during the last segment of the delay following an incorrect choice, and the stimulus at the onset of the delay following an incorrect choice was the same as that preceding food during the last segment of the delay following a correct choice, also facilitated discrimination learning relative to the nondifferential stimulus conditions.     

Independence of stimulus discriminability from absolute rate of reinforcement in a signal-detection procedure.

Three experiments are reported in which two pigeons were trained to detect differences in stimulus duration under varying levels of absolute rate of reinforcement. Two red stimuli, differing in duration, were arranged probabilistically on the center key of a three-key chamber. On completion of the center-key duration, the center keylight was extinguished and the two side keys were illuminated white. Correct responses were left-key pecks following the shorter duration and right-key pecks following the longer duration. In Experiment 1, relative rate of reinforcement for correct responses was held constant and absolute rate of reinforcement was varied in seven conditions from continuous reinforcement to a variable-interval 90-second schedule. In Experiment 2, relative rate of reinforcement was manipulated across three different absolute rates of reinforcement (continuous reinforcement, variable-interval 15-second, and variable-interval 45-second). Stimulus discriminability was unaffected by changes in absolute or relative rates of reinforcement. Experiment 3 showed that discriminability was also unaffected by arranging the same consequences (three-second blackout) for unreinforced correct responses and errors.     

On the rate of acquisition of visual information about space,time, and intensity

The temporal course of acquisition of visual information was mapped by measuring the speed and accuracy With which Ss could discriminate the position, intensity, or duration of a stimulus that was varied independently on these three binary dimensions. The speed-accuracy tradeoff was different for each of the three discrimination tasks, with performance substantially superior in the position discrimination task. In the intensity and duration discrimination tasks, there were systematic changes with RT in the relative influences of the three stimulus variables over the two alternative responses, suggesting .three phases of the corresponding perceptual processes: an initial dominance by position of the stimulus, a subsequent, joint influence by both intensity and duration, and finally an exclusive control by the correct dimension. The results demonstrate that Ss can selectively attend to various dimensions of the same stimulus, with concommitantly varying time constants. The perceptual processes being tapped by a discrimination task depend upon instructions and upon RT.     

Six-month-old infants use analog magnitudes to represent duration

While many studies have investigated duration discrimination in human adults and in nonhuman animals, few have investigated this ability in infants. Here, we report findings that 6-month-old infants are able to discriminate brief durations, and, as with other animal species, their discrimination function is characterized by Weber's Law: proportionate difference rather than absolute difference between stimuli determined successful discrimination. Importantly, paralleling results found with nonhuman animals, the Weber function that we found for infants' discrimination of time is the same as that found for their discrimination of number. Infants discriminated durations of an audiovisual event differing by a 1:2 ratio, but not those differing by a 2:3 ratio, over a range of durations. This suggests that (a) in human as in nonhuman animals, the same mental mechanism may underlie the ability to measure duration as to represent number, and (b) we may share this mental mechanism with other animal species.     

Stimulus control in the experimental study of cooperation

The cooperative responses of pairs of human subjects were reinforced under several stimulus conditions in two settings designed to require a "social" response, i.e., where at least one of the two persons is responding to the behavior of the other. The first task, designed by Lindsley and Cohen, required individual responses within 0.5 sec of one another for reinforcement. The second (modified) task required a delay of 3 sec between individual responses. To determine dependence of cooperation on social stimuli, rates of cooperative behavior on these tasks were compared in the presence and absence of a stimulus indicating to each subject the other's response and a stimulus which indicated the duration of the timeout after reinforcement. The results indicated that only in the modified task was a high rate of cooperation always contingent upon the presence of the social stimuli.     

Switching or sharing in dual-task line-length discrimination?

In two experiments, we tested whether subjects switched or shared attention between two simultaneously relevant line-length discrimination tasks. Switching models that allowed within- as well as between-trial switching were considered. In the first experiment, stimulus duration was varied randomly from trial to trial. With varied durations, many switching models predict negative contingencies: for a given duration and attentional allocation, accurate responses on one task should be associated with inaccurate responses on the other task. The results, however, showed no negative contingencies, which is consistent with sharing models. In the second experiment, stimulus duration was reduced to 20 msec, yet responses were more than 75% correct overall. This implies that information was obtained about both of the tasks within single trials, contradicting those switching models which predict that information can be obtained from, at most, one task within a period of 20 msec or less. In short, the results of both experiments support sharing models.     

Effects on choice of reinforcement delay and conditioned reinforcement

Pigeons chose between fixed-interval schedules of different durations presented in the terminal links of concurrent-chains schedules. The pair of schedules was always in the ratio of 2:1, but the absolute duration of the fixed intervals varied. In one set of conditions, the different terminal-link schedules were associated with different keylight stimuli (cued conditions). In a second set of conditions, the different terminal-link schedules were associated with the same stimulus (uncued conditions). Results from the cued conditions replicated previous findings that preference for the shorter fixed-interval schedule increased with fixed-interval duration. Preferences in the uncued conditions were lower than in the corresponding cued conditions but also increased with fixed-interval length. In addition, the degree of control under the uncued conditions was correlated with the extent to which the schedule during the terminal link was discriminated immediately upon entry into the terminal link. The pattern of results in both conditions was inconsistent with the notion that choice behavior matches relative immediacy of reinforcement. Reanalysis of previous evidence for matching (Chung and Herrnstein, 1967) showed that matching in fact did not occur, as the preferences of their subjects for the shorter of two delays also increased with the absolute size of the delays.     

The filled-duration illusion: The function of temporal and nontemporal set

Studies of the perception of time have demonstrated that large visual stimuli presented for the same duration as small stimuli will appear to have been presented longer. Two theoretical approaches have been offered to explain this effect. Both have emphasized the absolute amount of information in the stimulus. Thus, both would predict the effect when size is varied between sessions. The present study found no effect of size when size was held constant within sessions, but a substantial effect resulted when size was varied within sessions. Also examined was the range of stimulus durations used within a session. In one condition, patterns of various sizes were presented at 15, 30, or 45 msec, while in a second condition, the same patterns were presented to a different group of subjects at 15, 85, or 155 msec. The size effect was significantly reduced when longer time intervals were presented. A third group of subjects was shown patterns at even longer times but with a reduced range (155, 170, and 185 msec). Here the size effect was of equal magnitude to that of the 15-msec interval condition. The implications of these results are that current theoretical explanations of the filled-duration illusion rely on incorrect assumptions.     

Timeout postponement without increased reinforcement frequency

Three experiments were conducted to examine pigeons' postponement of signaled extinction periods (timeouts) from a schedule of food reinforcement when such responding neither decreased overall timeout frequency nor increased the overall frequency of food reinforcement. A discrete-trial procedure was used in which a response during the first 5 s of a trial postponed an otherwise immediate 60-s timeout to a later part of that same trial but had no effect on whether the timeout occurred. During time-in periods, responses on a second key produced food according to a random-interval 20-s schedule. In Experiment 1, the response-timeout interval was 45 s under postponement conditions and 0 s under extinction conditions (responses were ineffective in postponing timeouts). The percentage of trials with a response was consistently high when the timeout-postponement contingency was in effect and decreased to low levels when it was discontinued under extinction conditions. In Experiment 2, the response-timeout interval was also 45 s but postponement responses increased the duration of the timeout, which varied from 60 s to 105 s across conditions. Postponement responding was maintained, generally at high levels, at all timeout durations, despite sometimes large decreases in the overall frequency of food reinforcement. In Experiment 3, timeout duration was held constant at 60 s while the response-timeout interval was varied systematically across conditions from 0 s to 45 s. Postponement responding was maintained under all conditions in which the response-timeout interval exceeded 0 s (the timeout interval in the absence of a response). In some conditions of Experiment 3, which were designed to control for the immediacy of food reinforcement and food-correlated (time-in) stimuli, responding postponed timeout but the timeout was delayed whether a response occurred or not. Responding was maintained for 2 of 3 subjects, suggesting that behavior was negatively reinforced by timeout postponement rather than positively reinforced by the more immediate presentation of food or food-correlated (time-in) stimuli.     

Orientation,length and exposure duration in the identification of line segments

Summary Single line segments titled with 0, 45, 90, and 135 degrees orientation were briefly exposed as stimuli and the subjects had to identify the orientation of a presented line segment. The length of the lines and the exposure duration were varied. The multicomponent theory (Rumelhart, 1970, 1971) was applied to the observed frequencies of a correct response and of some incorrect responses given to a stimulus with a particular orientation, length, and exposure duration. Within the framework of this theory several hypotheses concerning the effects of orientation, length and exposure duration in identification were tested. The main results are: (1) The rate of component detection was higher for the horizontal and vertical orientations than for the two oblique orientations. (2) The number of components in a line segment quite accurately followed a linear function of the line's physical length, and the negatively valued additive constant in this function could not be neglected. (3) The information in the visual information store decayed more rapidly after longer exposure durations and more slowly after shorter exposure durations. These findings are discussed in relation to other research on the subject.