Reinforcer control and human signal-detection performance

Eight humans participated in a two-choice signal-detection task in which stimulus disparity was varied over four levels. Two procedures arranged asymmetrical numbers of reinforcers received for correct left- and right-key responses (the reinforcer ratio). The controlled procedure ensured that the obtained reinforcer ratio remained constant over changes in stimulus disparity, irrespective of subjects' performances. In the uncontrolled procedure, the asymmetrical reinforcer ratio could covary with subjects' performances. The receiver operating characteristic (ROC) patterns obtained from the controlled procedure approximated isobias functions predicted by criterion location measures of bias. The uncontrolled procedure produced variable ROC patterns that were somewhat like the isobias predictions made by likelihood ratio measures of bias; however, the obtained reinforcer ratio became more extreme as discriminability decreased. The obtained pattern of bias was directly related to the obtained reinforcer ratio. This research indicates that criterion location measures seem to be preferable indices of response bias.     

Independence of stimulus discriminability from absolute rate of reinforcement in a signal-detection procedure.

Three experiments are reported in which two pigeons were trained to detect differences in stimulus duration under varying levels of absolute rate of reinforcement. Two red stimuli, differing in duration, were arranged probabilistically on the center key of a three-key chamber. On completion of the center-key duration, the center keylight was extinguished and the two side keys were illuminated white. Correct responses were left-key pecks following the shorter duration and right-key pecks following the longer duration. In Experiment 1, relative rate of reinforcement for correct responses was held constant and absolute rate of reinforcement was varied in seven conditions from continuous reinforcement to a variable-interval 90-second schedule. In Experiment 2, relative rate of reinforcement was manipulated across three different absolute rates of reinforcement (continuous reinforcement, variable-interval 15-second, and variable-interval 45-second). Stimulus discriminability was unaffected by changes in absolute or relative rates of reinforcement. Experiment 3 showed that discriminability was also unaffected by arranging the same consequences (three-second blackout) for unreinforced correct responses and errors.     

Discriminability and sensitivity to reinforcer magnitude in a detection task

Three pigeons discriminated between two sample stimuli (intensities of red light). The difficulty of the discrimination was varied over four levels. At each level, the relative reinforcer magnitude for the two correct responses was varied across conditions, and the reinforcer rates were equal. Within levels, discriminability between the sample stimuli did not change systematically as reinforcer magnitude varied. Across levels, the sensitivity of behavior to changes in the reinforcer-magnitude ratio decreased as the discriminability between the sample stimuli increased. Subsequent analysis showed that this relation was limited to performance following only one of the sample stimuli, the dim red light that remained constant across all conditions. Extant behavioral models of signal detection cannot easily accommodate these results.     

Stimulus disparity and punisher control of human signal-detection performance

The present experiment examined the effects of varying stimulus disparity and relative punisher frequencies on signal detection by humans. Participants were placed into one of two groups. Group 3 participants were presented with 1:3 and 3:1 punisher frequency ratios, while Group 11 participants were presented with 1:11 and 11:1 punisher frequency ratios. For both groups, stimulus disparity was varied across three levels (low, medium, high) for each punisher ratio. In all conditions, correct responses were intermittently reinforced (1:1 reinforcer frequency ratio). Participants were mostly biased away from the more punished alternative, with more extreme response biases found for Group 11 participants compared to Group 3. For both groups, estimates of discriminability increased systematically across the three disparity levels and were unaffected by the punisher ratios. Likewise, estimates of response bias and sensitivity to the punisher ratios were unaffected by changes in discriminability, supporting the assumption of parameter invariance in the Davison and Tustin (1978) model of signal detection. Overall, the present experiment found no relation between stimulus control and punisher control, and provided further evidence for similar but opposite effects of punishers to reinforcers in signal-detection procedures.     

Effects of varying stimulus disparity and the reinforcer ratio in concurrent-schedule and signal-detection procedures.

The present study measured the effects of stimulus and reinforcer variations on pigeons' behavior in two different choice procedures. Two intensities of white light were presented as the stimuli on the main key in a switching-key concurrent schedule and as the sample stimuli in a signal-detection procedure. Under both procedures, the scheduled rate of reinforcement was varied across conditions to produce various ratios of obtained reinforcement. These ratios were obtained for seven pairs of light intensities. In the concurrent schedules, the effects of reinforcer-ratio variations were positively correlated with the physical disparity between the two light intensities. In the signal-detection procedure, changes in the reinforcer ratio produced greater effects on performance when stimulus disparity was very low or very high compared to those found at intermediate levels of stimulus disparity. This discrepancy creates a dilemma for existing behavioral models of signal-detection performance.     

Effects of relative reinforcer frequency on complex color detection

Pigeons were trained under a discrete-trials detection procedure in which one of a set of color stimuli was presented on the center key and a single response turned off the stimulus and illuminated two side keys. Single responses to one or the other side key produced occasional reinforcers depending on the value of the color stimulus. In Experiment 1, one color-stimulus set comprised 559, 564, 569, and 574 nm, and right-key pecks were occasionally reinforced following presentations of members of this set. The other stimulus set comprised 579, 584, 589, and 594 nm, and left-key pecks were occasionally reinforced following presentations of members of this set. Across seven experimental conditions, the left/(left + right) relative reinforcer frequency was varied from .1 to .9. In Experiment 2, one stimulus set contained only one member, 574 nm, and right-key responses were occasionally reinforced following its presentation. Over 12 experimental conditions, two manipulations were carried out. First, the number of stimuli comprising the other stimulus set was increased from one (579 nm) to two (579 and 584 nm) to three (579, 584, and 589 nm) and to four (579, 584, 589, and 594 nm), and left-key responses were reinforced occasionally following center-key presentations of members of this set. Second, for each stimulus combination, the left/(left + right) relative reinforcer frequency was varied from .1 to .5 to .9 across three experimental conditions. The principal finding of Experiments 1 and 2 was that reinforcers and stimuli interacted in their effects on behavior. In Experiment 3, pairs of adjacent stimuli (5 nm apart) in the range 559 to 594 nm were presented in each experimental condition, and the left/(left + right) relative reinforcer frequency was held constant at .5. The data from all three experiments were analyzed according to a detection model describing performance in multiple-stimulus two-response procedures. This model provided independent measures of stimulus discriminability, contingency discriminability, and bias. The analysis showed that (a) consistent with the color-naming function, pigeons were better able to discriminate between higher nanometer values than lower nanometer values; (b) their ability to discriminate between the stimuli was independent of the number of wavelengths comprising each stimulus set; (c) they allocated delivered reinforcers very accurately to the previously emitted response; and (d) no consistent biases emerged.     

Stimulus Presentation Ratios And The Outcomes For Correct Responses In Signal-detection Procedures

Three pigeons were trained to discriminate between two line orientations (S₁ and S₂). A left-key peck was correct when S₁ was presented, and a right key-peck was correct when S₂ was presented. In all procedures, correct responses were occasionally reinforced with food paired with the presentation of the magazine light. Incorrect responses produced a blackout. Six detection procedures were used. In the first, the signal presentation ratio was varied across conditions and the reinforcer ratio was allowed to covary. In the second, the signal presentation ratio was held constant at 1:1 and the reinforcer ratio varied across conditions. In the third, the signal presentation ratio was varied across conditions and the reinforcer ratio was held constant at 1:1. In these three procedures, correct responses that were not scheduled for reinforcement were followed by blackout. The remaining three procedures repeated those described above with one procedural change: Nonreinforced but correct trials were followed by the presentation of the magazine light. Birds showed systematic preferences for the key associated with the stimulus presented or reinforced most often. There was no change in the birds' performance over changes in the feedback for nonreinforced but correct responses.     

On the discriminability of stimulus duration.

The performance of pigeons trained to detect differences in the duration of stimuli was analysed using a matching model of signal detection. Two white stimuli, S1 and S2, differing in duration, were arranged with equal probability on the center key of a three-key chamber. S1 was systematically varied from 5 seconds to 25 seconds while S2 remained constant at 30 seconds. On completion of the center-key stimulus, a peck on the center key turned on the two red side keys. A left-key response was "correct" when S1 had been in effect on the center key and a right-key response was "correct" on S2 trials. A correct response produced a 3-second magazine light accompanied intermittently by food. Incorrect responses produced 3-second blackouts. Detection performance was measured under two procedures. In the first, the obtained reinforcement ratio was uncontrolled by allowing the number of food reinforcements obtained for correct left- and right-key responses to vary as the stimuli were changed. In the second procedure, the presentation of food reinforcement was controlled by holding the obtained reinforcement ratio constant. Discriminability changed as a function of stimulus differences under both procedures. No such trend was found in response bias.     

Human Signal-detection Performance: Effects Of Signal Presentation Probabilities And Reinforcer Distributions

University students participated in one of four standard two-choice signal-detection experiments in which signal presentation probability was varied and the reinforcement distribution was held constant and equal. In Experiments 1, 3 and 4, subjects' performance showed a systematic response bias for reporting the stimulus presented least often. Experiments 1 and 4 showed that this effect was reliable with extended training and monetary, rather than point, reinforcement. In Experiment 2, all correct responses were signaled in some way, and this produced the opposite relationship between signal presentation probability and response bias. Experiments 1 and 3 found that explicitly deducting money (intended as punishment) for equal numbers of incorrect responses on each alternative, or varying the obtained overall rate of reinforcement, produced no clear change in response bias. The bias, shown by humans, for reporting the stimulus presented least often remains a challenge for theories of stimulus detection.     

Reinforcer control by comparison-stimulus color and location in a delayed matching-to-sample task

Six pigeons were trained in a delayed matching-to-sample task involving bright- and dim-yellow samples on a central key, a five-peck response requirement to either sample, a constant 1.5-s delay, and the presentation of comparison stimuli composed of red on the left key and green on the right key or vice versa. Green-key responses were occasionally reinforced following the dimmer-yellow sample, and red-key responses were occasionally reinforced following the brighter-yellow sample. Reinforcer delivery was controlled such that the distribution of reinforcers across both comparison-stimulus color and comparison-stimulus location could be varied systematically and independently across conditions. Matching accuracy was high throughout. The ratio of left to right side-key responses increased as the ratio of left to right reinforcers increased, the ratio of red to green responses increased as the ratio of red to green reinforcers increased, and there was no interaction between these variables. However, side-key biases were more sensitive to the distribution of reinforcers across key location than were comparison-color biases to the distribution of reinforcers across key color. An extension of Davison and Tustin's (1978) model of DMTS performance fit the data well, but the results were also consistent with an alternative theory of conditional discrimination performance (Jones, 2003) that calls for a conceptually distinct quantitative model.     

Sample-stimulus discriminability and sensitivity to reinforcement in delayed matching to sample

Five pigeons were trained in a delayed matching-to-sample task with red and green stimuli. The retention interval between sample-stimulus presentation and the availability of the choice stimuli was varied between 0.01 s and 12 s within each session. The probability of food produced by correct-red and correct-green responses was varied across conditions. Sample-stimulus discriminability and response bias were measured at four different retention intervals. The results of these analyses showed an interaction between the discriminability of the sample stimuli and the control exerted by differential reinforcement. At longer retention intervals, sample discriminability decreased and sensitivity of choice behavior to changes in the red/green reinforcer ratio increased. An analogous relation has been reported in conditional discriminations in which the physical disparity of stimuli has been varied. This correspondence suggests that increasing the delay between presentation of one of two stimuli and an opportunity to respond discriminatively to it may be functionally similar to increasing the physical similarity of the two stimuli.     

Stimulus control and response bias in an analogue prey-detection procedure

The present study compared the performance of 6 pigeons trained to detect luminance differences in two different signal-detection procedures. Exposed to a three-key array, the pigeons were trained to peck the left key when the brighter of two light intensities had been presented on the center key and to peck the right key when the dimmer of two light intensities had been presented on the center key. Procedure A was a standard signal-detection procedure in which left/bright and right/dim responses produced food reinforcement and left/dim and right/bright responses produced periods of timeout. Procedure B was designed to simulate some of the contingencies operating in a prey-detection situation. Left-key responses produced reinforcement following the brighter center-key stimulus and a period of timeout following the dimmer center-key stimulus. Right-key responses always produced a short period of timeout irrespective of the stimulus. Within each procedure, the duration of timeout arranged for false alarms (left/dim responses) was varied between 3 s and 120 s. Measures of accuracy and response bias were compared between the two procedures. The timeout manipulation produced systematic, but relatively small, changes in these measures when right/dim responses (i.e., correct rejections) produced reinforcement (Procedure A). Arranging timeout for right/dim responses in Procedure B produced greater variability in accuracy and response bias than did arranging reinforcement, but this variability was not related to timeout duration. Overall, discrimination accuracy was considerably higher when right/dim responses produced timeout than when they resulted in reinforcement, and accuracy was accompanied by a large bias toward the response associated with reinforcement. These results are consistent with a recently proposed model of signal detection.     

The interaction between stimulus and reinforcer control on remembering.

In a symbolic matching-to-sample task, 6 pigeons obtained food by pecking a red side key when the brighter of two white lights had been presented on the center key and by pecking a green side key when the dimmer of two white lights had been presented on the center key. Across Part 1 and Parts 6 to 10, the delay between sample-stimulus presentation and the availability of the choice keys was varied between 0 s and 25 s. Across Parts 1 to 5, the delay between the emission of a correct choice and the delivery of a reinforcer was varied between 0 s and 30 s. Although increasing both types of delay decreased stimulus discriminability, lengthening the stimulus-choice delay produced a greater decrement in choice accuracy than did lengthening the choice-reinforcer delay. Additionally, the relative reinforcer rate for correct choice was varied across both types of delay. The sensitivity of behavior to the distribution of reinforcers decreased as discriminability decreased under both procedures. These data are consistent with the view, based on the generalized matching law, that sample stimuli and reinforcers interact in their control over remembering.     

Stimulus control of differential-reinforcement-of-low-rate responding

Five pigeons were given single-stimulus training on an 8-sec differential-reinforcement-of-low-rate schedule followed by steady-state generalization training using 12 wavelength stimuli. Three birds had a high percentage of reinforced responses on the training schedule and flat generalization gradients of total responses. The birds with fewer reinforced responses had much steeper generalization gradients. Generalization gradients plotted as a function of both stimulus wavelength and interresponse time showed that for most birds, stimulus control was restricted to responses with long interresponse times. Responses with very short interresponse times were not under stimulus control and there was some evidence of inhibitory control of short interresponse times. Interresponse-times-per-opportunity functions, plotted as a function of stimulus wavelength, showed that stimulus wavelength controlled the temporal distribution of responses, rather than the overall rate of response. The data indicate that the differential-reinforcement-of-low-rate schedule generates several response categories that are controlled in different ways by wavelength and time-correlated stimuli, and that averaging responses regardless of interresponse-time length obscures this control.     

Applying signal-detection theory to the study of observer accuracy and bias in behavioral assessment

We evaluated the feasibility and utility of a laboratory model for examining observer accuracy within the framework of signal-detection theory (SDT). Sixty-one individuals collected data on aggression while viewing videotaped segments of simulated teacher-child interactions. The purpose of Experiment 1 was to determine if brief feedback and contingencies for scoring accurately would bias responding reliably. Experiment 2 focused on one variable (specificity of the operational definition) that we hypothesized might decrease the likelihood of bias. The effects of social consequences and information about expected behavior change were examined in Experiment 3. Results indicated that feedback and contingencies reliably biased responding and that the clarity of the definition only moderately affected this outcome.     

Stimulus control in the use of landmarks by pigeons in a touch-screen task

Pigeons were tested in a search task on the surface of a monitor on which their responses were registered by a touch-sensitive device. A graphic landmark array was presented consisting of a square outline (the frame) and a colored “landmark.” The unmarked goal, pecks at which produced reward, was located near the center of one edge of the frame, and the landmark was near it. The entire array was displaced without rotation on the monitor from trial to trial. On occasional no-reward tests, the following manipulations were made to the landmark array: (a) either the frame or the landmark was removed; (2) either one edge of the frame or the landmark was shifted; and (3) two landmarks were presented with or without the frame present. On these two-landmark tests, the frame, when present, defined which was the “correct” landmark. When the frame was absent, the “correct” landmark was arbitrarily determined. Results showed that pecks of 2 pigeons were controlled almost solely by the landmark, pecks of 3 were controlled primarily by the landmark but the frame could distinguish the correct landmark, and 1 bird's behavior was controlled primarily by the frame. Stimulus control in this search task is thus selective and differs across individuals. Comparisons to other search tasks and to other stimulus control experiments are made.     

Human symbolic matching-to-sample performance: Effects of reinforcer and sample-stimulus probabilities

Four experiments, each with 6 human subjects, varied the distribution of reinforcers for correct responses and the probability of sample-stimulus presentation in symbolic matching-to-sample procedures. Experiment 1 held the sample-stimulus probability constant and varied the ratio of reinforcers obtained for correct responses on the two alternatives across conditions. There was a positive relation between measures of response bias and the ratio of reinforcers. Experiment 2 held the ratio of reinforcers constant and varied the sample-stimulus probability across conditions. Unlike previous studies that used pigeons as subjects, there was a negative relation between bias and the ratio of sample-stimulus presentations. In Experiment 3, the sample-stimulus probability and the reinforcer ratio covaried across conditions. Response bias did not vary systematically across conditions. In Experiments 1 to 3, correct responses were reinforced intermittently. Experiment 4 used the same procedure as Experiment 3, but all correct responses now produced some scheduled consequence. There was a positive relation between response bias and the ratio of reinforcers. The results suggest that human performance in these tasks was controlled by both the relative frequency of reinforced responses and the relative frequency of nonreinforced responses.     

The effects of number of sample stimuli and number of choices in a detection task on measures of discriminability

Six pigeons were trained on a conditional discrimination task involving the discrimination of various intensities of yellow light. The research asked whether stimulus—response discriminability measures between any pair of stimuli would remain constant when a third or fourth sample and reinforced response were added. The numbers of different sample stimuli presented and different responses reinforced were two (Part 1), three (Parts 2 and 4), and four (Part 3). Across conditions within parts, the ratios of reinforcers obtainable for correct responses were varied over at least five levels. In Part 5, the numbers of sample stimuli and reinforced responses were varied among two, three, and four, and the reinforcer ratio between consecutive remaining samples was constant at 2:1. It was found that once a particular response had been reinforced, subjects continued to emit that response when the conditional stimulus for that response was no longer presented. Data analysis using a generalization-based detection model indicated that this model was able to describe the data effectively. Four findings were in accord with the theory. First, estimates of stimulus—response discriminability usually decreased as the arranged physical disparity between the sample stimuli decreased. Second, stimulus—response discriminability measures were independent of response—reinforcer discriminability measures, preserving parameter invariance between these measures. Third, stimulus—response discriminability measures for constant pairs of conditional stimuli did not change systematically as conditional stimulus—response alternatives were added. Fourth, log stimulus—response discriminability values between physically adjacent conditional stimuli summed to values that were not significantly different from estimates of the discriminability values for conditional stimuli that were spaced further apart.     

The effect of reinforcement probability on time discrimination in the midsession reversal task

We examined how biasing time perception affects choice in a midsession reversal task. Given a simultaneous discrimination between stimuli S1 and S2, with choices of S1 reinforced during the first, but not the second half of the trials, and choices of S2 reinforced during the second, but not the first half of the trials, pigeons show anticipation errors (premature choices of S2) and perseveration errors (belated choices of S1). This suggests that choice depends on timing processes, on predicting when the contingency reverses based on session duration. We exposed 7 pigeons to a midsession reversal task and manipulated the reinforcement rate on each half of the session. Compared to equal reinforcement rates on both halves of the session, when the reinforcement rate on the first half was lower than on the second half, performance showed more anticipation and less perseveration errors, and when the reinforcement rate on the first half was higher than on the second half, performance showed a remarkable reduction of both types of errors. These results suggest that choice depends on both time into the session and the outcome of previous trials. They also challenge current models of timing to integrate local effects.