Effects of stimulus frequency and reinforcement variables on reaction time

Pigeons pecked at one of two black forms, “+” or “O,” either of which could appear alone on a white computer monitor screen. In baseline series of sessions, each form appeared equally often, and two pecks at it produced food reinforcement on 10% of trials. Test series varied the relative probability or duration of reinforcement or frequency of appearance of the targets. Peck reaction times, measured from target onset to the first peck, were found to vary as a function of reinforcement probability but not as a function of relative target frequency or of reinforcement duration. Reaction times to the two targets remained approximately equal as long as the probability of reinforcement, per trial, was equal for the targets, even if the relative frequency of the targets differed by as much as 19 to 1. The results address issues raised in visual search experiments and indicate that attentional priming is unimportant when targets are easy to detect. The results also suggest that equalizing reinforcement probability per trial for all targets removes differential reinforcement as an important variable. That reaction time was sensitive to the probability but not the duration of reinforcement raises interesting questions about the processes reflected in reaction time compared with rate as a response measure.     

Response and time allocation in concurrent second-order schedules

Six pigeons were trained on two-key concurrent variable-interval schedules in which the required response was the completion of a fixed number of key pecks. When the required number of pecks was equal on the two keys, response- and time-allocation ratios under-matched obtained reinforcement rate ratios. A similar result was found when the required number of pecks was unequal, except that performance, measured in response terms, was biased to the shorter required number of pecks and was less sensitive to reinforcement-rate changes. No such differences were found in the data on time spent responding. When the variable-interval schedules were kept constant and the required numbers of pecks were systematically varied, response ratios changed inversely with the ratio of the required number of pecks, but time-allocation ratios varied directly with the same independent variable. Thus, on response measures, pigeons “prefer” the schedule with the smaller peck requirement, but on time measures they “prefer” the schedule with the larger peck requirement. This finding is inconsistent with a commonsense notion of choice, which sees response and time-allocation measures as equivalent.     

Landmark Discrimination Learning in the Dog

Allocentric spatial memory was studied in dogs of varying ages and sources using a landmark discrimination task. The primary goal of this study was to develop a protocol to test landmark discrimination learning in the dog. Using a modified version of a landmark test developed for use in monkeys, we successfully trained dogs to make a spatial discrimination on the basis of the position of a visual landmark relative to two identical discriminanda. Task performance decreased, however, as the distance between the landmark and the “discriminandum” was increased. A subgroup of these dogs was also tested on a delayed nonmatching to position spatial memory task (DNMP), which relies on egocentric spatial cues. These findings suggest that dogs can acquire both allocentric and egocentric spatial tasks. These data provide a useful tool for evaluating the ability of canines to use allocentric cues in spatial learning.     

Strategies in landmark use by children, adults, and marmoset monkeys

Common marmosets (Callithrix jacchus jacchus), human children, and human adults learned to find a goal that was located in the center of a square array of four identical landmarks. The location of the landmark array and corresponding goal varied across trials, so the task could not be solved without using the landmark array. In Experiment 1, a matrix of discrete goal locations was presented and the landmarks surrounded and were adjacent to the correct location during training. After training, an expansion test was given in which the distance between landmarks was increased. Marmosets, children (ages 5–9), and adults all readily learned to use the landmarks to search accurately during training. On the expansion test, adults uniformly searched in the center of the array. Monkeys and children concentrated their searching near the landmarks rather than in the center. The monkeys, but not the children, searched more often on the directionally appropriate side of the landmarks than on other sides of the landmarks. In Experiment 2, children (ages 3–5) were trained with a continuous search space and with the goal farther from the landmarks so that a beaconing strategy rule could not be used. Several of the children failed to acquire the training task. Of those who learned to find the goal, three searched in the middle on expansion tests but most searched nearer to the landmarks. The “middle rule” strategy that is uniformly used by adult humans does not appear to be a preferred strategy for children or non-human primates.     

Food delivery as a conditional stimulus: Feature-learning and memory in pigeons

Three experiments investigated the learning and memory of discriminations based on presence versus absence of a pre-trial food delivery. In Experiment 1 half the illuminations of a response key were followed by food regardless of the subject's behavior. In one group an extra food delivery preceded only reinforced trials (feature-positive condition), whereas in a second group it preceded only nonreinforced trials (feature-negative condition). Key pecks and approaches revealed more rapid and superior discrimination learning in the first group. Experiment 2 replicated the results of Experiment 1 but yielded no evidence that greater “unexpectedness” of pretrial food conditions facilitates discriminative performance. In Experiment 3, individual pigeons trained on a conditional discrimination exhibited a within-subject feature-positive superiority. Delay between pretrial and trial stimuli interacted with feature-positive versus feature-negative training in both the between-group (Experiment 2) and within-subject (Experiment 3) procedures: performance was decremented at both short and long delays in the feature-positive condition but was decremented only at longer delays in the feature-negative condition. The feature-positive superiority obtained here is incompatible with explanations based on either the general concept of “perceptual organization” or on the conditional nature of feature-negative discriminations.     

Delayed alternation in the pigeon

Pigeons were studied in a delayed-response task requiring alternation of key pecks on two response keys. Blackouts of from 1 to 10 sec intervened between successive choices on the two keys.

The following results were obtained: (1) Birds performed at well above chance accuracy on all the delays tested. Accuracy was generally lowest at 1- and 10-sec delays. (2) Overt postural orientations during the delay interval appeared to mediate accurate key-pecking behavior. (3) The shape of the delay vs. accuracy function was discussed in terms of the possibly confounding influences of (a) stimulus “trace” variables, and (b) aversive effects of the time outs produced by incorrect responding.

     

Bias in self-evaluation: Signal probability effects

Two experiments examined apparent signal probability effects in simple verbal self-reports. After each trial of a delayed matching-to-sample task, young adults pressed either a “yes” or a “no” button to answer a computer-presented query about whether the most recent choice met a point contingency requiring both speed and accuracy. A successful matching-to-sample choice served as the “signal” in a signal-detection analysis of self-reports. Difficulty of matching to sample, and thus signal probability, was manipulated via the number of nonmatching sample and comparison stimuli. In Experiment 1, subjects exhibited a bias (log b) for reporting matching-to-sample success when success was frequent, and no bias or a bias for reporting failure when success was infrequent. Contingencies involving equal conditional probabilities of point consequences for “I succeeded” and “I failed” reports had no systematic effect on this pattern. Experiment 2 found signal probability effects to be evident regardless of whether referent-response difficulty was manipulated in different conditions or within sessions. These findings indicate that apparent signal probability effects in self-report bias that were observed in previous studies probably were not an artifact of contingencies intended to improve self-report accuracy or of the means of manipulating signal probability. The findings support an analogy between simple self-reports and psychophysical judgments and bolster the conclusion of Critchfield (1993) that signal probability effects can influence simple self-reports much as they do reports about external stimuli in psychophysical experiments.     

Auto-maintenance in the pigeon: sustained pecking despite contingent non-reinforcement

If a response key is regularly illuminated for several seconds before food is presented, pigeons will peck it after a moderate number of pairings; this “auto-shaping” procedure of Brown and Jenkins (1968) was explored further in the present series of four experiments. The first showed that pecking was maintained even when pecks turned off the key and prevented reinforcement (auto-maintenance); the second controlled for possible effects of generalization and stimulus change. Two other experiments explored procedures that manipulated the tendency to peck the negatively correlated key by introducing alternative response keys which had no scheduled consequences. The results indicate that pecking can be established and maintained by certain stimulus-reinforcer relationships, independent of explicit or adventitious contingencies between response and reinforcer.     

Superstitious key pecking after three peck-produced reinforcements

The first three pecks on a response key by experimentally naive pigeons produced grain reinforcements. Thereafter, for approximately 50 experimental sessions and under a variety of schedule conditions, grain was presented independently of the subjects' behaviors. The pigeons continued to peck the response key “superstitiously” throughout the 50 sessions. The results suggest that superstitions are commonplace—not relatively infrequent or abnormal events—in the behavior of pigeons.     

Stimulus-reinforcer contingencies and local behavioral contrast

Four pigeons were exposed to a series of multiple schedules of variable-interval reinforcement in which pecks were required on one key (operant key) and components were signalled on a second key (signal key). Four additional pigeons experienced identical conditions, except that a yoking procedure delivered food on variable-time schedules, with no key pecks required. One of the components of the multiple schedule was constant throughout the experiment as a variable-interval (or variable-time) 30-second schedule. Operant-key responding during the constant component was uniform throughout the component, uninfluenced by changes in the duration of the variable component, and only slightly influenced by changes in reinforcement frequency correlated with the variable component. By comparison, signal-key response rate during the constant component was highest at the onset of the component, was higher when the variable component was 60-sec long than when it was 1-sec long, and was higher when no reinforcement occurred in the variable component than when reinforcement was scheduled in the variable component. These characteristics of signal-key pecking matched characteristics of local positive behavioral contrast. These data are taken to support the “additivity theory” of behavioral contrast and to suggest that Pavlovian stimulus-reinforcer relations contribute primarily to the phenomenon of local positive contrast.     

Feedback effects on sequential ordering in humans

Under various feedback conditions, 38 college undergraduates were asked to rearrange abstract graphic characters on a computer screen, placing them in arbitrarily designated “correct” sequences. Two sets of seven horizontally arrayed stimuli were used. In Experiment 1, subjects in Group 1 learned to arrange the first set under Selection Feedback in which a “+” appeared above each character after it was selected in the correct order and to arrange the second set under Order Feedback in which a correct response produced a copy of the character in its correct ordinal position at the top of the screen. For Group 2 the order of these conditions was reversed. In Experiment 2, for subjects in Group 3, correct responses produced neither of these types of feedback. Subjects in Group 4 received Order Feedback only until the first set was correctly ordered once. Order Feedback was more effective than Selection Feedback during initial acquisition of the first set but not during maintenance; no differences were found for the second set. Only 2 of 9 subjects successfully put the characters in correct sequential order under the No Feedback condition. When, in Experiment 2, Order Feedback was eliminated after the first correctly arranged sequence, the steady-state criteria were met more slowly than in Experiment 1.     

Comparing the effects of two correction procedures on human acquisition of sequential behavior patterns

Thirty-one college undergraduates learned to touch abstract stimuli on a computer screen in arbitrarily designated “correct” sequential orders. Four sets of seven stimuli were used; the stimuli were arrayed horizontally on the screen in random sequences. A correct response (i.e., touching first the stimulus designated as first) resulted in that stimulus appearing near the top of the screen in its correct sequential position (left to right), and remaining there until the end of the trial. Incorrect responses (i.e., touching a stimulus out of sequence) terminated the trial. New trials displayed either the same sequence as the one on which an error had occurred (same-order correction procedure), or a new random sequence (new-order correction procedure). Whenever all responses occurred in the correct sequence, the next trial displayed a new random sequence. Each phase ended when five consecutive correct response sequences occurred. Initially, the same-order correction procedure increased control by the position as well as by the shape of the stimuli; also, it produced more errors, more total trials, more trials to mastery, and more individual patterns of reacquisition than were produced by the new-order procedure.     

A test of the reinforcing properties of stimuli correlated with nonreinforcement

The information hypothesis of conditioned reinforcement predicts that a stimulus that “reduces uncertainty” about the outcome of a trial will acquire reinforcing properties, even when the stimulus reliably predicts nonreinforcement. Four pigeons' key pecks produced one of two 5-sec stimuli with 0.50 probability according to a discriminated variable-interval schedule. One stimulus was followed by reinforcement; a second stimulus was followed by blackout. To the same extent, therefore, both stimuli reduced uncertainty about the possibility that food would arrive at the termination of the schedule interval. When a second key in the chamber was lighted, each peck on it could produce the stimulus preceding reinforcement, the stimulus preceding nonreinforcement, a novel stimulus, or no stimulus, across separate conditions. The stimulus preceding food maintained responding at substantial levels on the second, stimulus-producing, key. Such responding was not maintained by other stimuli. These data, replicated when the stimuli were reversed on the variable-interval schedule, do not support the prediction that uncertainty-reducing stimuli are necessarily conditioned reinforcers.     

Free-operant choice behavior: A molecular analysis

Pigeons' pecks to two concurrent initial-link stimuli occasionally produced one of two mutually exclusive terminal links. Further responding to the terminal-link stimulus produced food under fixed-interval or fixed-ratio schedules. In such concurrent chained schedules, investigators rarely use a changeover delay to control superstitious switching, although it is customary to use a changeover delay in simple concurrent schedules in which choice responses produce food directly. When terminal-link fixed-interval schedules were equal or similar in duration and no changeover delay was employed, conditional probabilities of choice for a schedule were found to be lower if the last choice was for that schedule than if the last choice was for the other schedule (“switching dependency”). Imposition of a changeover delay with equal or unequal terminal links produced the opposite pattern: conditional probabilities of choice for a schedule were higher if the last choice was for that schedule than if the last choice was for the other schedule. Turning off all chamber lights during the changeover delay interval attenuated these “repetition dependencies.” The results indicate that excessive switching can complicate the interpretation of data from concurrent chains much as from simple concurrent schedules, and that using blackouts to control switching may be preferable to using a changeover delay.     

Unbiased and unnoticed verbal conditioning: the double agent robot procedure

Subjects who were told they were “experimenters” attempted to reinforce fluent speech in a supposed subject with whom they spoke via intercom. The supposed subject was to say nouns, one at a time, on request by the “experimenter”, who reinforced fluent pronunciation with points. Actually, the “experimenter” was talking to a multi-track tape recording, one track of which contained fluently spoken nouns, the other track containing disfluently spoken nouns. If the “experimenter's” request for the next noun was in a specified form a word from the fluent track was played to him as reinforcement; requests in any other form produced the word from the disfluent track. Repeated conditioning of specific forms of requests was accomplished with two subject-“experimenters,” who were unable to describe changes in their own behavior, or the contingencies applied. This technique improved upon an earlier method that had yielded similar results, but was less thoroughly controlled against possible human bias.     

Selective attention: the effects of combining stimuli which control incompatible behavior

Four rhesus monkeys learned both a color and tilt discrimination. The stimuli were combined to produce incompatible behavior. The behavior controlled by one set of stimuli was reinforced until “errors” virtually disappeared. The stimuli were tested separately again. Sixteen replications of the entire procedure indicated that the stimuli producing “errors” were ignored.     

Concurrent performances: synthesizing rate constancies by manipulating contingencies for a single response

An earlier experiment scheduled variable-interval reinforcement for pigeons' pecks on one key, and variable-interval reinforcement alternating with extinction, in a multiple schedule, for pecks on a second key. During the second key's extinction component, first-key pecking was relatively slow and continuous, rarely interrupted by second-key pecking; during the variable-interval component, first-key pecking was frequently interrupted by second-key pecking. When changeover delays operated, so that reinforced pecks on one key could not follow closely upon changeovers from the other key, rapid first-key pecking between interruptions compensated sufficiently for the time lost in second-key pecking that the overall rate of first-key pecking remained roughly constant across the alternating multiple-schedule components. The present experiments duplicated, on a single key, the temporal pattern of first-key pecking generated in the earlier experiments: components of continuous key availability were alternated with components of interrupted key availability. Approximately constant overall rates of responding were observed with a single-key equivalent of a changeover delay scheduled after interruptions and with manipulations of the on-off durations of the interruption cycle. Rate constancies in the original concurrent situation presumably depended on analogous contingencies that operated upon the concurrent responses, rather than on any constant “reserve” of responses.     

Probability learning as a function of momentary reinforcement probability

Pigeons were trained on a probability learning task where the overall reinforcement probability was 0.50 for each response alternative but where the momentary reinforcement probability differed and depended upon the outcome of the preceding trial. In all cases, the maximum reinforcement occurred with a “win-stay, lose-shift” response pattern. When both position and color were relevant cues, the optimal response pattern was learned when the reinforcement probability for repeating the just-reinforced response was 0.80 but not when the probability was 0.65. When only color was relevant, learning occurred much more slowly, and only for subjects trained on large fixed ratio requirements.     

Avoidance of risk as a determinant of cooperation

Pairs of subjects could either cooperate or respond on a lower paying individual task. Whenever both subjects chose to cooperate, either subject could make a response that took $1.00 of the other's earnings. In Exp. I, a stimulus signalled when a “take” response had been made. Either subject could avoid the loss by switching to the individual task within 5 sec after the stimulus appeared. Rates of cooperation were high when losses could be avoided but decreased again when the avoidance condition was removed. In Exp. II, a response prevented “takes” from occurring for a specified time interval after the response. This procedure also maintained cooperation. When each avoidance response subtracted from earnings, both avoidance responding and cooperation were eliminated.     

Sequential effects in dimensional contrast

Two experiments examined pigeons' response rates during short trials signaled by stimuli closely spaced along a wavelength continuum. In Experiment 1 separate halves of the continuum were correlated with different reinforcement schedules. In Experiment 2, the middle stimulus was accompanied by a lower probability of reinforcement than were the remaining wavelengths. Both procedures resulted in dimensional contrast “shoulders,” seen as relatively enhanced or depressed response rates in the presence of stimuli between the extreme of the continuum and the border separating the positive and negative stimuli. Sequential analyses addressed possible contributions of the following interactions: (a) local contrast, seen when rate during a given schedule depends on the schedule in the just-preceding trial; (b) modification of local contrast by the similarity of the signaling stimuli (P. Blough, 1983); and (c) schedule-independent rate contrast, seen when rate in a given trial depends on the rate controlled by the stimulus that accompanied the just-preceding trial (Malone & Rowe, 1981). Dimensional contrast functions were similar when isolated according to the schedule, to the similarity of the signaling stimulus, and to the response rate of the just-preceding trial. The interactions noted above do not appear to make important contributions to this effect.