Effects of relative reinforcer frequency on complex color detection

Pigeons were trained under a discrete-trials detection procedure in which one of a set of color stimuli was presented on the center key and a single response turned off the stimulus and illuminated two side keys. Single responses to one or the other side key produced occasional reinforcers depending on the value of the color stimulus. In Experiment 1, one color-stimulus set comprised 559, 564, 569, and 574 nm, and right-key pecks were occasionally reinforced following presentations of members of this set. The other stimulus set comprised 579, 584, 589, and 594 nm, and left-key pecks were occasionally reinforced following presentations of members of this set. Across seven experimental conditions, the left/(left + right) relative reinforcer frequency was varied from .1 to .9. In Experiment 2, one stimulus set contained only one member, 574 nm, and right-key responses were occasionally reinforced following its presentation. Over 12 experimental conditions, two manipulations were carried out. First, the number of stimuli comprising the other stimulus set was increased from one (579 nm) to two (579 and 584 nm) to three (579, 584, and 589 nm) and to four (579, 584, 589, and 594 nm), and left-key responses were reinforced occasionally following center-key presentations of members of this set. Second, for each stimulus combination, the left/(left + right) relative reinforcer frequency was varied from .1 to .5 to .9 across three experimental conditions. The principal finding of Experiments 1 and 2 was that reinforcers and stimuli interacted in their effects on behavior. In Experiment 3, pairs of adjacent stimuli (5 nm apart) in the range 559 to 594 nm were presented in each experimental condition, and the left/(left + right) relative reinforcer frequency was held constant at .5. The data from all three experiments were analyzed according to a detection model describing performance in multiple-stimulus two-response procedures. This model provided independent measures of stimulus discriminability, contingency discriminability, and bias. The analysis showed that (a) consistent with the color-naming function, pigeons were better able to discriminate between higher nanometer values than lower nanometer values; (b) their ability to discriminate between the stimuli was independent of the number of wavelengths comprising each stimulus set; (c) they allocated delivered reinforcers very accurately to the previously emitted response; and (d) no consistent biases emerged.     

Visually guided catching and tracking skills in pigeons: A preliminary analysis

Research on reaching, tracking, and catching in the pigeon has been hampered by limitations of technology. A new system was developed in which the target was a small rectangle presented on a video display terminal and the pecking response was detected with touch technology. The target moved up and down vertically with sinusoidal velocity. A coincidence between the location of the pigeon's beak and the cursor produced reinforcement. The pigeon pecked ahead and behind the target, but most pecks occurred behind the target so the dominant tracking strategy was lagging. The pigeon was adept at “catching” the target at many locations throughout the trajectory. Transfer of motor learning was tested on probe trials during which the trajectory changed from vertical to horizontal. On transfer trials the pigeons' dominant pattern of pecking immediately shifted from vertical to horizontal. The motor skill displayed by the pigeons was flexible and adaptive, suggesting that the pigeons had learned to track the cursor.     

Connectionist models of conditioning: A tutorial

Models containing networks of neuron-like units have become increasingly prominent in the study of both cognitive psychology and artificial intelligence. This article describes the basic features of connectionist models and provides an illustrative application to compound-stimulus effects in respondent conditioning. Connectionist models designed specifically for operant conditioning are not yet widely available, but some current learning algorithms for machine learning indicate that such models are feasible. Conversely, designers for machine learning appear to have recognized the value of behavioral principles in producing adaptive behavior in their creations.     

Functional classes and equivalence relations

Three adult subjects were taught a set of two-choice simultaneous discriminations, with three positive and three negative stimuli; all possible combinations of positive and negative stimuli yielded nine different pairs. The discriminations were repeatedly reversed and rereversed, the former positive stimuli becoming negative and the former negative stimuli becoming positive. With all subjects, a reversal of the contingencies for one pair of stimuli became sufficient to change their responses to all of the other pairs. The reversals had produced functional stimulus classes. Then, all subjects showed conditional discriminations emerging between members of a functional class; given a sample from one class and comparisons from both classes, they selected the comparison that was in the same class as the sample. Next, 2 of the subjects showed that the within-class conditional relations possessed the symmetric and transitive properties of equivalence relations; after having been taught to relate new stimuli to existing class members, the subjects then matched other class members to the new stimuli. Subsequent tests of two-choice discriminations showed that the conditional discriminations had transferred functional class membership to the new stimuli. The 3rd subject, who did not show equivalence relations among functional class members, was also found to have lost the within-class conditional relations after the equivalence tests.     

The role of observing and attention in establishing stimulus control.

Early theorists (Skinner, Spence) interpreted discrimination learning in terms of the strengthening of the response to one stimulus and its weakening to the other. But this analysis does not account for the increasing independence of the two performances as training continues or for increases in control by dimensions of a stimulus other than the one used in training. Correlation of stimuli with different densities of reinforcement produces an increase in the behavior necessary to observe them, and greater observing of and attending to the relevant stimuli may account for the increase in control by these stimuli. The observing analysis also encompasses errorless training, and the selective nature of observing explains the feature-positive effect and the relatively shallow gradients of generalization generated by negative discriminative stimuli. The effectiveness of the observing analysis in handling these special cases adds to the converging lines of evidence supporting its integrative power and thus its validity.     

Symmetry and transitivity of conditional relations in monkeys (Cebus apella) and pigeons (Columba livia)

In Experiment 1 six monkeys were tested with discriminative relations that were backward relative to their training in a 0-second conditional (“symbolic”) matching procedure. Although there was some indication of backward associations, the evidence was generally weak, and statistical evaluations did not reach conventional significance levels. Unlike children, who show backward associations to the point of symmetry, monkeys and pigeons display at best only weak and transient backward associations. In Experiment 2 associative transitivity was assessed across two sets of conditional matching tasks. All four monkeys tested demonstrated strong transitivity. In contrast, in Experiment 3 there was no evidence of transitivity in three pigeons tested under conditions closely comparable to those of Experiment 2. These results may identify some key features of interspecies differences and contribute to analyses of serial learning in animals.     

Temporal proximity and reinforcement sensitivity in multiple schedules

Distributions of reinforcers between two components of multiple variable-interval schedules were varied over a number of conditions. Sensitivity to reinforcement, measured by the exponent of the power function relating ratios of responses in the two components to ratios of reinforcers obtained in the components, did not differ between conditions with 15-s or 60-s component durations. The failure to demonstrate the “short-component effect,” where sensitivity is high for short components, was consistent with reanalysis of previous data. With 60-s components, sensitivity to reinforcement decreased systematically with time since component alternation, and was higher in the first 15-s subinterval of the 60-s component than for the component whose total duration was 15 s. Varying component duration and sampling behavior at different times since component transition may not be equivalent ways of examining the effects of average temporal distance between components.     

Sequential effects in dimensional contrast

Two experiments examined pigeons' response rates during short trials signaled by stimuli closely spaced along a wavelength continuum. In Experiment 1 separate halves of the continuum were correlated with different reinforcement schedules. In Experiment 2, the middle stimulus was accompanied by a lower probability of reinforcement than were the remaining wavelengths. Both procedures resulted in dimensional contrast “shoulders,” seen as relatively enhanced or depressed response rates in the presence of stimuli between the extreme of the continuum and the border separating the positive and negative stimuli. Sequential analyses addressed possible contributions of the following interactions: (a) local contrast, seen when rate during a given schedule depends on the schedule in the just-preceding trial; (b) modification of local contrast by the similarity of the signaling stimuli (P. Blough, 1983); and (c) schedule-independent rate contrast, seen when rate in a given trial depends on the rate controlled by the stimulus that accompanied the just-preceding trial (Malone & Rowe, 1981). Dimensional contrast functions were similar when isolated according to the schedule, to the similarity of the signaling stimulus, and to the response rate of the just-preceding trial. The interactions noted above do not appear to make important contributions to this effect.     

Stimulus class membership established via stimulus-reinforcer relations

In an arbitrary matching-to-sample procedure, two mentally retarded subjects learned conditional discriminations with two sets of stimuli. Each set included a spoken name (N1 or N2), an object (O1 or O2), and a printed symbol (S1 or S2). One subject selected conditionally (a) O1 upon N1, and O2 upon N2, and (b) S1 upon O1, and S2 upon O2. The other subject selected conditionally (a) S1 upon N1, and S2 upon N2, and (b) O1 upon S1, and O2 upon S2. For both subjects, selections of O1 and S1 produced one type of food, F1; selections of O2 and S2 produced a different type of food, F2. Both subjects also learned identity-matching performances, selecting O1, O2, S1, S2, F1, and F2 conditionally upon those stimuli as samples; F1 followed selections of O1, S1, and F1; F2 followed selections of O2, S2, and F2. Matching performances consistent with stimulus class formation involving the names, objects, symbols, and foods were demonstrated on probe trials, even though these performances had not been taught explicitly. Next, new objects, X1 and X2, were presented on identity-matching trials, producing F1 and F2, respectively. Without further training, X1 was selected conditionally upon N1, S1, and O1, and X2 was selected upon N2, S2, and O2. When the contingencies were changed so that selections of X1 and X2 were now followed by F2 and F1, respectively, X2 was selected conditionally upon N1, S1, and O1, and X1 was selected upon N2, S2, and O2. Class membership of X1 and X2 had apparently changed. This study provides evidence that reinforcers may become members of stimulus classes, and that new stimuli may become class members through relations with reinforcers.     

Variable-interval schedules of timeout from avoidance

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other occasionally produced a 2-min timeout during which the shock-postponement schedule was suspended and its correlated stimuli were removed. Throughout, the shock-postponement schedule maintained proficient levels of avoidance. Nevertheless, in Experiment 1 responding on the timeout lever was established rapidly, was maintained at stable levels on variable-interval schedules, was extinguished by withholding timeout, was reestablished when timeout was reintroduced, and was brought under discriminative control with a multiple variable-interval extinction schedule of timeout. These results are in contrast with Verhave's (1962) conclusion that timeout is an ineffective reinforcer when presented to rats on intermittent schedules. In Experiment 2 the consequence of responding on the timeout lever was altered so that the shock-postponement schedule remained in effect even though the stimulus conditions associated with timeout were produced for 2 min. Responding extinguished, indicating that suspension of the shock-postponement schedule, not stimulus change, was the source of reinforcement. By establishing the reinforcing efficacy of timeout with standard variable-interval schedules, these experiments illustrate a procedure for studying negative reinforcement in the same way as positive reinforcement.