Repeated acquisition in the analysis of rule-governed behavior

Five children, ranging in age from 3½ years to 5½ years, were taught various four-response chains using conditioned reinforcement. Experiment 1 investigated the effects of presenting “instruction” stimuli—a sequence of lights over the correct response buttons—to assess their role in facilitating the acquisition of a chain of responses. Without the “instruction” stimuli, children made many errors before responses were brought under the control of the programmed contingencies. When confronted with the same contingencies later in the day, these subjects made fewer errors. In contrast, in the presence of the “instruction” stimuli, subjects made virtually no errors. However, when the “instruction” stimuli were discontinued in the subsequent session, all 5 subjects made errors. In Experiment 2, the subjects were taught to verbalize the contingencies during the phase without the “instruction” stimuli. This resulted in errorless performance during the subsequent exposure to the same procedure, but errors nevertheless occurred again during reexposure to the procedure with the “instruction” stimuli discontinued.     

Stimulus properties of conspecific behavior

Two experiments identified the conditions in which the behavior of one bird acquired discriminative control of the behavior of a second bird. The schedule-controlled behaviors of the “stimulus” bird were differentially correlated with the components of a multiple schedule according to which the pecking of an “experimental” bird produced food. In Experiment 1, three pairs of pigeons acquired a successive discrimination and two reversals with the conspecific stimuli. Experiment 2 included a control condition in which no systematic relationship existed between the conspecific stimuli and the component schedules. While differential responding during the components of the multiple schedule was again found when the conspecific stimuli were available, differential responding did not occur in the control condition. Test conditions included in the experiments indicated that (a) the differential responding was not dependent on the discriminative properties of reinforcement, (b) the pecking of the stimulus and experimental birds was temporally interrelated, (c) the visual conspecific stimuli were critical to the maintenance of the discrimination, and (d) the observed stimulus control immediately generalized to an unfamiliar conspecific.     

Bias in self-evaluation: Signal probability effects

Two experiments examined apparent signal probability effects in simple verbal self-reports. After each trial of a delayed matching-to-sample task, young adults pressed either a “yes” or a “no” button to answer a computer-presented query about whether the most recent choice met a point contingency requiring both speed and accuracy. A successful matching-to-sample choice served as the “signal” in a signal-detection analysis of self-reports. Difficulty of matching to sample, and thus signal probability, was manipulated via the number of nonmatching sample and comparison stimuli. In Experiment 1, subjects exhibited a bias (log b) for reporting matching-to-sample success when success was frequent, and no bias or a bias for reporting failure when success was infrequent. Contingencies involving equal conditional probabilities of point consequences for “I succeeded” and “I failed” reports had no systematic effect on this pattern. Experiment 2 found signal probability effects to be evident regardless of whether referent-response difficulty was manipulated in different conditions or within sessions. These findings indicate that apparent signal probability effects in self-report bias that were observed in previous studies probably were not an artifact of contingencies intended to improve self-report accuracy or of the means of manipulating signal probability. The findings support an analogy between simple self-reports and psychophysical judgments and bolster the conclusion of Critchfield (1993) that signal probability effects can influence simple self-reports much as they do reports about external stimuli in psychophysical experiments.     

Discrimination learning with and without “errors”

Responses to S− (“errors”) are not a necessary condition for the formation of an operant discrimination of color. Errors do not occur if discrimination training begins early in conditioning and if S+ and S− initially differ with respect to brightness, duration and wavelength. After training starts, S−'s duration and brightness is progressively increased until S+ and S− differ only with respect to wavelength. Errors do occur if training starts after much conditioning in the presence of S+ has occurred or if S+ and S− differ only with respect to wavelength throughout training. Performance following discrimination learning without errors lacks three characteristics that are found following learning with errors. Only those birds that learned the discrimination with errors showed (1) “emotional” responses in the presence of S−, (2) an increase in the rate (or a decrease in the latency) of its response to S+, and (3) occasional bursts of responses to S−.     

Neural coding of reward magnitude in the orbitofrontal cortex of the rat during a five-odor olfactory discrimination task

The orbitofrontal cortex (OBFc) has been suggested to code the motivational value of environmental stimuli and to use this information for the flexible guidance of goal-directed behavior. To examine whether information regarding reward prediction is quantitatively represented in the rat OBFc, neural activity was recorded during an olfactory discrimination “go”/“no-go” task in which five different odor stimuli were predictive for various amounts of reward or an aversive reinforcer. Neural correlates related to both actual and expected reward magnitude were observed. Responses related to reward expectation occurred during the execution of the behavioral response toward the reward site and within a waiting period prior to reinforcement delivery. About one-half of these neurons demonstrated differential firing toward the different reward sizes. These data provide new and strong evidence that reward expectancy, regardless of reward magnitude, is coded by neurons of the rat OBFc, and are indicative for representation of quantitative information concerning expected reward. Moreover, neural correlates of reward expectancy appear to be distributed across both motor and nonmotor phases of the task.     

Recognition by the pigeon of stimuli varying in two dimensions

Pigeons served in four experiments, each of which involved about 44,000 discrete 1.2-sec trials under steady-state conditions. The first experiment scaled a short segment of the visual wavelength continuum; this dimension was then combined in a conditional discrimination with each of three others; time after reinforcement, tone frequency, and line tilt. In the two-stimulus experiments, the birds' responses were reinforced in the presence of only one stimulus combination: “582 nm” together with “2 min after reinforcement”, “3990 Hz”, or “vertical line”. Many other stimulus combinations also appeared equally often and went without reinforcement. The wavelength stimuli conformed to an equal-interval scale, and per cent response was generally linear with wavelength, when scaled on cumulative normal coordinates. The components of the compound stimulus were found to interact in a multiplicative fashion; when one component differed greatly from its reinforcement value, changes in the other component had relatively little effect. For the “time”-“wavelength” compound, this interaction appeared to be modified by the effects of set or attention. Certain response latency data are reported, and other combination rules are discussed.     

Eccentric stimuli on multiple fixed-interval schedules

The effects of presenting a different (“eccentric”) stimulus for one interval during either or both components of a cyclic multiple fixed-interval fixed-interval schedule, with 12 short and four long intervals per cycle, were studied in three experiments. Eccentric stimuli in the short-interval component reliably produced a persistent, substantial elevation in key-peck rate. The effect appears to depend on schedule context and an initial “disinhibiting” effect of the eccentric stimulus.     

Effects of chlorpromazine and d-amphetamine on escape and avoidance behavior under a temporally defined schedule of negative reinforcement

Chlorpromazine hydrochloride and d-amphetamine sulphate were administered to two rats responding on a baseline temporally defined schedule of negative reinforcement which produced both “escape-like” and “avoidance-like” behavior. The effects of these drugs appeared similar to those expected on the more customary sort of non-cued avoidance schedule.     

Choice behavior on discrete trials: a demonstration of the occurrence of a response strategy

Three pairs of pigeons were trained to peck at two keys presented simultaneoulsy in discrete trials with intertrial intervals of 1, 22, or 120 sec. Left-key responses incremented the probability of reinforcement for the first right-key response and, conversely, right-key responses incremented the probability of reinforcement for the first left-key response. In terms of relative response rates, it was found that all birds' choices were described by a momentary maximizing strategy, but this fact was not reflected in the detailed sequential statistics for birds with the longer (22 or 120 sec) intertrial intervals. It was hypothesized that choice behavior, in general, may be accurately described by a momentary maximizing sequence, but that prior failures to demonstrate this were due to “errors” in executing the momentary maximizing sequence. These misappropriated responses, which are hypothesized to be randomly distributed among the responses defining the momentary maximizing sequence, caused successive choices to appear to be statistically independent when, in fact, they were not.     

The effect of overtraining on behavioral contrast and the peak-shift

Following initial discrimination training between two wavelength stimuli and a subsequent generalization test to the wavelength dimension, Group 1 was “overtrained” for 105 days on the original discrimination. Group 2 was “overtrained” with the original positive stimulus and a new negative stimulus, a white line. Group 3 was “overtrained” with the original negative stimulus and a new positive stimulus, the white line. Each 15 days of extended training were followed by a wavelength generalization test similar to the first test. The results suggest that there is no consistent relationship between the response rate in positive stimulus immediately before the generalization test and whether or not a peak shift occurs during the test.     

Reinforcement omission on temporal go-no-go schedules

Either a partial blackout, or the blackout plus a “feeder flash”, occurred in lieu of reinforcement on two procedures that produced opposite patterns of responding after reinforcement. Response rate was elevated after reinforcement omission on the procedure that produced a “pause-and-respond” pattern following reinforcement, but depressed after reinforcement omission on the procedure that produced a “respond-and-pause” pattern. The effect of blackout plus feeder flash was generally intermediate between the effects of blackout and the effects of reinforcement. These results are consistent with an interpretation of reinforcement omission effects in terms of the discriminative temporal control exerted by reinforcement and stimuli similar to it.     

Unbiased and unnoticed verbal conditioning: the double agent robot procedure

Subjects who were told they were “experimenters” attempted to reinforce fluent speech in a supposed subject with whom they spoke via intercom. The supposed subject was to say nouns, one at a time, on request by the “experimenter”, who reinforced fluent pronunciation with points. Actually, the “experimenter” was talking to a multi-track tape recording, one track of which contained fluently spoken nouns, the other track containing disfluently spoken nouns. If the “experimenter's” request for the next noun was in a specified form a word from the fluent track was played to him as reinforcement; requests in any other form produced the word from the disfluent track. Repeated conditioning of specific forms of requests was accomplished with two subject-“experimenters,” who were unable to describe changes in their own behavior, or the contingencies applied. This technique improved upon an earlier method that had yielded similar results, but was less thoroughly controlled against possible human bias.     

Reinforcement by an imprinting stimulus versus water on simple schedules in ducklings

Ducklings (5 to 28 days old) were trained to peck a pole on fixed-ratio, fixed-interval, and multiple schedules using brief presentation of an imprinting stimulus as the response-contingent event. Other ducklings of the same age were trained similarly except that reinforcement consisted of access to water. With water reinforcement the typical fixed-ratio (“break-run”), fixed-interval (“scallop”), and multiple schedule response patterns were readily established and consistently maintained. With the imprinting stimulus these schedule effects were inconsistent in some subjects and virtually nonexistent in others, despite extended training. Schedule control with the imprinting stimulus was not improved by the use of a reinforcement signaling procedure which enhances responding reinforced by electrical brain stimulation on intermittent schedules. However, the overall rates of responding and the extinction functions generated after reinforcement with water versus the imprinting stimulus were comparable. These findings imply that control by temporal and discriminative stimuli may be relatively weak when a young organism's behavior is reinforced by presentation of an imprinting stimulus.     

Conditioning of within-trial patterns of key pecking in pigeons

The possibility of conditioning systematic patterns of responding during brief discrete trials was studied by requiring hungry pigeons to key peck and then pause or to pause and then key peck in order to gain access to food. These schedules were highly effective in promoting decelerated and accelerated rates of responding, respectively, within individual trials; indeed, performance was quite similar to that observed when explicit external stimuli were correlated with “peck” and “pause” portions of the daily trials. Finally, schedules of reinforcement that did not selectively reinforce peck-pause or pause-peck patterns neither generated these patterns nor maintained them at the previous high levels. The results, therefore, confirm Shimp's (1976) proposal that organized groupings of discrete responses may function as operants—even in the absence of strict response-reinforcer contiguity.     

Conditioned reinforcement and discrimination in second-order schedules

Pigeons were exposed to multiple second-order schedules in which responding on the “main key” was reinforced according to either a variable-interval or fixed-interval schedule by production of a brief stimulus on the “brief-stimulus key”. A response was required to the brief stimulus during its fourth (final) presentation to produce food; responses to the earlier brief stimuli indicated the extent to which the final brief stimulus was discriminated from preceding ones. Main-key response rates were higher in early components of paired brief-stimulus schedules, in which each brief stimulus was the same as that paired with reinforcement, than in comparable unpaired brief-stimulus or tandem schedules. Poor discrimination occurred between paired brief stimuli (Experiment I). When chain stimuli on the main key induced a discrimination between the first two and second two brief stimuli, the response-rate enhancement in the paired brief-stimulus schedule persisted (Experiment II). Rate enhancement diminished when the initial link of the chain included the first three components (Experiment IV). Eliminating the contingency between responding and brief-stimulus production also diminished rate enhancement (Experiment III). The results show that the discriminative and conditioned reinforcing effects of food-paired brief stimuli may be selectively manipulated and suggest that the reinforcing effects are modulated by other reinforcers in the situation.     

Food delivery as a conditional stimulus: Feature-learning and memory in pigeons

Three experiments investigated the learning and memory of discriminations based on presence versus absence of a pre-trial food delivery. In Experiment 1 half the illuminations of a response key were followed by food regardless of the subject's behavior. In one group an extra food delivery preceded only reinforced trials (feature-positive condition), whereas in a second group it preceded only nonreinforced trials (feature-negative condition). Key pecks and approaches revealed more rapid and superior discrimination learning in the first group. Experiment 2 replicated the results of Experiment 1 but yielded no evidence that greater “unexpectedness” of pretrial food conditions facilitates discriminative performance. In Experiment 3, individual pigeons trained on a conditional discrimination exhibited a within-subject feature-positive superiority. Delay between pretrial and trial stimuli interacted with feature-positive versus feature-negative training in both the between-group (Experiment 2) and within-subject (Experiment 3) procedures: performance was decremented at both short and long delays in the feature-positive condition but was decremented only at longer delays in the feature-negative condition. The feature-positive superiority obtained here is incompatible with explanations based on either the general concept of “perceptual organization” or on the conditional nature of feature-negative discriminations.     

Discrimination learning as a function of stimulus location along an auditory intensity continuum

Eight groups of rats were trained on an auditory intensity discrimination in which the discriminative stimuli were separated by 10 decibels (db). Four pairs of stimuli were selected from different regions along a 60–100 db (SPL) intensity continuum. Counterpart groups were trained on each stimulus pair, with the relative intensity positions of the reinforced stimulus (S^D) and the non-reinforced stimulus (S^Δ) reversed for the two groups. Discrimination acquisition curves were compared to determine whether stimuli separated by equal logarithmic units were of comparable “difficulty”, and to determine the relative effectiveness of an S^D serving as the more versus less intense member of a stimulus pair. It was concluded that: (1) When S^D is the more intense, auditory intensities of constant logarithmic separation are graded in “difficulty” along the intensity continuum; high intensity discriminative stimuli are most readily discriminated. When S^Δ is the more intense, this graded effect is not evident. (2) For a given continuum location, discrimination is inferior when S^Δ is the more intense. This effect is most pronounced at the high intensity end of the continuum and is chiefly attributable to differences in the rate of S^Δ responding.     

Sign-tracking with an interfood clock

Food was presented to pigeons, irrespective of their behavior. The fixed 60-s interfood interval was segmented into ten 6-s periods, each signaled by a distinctive stimulus color, ordered by wavelength. This “interfood clock” reliably generated and maintained successively higher rates of key pecking at stimuli successively closer to food. Under extinction, key pecking ceased. When the standard stimulus sequence was changed to a different sequence for each bird, accelerated responding again emerged and was sustained under each of the new color sequences. However, responding was neither maintained nor acquired when each successive interfood interval provided a different random sequence of the ten stimuli. Thus, the interfood clock generated and maintained sign-tracking under stimulus control, and the resulting behavior was attributable neither to stimulus generalization nor to a simple temporal gradient.     

Teaching serial position sequences to monkeys with a delayed matching-to-sample procedure

Comparison was made of two methods for training monkeys to “observe” a two-member serial position sequence by pressing two consecutively lighted keys and then to “report” the sequence by pressing the same two keys in the same order but without the lights. A fading technique involving gradual elimination of brightness cues from “reporting” keys was found more effective than a no-fading procedure in which the cues remained bright during training and then were suddenly removed. Animals that failed to learn to report a new sequence with the no-fading procedure sometimes developed behavior incompatible with that desired. They made repeated and specific errors that prematurely terminated trials of the sequence to-be-learned, even though the correct key was cued by a bright light. They behaved appropriately, however, on succeeding trials of other sequences. Thus, the errors were followed by trials on which reinforcement occurred. Manipulation of this contingency indicated its importance in maintaining the stereotyped error patterns.     

Adaptation, teleology, and selection by consequences

This paper presents and defends the view that reinforcement and natural selection are selection processes, that selection processes are neither mechanistic nor teleological, and that mentalistic and vitalistic processes are teleological but not mechanistic. The differences between these types of processes are described and used in discussing the conceptual and methodological significance of “selection type theories” and B. F. Skinner's radical behaviorist view that “operant behavior is the field of intention, purpose, and expectation. It deals with that field precisely as the theory of evolution has dealt with another kind of purpose” (1986, p. 716). The antimentalism of radical behaviorism emerges as a post-Darwinian extension of Francis Bacon's (and Galileo's) influential view that “[the introduction of final causes] rather corrupts than advances the sciences” (Bacon, 1905, p. 302).