Maintenance and transfer of self-reinforcement functions

In order to examine conditions maintaining self-reinforcement functions, pigeons were trained to reward their own performances and then tested for adherence to work requirements under decreasing likelihood of punishment for undeserved self-reward. Contingent self-reinforcement was stably maintained given moderate to high probability that unmerited self-reward would incur punishing consequences. In successive reversals of treatment and test conditions, the amount of behavior performed for each self-reward covaried with self-reinforcement rate. A further experiment demonstrated that self-reinforcing practices, involving both performance and consummatory contingencies, transfer to new activities for which the animal had never been trained to reward himself contingently. Adherence to performance requirements was more stringent, however, than to limitations on amount of rcinforcers consumed from freely available provisions.     

Discriminative activation and maintenance of contingent self-reinforcement

The present experiment studied the process by which environmental events come to exercise some degree of control over the activation of contingent self-reinforcement. Through differential training, animals learned to impose performance requirements for self-reward in certain environmental contexts but not in others. Both contextual influences and periodic negative consequences for noncontingent self-reward increased adherence to response requisites for self-reinforcement.     

Self-reinforcement: An assessment of external influences

Two experiments were carried out in which low- and high-demand characteristics of self-reward were assessed in a multiple-baseline design across subjects. In the first experiment, arithmetic performance of four children was systematically assessed under no-reinforcement, self-reward/low-demand and self-reward/high-demand phases. In the second experiment, the performance of four children on a less meaningful task was assessed under the same conditions employed in Experiment 1. Results indicated that performance rates were relatively stable during the no-reinforcement phase but that these rates dropped markedly during the self-reward/low-demand phase for all eight subjects. Further, increased rates of performance were achieved under the self-reward/high-demand phase for all subjects. Results are discussed in terms of the assessment of the self-reinforcement procedure, current definitions of self-reinforcement, and the need to view self-reinforcement as existing on a continuum of external demand characteristics.This project was supported in whole by a faculty research grant awarded to the first author and completed while the second author was a visiting associate professor at Western Psychiatric Institute and Clinic.     

"Turning back the clock" on serial-stimulus sign tracking.

Two experiments examined the effects of a negative (setback) response contingency on key pecking engendered by a changing light-intensity stimulus clock (ramp stimulus) signaling fixed-time 30-s food deliveries. The response contingency specified that responses would immediately decrease the light-intensity value, and, because food was delivered only after the highest intensity value was presented, would delay food delivery by 1 s for each response. The first experiment examined the acquisition and maintenance of responding for a group trained with the contingency in effect and for a group trained on a response-independent schedule with the ramp stimulus prior to introduction of the contingency. The first group acquired low rates of key pecking, and, after considerable exposure to the contingency, those rates were reduced to low levels. The rates of responding for the second group were reduced very rapidly (within four to five trials) after introduction of the setback contingency. For both groups, rates of responding increased for all but 1 bird when the contingency was removed. A second experiment compared the separate effects of each part of the response contingency. One group was exposed only to the stimulus setback (stimulus only), and a second group was exposed only to the delay of the reinforcer (delay only). The stimulus-only group's rates of responding were immediately reduced to moderate levels, but for most of the birds, these rates recovered quickly when the contingency was removed. The delay-only groups's rates decreased after several trials, to very low levels, and recovery of responding took several sessions once the contingency was removed. The results suggest that (a) sign-tracking behavior elicited by an added clock stimulus may be reduced rapidly and persistently when a setback contingency is imposed, and (b) the success of the contingency is due both to response-dependent stimulus change and response-dependent alterations in the frequency of food delivery. The operation of the contingency is compared with the effects of secondary reinforcement and punishment procedures.     

The role of the peck-food contingency on fixed-interval schedules

Pigeons were trained to peck on a fixed-interval schedule of food reinforcement and then exposed to three schedules in which there was either no, or an indirect, relation between pecking and food delivery: (a) a conjunctive schedule in which food was delivered at fixed intervals, providing at least one peck was emitted in the interval; (b) a recycling version of the conjunctive schedule that essentially eliminated occasional peck-food contiguities (recycling conjunctive); (c) delivery of food at fixed intervals independently of the birds' behavior (fixed time). The rates and patterns of pecking sustained by these procedures depended on interfood interval and relative proximity of pecks to food.     

Discrimination of methadone and cocaine by pigeons without explicit discrimination training.

Pigeons were trained to peck a key on a variable-interval 2-min schedule of food reinforcement. Prior to each session, either 2.0 mg/kg methadone (n = 3), 3.0 mg/kg cocaine (n = 4), or 5.6 mg/kg cocaine (n = 2) was administered. When each pigeon's rate of pecking was stable, a range of doses of the training drug and saline were administered prior to 20-min extinction sessions separated by at least four training sessions. Rate of pecking during these extinction tests was generally an increasing function of dose, with the lowest rates obtained following saline and low doses and the highest rates obtained following doses near the training doses. Dose functions from pigeons trained with 5.6 mg/kg cocaine were steeper than those from pigeons trained with 3.0 mg/kg cocaine. Pigeons trained with methadone or 3.0 mg/kg cocaine were then given discrimination training, in which food reinforcement followed drug administration and 20-min extinction sessions followed saline administration. Rates of pecking under these conditions quickly diverged until near-zero rates were obtained following saline and high rates were obtained following drug. Discrimination training steepened dose functions for the training drugs, and the effects of several other substituted drugs depended on the pharmacology of the training drug. The pigeons trained with 5.6 mg/kg cocaine were tested with d-amphetamine, methadone, and morphine prior to discrimination training. d-Amphetamine increased rates dose dependently, and methadone and morphine did not. The results suggest that discriminative control by methadone and cocaine was established without explicit discrimination training.     

Interval reinforcement of choice behavior in discrete trials

Pigeons were trained to peck at red or green keys presented simultaneously in discrete trials. In one experiment, reinforcements were arranged by concurrent variable-interval schedules. The proportion of responses to green approximately matched the proportion of reinforcements produced by pecking green. Detailed analysis of responding revealed a systematic decrease in the probability of switching from green to red within sequences of trials after reinforcement. This trend corresponded to sequential changes in the relative frequency of reinforcement, and not to sequential changes in probability of reinforcement. In a second experiment, reinforcements were scheduled probabilistically every seventh trial. Even though there were no contingencies on pecking during the first six post-reinforcement trials, choices of green on the first response after reinforcement matched the proportion of reinforcements for pecking green. These results extend the generality of overall matching under concurrent reinforcement.     

Overt activity during conditioned suppression: a search for punishment artifacts

Pigeons, previously trained to peck a key (using food as the reinforcer), were permitted unlimited access to food and, concurrently, key pecking was allowed to go unreinforced until all pecking ceased. A tone ending with electrical shock was then repeatedly presented in an effort to establish the tone as a potentially suppressing stimulus. When key pecking was later reestablished, tone presentation (without shock) sharply reduced the rate of pecks. At selected points throughout the experiment, special observation procedures supplemented the recordings of key pecks and provided detailed fine-grain protocols of the birds' overt movements during the periods before, during, and after tone presentations. Results indicated that neither punishment of key pecks nor punishment of other overt movements was a necessary precursor to the conditioned suppression observed in the final stage. As such, the findings support interpretations of conditioned suppression that characterize the phenomenon as reflecting a conditioned emotional reaction that either directly or indirectly inhibits overt activity.     

On conditioned reinforcing effects of negative discriminative stimuli

Observing responses by pigeons were studied during sessions in which a food key and an observing key were available continuously. A variable-interval schedule and extinction alternated randomly on the food key. In one condition, food-key pecking during extinction decreased reinforcement frequency during the next variable-interval component, and in the other condition such pecking did not affect reinforcement frequency. Observing responses either changed both keylight colors from white to green (S+) or to red (S−) depending on the condition on the food key, or the observing responses never produced the S+ but produced the S− when extinction was in effect on the food key. Observing responses that produced only S− were maintained only when food-key pecking during extinction decreased reinforcement frequency in the subsequent variable-interval component. The red light conformed to conventional definitions of a negative discriminative stimulus, rendering results counter to previous findings that production of S− alone does not maintain observing. Rather than offering support for an informational account of conditioned reinforcement, the results are discussed in terms of a molar analysis to account for how stimuli acquire response-maintaining properties.     

Relative preference for external and self-controlled reinforcement in monkeys

Monkeys were tested for their relative preference for self-monitored and externally imposed systems of reinforcement. One monkey clearly preferred to self-reward his own performances. whereas a second monkey favored slightly the externally administered system but displayed a small gradual shift toward self-reward as the sessions progressed. Unlike previous findings. unmerited self-reward did not result in rapid discarding of self-imposed performance requirements. The level of work output and the interactive effects of multiple reinforcement control are possible operative factors in sustaining self-imposed contingencies long after punishment for unmerited self-reward has been discontinued.     

Pigeons' discrimination of Michotte's launching effect

We trained four pigeons to discriminate a Michotte launching animation from three other animations using a go/no-go task. The pigeons received food for pecking at one of the animations, but not for pecking at the others. The four animations featured two types of interactions among objects: causal (direct launching) and noncausal (delayed, distal, and distal & delayed). Two pigeons were reinforced for pecking at the causal interaction, but not at the noncausal interactions; two other pigeons were reinforced for pecking at the distal & delayed interaction, but not at the other interactions. Both discriminations proved difficult for the pigeons to master; later tests suggested that the pigeons often learned the discriminations by attending to subtle stimulus properties other than the intended ones.     

Self-reinforcement: Appealing misnomer or effective mechanism?1

The purpose of these two experiments was to test the motivational explanation for the effectiveness of self-reinforcement techniques. In each experiment, the classic self-reinforcement procedure (a desirable consequence following a target behavior) was compared with variations. In Experiment 1, undergraduate females (n = 61) showed no more self-recorded exercise when they performed a classic self-reinforcement procedure (high-probability activities following exercising) than when they only self-monitored exercise or engaged in one of three related procedures (high-probability activities preceding exercise, or low-probability activities following or preceding exercise). Indeed, only this last group produced significantly more self-recorded exercise. In Experiment 2, undergraduates (n = 62) attempted more workbook questions following a reading session in which they read a page and then ate a chosen food (classic self-reinforcement procedure) or ate a chosen food and then read a page, than when they read and ate in no particular order. More importantly, more workbook questions were answered correctly by the group who ate a chosen food and then read a page than by the other two groups. No differential results were produced by subjects' level of food deprivation. The classic self-reinforcement procedure showed no advantage over the variations in either experiment. These results call into question the motivational explanation for the effectiveness of self-reinforcement techniques. An alternative stimulus or cuing explanation is offered: that self-reinforcement is effective because it cues the long-term environmental consequences that actually control the frequency of the target behavior.     

Automaintenance without stimulus-change reinforcement: Temporal control of key pecks

Yoked pairs of experimentally naive pigeons were exposed to a modified autoshaping procedure in which key pecking by the leader birds postponed both keylight termination and access to grain for the leader and the follower bird. Key pecking developed and was maintained in all birds and continued through two reversals of roles in the yoked procedure. Although temporal control developed more slowly in follower birds, asymptotic temporal distributions of key pecking were similar for all birds in both leader and follower roles; maximum responding occurred soon after keylight onset and decreased to a minimum prior to reinforcement. Response distributions for both leader and follower birds were described by Killeen's (1975) mathematical model of temporal control. Follower birds received response-independent reinforcement, and the development by these birds of temporal distributions which are minimal immediately prior to reinforcement is without precedent in Pavlovian appetitive conditioning. However, maintenance of key pecking by the leader birds, whose responses postponed both stimulus-change and food reinforcement, supports an interpretation of autoshaped and automaintained key pecking as responding elicited by signaled grain presentation.     

High-order concept formation in the pigeon

After 30 days of operant training, with pecking responses to aerial photographs containing man-made objects reinforced with food, and no food reinforcement for pecking on photographs not containing man-made objects, a discrimination to the two classes of photographs was obtained. The discriminative response generalized to photographs with which the pigeons had no previous experience. This study demonstrates that pigeons are capable of forming relatively high-order concepts. Some possible stimulus properties controlling the discrimination are discussed.     

Transfer of control of the pigeon's key peck from food reinforcement to avoidance of shock

Eight pigeons were initially trained to peck a white key for food under a variable-interval 1-min schedule of reinforcement. Then, a shock-avoidance schedule was initiated and food was no longer available in the experimental situation. Under the avoidance schedule, each peck on the key postponed shock for 40 sec. A warning signal, consisting of tone and red houselights, was presented after 30 sec without a response. If no response occurred, a shock was delivered 10 sec after warning-signal onset. Shocks were delivered every 10 sec in the presence of the warning signal until a response was made. The warning signal was terminated only by a response. Key pecking of all eight pigeons came under control of the avoidance schedule and responding continued throughout the 20-day avoidance training period.     

Mechanisms underlying the effects of unsignaled delayed reinforcement on key pecking of pigeons under variable-interval schedules.

Three experiments were conducted to test an interpretation of the response-rate-reducing effects of unsignaled nonresetting delays to reinforcement in pigeons. According to this interpretation, rates of key pecking decrease under these conditions because key pecks alternate with hopper-observing behavior. In Experiment 1, 4 pigeons pecked a food key that raised the hopper provided that pecks on a different variable-interval-schedule key met the requirements of a variable-interval 60-s schedule. The stimuli associated with the availability of the hopper (i.e., houselight and keylight off, food key illuminated, feedback following food-key pecks) were gradually removed across phases while the dependent relation between hopper availability and variable-interval-schedule key pecks was maintained. Rates of pecking the variable-interval-schedule key decreased to low levels and rates of food-key pecks increased when variable-interval-schedule key pecks did not produce hopper-correlated stimuli. In Experiment 2, pigeons initially pecked a single key under a variable-interval 60-s schedule. Then the dependent relation between hopper presentation and key pecks was eliminated by arranging a variable-time 60-s schedule. When rates of pecking had decreased to low levels, conditions were changed so that pecks during the final 5 s of each interval changed the keylight color from green to amber. When pecking produced these hopper-correlated stimuli, pecking occurred at high rates, despite the absence of a peck-food dependency. When peck-produced changes in keylight color were uncorrelated with food, rates of pecking fell to low levels. In Experiment 3, details (obtained delays, interresponse-time distributions, eating times) of the transition from high to low response rates produced by the introduction of a 3-s unsignaled delay were tracked from session to session in 3 pigeons that had been initially trained to peck under a conventional variable-interval 60-s schedule. Decreases in response rates soon after the transition to delayed reinforcement were accompanied by decreases in eating times and alterations in interresponse-time distributions. As response rates decreased and became stable, eating times increased and their variability decreased. These findings support an interpretation of the effects of delayed reinforcement that emphasizes the importance of hopper-observing behavior.     

Categorization of natural movements by pigeons: visual concept discrimination and biological motion

In three experiments, pigeons were exposed to a discriminated autoshaping procedure in which categories of moving stimuli, presented on videotape, were differentially associated with reinforcement. All stimuli depicted pigeons making defined responses. In Experiment 1, one category consisted of several different scenes of pecking and the other consisted of scenes of walking, flying, head movements, or standing still. Four of the 4 birds for which pecking scenes were positive stimuli discriminated successfully, whereas only 1 of the 4 for which pecking was the negative category did so. In the pecking-positive group, there were differences between the pecking rates in the presence of the four negative actions, and these differences were consistent across subjects. In Experiment 2, only the categories of walking and pecking were used; some but not all birds learned this discrimination, whichever category was positive, and these birds showed some transfer to new stimuli in which the same movements were represented only by a small number of point lights (Johansson's “biological motion” displays). In Experiment 3, discriminations between pecking and walking movement categories using point-light displays were trained. Four of the 8 birds discriminated successfully, but transfer to fully detailed displays could not be demonstrated. Pseudoconcept control groups, in which scenes from the same categories of motion were used in both the positive and negative stimulus sets, were used in Experiments 1 and 3. None of the 8 pigeons trained under these conditions showed discriminative responding. The results suggest that pigeons can respond differentially to moving stimuli on the basis of movement cues alone.     

Mirror pecking and timeout under a multiple fixed-ratio schedule of food delivery

Pigeons were trained to peck a key under a multiple fixed-ratio 25 fixed-ratio 175 schedule of food presentation. In the first condition, either a mirror or the opportunity to produce a 30-second timeout were available. In a second condition, mirror and timeout availability were reversed for the two groups. Following a return to the initial condition, mirror and timeout keys were presented together for all birds. Mirror and timeout responses occurred predominantly in the pause in the larger fixed-ratio component, regardless of whether the opportunities for the two responses were available singly or together. Mirror responding occurred in a greater proportion of the pauses than did timeouts. When the opportunities for both mirror pecking and timeout were available concurrently, they occurred with probabilities similar to those under the single conditions. Within the pause itself, mirror responses most frequently occurred immediately after reinforcement. Timeouts occurred most frequently toward the end of the pause, and some timeouts occurred in the early part of the run. Longer preratio pausing occurred in the larger fixed-ratio component in the conditions in which the mirror was present, whether or not any mirror pecks were recorded.     

Maintenance of key pecking by response-independent food presentation: the role of the modality of the signal for food

Three pigeons were exposed to a series of procedures in which periods of response-independent food presentation, on a variable-time schedule, alternated with periods in which food was never presented. The stimuli that signalled periods of food availability or non-availability varied from one procedure to the next, and were sometimes key colors, sometimes tones, and sometimes compounds of both. Key pecking was initiated and maintained when key color was a signal for food; key pecking was not initiated when a tone was the signal for food. However, control of key pecking that was already established could be transferred from key color to tone, and subsequently, initiated by the tone. It is suggested that for pigeons, pre-experimental relationships exist among food, visual stimuli, and pecking, and that a similar relationship, which includes auditory stimuli, must be induced in the laboratory.     

The relative law of effect: effects of shock intensity on response strength in multiple schedules

Key pecking of four birds was reinforced with food according to a two-component multiple variable-interval 1-minute variable-interval 4-minute schedule. In addition, key pecking was punished by a brief shock according to a variable-interval 30-second schedule during both components of the multiple schedule. The intensity of the shock was varied. For all birds, punishment had a stronger suppressive effect on the responding maintained by the leaner food schedule, and the ratio of responding during the two components of the multiple schedule became closer to the ratio of reinforcement as shock intensity was increased, as the relative law of effect predicts. At the higher shock intensity, there was some evidence that the ratio of responses overmatched the ratio of reinforcements.