Duration-reduction of avoidance sessions as negative reinforcement

Five rats were exposed to a shock-postponement procedure in which responses on each of two levers initially had equivalent effects. After an initial training sequence that ensured at least some responding on each lever, an additional consequence was made conjointly operative on the previously less-preferred lever for each animal. Each response on this lever continued to postpone shock, but also reduced the session duration by one minute. The conjoint contingencies were operative until, through session-shortening responses and the passage of time, the session was scheduled to end in two minutes; during the final two minutes the session-shortening contingency was disabled while the shock-postponement contingency continued to be operative on both levers. When responding shifted to a predominance on the session-shortening lever, the conjoint contingency was shifted to the other lever; for four of the five rats this reversal was followed by two additional reversals. Two of the rats' responding showed clear, strong, and unambiguous sensitivity to the session-shortening contingency. The responding of two others was also systematically controlled by that contingency, but the effects were less clearcut. The fifth animal showed an initial shift when session-shortening was introduced, but its subsequent behavior proved insensitive to reversals of procedure. The results clearly indicate a sensitivity of behavior to events on a time scale quite distinct from that of immediate consequences. They also support an interpretation of avoidance sessions, considered in their entirety, as events whose contingent relationship to behavior can affect that behavior—even in the absence of stimuli that delineate those relationships. Finally, these results support an interpretation of aversively based conditioning within a broader context, analogous to the “open versus closed economy” interpretation of appetitively controlled behavior.     

Avoidance of risk as a determinant of cooperation

Pairs of subjects could either cooperate or respond on a lower paying individual task. Whenever both subjects chose to cooperate, either subject could make a response that took $1.00 of the other's earnings. In Exp. I, a stimulus signalled when a “take” response had been made. Either subject could avoid the loss by switching to the individual task within 5 sec after the stimulus appeared. Rates of cooperation were high when losses could be avoided but decreased again when the avoidance condition was removed. In Exp. II, a response prevented “takes” from occurring for a specified time interval after the response. This procedure also maintained cooperation. When each avoidance response subtracted from earnings, both avoidance responding and cooperation were eliminated.     

Some factors that influence the acquisition of complex, stereotyped, response sequences in pigeons

Two pigeons were required to peck six to nine illuminated response keys. A response on any one of the keys darkened that key. When each key had been darkened, a reinforcer was delivered. No specific sequence of key pecking was ever required. The keys were presented in various matrices: three by two, three by three, horizontal rows, and vertical columns. The keys either presented the same stimulus, white light; or each key presented a different stimulus, a color or form. The results indicated that although there were 720 to 362,880 different sequences that would produce reinforcement, each bird developed a particular, stereotyped sequence that dominated its behavior. Variability among the birds across phases yielded less than 60 sequences, .0001 to 6 percent of the possible sequences. The data suggest that a reinforcement contingency that includes “free choice” of response sequence will produce stereotypical response sequences that function as complex “units” of behavior.     

Effects of response variability on the sensitivity of rule-governed behavior

Two experiments examined the relation between response variability and sensitivity to changes in reinforcement contingencies. In Experiment 1, two groups of college students were provided complete instructions regarding a button-pressing task; the instructions stated “press the button 40 times for each point” (exchangeable for money). Two additional groups received incomplete instructions that omitted the pattern of responding required for reinforcement under the same schedule. Sensitivity was tested in one completely instructed and one incompletely instructed group after responding had met a stability criterion, and for the remaining two groups after a short exposure to the original schedule. The three groups of subjects whose responding was completely instructed or who had met the stability criterion showed little variability at the moment of change in the reinforcement schedule. The responding of these three groups also was insensitive to the contingency change. Incompletely instructed short-exposure responding was more variable at the moment of schedule change and was sensitive to the new contingency in four of six cases. In Experiment 2, completely and incompletely instructed responding first met a stability criterion. This was followed by a test that showed no sensitivity to a contingency change. A strategic instruction was then presented that stated variable responding would work best. Five of 6 subjects showed increased variability after this instruction, and all 6 showed sensitivity to contingency change. The findings are discussed from a selectionist perspective that describes response acquisition as a process of variation, selection, and maintenance. From this perspective, sensitivity to contingency changes is described as a function of variables that produce response variability.     

Partial avoidance contingencies: Absolute omission and punishment probabilities

Avoidance contingencies were defined by the absolute probability of the conjunction of responding or not responding with shock or no shock. The “omission” probability (ρ₀₀) is the probability of no response and no shock. The “punishment” probability (ρ₁₁) is the probability of both a response and a shock. The traditional avoidance contingency never omits shock on nonresponse trials (ρ₀₀=0) and never presents shock on response trials (ρ₁₁=0). Rats were trained on a discrete-trial paradigm with no intertrial interval. The first lever response changed an auditory stimulus for the remainder of the trial. Shocks were delivered only at the end of each trial cycle. After initial training under the traditional avoidance contingency, one group of rats experienced changes in omission probability (ρ₀₀>0), holding punishment probability at zero. The second group of rats were studied under different punishment probability values (ρ₁₁>0), holding omission probability at zero. Data from subjects in the omission group looked similar, showing graded decrements in responding with increasing probability of omission. These subjects approximately “matched” their nonresponse frequencies to the programmed probability of shock omission on nonresponse trials, producing a very low and approximately constant conditional probability of shock given no response. Subjects in the punishment group showed different sensitivity to increasing absolute punishment probability. Some subjects decreased responding to low values as punishment probability increased, while others continued to respond at substantial levels even when shock was inevitable on all trials (noncontingent shock schedule). These results confirm an asymmetry between two dimensions of partial avoidance contingencies. When the consequences of not responding included occasional omission of shock, all subjects showed graded sensitivity to changes in omission frequency. When the consequences of responding included occasional shock delivery, some subjects showed graded sensitivity to punishment frequency while others showed control by overall shock frequency as well.     

Competitive fixed-interval performance in humans

Two persons responded in the same session in separate cubicles, but under a single schedule of reinforcement. Each time reinforcement was programmed, only the first response to occur, that is, the response of only one of the subjects, was reinforced. “Competitive” behavior that developed under these conditions was examined in three experiments. In Experiment 1 subjects responded under fixed-interval (FI) 30-s, 60-s, and 90-s schedules of reinforcement. Under the competition condition, relative to baseline conditions, the response rates were higher and the pattern was “break-and-run.” In Experiment 2, subjects were exposed first to a conventional FI schedule and then to an FI competition schedule. Next, they were trained to respond under either a differential-reinforcement-of-low-rate (DRL) or fixed-ratio (FR) schedule, and finally, the initial FI competition condition was reinstated. In this second exposure to the FI competition procedure, DRL subjects responded at lower rates than were emitted during the initial exposure to that condition and FR subjects responded at higher rates. For all subjects, however, responding gradually returned to the break-and-run pattern that had occurred during the first FI competition condition. Experiment 3 assessed potential variables contributing to the effects of the competitive FI contingencies during Experiments 1 and 2. Subjects were exposed to FI schedules where (a) probability of reinforcement at completion of each fixed interval was varied, or (b) a limited hold was in effect for reinforcement. Only under the limited hold was responding similar to that observed in previous experiments.     

MMPI correlates of performance on a behavior therapy ward.

In a behavioral treatment program for acute psychiatric patients, points were earned for adaptive behavior (e.g. self-care, attending ward activities) and lost for maladaptive behavior (e.g. assaults, verbal abuse). Points earned could be spent for a variety of goods and services (e.g. passes, extra staff time). Statistically significant correlations were found between MMPI scale scores and point-earning behavior. High scores on the F, 5, 6 and 8 scales were associated with low point gain for adaptive behavior, high point loss for maladaptive behavior, a high proportion of points spent to points earned, and a low overall net point earnings. Low score on F scale in combination with high score on 2 scale best predicted point-gain behavior, whereas high score on 8 scale in combination with low score on 1 scale best predicted point-loss behavior. Overall net points were best predicted by low score on F scale in combination with high scores on 0 and 9 scales. When subjects were grouped into common psychiatric profile types, differences were found in point-gain behaviors for items related to personal care and attending ward activities. At least some of these differences could be attributed to two factors: high scores on the 2, 3 and 7 scales were associated with higher than average point earnings, while high scores on the 8 scale were associated with lower than average point earnings.     

Escape from an effortful situation

This experiment investigated the tendency to escape from a situation requiring effortful responding. Five human subjects responded in a situation where the response mechanism required 20-lb force to operate; responses were reinforced according to a variable-interval schedule. A subject escaped from this situation by emitting a vocal response which produced a 60-sec “easy period”. During the easy period the reinforcement contingency was switched to a response mechanism requiring 1 lb to operate. It was found that: (1) Escape responding could be conditioned and maintained by producing the easy period; the easy period did not maintain escape responding when the force requirement in the normal situation was equated with it. (2) The rate of escape responding was a function of the magnitude of the force normally required. (3) When easy periods were scheduled after fixed ratios, pausing from the end of the previous easy period to the first escape response was noted. It was concluded that a situation requiring high-force responding is a negative reinforcer. The pattern of fixed-ratio responding suggests that this reinforcer produces typical schedule control in human subjects.     

Bias in self-evaluation: Signal probability effects

Two experiments examined apparent signal probability effects in simple verbal self-reports. After each trial of a delayed matching-to-sample task, young adults pressed either a “yes” or a “no” button to answer a computer-presented query about whether the most recent choice met a point contingency requiring both speed and accuracy. A successful matching-to-sample choice served as the “signal” in a signal-detection analysis of self-reports. Difficulty of matching to sample, and thus signal probability, was manipulated via the number of nonmatching sample and comparison stimuli. In Experiment 1, subjects exhibited a bias (log b) for reporting matching-to-sample success when success was frequent, and no bias or a bias for reporting failure when success was infrequent. Contingencies involving equal conditional probabilities of point consequences for “I succeeded” and “I failed” reports had no systematic effect on this pattern. Experiment 2 found signal probability effects to be evident regardless of whether referent-response difficulty was manipulated in different conditions or within sessions. These findings indicate that apparent signal probability effects in self-report bias that were observed in previous studies probably were not an artifact of contingencies intended to improve self-report accuracy or of the means of manipulating signal probability. The findings support an analogy between simple self-reports and psychophysical judgments and bolster the conclusion of Critchfield (1993) that signal probability effects can influence simple self-reports much as they do reports about external stimuli in psychophysical experiments.     

Determination of a behavioral transfer function: White-noise analysis of session-to-session response-ratio dynamics on concurrent VI VI schedules

Six pigeons were exposed to concurrent variable-interval schedules in which the programmed reinforcer ratios changed from session to session according to a pseudorandom binary sequence. This procedure corresponded to the stochastic identification paradigm (“white-noise experiment”) of systems theory and enabled the relation between log response ratios in the current session and log reinforcer ratios in all previous sessions to be determined. Such dynamic relations are called linear transfer functions. Both nonparametric and parametric representations of these, in the form of “impulse-response functions,” were determined for each bird. The session-to-session response ratios resulting from the session-to-session pseudorandom binary variations in reinforcer ratios were well predicted by the impulse-response functions identified for each pigeon. The impulse-response functions were well fitted by a second-order dynamic model involving only two parameters: a time constant and a gain. The mean time constant was 0.67 sessions, implying that the effects of abrupt changes in log reinforcer ratios should be 96% complete within about five sessions. The mean gain was 0.53, which was surprisingly low inasmuch as it should equal the sensitivity to reinforcement ratio observed under steady-state conditions. The same six pigeons were subjected to a similar experiment 10 months following the first. Despite individual differences in impulse-response functions between birds within each experiment, the impulse-response functions determined from the two experiments were essentially the same.     

Repeated acquisition in the analysis of rule-governed behavior

Five children, ranging in age from 3½ years to 5½ years, were taught various four-response chains using conditioned reinforcement. Experiment 1 investigated the effects of presenting “instruction” stimuli—a sequence of lights over the correct response buttons—to assess their role in facilitating the acquisition of a chain of responses. Without the “instruction” stimuli, children made many errors before responses were brought under the control of the programmed contingencies. When confronted with the same contingencies later in the day, these subjects made fewer errors. In contrast, in the presence of the “instruction” stimuli, subjects made virtually no errors. However, when the “instruction” stimuli were discontinued in the subsequent session, all 5 subjects made errors. In Experiment 2, the subjects were taught to verbalize the contingencies during the phase without the “instruction” stimuli. This resulted in errorless performance during the subsequent exposure to the same procedure, but errors nevertheless occurred again during reexposure to the procedure with the “instruction” stimuli discontinued.     

Rule-governed behavior and sensitivity to changing consequences of responding

Humans were presented with a task that required moving a light through a matrix. Button presses could produce light movements according to a multiple fixed-ratio 18/differential-reinforcement-of-low-rate 6-s schedule, with components alternating every 2 min. Moving the light through the maze earned points worth chances on money prizes. In Experiment 1 four conditions were assessed through between-subject comparisons: minimal instructions, instructions to press rapidly, instructions to press slowly, and instructions that sometimes rapid responding would work while at other times a slow rate would work best. Subjects responded in three successive sessions of 32 min each. The results suggested that instructions affected the nature of the contact made with the programmed consequences and thus subsequent performance. In some cases, responding seemed to result from added contingencies introduced by stating rules. In Experiment 2 the relative contribution of these two effects was assessed by presenting and then withdrawing two lights that had been paired with two specific instructions: “Go Fast” or “Go Slow.” There were three conditions. In one condition, only the Go Fast light was on; in a second, only the Go Slow light was on; and in a third, the lights alternated each minute. In each condition, half the subjects had all instruction lights turned off after the first session. The results once again showed an effect of instructions on contact with the programmed consequences. However, responding sometimes continued in a manner consistent with added contingencies for rule-following even when the programmed consequences had been contacted and would have controlled a different type of responding in the absence of instructions. The relevance of added contingencies for rule-following in determining the effects of explicitly programmed consequences is emphasized.     

Contrast and reallocation of extraneous reinforcers between multiple-schedule components

Four pigeons responded in components of multiple schedules in which two responses were available and reinforced with food. Pecks on the left key (“main” key) were reinforced at a constant rate in one component and at a rate that varied over conditions in the other component. When reinforcer rate was varied, behavioral contrast occurred in the constant component. On the right key (“extra” key), five variable-interval schedules and one variable-ratio schedule, presented conjointly, arranged reinforcers for responses in all conditions. These conjoint schedules were common to both multiple-schedule components—rather than unique to particular components—and reinforcers from these schedules could therefore be arranged in one component and obtained during the other component. In this way, the additional reinforcers were analogous to the “extraneous” reinforcers thought to maintain behavior other than pecking in conventional multiple schedules. Response rate on the extra key did not change systematically over conditions in the constant component, and in the varied component extra responding was inversely related to main-key reinforcement. All subjects obtained more extra-key reinforcers in whichever component arranged fewer main-key reinforcers. Consistent with the theory that reallocation of extraneous reinforcers may cause behavioral contrast, absolute reinforcer rate for the extra key in the constant component was low in conditions that produced positive contrast on the main key and high in those that produced negative contrast. Also consistent with this theory, behavioral contrast was reduced in two conditions that canceled extra-key reinforcers that had been arranged but not obtained at the end of components. Thus, a constraint on reallocation markedly reduced the extent of contrast.     

College students' responding to and rating of contingency relations: The role of temporal contiguity

Two experiments investigated the role of temporal contiguity in college students' responding to and rating of contingency relations during operant conditioning. Schedules were devised that determined when but not whether appetitive or aversive events would occur. Subjects' reports concerning the schedules were obtained by means of a 200-point rating scale, anchored by the phrases “prevents the light from occurring” (−100) and “causes the light to occur” (+100). When tapping a telegraph key advanced the time of point gain, responding was maintained or increased and subjects gave positive ratings. When tapping a telegraph key advanced the time of point loss, subjects also gave positive ratings, but responding now decreased. When key tapping delayed the time of point gain, responding decreased and subjects gave negative ratings. When key tapping delayed the time of point loss, subjects also gave negative ratings, but responding now increased. These findings implicate response-outcome contiguity as an important contributor to causal perception and to reinforcement and punishment effects. Other accounts—such as those stressing the local probabilistic relation between response and outcome or the molar correlation between response rate and outcome rate—were seen to be less preferred interpretations of these and other results.     

Effect of reinforcement duration on fixed-interval responding

Five different reinforcement durations occurred randomly within each session on fixed interval 60-sec. Postreinforcement pause was directly related (and “running” rate inversely related) to the duration of reinforcement initiating each fixed interval.     

Feedback effects on sequential ordering in humans

Under various feedback conditions, 38 college undergraduates were asked to rearrange abstract graphic characters on a computer screen, placing them in arbitrarily designated “correct” sequences. Two sets of seven horizontally arrayed stimuli were used. In Experiment 1, subjects in Group 1 learned to arrange the first set under Selection Feedback in which a “+” appeared above each character after it was selected in the correct order and to arrange the second set under Order Feedback in which a correct response produced a copy of the character in its correct ordinal position at the top of the screen. For Group 2 the order of these conditions was reversed. In Experiment 2, for subjects in Group 3, correct responses produced neither of these types of feedback. Subjects in Group 4 received Order Feedback only until the first set was correctly ordered once. Order Feedback was more effective than Selection Feedback during initial acquisition of the first set but not during maintenance; no differences were found for the second set. Only 2 of 9 subjects successfully put the characters in correct sequential order under the No Feedback condition. When, in Experiment 2, Order Feedback was eliminated after the first correctly arranged sequence, the steady-state criteria were met more slowly than in Experiment 1.     

Unbiased and unnoticed verbal conditioning: the double agent robot procedure

Subjects who were told they were “experimenters” attempted to reinforce fluent speech in a supposed subject with whom they spoke via intercom. The supposed subject was to say nouns, one at a time, on request by the “experimenter”, who reinforced fluent pronunciation with points. Actually, the “experimenter” was talking to a multi-track tape recording, one track of which contained fluently spoken nouns, the other track containing disfluently spoken nouns. If the “experimenter's” request for the next noun was in a specified form a word from the fluent track was played to him as reinforcement; requests in any other form produced the word from the disfluent track. Repeated conditioning of specific forms of requests was accomplished with two subject-“experimenters,” who were unable to describe changes in their own behavior, or the contingencies applied. This technique improved upon an earlier method that had yielded similar results, but was less thoroughly controlled against possible human bias.     

Stimulus control of avoidance behavior

The introduction of a warning signal preceding shocks greatly increased the effectiveness of avoidance responding. Periods of “warm-up” at the beginning of the session were eliminated, and the number of shocks received by the subjects was greatly reduced. With response-shock interval constant, response rate increased as the interval between the response and the onset of the warning signal was shortened. The response tended to occur shortly after the onset of the warning signal regardless of the duration of these “safe” periods. A greatly elevated response rate was maintained even when the duration of the safe period was reduced to 0.3 sec. Thus, the pre-shock signal obtained nearly exclusive control of the responding and overrode the usual “temporal discrimination” of the response-shock interval.     

Species differences in temporal control of behavior II: human performance

Human subjects responded on two panels. A differential-reinforcement-of-low-rate schedule with a limited-hold contingency operated on Panel A. In Condition 1, responses on Panel B produced a stimulus on the panel that signalled whether reinforcement was available on Panel A. In Condition 2, responses on Panel B briefly illuminated a digital clock. In both conditions, performance on Panel A was very efficient; with few exceptions, Panel A was pressed only when reinforcement was available. Thus, in effect, a fixed-interval schedule operated on Panel B. In Condition 1, a “break-and-run” response pattern occurred on Panel B; with increasing temporal parameters, the duration of the postreinforcement pause on Panel B increased linearly while overall response rate and running rate (calculated by excluding the postreinforcement pauses) remained approximately constant. In Condition 2, the response pattern on Panel B was scalloped; the postreinforcement pause was a negatively accelerated increasing function of schedule value, while overall response rate and running rate were negatively accelerated decreasing functions of schedule value. The performance of subjects in Condition 2, but not in Condition 1, was highly sensitive to the contingencies in operation, and resembled that of other species on the fixed-interval schedule.     

Acquisition and maintenance of trusting behavior

This study determined whether a two-person exchange situation contained natural contingencies for trusting behavior or whether external contingencies were necessary. Pairs of college students worked matching-to-sample problems for money. On each trial there was one problem and the subjects determined which of them would solve it. Trusting behavior was defined as an increase in the number of consecutive problems each subject allowed his partner to work during sessions that also ended with an equitable distribution. Simply, trust was a temporary deviation from equity. A subject could give the problem to the other person (cooperate), or not respond and let the other person take the problem (share). Other possibilities were for both subjects to try to take the problem (complete), or for neither subject to respond and thereby let the person who worked the last problem also work the next one (passive trust). When only four lever pulls were required to distribute a problem (no external contingencies to reach either equity or trust) subjects reached equity, but only minimal trust (strict alternation of single problems) developed in 18 sessions. When 30 or 60 lever pulls were required to distribute a problem (smaller response requirement for passive trust and therefore a contingency for trust), trusting behavior developed after a few sessions (fixed ratio 30) or after several trials of the first session (fixed ratio 60) and it ordinarily expanded gradually to 10 to 15 consecutive problems through passive trust. The aversiveness of the inequity involved in trusting appears to necessitate a contingency for acquisition. Once trust develops, however, this aversiveness is reduced as subjects learn the inequity is only temporary (e.g., once trust was acquired at fixed ratio 60 it was maintained at fixed ratio 4, which would not initially produce it), and the direction of the inequity appears to become of questionable importance (e.g., being behind was alternated over rather than within sessions and usually not in a systematic manner).