Reinforcement by an imprinting stimulus versus water on simple schedules in ducklings

Ducklings (5 to 28 days old) were trained to peck a pole on fixed-ratio, fixed-interval, and multiple schedules using brief presentation of an imprinting stimulus as the response-contingent event. Other ducklings of the same age were trained similarly except that reinforcement consisted of access to water. With water reinforcement the typical fixed-ratio (“break-run”), fixed-interval (“scallop”), and multiple schedule response patterns were readily established and consistently maintained. With the imprinting stimulus these schedule effects were inconsistent in some subjects and virtually nonexistent in others, despite extended training. Schedule control with the imprinting stimulus was not improved by the use of a reinforcement signaling procedure which enhances responding reinforced by electrical brain stimulation on intermittent schedules. However, the overall rates of responding and the extinction functions generated after reinforcement with water versus the imprinting stimulus were comparable. These findings imply that control by temporal and discriminative stimuli may be relatively weak when a young organism's behavior is reinforced by presentation of an imprinting stimulus.     

Conditioned suppression, punishment, and aversion

Three experiments were conducted to assess the aversive properties of a visual stimulus in the presence of which one group of birds received response-contingent shock (discriminated punishment) while a yoked group of birds received non-contingent shocks (conditioned suppression). In Experiment 1, presentation of the visual stimulus contingent on key pecking reduced the response rate (conditioned punishment effect) for birds under the conditioned suppression procedure but did not reduce the response rate of birds under the discriminative punishment procedure. Non-contingent shocks also produced greater suppression of responding maintained by positive reinforcement in the presence of a visual stimulus than did response-contingent shocks. In Experiment 2, a greater shock intensity (2 mA) was used. All the differences between the two groups found in Experiment 1 were also found in Experiment 2. Experiment 3 demonstrated that response-contingent shock did not result in a conditioned punishment effect even when positive reinforcers were unavailable during the discriminative punishment schedule. The exteroceptive stimulus that was paired with shock in the conditioned suppression procedure acquired the ability to punish behavior. The exteroceptive stimulus in the discriminative punishment schedule did not acquire this ability.     

The effects of punishment intensity on squirrel monkeys

Responses of squirrel monkeys were maintained by a variable-interval schedule of food reinforcement. Concurrently, punishment consisting of a brief electric shock followed each response. As has been found for pigeons and rats, punishment did not produce extreme, all-or-none reactions. By gradually increasing the punishment intensity it was possible to produce response rates intermediate to no suppression and complete suppression. Similarly, the moment-to-moment response rate was free of extreme fluctuations. A “warm-up” effect occurred in which the punished responses were especially suppressed during the initial part of a session. The pre-punishment performance was negatively accelerated within a session, and punishment reduced the degree of negative acceleration. When punishment was discontinued, responding recovered immediately except when suppression had been complete or prolonged. When the punishment intensity was decreased gradually, more suppression resulted at a given intensity than when intensity was increased gradually. This suggests a “behavioral inertia” effect wherein behavior at a new punishment intensity is biased toward the behavior at the previous value. A corollary generalization is that the larger the change in intensity, the less the behavior at the new value will be biased toward the behavior at the previous value.     

Comparing the effects of two correction procedures on human acquisition of sequential behavior patterns

Thirty-one college undergraduates learned to touch abstract stimuli on a computer screen in arbitrarily designated “correct” sequential orders. Four sets of seven stimuli were used; the stimuli were arrayed horizontally on the screen in random sequences. A correct response (i.e., touching first the stimulus designated as first) resulted in that stimulus appearing near the top of the screen in its correct sequential position (left to right), and remaining there until the end of the trial. Incorrect responses (i.e., touching a stimulus out of sequence) terminated the trial. New trials displayed either the same sequence as the one on which an error had occurred (same-order correction procedure), or a new random sequence (new-order correction procedure). Whenever all responses occurred in the correct sequence, the next trial displayed a new random sequence. Each phase ended when five consecutive correct response sequences occurred. Initially, the same-order correction procedure increased control by the position as well as by the shape of the stimuli; also, it produced more errors, more total trials, more trials to mastery, and more individual patterns of reacquisition than were produced by the new-order procedure.     

Partial avoidance contingencies: Absolute omission and punishment probabilities

Avoidance contingencies were defined by the absolute probability of the conjunction of responding or not responding with shock or no shock. The “omission” probability (ρ₀₀) is the probability of no response and no shock. The “punishment” probability (ρ₁₁) is the probability of both a response and a shock. The traditional avoidance contingency never omits shock on nonresponse trials (ρ₀₀=0) and never presents shock on response trials (ρ₁₁=0). Rats were trained on a discrete-trial paradigm with no intertrial interval. The first lever response changed an auditory stimulus for the remainder of the trial. Shocks were delivered only at the end of each trial cycle. After initial training under the traditional avoidance contingency, one group of rats experienced changes in omission probability (ρ₀₀>0), holding punishment probability at zero. The second group of rats were studied under different punishment probability values (ρ₁₁>0), holding omission probability at zero. Data from subjects in the omission group looked similar, showing graded decrements in responding with increasing probability of omission. These subjects approximately “matched” their nonresponse frequencies to the programmed probability of shock omission on nonresponse trials, producing a very low and approximately constant conditional probability of shock given no response. Subjects in the punishment group showed different sensitivity to increasing absolute punishment probability. Some subjects decreased responding to low values as punishment probability increased, while others continued to respond at substantial levels even when shock was inevitable on all trials (noncontingent shock schedule). These results confirm an asymmetry between two dimensions of partial avoidance contingencies. When the consequences of not responding included occasional omission of shock, all subjects showed graded sensitivity to changes in omission frequency. When the consequences of responding included occasional shock delivery, some subjects showed graded sensitivity to punishment frequency while others showed control by overall shock frequency as well.     

Avoidance of risk as a determinant of cooperation

Pairs of subjects could either cooperate or respond on a lower paying individual task. Whenever both subjects chose to cooperate, either subject could make a response that took $1.00 of the other's earnings. In Exp. I, a stimulus signalled when a “take” response had been made. Either subject could avoid the loss by switching to the individual task within 5 sec after the stimulus appeared. Rates of cooperation were high when losses could be avoided but decreased again when the avoidance condition was removed. In Exp. II, a response prevented “takes” from occurring for a specified time interval after the response. This procedure also maintained cooperation. When each avoidance response subtracted from earnings, both avoidance responding and cooperation were eliminated.     

Punishment: the interactive effects of delay and intensity of shock

A discrete-trial punishment procedure, with rats, was used to examine how delay-of-shock intervals of 0 to 28 sec and shock intensity interact to decrease the frequency and increase the latency of a positively reinforced response. For delay-of-shock intervals of 0, 7, 14, and 28 sec, there was a range of shock intensities, for some subjects, over which the punishing effect of shock was an increasing, monotonic function of shock intensity. For other subjects this transition was abrupt. Functions relating response frequency and latency measures to shock intensity were displaced toward higher values on the shock intensity axis with an increase in delay-of-shock interval. The effects of “gradual” and “abrupt” introduction to “severe” shock, as well as re-exposure to previously used shock intensities, were examined under both the immediate and delay-of-shock conditions. With delay-of-shock intervals of 7, 14, or 28 sec, shock intensities of approximately 0.50 milliamperes or greater were necessary to decrease substantially the number and increase the latency of the lever-pressing response. For the immediate punishment group this intensity was approximately 0.20 ma. These facts were related to Annau and Kamin's (1961) conditioned emotional response experiment in which a shock intensity of 0.49 ma or greater was required to suppress the rate of a positively reinforced response.     

Recognition by the pigeon of stimuli varying in two dimensions

Pigeons served in four experiments, each of which involved about 44,000 discrete 1.2-sec trials under steady-state conditions. The first experiment scaled a short segment of the visual wavelength continuum; this dimension was then combined in a conditional discrimination with each of three others; time after reinforcement, tone frequency, and line tilt. In the two-stimulus experiments, the birds' responses were reinforced in the presence of only one stimulus combination: “582 nm” together with “2 min after reinforcement”, “3990 Hz”, or “vertical line”. Many other stimulus combinations also appeared equally often and went without reinforcement. The wavelength stimuli conformed to an equal-interval scale, and per cent response was generally linear with wavelength, when scaled on cumulative normal coordinates. The components of the compound stimulus were found to interact in a multiplicative fashion; when one component differed greatly from its reinforcement value, changes in the other component had relatively little effect. For the “time”-“wavelength” compound, this interaction appeared to be modified by the effects of set or attention. Certain response latency data are reported, and other combination rules are discussed.     

The associative relation underlying autoshaping in the pigeon

Fifteen pigeons were exposed to either response-independent or response-dependent schedules of water reinforcement, whereby water was injected directly into the unrestrained pigeons' mandibles. Key-contact responses were released by a lighted key correlated with water, but not by a lighted key uncorrelated with water. A negative response-reinforcer contingency suppressed autoshaped key-contact responses, resulting in responding directed away from the lighted key. In all pigeons, water injected directly into the mandibles elicited a consummatory fixed-action pattern of “mumbling” and swallowing. The lighted key correlated with water released a broader set of both appetitive and consummatory responses: approach to the lighted key, “bowing”, “rooting”, “mumbling”, and swallowing. Key-contact responses were “rooting” and “mumbling” motions of the beak on the surface of the key. Views of autoshaping based on stimulus substitution or stimulus surrogation do not fully explain the origin of autoshaped responses not previously elicited by the reinforcer. The present findings are consonant with views of conditioning that emphasize the large degree of biological pre-organization in conditioned response patterns, and the importance of associative factors in the control of such patterns.     

RESPONDING MAINTAINED UNDER INTERMITTENT SCHEDULES OF ELECTRIC-SHOCK PRESENTATION: “SAFETY” OR SCHEDULE EFFCTS?

Four experiments were conducted in which lever pressing by squirrel monkeys was maintained under multiple, mixed, or chained schedules of electric-shock presentation. In the first two experiments, a multiple schedule was employed in which a fixed-interval schedule of shock presentation alternated with a signaled two-minute component. Initially, no events were scheduled during the two-minute component (a safety period). In the first experiment, the safety period was “degraded” by introducing and systematically increasing the frequency of periodic shocks presented during that component. In the second experiment, the proportion of overall safe time to unsafe time was decreased by decreasing the value of the fixed-interval schedule while holding constant shock frequency during the two-minute component. In the third experiment, the overall arrangement was changed from a multiple to a mixed schedule in an attempt to determine whether fixed-interval responding would be maintained when a single exteroceptive stimulus was associated with both components. In the fourth experiment, the overall arrangement was changed from a multiple to a chained schedule in an effort to determine whether fixed-interval responding would be maintained when its consequence was presentation of a signaled “unsafe” period. Fixed-interval responding was well maintained under all experimental conditions; the varied relationships obtained lend more support to conceptualizations of shock-maintained behavior as exemplifying schedule-controlled behavior than to suggestions that such behavior may be readily accounted for by “safety theory.”     

The effect of overtraining on behavioral contrast and the peak-shift

Following initial discrimination training between two wavelength stimuli and a subsequent generalization test to the wavelength dimension, Group 1 was “overtrained” for 105 days on the original discrimination. Group 2 was “overtrained” with the original positive stimulus and a new negative stimulus, a white line. Group 3 was “overtrained” with the original negative stimulus and a new positive stimulus, the white line. Each 15 days of extended training were followed by a wavelength generalization test similar to the first test. The results suggest that there is no consistent relationship between the response rate in positive stimulus immediately before the generalization test and whether or not a peak shift occurs during the test.     

Active role of the shock-avoidance contingency in shuttlebox-avoidance learning in Fischer344 rats

Different groups of Fischer344 rats acquired and maintained shuttlebox-avoidance responses under one, two, all three, or none of the shock-avoidance, warning-signal (WS) termination, and shock-escape contingencies. The intertrial interval was fixed at either 30 or 60 s. Removal of the shock-avoidance contingency led to a marked disruption of avoidance behavior at both intervals. This result was not produced by a selective punishment effect of the avoidance response by an inevitable shock. Thus, the shock-avoidance contigency provided particularly effective control over the avoidance learning. The WS termination contingency appeared to have little overall effect at the 60-s intertrial interval. It is suggested that response-contingent stimulus change served only as a discriminative cue for the absence of a shock.     

Stimulus control of avoidance behavior

The introduction of a warning signal preceding shocks greatly increased the effectiveness of avoidance responding. Periods of “warm-up” at the beginning of the session were eliminated, and the number of shocks received by the subjects was greatly reduced. With response-shock interval constant, response rate increased as the interval between the response and the onset of the warning signal was shortened. The response tended to occur shortly after the onset of the warning signal regardless of the duration of these “safe” periods. A greatly elevated response rate was maintained even when the duration of the safe period was reduced to 0.3 sec. Thus, the pre-shock signal obtained nearly exclusive control of the responding and overrode the usual “temporal discrimination” of the response-shock interval.     

Species differences in temporal control of behavior II: human performance

Human subjects responded on two panels. A differential-reinforcement-of-low-rate schedule with a limited-hold contingency operated on Panel A. In Condition 1, responses on Panel B produced a stimulus on the panel that signalled whether reinforcement was available on Panel A. In Condition 2, responses on Panel B briefly illuminated a digital clock. In both conditions, performance on Panel A was very efficient; with few exceptions, Panel A was pressed only when reinforcement was available. Thus, in effect, a fixed-interval schedule operated on Panel B. In Condition 1, a “break-and-run” response pattern occurred on Panel B; with increasing temporal parameters, the duration of the postreinforcement pause on Panel B increased linearly while overall response rate and running rate (calculated by excluding the postreinforcement pauses) remained approximately constant. In Condition 2, the response pattern on Panel B was scalloped; the postreinforcement pause was a negatively accelerated increasing function of schedule value, while overall response rate and running rate were negatively accelerated decreasing functions of schedule value. The performance of subjects in Condition 2, but not in Condition 1, was highly sensitive to the contingencies in operation, and resembled that of other species on the fixed-interval schedule.     

Auto-maintenance in the pigeon: sustained pecking despite contingent non-reinforcement

If a response key is regularly illuminated for several seconds before food is presented, pigeons will peck it after a moderate number of pairings; this “auto-shaping” procedure of Brown and Jenkins (1968) was explored further in the present series of four experiments. The first showed that pecking was maintained even when pecks turned off the key and prevented reinforcement (auto-maintenance); the second controlled for possible effects of generalization and stimulus change. Two other experiments explored procedures that manipulated the tendency to peck the negatively correlated key by introducing alternative response keys which had no scheduled consequences. The results indicate that pecking can be established and maintained by certain stimulus-reinforcer relationships, independent of explicit or adventitious contingencies between response and reinforcer.     

Timing in free-operant and discrete-trial avoidance

A procedure (“discrete-trial” avoidance) was devised to differentiate between the two main theories of responding in Sidman's “free-operant” avoidance procedure. One theory, a version of two-factor theory, holds that responding is reinforced by the removal of a conditioned aversive stimulus. The conditioned aversive stimulus is held to be temporal, which accounts for the spaced responding, or timing, that Sidman's procedure produces. The other theory holds that the reinforcement for both responding and timing is shock-frequency reduction. The new procedure eliminated this reinforcement for timing, but retained the conditions for the formation of conditioned aversive temporal stimuli. According to one theory, the new procedure should have sustained timing as well as Sidman's, while according to the other, it should have sustained no timing. The results confirmed neither theory. Timing was found with both procedures, but unequally in degree and kind. Large variations in the precision of timing did not appear to be correlated with successful avoidance for either procedure.     

The operant control of vocalization in the dog

Control over the vocal responses of three dogs was established using operant-conditioning procedures. Several points of interest were observed in the data. First, fixed-ratio schedules of reinforcement generated a vocal response topography which was similar in detail to that of a “motor” bar-nosing response. Second, vocal responding was brought under the control of external visual stimuli as a result of differential reinforcement. Third, good stimulus control was maintained on a multiple schedule containing a vocal-response component and a bar-response component. Fourth, the stimulus control on the multiple schedule transferred with minimal disruption to a chain schedule requiring a sequence of 10 bar responses followed by 10 vocal responses. Fifth, because vocal and bar responses are not mutually exclusive, concurrent responding tended to develop on the chain schedule.

These results were discussed with reference to the advisability of applying the terms operant and respondent to unconditioned behavior, and, particularly, to unconditioned verbal behavior.

     

Conditioned reinforcement and discrimination in second-order schedules

Pigeons were exposed to multiple second-order schedules in which responding on the “main key” was reinforced according to either a variable-interval or fixed-interval schedule by production of a brief stimulus on the “brief-stimulus key”. A response was required to the brief stimulus during its fourth (final) presentation to produce food; responses to the earlier brief stimuli indicated the extent to which the final brief stimulus was discriminated from preceding ones. Main-key response rates were higher in early components of paired brief-stimulus schedules, in which each brief stimulus was the same as that paired with reinforcement, than in comparable unpaired brief-stimulus or tandem schedules. Poor discrimination occurred between paired brief stimuli (Experiment I). When chain stimuli on the main key induced a discrimination between the first two and second two brief stimuli, the response-rate enhancement in the paired brief-stimulus schedule persisted (Experiment II). Rate enhancement diminished when the initial link of the chain included the first three components (Experiment IV). Eliminating the contingency between responding and brief-stimulus production also diminished rate enhancement (Experiment III). The results show that the discriminative and conditioned reinforcing effects of food-paired brief stimuli may be selectively manipulated and suggest that the reinforcing effects are modulated by other reinforcers in the situation.     

Prepositional phrases spoken and heard

The relation between verbal and nonverbal behavior with common syntactic properties was investigated, using retarded and nonretarded children. Reinforcement was contingent on either verbal or nonverbal responses whereas responses of the other repertoire had no experimental consequences. Changes sometimes occurred in the unreinforced (collateral) repertoire, but they were always changes in the stimulus control of pre-existing topographies. A contingency involving responses of one repertoire never instated new topographies in the collateral repertoire. This suggested that the problem of “cross-modality generalization” should be reformulated to distinguish explicitly between instating new topographies and changing the stimulus control of pre-existing topographies. The result confirmed Skinner's hypothesis about “the same response spoken and heard” and clarified some anomalies in previous studies.     

Conditioned suppression of free-operant avoidance

The responses of white rats were maintained on an unsignalled free-operant avoidance schedule. Superimposed on the avoidance schedule was a blinking white light followed immediately by response-independent electric shock. Duration of the light stimulus was either 1 or 3 min. Avoidable shock was 1.5 mA; response-independent shock was 7.5 mA. Suppression of responding during the light stimulus (both durations) developed over sessions. Responding immediately following the response-independent light-shock sequence was neither suppressed nor accelerated. The similarity is noted between the present result and findings of “positive conditioned suppression”.