Discriminative functions based on a delay in the reinforcement relation

Pigeons were shown to come under discrimination control when the S^D and S^Δ were temporally separated from reinforcement and non-reinforcement. S^D and S^Δ consisted of distinctive key illuminations presented separately. Responding on an FR 5 in the presence of S^D or S^Δ produced a third stimulus containing a schedule requirement. If this third (or interpolated) stimulus was preceded by S^D, responding in its presence produced reinforcement followed by a time-out (TO). If, on the other hand, the third stimulus was preceded by S^Δ, responding produced TO alone. In this fashion, the same stimulus and the same response requirement were imposed between S^D and the reinforcement as between S^Δ and the TO. In Experiment I, the schedule employed during the interpolated stimulus was FR; in Experiment II, FI. Discrimination reversal was accomplished in both experiments.     

The effect of multiple SΔ periods on responding on a fixed-interval schedule: III. Effect of changes in pattern of interruptions, parameters and stimuli

Pigeons responding under fixed-interval schedules of reinforcement were interrupted by S^Δ periods during the course of the intervals. Whether intervals were interrupted by 1, 2, or 5 S^Δ periods, the general scalloped pattern of FI responding persisted. Parameter values up to 27¾ hr for the FI and 2¾ hr for the individual S^Δ interruptions were studied. The results further weaken the hypothesis that the FI pattern of responding depends crucially on control of responding by continuously chained mediating behavior.     

The effect of multiple SΔ periods on responding on a fixed-interval schedule

The effect of repeated interruption of FI responding by short S^Δ presentations on the pattern of increasing frequency of responding through the interval has been studied. Although the S^Δ profoundly changed the pattern of responding during their presentation, the general scalloped pattern of FI responding survived. The implication of these findings for understanding the role of chaining of responses in FI patterns is discussed. It is suggested that chaining is not a necessary condition for the scalloped pattern.     

The role of instrumental responding and contiguity of stimuli in the development of infant secondary reinforcement

Ten-month-old infants received contingent pairings of a tone (T+) and food reinforcer. Groups S^r and S^D received the food on an FI 23-sec schedule for target touching, the former group receiving T+ immediately after the response and 1.5 sec prior to food and the latter group receiving T+ at the end of the intertrial interval. Group SC received food reinforcers 1.5 sec after T+ with no response required. A second tone (Tⁿ) was heard by all groups once during each intertrial interval, at randomly determined points. All groups subsequently were given a spatial discrimination task, receiving T+ for one alternative and Tⁿ for the other. Group S^r gave significantly more responses for T+ than for Tⁿ, but neither of the other two groups produced a superiority for T+. Thus, both contiguity with a primary reinforcer and the presence of an operant during training appear to be necessary for a neutral signal to acquire the ability to enhance responding.     

Generalization and discrimination as a function of the SD — SΔ intensity difference

Three groups of four rats were trained on an auditory-intensity discrimination for 21 days. The S^D-S^Δ intensity difference for Group I was 10 db; for Group II, 20 db; and for Group III, 30 db. Following the initial discrimination training, the animals were tested for generalization of the bar-press response to seven novel S^Δ's which were presented intermingled with the original S^D and S^Δ values. Conclusions: (1.) The amount of simple discrimination training required to obtain fairly stable differences in S^D and S^Δ responding is an inverse function of the magnitude of the stimulus difference between S^D and S^Δ. (2.) Generalization gradients obtained immediately following simple discrimination training exhibit a maximum displaced from S^D in a direction also away from S^Δ. (3.) Gradients obtained following continued exposure to the multivalued S^Δ situation show a fairly stable maximum at the S^D value. (4.) Although the gradients tend to fall off systematically on either side of the continuum as distance from S^D is increased, they decrease most rapidly on the S^Δ limb of the gradient.     

Stress-induced breakdown of an appetitive discrimination

Rats trained to discriminate between S^D and S^Δ for food reinforcement showed marked impairments in this discrimination when strong, unavoidable shocks occurred at the termination of a third stimulus. The predominant feature of this impairment was a supernormal rate of unreinforced (S^Δ) behavior. Shocks delivered without exteroceptive warning also led to a discriminative breakdown. The effect was a direct function of shock intensity. When behavior was strongly suppressed in the third stimulus by response-correlated shock (“punishment”), instead of unavoidable shock, breakdowns were only temporary; as soon as responding recovered from its overall suppression, discriminative performance returned to normal. The discriminative deterioration may be interpreted as an emotional by-product of frequent aversive stimulation, but accidental contingencies could also have played a role.     

Effects of a DRL contingency added to a fixed-interval reinforcement schedule

Following 30 days of reinforcement for the bar press response of two white rats on 30-sec fixed-interval (FI), a DRL component was added so that a minimal interresponse time (IRT) for the reinforced response, in addition to the FI variable, was necessary for reinforcement. Marked control over response rate by the superimposed DRL requirement was demonstrated by an inverse hyperbolic function as the DRL component was increased from 1 to 24 sec within the constant 30-sec FI interval. Interresponse time and post-reinforcement (post-S^R) “break” distributions taken at one experimental point (DRL = 24 sec) suggested that a more precise temporal discrimination was initiated by an S^R than by a response, since the relative frequency of a sequence of two reinforced responses appeared greater than that of a sequence of a non-reinforced response followed by a reinforced one. This latter finding was confirmed with new animals in a follow-up experiment employing a conventional 24-sec DRL schedule.     

Some notes on time out from reinforcement

Pigeons produced food on a fixed-ratio schedule by pecking at one key, and an S^Δ period by pecking at a second (switching) key. Switching behavior was examined as a function of (a) size of the fixed ratio, (b) whether the S^Δ was of fixed duration or could be determined by the bird, (c) the introduction of a novel food S^D, (d) extinction of food responding, and (e) the stimuli associated with the S^D and S^Δ conditions. No monotonic relationship was obtained between ratio size and switching behavior. Switching behavior was, however, influenced by many variables. The results suggest that an interpretation of switching behavior in terms of its being reinforced by the removal of aversive conditions, is open to considerable question.     

The effects of delayed reinforcement on free-operant responding

In previous studies of delayed reinforcement, response rate has been found to vary inversely with the response-reinforcer interval. However, in all of these studies the independent variable, response-reinforcer time, was confounded with the number of reinforcers presented in a fixed period of time (reinforcer frequency). In the present study, the frequency of available reinforcers was held constant, while temporal separation between response and reinforcer was independently manipulated. A repeating time cycle, T, was divided into two alternating time periods, t^D and t^Δ. The first response in t^D was reinforced at the end of the prevailing T cycle and extinction prevailed in t^Δ. Two placements for t^D were defined, an early t^D placement in which t^D precedes t^Δ and a late t^D placement in which t^D follows t^Δ. The duration of the early and late t^D was systematically decreased from 30 seconds (i.e., t^D = T) to 0.1 second. Manipulation of t^D placement and duration controlled the temporal separation between response and reinforcement, but it did not affect the frequency of programmed reinforcers, which was 1/T. The results show that early and late t^D placements of equal duration have similar overall effects upon response rate, reinforcer frequency, responses per reinforcer, and obtained response-reinforcer temporal separation. A stepwise regression analysis using log response rate as the dependent variable showed that the obtained delay was a significant first-step variable for six of eight subjects, with obtained reinforcer frequency significant for the remaining two subjects.     

Discrimination learning as a function of stimulus location along an auditory intensity continuum

Eight groups of rats were trained on an auditory intensity discrimination in which the discriminative stimuli were separated by 10 decibels (db). Four pairs of stimuli were selected from different regions along a 60–100 db (SPL) intensity continuum. Counterpart groups were trained on each stimulus pair, with the relative intensity positions of the reinforced stimulus (S^D) and the non-reinforced stimulus (S^Δ) reversed for the two groups. Discrimination acquisition curves were compared to determine whether stimuli separated by equal logarithmic units were of comparable “difficulty”, and to determine the relative effectiveness of an S^D serving as the more versus less intense member of a stimulus pair. It was concluded that: (1) When S^D is the more intense, auditory intensities of constant logarithmic separation are graded in “difficulty” along the intensity continuum; high intensity discriminative stimuli are most readily discriminated. When S^Δ is the more intense, this graded effect is not evident. (2) For a given continuum location, discrimination is inferior when S^Δ is the more intense. This effect is most pronounced at the high intensity end of the continuum and is chiefly attributable to differences in the rate of S^Δ responding.     

The effects of unavoidable shocks on a multiple schedule having an avoidance component

Two dogs were maintained on a multiple schedule having both a food reinforced and an avoidance component (Mult VI 1′ S^Δ Avoid_{SS20 RS20} S^Δ). The effects of superimposing an Estes-Skinner procedure for delivering unavoidable shocks on all components of the multiple schedule were observed. The buzzer-shock pairing of the Estes-Skinner procedure produced an increased rate of responding on the avoidance component of the schedule and also on the S^Δ components. No persistent change in rate was observed on the food component during the pre-shock stimulus. Control performances on all components could be regained by either extinguishing or eliminating the buzzer-shock pairing. Extinction of the avoidance responding had little effect on the increased rates of responding produced by the Estes-Skinner procedure on the S^Δ and avoidance extinction components and did not lead to a conditioned suppression of the food reinforced responding. Rate of responding during the pre-shock stimulus was observed to be relatively independent of changes in the maintaining schedules. Responding during the pre-shock stimulus could be conditioned and maintained after an extensive history of avoidance extinction.     

Antecedent reinforcement contingencies in the stimulus control of an auditory discrimination

In order to assess possible confounding of discriminative stimulus effects with those produced by the reinforcing stimulus, three groups of four rats each were trained for 45 hr on a variable-interval 1-min reinforcement program. Two groups were run on a multiple variable-interval extinction schedule in which the reinforcement stimulus (S^D) and the nonreinforcement stimulus (S^Δ) were two intensities of a 4-kHz (cps) tone separated by 40 or 10 db. The third group was run on a mixed schedule with a single intensity constantly present. The mixed-schedule animals showed no discrimination of the reinforcement program. Under the multiple schedule, the highest S^Δ rates were obtained after S^D intervals, regardless of the reinforcement availability in the S^D interval. These local rate variations in S^Δ were small in proportion to those produced by the S^D versus S^Δ intensities.     

The dissociation of discriminative and conditioned reinforcing functions of stimuli with changes in deprivation

Pigeons were studied in two experiments designed to explore the effects of deprivation level upon responding in each link of a two-link chained schedule. The stimulus associated with the terminal link of the chain can be both a discriminative stimulus (S^D) for responding in the presence of the stimulus and a conditioned reinforcer (S^r) for responding in the preceding link. Previous findings have indicated that the S^r function was more readily weakened by satiation than was the S^D function, i.e., the rate of responding decreased more rapidly in the initial link of the chain than in the terminal link. The first of the present experiments, in which tests were conducted after a series of sessions, produced different results: rates of responding in the two links declined simultaneously. The second experiment supported the hypothesis that the effects of satiation interact with the duration of maintenance on the satiation procedure: in early sessions the S^r function was more readily disrupted, but in later sessions the rates of responding in the two links declined simultaneously. Subsequent to this extensive series of identical sessions, the pigeons' deprivation level was altered before a session by pre-feeding the pigeons up to their normal post-session weights. The rates of responding failed to reflect fully this change in deprivation in the first such session, suggesting that the pigeons' behavior had become partially independent of deprivation level.     

Operant control of vocal behavior in the cat

Evidence of operant control of vocal behavior in the cat is presented: (1) On mult FR 12 S^Δ schedule, cats miaowed rapidly during periods of S^D and much less or not at all during S^Δ. (2) This control was re-established following reversal of stimuli. (3) The frequency distribution of response durations was shifted to both shorter and longer values by the differential reinforcement of shorter or longer response durations respectively. Since both the frequency and duration of vocal responses were shown to be under the control of the schedule of reinforcement, it is concluded that at least some of the vocal behavior of the cat is susceptible to operant control.     

Effect of prior Pavlovian discrimination training upon learning an operant discrimination

The effect of Pavlovian discrimination training with two stimuli upon subsequent learning of an operant discrimination involving those stimuli was studied. After preliminary lever press training, the lever was removed and thirsty rats received noncontingent pairings between S₁ (a tone or a clicker) and water reinforcements, whereas S₂ (a clicker or a tone) occurred always without reinforcement. This procedure presumably established S₁ as a positive CS for respondent behavior, whereas S₂ was established as an inhibitory CS. Following this training, the lever was reintroduced and the rats were trained on an operant (lever pressing) discrimination involving S₁ and S₂. For the Consistent Ss, S₁ was the S^D and S₂ the S^Δ in the operant discrimination; for the Reversed Ss, S₂ served as S^D and S₁ as S^Δ. The Consistent Ss learned the operant discrimination significantly faster than did the Reversed Ss. The result emphasizes the importance of respondents, conditioned to S^D and S^Δ, which modulate operant performance to these stimuli.     

Conditioning history and human fixed-interval performance

The effects of two different conditioning histories upon human operant behavior under a FI 10-sec schedule of point reinforcements were investigated. Relatively high rates of continuous inter-reinforcement responding characterized the FI 10-sec performance of subjects with an FR 40 conditioning history. Low response rates with little or no inter-reinforcement responding were emitted by Ss with a DRL 20-sec conditioning history.     

Behavioral effects of pairing an SD with a decreasing limited-hold reinforcement schedule

Four pigeons were trained on a multiple reinforcement schedule consisting of two limited-hold schedules, one in which a discriminative stimulus (S^D) accompanied the periodic reinforcement contingency, and one in which the discriminative stimulus was omitted. The duration of the limited-hold in each component of the multiple schedule was reduced in parallel steps. It was shown that behavioral differences between the two schedules were attenuated by this manipulation of temporal parameters. When S^D was reduced in duration, three out of four pigeons responded with extremely high S^Δ rates, despite the regular pairing of S^Δ with the reinforcement contingency. These high rates qualitatively resembled the rapid rates emitted on the analogous no-S^D component.     

The effect of multiple SΔ periods on responding on fixed-interval schedule: II. in a primate

A squirrel monkey was subjected to a fixed-interval pattern of reinforcement. During the course of each interval a bright white light was repeatedly presented. In the presence of the white light, a response was never immediately followed by food; the white light thus functioned as S^Δ. Responding was interrupted during the S^Δ periods, but in the squirrel monkey as in the pigeon, these interruptions did not destroy the characteristic scalloped pattern of the cumulatively recorded responding through each interval.     

Pavlovian conditioned stimulus effects upon instrumental choice behavior are reinforcer specific

In a transfer-of-control experiment with rats, Pavlovian CSs were tested for the specificity of their effects. The instrumental behavior consisted of a discriminative, conditional two-lever choice task in which qualitatively different appetitive reinforcers were contingent upon the two correct choices. In a Pavlovian phase, subjects experienced conditioning to establish either a CS⁺ or CS^? for one reinforcer or a CS⁺ or CS^? for the other reinforcer. Finally, in a test, these CSs were presented when there was the opportunity to make choice responses. The CS⁺s evoked choices of the lever which had eventuated in the reinforcer that had served as the Pavlovian US, while the CS^?s showed only a slight tendency to evoke the other choice responses. When the CSs were compounded with the original S^Ds, the CS⁺s had little effect upon the vigor of responding while the CS^?s reduced the vigor of responding to the S^D for the reinforcer that was the same as the US used in establishing the CS^?. The results are discussed in terms of associative mediational theory and the reinforcer specificity of Pavlovian conditioned excitation and inhibition.     

Rate changes after unscheduled omission and presentation of reinforcement

Changes in response rate similar to frustration effects were studied in a two-lever situation. Responding on one lever on a fixed-interval schedule produced access to water for 5 sec and an exteroceptive stimulus. In the presence of this stimulus, responding on another lever on a fixed-interval schedule produced access to water for 5 sec and terminated the stimulus. Occasional omission of a previously scheduled reinforcer after responding on the first lever resulted consistently in increases in rate on the second lever during the immediately succeeding interval. In another procedure, occasional presentation of a previously unscheduled reinforcer after responding on the first lever resulted consistently in decreases in rate on the second lever during the immediately succeeding interval. Changes occurred after the first omissions or presentations and were about the same in magnitude as the procedure continued over several sessions. Typically, an increase or decrease in rate was maintained throughout an entire 100-sec interval. Changes in rate on the second lever of approximately the same magnitude also occurred when rate on the first lever was near-zero under a schedule that differentially reinforced behavior other than lever pressing.