Discriminative functions based on a delay in the reinforcement relation

Pigeons were shown to come under discrimination control when the S^D and S^Δ were temporally separated from reinforcement and non-reinforcement. S^D and S^Δ consisted of distinctive key illuminations presented separately. Responding on an FR 5 in the presence of S^D or S^Δ produced a third stimulus containing a schedule requirement. If this third (or interpolated) stimulus was preceded by S^D, responding in its presence produced reinforcement followed by a time-out (TO). If, on the other hand, the third stimulus was preceded by S^Δ, responding produced TO alone. In this fashion, the same stimulus and the same response requirement were imposed between S^D and the reinforcement as between S^Δ and the TO. In Experiment I, the schedule employed during the interpolated stimulus was FR; in Experiment II, FI. Discrimination reversal was accomplished in both experiments.     

Behavioral effects of pairing an SD with a decreasing limited-hold reinforcement schedule

Four pigeons were trained on a multiple reinforcement schedule consisting of two limited-hold schedules, one in which a discriminative stimulus (S^D) accompanied the periodic reinforcement contingency, and one in which the discriminative stimulus was omitted. The duration of the limited-hold in each component of the multiple schedule was reduced in parallel steps. It was shown that behavioral differences between the two schedules were attenuated by this manipulation of temporal parameters. When S^D was reduced in duration, three out of four pigeons responded with extremely high S^Δ rates, despite the regular pairing of S^Δ with the reinforcement contingency. These high rates qualitatively resembled the rapid rates emitted on the analogous no-S^D component.     

Food-paired stimuli as conditioned reinforcers: effects of d-amphetamine.

Seven pigeons were studied in two experiments in which key pecks were reinforced under a second-order schedule wherein satisfaction of variable-interval schedule requirements produced food or a brief stimulus. In the second part of each session, responses produced only the brief stimulus according to a variable-interval schedule (food extinction). For the 4 pigeons in Experiment 1, the response key was red throughout the session. In separate phases, the brief stimulus was either paired with food, not paired with food, or not presented during extinction. d-Amphetamine (0.3 to 10.0 mg/kg) dose-dependently reduced food-maintained responding during the first part of the session and, at intermediate dosages, increased responding during the extinction portion of the session. The magnitude of these increases, however, did not consistently depend on whether the brief stimulus was paired, not paired, or not presented. It was also true that under nondrug conditions, response rates during extinction did not differ reliably depending on pairing operations for the brief stimulus. In Experiment 2, 3 different pigeons responded under a procedure wherein the key was red in the component with food presentations and blue in the extinction component (i.e., multiple schedule). Again, d-amphetamine produced dose-related decreases in responding during the first part of a session and increases in responding in the second part of the session. These increases, however, were related to the pairing operations; larger increases were observed when the brief stimulus was paired with food than when it was not or when it was not presented at all. Under nondrug conditions, the paired brief stimulus controlled higher response rates during extinction than did a nonpaired stimulus or no stimulus. These findings suggest that d-amphetamine can enhance the efficacy of conditioned reinforcers, and that this effect may be more robust if conditioned reinforcers occur in the context of a signaled period of extinction.     

Antecedent reinforcement contingencies in the stimulus control of an auditory discrimination

In order to assess possible confounding of discriminative stimulus effects with those produced by the reinforcing stimulus, three groups of four rats each were trained for 45 hr on a variable-interval 1-min reinforcement program. Two groups were run on a multiple variable-interval extinction schedule in which the reinforcement stimulus (S^D) and the nonreinforcement stimulus (S^Δ) were two intensities of a 4-kHz (cps) tone separated by 40 or 10 db. The third group was run on a mixed schedule with a single intensity constantly present. The mixed-schedule animals showed no discrimination of the reinforcement program. Under the multiple schedule, the highest S^Δ rates were obtained after S^D intervals, regardless of the reinforcement availability in the S^D interval. These local rate variations in S^Δ were small in proportion to those produced by the S^D versus S^Δ intensities.     

Some notes on time out from reinforcement

Pigeons produced food on a fixed-ratio schedule by pecking at one key, and an S^Δ period by pecking at a second (switching) key. Switching behavior was examined as a function of (a) size of the fixed ratio, (b) whether the S^Δ was of fixed duration or could be determined by the bird, (c) the introduction of a novel food S^D, (d) extinction of food responding, and (e) the stimuli associated with the S^D and S^Δ conditions. No monotonic relationship was obtained between ratio size and switching behavior. Switching behavior was, however, influenced by many variables. The results suggest that an interpretation of switching behavior in terms of its being reinforced by the removal of aversive conditions, is open to considerable question.     

Conditioned suppression of an avoidance response by a stimulus paired with food

Three food-deprived Long-Evans rats were exposed to a non-discriminated shock avoidance procedure. Superimposed upon this operant avoidance baseline were periodic presentations of a conditioned stimulus that was paired with food, the unconditioned stimulus. These pairings resulted in increases in the rate of shock over that recorded when the conditioned stimulus was not present. A traditional suppression ratio failed to reveal any differential effect of the conditioned stimulus on the overall rate of avoidance responding, although all subjects showed a consistent pattern of pausing and postshock response bursts during presentations of the conditioned stimulus. When food was withheld during a final extinction phase, the conditioned stimulus ceased to occasion increases in shock rates and disruptive postshock response bursts were eliminated. An analysis of conditioned suppression procedures is proposed that stresses not only operant-Pavlovian or appetitive-aversive incompatibility, but also the manner in which the baseline schedule of reinforcement affects operant behavior changes that are elicited by the superimposed Pavlovian procedure.     

Interactions between the discriminative and aversive properties of punishment

Punishment acquires a discriminative property when it is selectively paired with either reinforcement or extinction. At the milder punishment intensities, the discriminative control exerted by punishment is similar to the discriminative control exerted by a response-produced neutral (nonaversive) stimulus. However, the effect of the aversive property is apparent as the intensity of the punishment is increased. The aversive property of the punishment acts to enhance the discriminative control when the punishment is selectively applied during extinction periods, and to attenuate the discriminative control when the punishment is selectively applied during reinforcement periods. One major difference was found between the control exerted by the punishment and the response-produced neutral simulus: Responding greatly increased after the S^Δ punishment but not after the S^Δ neutral stimulus; this increase in responding was independent of the punishment intensities studied.     

Second-order schedules and the problem of conditioned reinforcement

Thirteen pigeons were exposed to a variety of second-order schedules in which responding under a component schedule was reinforced according to a schedule of reinforcement. Under different conditions, completion of each component resulted in either (1) the brief presentation of a stimulus also present during reinforcement (pairing operation), (2) the brief presentation of a stimulus not present during reinforcement (nonpairing operation), or (3) no brief stimulus presentation (tandem). Brief-stimulus presentations engendered a pattern of responding within components similar to that engendered by food. Patterning was observed when fixed-interval and fixed-ratio components were maintained under fixed- and variable-ratio and fixed- and variable-interval schedules. There were no apparent differences in performance under pairing and nonpairing conditions in any study. The properties of the stimuli presented in brief-stimulus operations produced different effects on response patterning. In one study, similar effects on performance were found whether brief-stimulus presentations were response-produced or delivered independently of responding. Response patterning did not occur when the component schedule under which a nonpaired stimulus was produced occurred independently of the food schedule. The results suggest a reevaluation of the role of conditioned reinforcement in second-order schedule performance. The similarity of behavior under pairing and nonpairing operations is consistent with two hypotheses: (1) the major effect is due to the discriminative properties of the brief stimulus; (2) the scheduling operation under which the paired or nonpaired stimulus is presented can establish it as a reinforcer.     

Maintenance and suppression of responding under schedules of electric shock presentation

In squirrel monkeys previously trained under a continuous avoidance schedule, characteristic patterns of responding were maintained under a 3-min variable-interval schedule of shock presentation (response-produced shock). Responding in the presence of a periodically presented stimulus, the termination of which coincided with the delivery of a response-independent electric shock (Estes-Skinner procedure), was not reliably affected. When shocks followed every response during certain signalled portions of the session, and were presented under the variable-interval schedule during the rest of the session (multiple 1-response fixed-ratio, 3-min variable-interval schedule of shock presentation), responding was suppressed during the fixed-ratio component and maintained during the variable-interval component. Environmental consequences do not have immutable properties, and may either support or suppress behavior, depending on the schedule of presentation.     

Stress-induced breakdown of an appetitive discrimination

Rats trained to discriminate between S^D and S^Δ for food reinforcement showed marked impairments in this discrimination when strong, unavoidable shocks occurred at the termination of a third stimulus. The predominant feature of this impairment was a supernormal rate of unreinforced (S^Δ) behavior. Shocks delivered without exteroceptive warning also led to a discriminative breakdown. The effect was a direct function of shock intensity. When behavior was strongly suppressed in the third stimulus by response-correlated shock (“punishment”), instead of unavoidable shock, breakdowns were only temporary; as soon as responding recovered from its overall suppression, discriminative performance returned to normal. The discriminative deterioration may be interpreted as an emotional by-product of frequent aversive stimulation, but accidental contingencies could also have played a role.     

Solution of the intermediate size problem by pigeons

Pigeons learned to respond to the middle-sized member (S^D) of a set of three simultaneously presented stimuli with responses to the S^D reinforced on a VI 1 schedule. They were then tested for several days with other sets of three stimuli. One procedure presented reinforcements on a VI 1 schedule during the test independent of the stimulus chosen when a reinforcement was programmed. The tests were also given under extinction conditions. With the testing carried out with extinction, preference consistently was for the test stimulus most similar in physical size to the S^D. However, when the tests were with reinforcement, random responding resulted. Another effect of testing with reinforcement was an increase in incorrect responding with the training set. Such a test procedure was unsatisfactory for determining the effective aspect of the S^D. The conclusion, based on the data of the extinction series, was that pigeons learned the intermediate size problem on the basis of the discrimination of absolute stimulus properties.     

Pavlovian phenomena in conditioned acceleration—Stimulus summation

Four naive male rats were separately fear-conditioned to two stimuli (light and sound) by pairing these stimuli with shocks. During Sidman avoidance these unreinforced stimuli elicited increases in response rates (conditioned acceleration). In a multiple schedule with light-shock and sound-shock reinforcements in the first component, the rats were tested for summation effects in the second (avoidance) component by presenting the light twice, the sound twice and the compound twice. They were then extinguished by eliminating the shocks in the first component. The effects of compounding were evidenced by: (1) a sizeable and reliable amplitude increase to the compound stimulus during conditioning; (2) a smaller and less consistent decrease in latencies during conditioning in all four subjects; and (3) a greater resistance to extinction for the compound stimulus as measured by mean amplitudes for all four subjects and on six of seven extinction sessions. Latency data during extinction were inconclusive. An additional unexpected finding was what appeared to be a two-factor secondary extinction of the avoidance itself as a result of Pavlovian extinction. The responding in the first four minutes of avoidance was inhibited to such an extent that the number of shocks received during this period increased 700 per cent from the last three conditioning sessions to the first three extinction sessions.     

The effects of single-component extinction of a three-component serial CS on resistance to extinction of the conditioned avoidance response

Central to a fear interpretation of how avoidance responses are maintained in the absence of further CS-UCS pairings is the underlying assumption of an existing gradient of fear across the CS-UCS interval. Extrapolations based on this gradient lead to a number of differential predictions concerning the topography of avoidance responding during extinction. The present research was concerned with the differential effects of extinguishing separate components of the CS complex upon responding to the complete CS complex during extinction. In Phase 1 of the study, rats were classically conditioned to a three-component serial CS (S₁/S₂/S₃) followed by shock. Each subject was then given avoidance training in a one-way apparatus to a criterion of one successful avoidance. In Phase 2, subjects were divided into four groups, with three of the groups receiving nonreinforced exposure for 25 trials to one of the components of the serial CS (S₁, S₂, or S₃). The fourth group (S₀) was exposed for the same period of time to the apparatus cues. In Phase 3, the total stimulus complex was reintroduced in its original order, and animals were tested until extinction of the instrumental response was reached. The results are consistent with the hypothesis that a fear gradient exists in extinction and decreases in magnitude as the distance from the point of UCS onset increases.     

Discrimination learning as a function of stimulus location along an auditory intensity continuum

Eight groups of rats were trained on an auditory intensity discrimination in which the discriminative stimuli were separated by 10 decibels (db). Four pairs of stimuli were selected from different regions along a 60–100 db (SPL) intensity continuum. Counterpart groups were trained on each stimulus pair, with the relative intensity positions of the reinforced stimulus (S^D) and the non-reinforced stimulus (S^Δ) reversed for the two groups. Discrimination acquisition curves were compared to determine whether stimuli separated by equal logarithmic units were of comparable “difficulty”, and to determine the relative effectiveness of an S^D serving as the more versus less intense member of a stimulus pair. It was concluded that: (1) When S^D is the more intense, auditory intensities of constant logarithmic separation are graded in “difficulty” along the intensity continuum; high intensity discriminative stimuli are most readily discriminated. When S^Δ is the more intense, this graded effect is not evident. (2) For a given continuum location, discrimination is inferior when S^Δ is the more intense. This effect is most pronounced at the high intensity end of the continuum and is chiefly attributable to differences in the rate of S^Δ responding.     

Generalization and discrimination as a function of the SD — SΔ intensity difference

Three groups of four rats were trained on an auditory-intensity discrimination for 21 days. The S^D-S^Δ intensity difference for Group I was 10 db; for Group II, 20 db; and for Group III, 30 db. Following the initial discrimination training, the animals were tested for generalization of the bar-press response to seven novel S^Δ's which were presented intermingled with the original S^D and S^Δ values. Conclusions: (1.) The amount of simple discrimination training required to obtain fairly stable differences in S^D and S^Δ responding is an inverse function of the magnitude of the stimulus difference between S^D and S^Δ. (2.) Generalization gradients obtained immediately following simple discrimination training exhibit a maximum displaced from S^D in a direction also away from S^Δ. (3.) Gradients obtained following continued exposure to the multivalued S^Δ situation show a fairly stable maximum at the S^D value. (4.) Although the gradients tend to fall off systematically on either side of the continuum as distance from S^D is increased, they decrease most rapidly on the S^Δ limb of the gradient.     

Factors influencing responding under multiple schedules of conditioned and unconditioned reinforcement

Two experiments examined pigeons' responses under multiple schedules of conditioned and unconditioned reinforcement. In one component, responses produced food according to a fixed-interval schedule; in a second component, responses produced brief stimuli according to a fixed-ratio schedule. When brief-stimulus presentations were paired with food in the first component, rates in the second component were usually higher than 10 responses per minute. When pairing in the first component was eliminated, responding continued to be maintained in the second component. Elimination of food presentation from the first component substantially decreased responding in the second component, even though the brief stimulus had not been paired with food. Experiment II demonstrated that response rate was affected by the duration of both the second component and the brief stimulus. The results suggest that three conditions are important in maintaining responding with brief-stimulus presentations: (1) pairing the brief stimulus, at least initially, with food, (2) maintaining unconditioned reinforcement in one component, and (3) employing optimal brief-stimulus and component durations.     

OBSERVING RESPONSES AND SERIAL STIMULI: SEARCHING FOR THE REINFORCING PROPERTIES OF THE S−

The control exerted by a stimulus associated with an extinction component (S−) on observing responses was determined as a function of its temporal relation with the onset of the reinforcement component. Lever pressing by rats was reinforced on a mixed random-interval extinction schedule. Each press on a second lever produced stimuli associated with the component of the schedule in effect. In Experiment 1 a response-dependent clock procedure that incorporated different stimuli associated with an extinction component of a variable duration was used. When a single S− was presented throughout the extinction component, the rate of observing remained relatively constant across this component. In the response-dependent clock procedure, observing responses increased from the beginning to the end of the extinction component. This result was replicated in Experiment 2, using a similar clock procedure but keeping the number of stimuli per extinction component constant. We conclude that the S− can function as a conditioned reinforcer, a neutral stimulus or as an aversive stimulus, depending on its temporal location within the extinction component.     

The effect of multiple SΔ periods on responding on a fixed-interval schedule: III. Effect of changes in pattern of interruptions, parameters and stimuli

Pigeons responding under fixed-interval schedules of reinforcement were interrupted by S^Δ periods during the course of the intervals. Whether intervals were interrupted by 1, 2, or 5 S^Δ periods, the general scalloped pattern of FI responding persisted. Parameter values up to 27¾ hr for the FI and 2¾ hr for the individual S^Δ interruptions were studied. The results further weaken the hypothesis that the FI pattern of responding depends crucially on control of responding by continuously chained mediating behavior.     

Average uncertainty as a determinant of observing behavior

After discrimination training on a multiple variable-interval extinction schedule of food reinforcement, pigeons were placed on the uncued or mixed version of the same schedule and allowed to make an optional “observing response” that converted the uncued schedule to the corresponding cued schedule by providing a 20-sec exposure to the appropriate discriminative stimulus. The schedule consisted of one hundred 40-sec components, and the probability that any one of them would be a variable-interval component was systematically varied between 0.00 and 1.00. The results showed that the amount of observing behavior was an inverted “U” function of the probability of the variable-interval component. Few observing responses occurred at probabilities of 0.00 or 1.00, and maximum responding occurred at a value less than 0.50.     

Effects of conditioned reinforcement frequency in an intermittent free-feeding situation,

Key-pecking intermittently produced a set of brief exteroceptive stimulus changes under two-component multiple schedules of conditioned reinforcement. Throughout the study, free access to grain was concurrently provided on an intermittent basis via a variable-interval tape. Free food presentations scheduled by the tape were delivered if no peck had been emitted for 6 sec, and the brief stimulus changes produced by responding under the multiple schedules were those which accompanied food presentation. The second component of each multiple schedule was always associated with a 1-min, variable-interval schedule of conditioned reinforcement. The schedule associated with the first component was systematically varied and conditioned reinforcement was either absent (extinction) or programmed on a 1-, 3-, 6-, or 12-min variable-interval schedule. Under these conditions, rate of responding in the manipulated component decreased monotonically with a decrease in the frequency of conditioned reinforcement. In addition, contrast effects were often obtained in the constant, second component. These results are similar to those obtained with similar multiple schedules of primary reinforcement.