Effects of pairing stimuli with reinforcement on multiple schedule performance of children

A nonsense word was paired with reinforcement to determine if pairing affected emission of a response that produced the word in the signalled absence of reinforcement. Children were trained on a multiple schedule that consisted of a reinforcement component, conditioned reinforcement component, and control component, each set of contingencies being signalled by a different colored light. In the primary reinforcement component, lever presses produced reinforcers which, in some phases, were paired with a word. In the other two components, lever presses were not reinforced and a button was made accessible. Button presses in the conditioned reinforcement component produced the word to be (or being) paired, e.g., "yafeh", while button presses in the control component produced another word, e.g., "grunch". Button pressing increased when one of the words was being paired and decreased when pairing was discontinued, but directly related rate changes occurred also in the control component. The order of components was shown to be a contributing variable.     

Concurrent responses during multiple schedules in rats

Abstract.— Rats' lever pressing was reinforced with a sucrose solution. In the presence of a tone pressing one lever was reinforced on a FR 40 schedule, in the absence of the tone pressing another lever was reinforced on a FR 20 schedule. The components alternated every fourth minute. In both components, the duration of pauses in lever pressing after reinforcement was positively correlated with the duration of licking the empty sucrose-delivery mechanism. In the FR 40 component, the duration of pauses was also positively correlated with the number of presses on the non-reforced lever. Licking and non-reinforced lever pressing were decreased when the rate of reinforced lever pressing was high. The response interactions were similar to those described for concurrent reinforcement schedules.     

Deprivation and satiation: The interrelations between food and wheel running

Two experiments were designed to assess whether depriving rats of food would increase the reinforcement effectiveness of wheel running (Experiment 1) and whether satiation for wheel running would decrease the reinforcement effectiveness of food (Experiment 2). In Experiment 1, a progressive-ratio schedule was used to measure the reinforcement effectiveness of wheel running when rats were deprived or not deprived of food. Completion of a fixed number of lever presses released a brake on a running wheel for 60 s, and the response requirement was systematically increased until the rat stopped pressing or until 8 hr had elapsed. The ratio value reached (and the total number of lever presses) was an inverted-U function of food deprivation (percentage body weight). In Experiment 2, when wheel running preceded test sessions, fewer food-reinforced lever presses were maintained by the progressive-ratio schedule, and responding occurred at a lower rate on a variable-interval schedule. An interpretation of these results is that deprivation or satiation with respect to one event (such as food) alters the reinforcement effectiveness of a different event (such as access to wheel running).     

Economic substitutability of electrical brain stimulation, food, and water.

Concurrent variable-ratio schedules of electrical brain stimulation, food, and water were paired in various combinations as reinforcement of rats' lever presses. Relative prices of the concurrent reinforcers were varied by changing the ratio of the response requirements on the two levers. Economic substitutability, measured by the sensitivity of response ratio to changes in relative price, was highest with brain stimulation reinforcement of presses on both levers and lowest with food reinforcement of presses on one lever and water reinforcement of presses on the other. Substitutability with brain stimulation reinforcement of presses on one lever and either food or water reinforcement for presses on the other was about as high as with brain stimulation for presses on both levers. Electrical brain stimulation for rats may thus serve as an economic substitute for two reinforcers, neither of which is substitutable for the other.     

Time-based and count-based measurement of preference

Rats' pressing on two levers was reinforced according to two independent variable-interval schedules that were varied during the experiment. Since the levers were connected directly to the programming equipment, bypassing the standard pulseformers, reinforcement could occur while a lever was held down. Although the time a lever was pressed might, therefore, have varied independently of number of presses, these two measures covaried substantially, because the average duration of the presses remained roughly constant. This rough invariance may have resulted from the rats' tendency to make bursts of brief presses (i.e., to jiggle the levers), even though the contingencies encouraged holding. When duration did vary, presses on the two levers tended to vary together. As a result, relative time spent pressing corresponded closely to relative number of presses. Both of these measures conformed well to the matching law. Absolute behavioral frequency at a lever, measured either way, varied directly with proportion of reinforcement for that lever, in accordance with the generalized version of the matching law. Number of presses seemed, on balance, to be a slightly more reliable measure than pressing time. The substantial interchangeability may prove more significant than the slight disparity, however, because it supports the notion that all behavior can be measured on a common scale of time.     

The generality of selective observing

Four rats obtained food pellets by poking a key and 5-s presentations of the discriminative stimuli by pressing a lever. Every 1 or 2 min, the prevailing schedule of reinforcement for key poking alternated between rich (either variable-interval [VI] 30 s or VI 60 s) and lean (either VI 240 s, VI 480 s, or extinction) components. While the key was dark (mixed-schedule stimulus), no exteroceptive stimulus indicated the prevailing schedule. A lever press (i.e., an observing response), however, illuminated the key for 5 s with either a steady light (S+), signaling the rich reinforcement schedule, or a blinking light (S-), signaling the lean reinforcement schedule. One goal was to determine whether rats would engage in selective observing (i.e., a pattern of responding that maintains contact with S+ and decreases contact with S-). Such a pattern was found, in that a 5-s presentation of S+ was followed relatively quickly by another observing response (which likely produced another 5-s period of S+), whereas exposure to S- resulted in extended breaks from observing. Additional conditions demonstrated that the rate of observing remained high when lever presses were effective only when the rich reinforcement schedule was in effect (S+ only condition), but decreased to a low level when lever presses were effective only during the lean reinforcement component (S- only condition) or when lever presses had no effect (in removing the mixed stimulus or presenting the multiple-schedule stimuli). These findings are consistent with relativistic conceptualizations of conditioned reinforcement and extend the generality of selective observing to procedures in which the experimenter controls the duration of stimulus presentations, the schedule components both offer intermittent food reinforcement, and rats serve as subjects.     

The role of temporal intervals on reinforcer accumulation

Animals accumulate reinforcers when they forgo the opportunity to consume available food in favor of acquiring additional food for later consumption. Laboratory research has shown that reinforcer accumulation is facilitated when an interval (either spatial or temporal) separates earning from consuming reinforcers. However, there has been no systematic investigation on the interval separating consuming reinforcers from earning additional reinforcers. This oversight is problematic because this second interval is an integral part of much of the previous research on reinforcer accumulation. The purpose of the current study was to determine the independent contributions of these two temporal intervals on reinforcer accumulation in rats. Each left lever press earned a single food pellet; delivery of the accumulated pellet(s) occurred upon a right lever press. Conditions varied based on the presence of either an intertrial interval (ITI) that separated pellet delivery from the further opportunity to accumulate more pellets, or a delay‐to‐reinforcement that separated the right lever press from the delivery of the accumulated pellet(s). Delay and ITI values of 0, 5, 10 and 20 s were investigated. The delay‐to‐reinforcement conditions produced greater accumulation relative to the ITI conditions, despite accumulation increasing the density of reinforcement more substantially in the ITI conditions. This finding suggests that the temporal separation between reinforcer accumulation and subsequent delivery and consumption was a more critical variable in controlling reinforcer accumulation.     

Is induction produced by upcoming food-pellet reinforcement the outcome of an increase in overall activity or in operant responding?

Researchers have demonstrated that rats reliably increase their rates of pressing a lever for 1% liquid-sucrose reinforcement if they will soon have the opportunity to press a lever for food-pellet reinforcement. In the present experiments, the authors investigated if this increase in response rates occurred because the upcoming food pellets produced an increase in all behaviors (i.e., general arousal) or an increase in only the specific operant response (i.e., lever pressing). The results of Experiments 1 and 2 showed that the appearance of induction in rats' lever pressing for 1% sucrose reinforcement when food-pellet reinforcement was upcoming did not coincide with increases in the frequency of running in a wheel or making a nonreinforced nose-poke response. On the other hand, in Experiment 3, the authors found the appearance of induction coincided with increase nonreinforced lever presses on an adjacent lever. These results shed doubt on the idea that induction is a result of a general increase in all activity, and suggest instead that the increase in responding that occurs during induction is limited to the operant response.     

SINGLE‐SAMPLE DISCRIMINATION OF DIFFERENT SCHEDULES' REINFORCED INTERRESPONSE TIMES

Food‐deprived rats in Experiment 1 responded to one of two tandem schedules that were, with equal probability, associated with a sample lever. The tandem schedules' initial links were different random‐interval schedules. Their values were adjusted to approximate equality in time to completing each tandem schedule's response requirements. The tandem schedules differed in their terminal links: One reinforced short interresponse times; the other reinforced long ones. Tandem‐schedule completion presented two comparison levers, one of which was associated with each tandem schedule. Pressing the lever associated with the sample‐lever tandem schedule produced a food pellet. Pressing the other produced a blackout. The difference between terminal‐link reinforced interresponse times varied across 10‐trial blocks within a session. Conditional‐discrimination accuracy increased with the size of the temporal difference between terminal‐link reinforced interresponse times. In Experiment 2, one tandem schedule was replaced by a random ratio, while the comparison schedule was either a tandem schedule that only reinforced long interresponse times or a random‐interval schedule. Again, conditional‐discrimination accuracy increased with the temporal difference between the two schedules' reinforced interresponse times. Most rats mastered the discrimination between random ratio and random interval, showing that the interresponse times reinforced by these schedules can serve to discriminate between these schedules.     

The effect of reinforcement differences on choice and response distribution during stimulus compounding

In Experiments I and II, rats were trained to respond on one lever during light and another during tone. The absence of tone and light controlled response cessation. In the multiple schedule of Experiment I, all reinforcements were received for responding in tone or light; in the chain schedule of Experiment II, all reinforcements were received in no tone + no light for not responding. Experiment I subjects, for which tone and light were associated with response and reinforcement increase, responded significantly more to tone-plus-light than to tone or light alone (additive summation). Experiment II subjects, for which tone and light were associated with response increase and reinforcement decrease, responded comparably to tone, light, and tone + light. Thus, additive summation was observed when stimulus-response and stimulus-reinforcer associations in tone and light were both positive, but not when they were conflicting. All subjects in both experiments responded predominantly on the light-correlated lever during tone + light, even when light intensity was reduced in testing. Furthermore, when a light was presented to a subject engaged in tone-associated responding, all subjects immediately switched the locus of responding to the light-correlated lever. No change in locus occurred when a tone was presented to a subject engaged in light-associated responding, irrespective of the stimulus-reinforcer association conditioned to tone. The light-lever preference in tone + light indicates that the heightened responding observed in Experiment I was not the summation of tone-associated behavior with light-associated behavior. Rather, it appears to be the result of a facilitation of one operant (light-associated responding) by the reinforcement-associated cue for the other.     

Effects of fixed-time shocks and brief stimuli on food-maintained behavior of rats

When a fixed-time schedule of shocks was presented to rats lever pressing for food on a random-interval schedule, a pattern of behavior developed with a high rate of pressing after shock declining to near zero before the next shock was delivered. Once this pattern had stabilized, one-quarter of the shocks were replaced with brief auditory stimuli (tones) in a random sequence. Tone maintained behavior similar to shock, although tone was never paired with shock. Both tone and shocks elicited responding when presented at various times as probe stimuli, and responding was usually totally suppressed if neither stimulus occurred at the beginning of the fixed-time interval. When other stimuli were paired with tone and shock, only those paired with tone gained discriminative control and elicited responding. These findings suggest that stimuli that signal a shock-free, or safe, period will maintain the pattern of behavior generated by shock on a fixed-time schedule. There is a parallel between this phenomenon and the control of behavior on second-order schedules of positive reinforcement with nonpaired brief stimuli.     

Compounding of discriminative stimuli that maintain responding on separate response levers

In Experiment 1, rats' responses were reinforced on a fixed-interval 30-sec schedule in the presence of either a light or a tone and were not reinforced in their absence. Each stimulus was correlated with its own response lever, with only one lever present during a session. When light and tone were compounded in the presence of the tone-correlated lever, no change in responding occurred. However, when tone was compounded with light in the presence of the light-correlated lever, level of responding was greater than to light alone (response summation). Summation was also found when each stimulus was correlated with the same lever. Next, light and tone were again correlated with separate levers, but both levers were always simultaneously present. Compounding produced both summation and emission of most responses on the light-correlated lever. This prepotency of light was reduced (1) by leaving a houselight on throughout the session; and (2) by correlating each stimulus with a different schedule (either fixed-interval 4.7-sec or fixed-interval 30-sec). With a medium- and high-intensity houselight and with the different reinforcement schedules, similar results were obtained during compounding, regardless of whether compounding occurred in the presence of the light- or tone-correlated lever.     

Effect of prior Pavlovian discrimination training upon learning an operant discrimination

The effect of Pavlovian discrimination training with two stimuli upon subsequent learning of an operant discrimination involving those stimuli was studied. After preliminary lever press training, the lever was removed and thirsty rats received noncontingent pairings between S₁ (a tone or a clicker) and water reinforcements, whereas S₂ (a clicker or a tone) occurred always without reinforcement. This procedure presumably established S₁ as a positive CS for respondent behavior, whereas S₂ was established as an inhibitory CS. Following this training, the lever was reintroduced and the rats were trained on an operant (lever pressing) discrimination involving S₁ and S₂. For the Consistent Ss, S₁ was the S^D and S₂ the S^Δ in the operant discrimination; for the Reversed Ss, S₂ served as S^D and S₁ as S^Δ. The Consistent Ss learned the operant discrimination significantly faster than did the Reversed Ss. The result emphasizes the importance of respondents, conditioned to S^D and S^Δ, which modulate operant performance to these stimuli.     

Naloxone administration impairs autoshaped learning

Effects of naloxone on acquisition of autoshaped behavior were investigated. Rats deprived to 85% of free-feeding weights were trained to touch a retractable lever; delivery of a food pellet occurred on every trial following lever retraction. The lever was retracted immediately if a touch occurred within 15 s, or automatically after 15 s. Analyses were conducted on number and latencies of touches of the extended lever, nose-pokes (touches) directed at the retracted lever during intertrial intervals (a measure less constrained by ceiling effects than extended lever touches), and unconditioned exploratory rearing activity, measured as touches of a metal strip mounted above the grid floor of the apparatus. In an initial experiment, male Sprague-Dawley rats were given saline or naloxone (2.0 mg/kg, ip) 5 min before a training session of 12 trials. Two days later they were tested, in the absence of drug, in a session of 36 (three blocks of 12) trials. Naloxone depressed training levels of lever responding, in addition to slowing acquisition rate. No effect of naloxone was observed on rearing activity. Previous work showed that injection of saline 5 min before behavioral testing increases the rate of autoshaping compared to injections 30 min before (Messing & Sparber, 1984). Thus, effects of naloxone on acquisition of lever-directed behaviors may have been confounded by behavioral depressant effects and/or by an injection effect such a short time before testing. In a second experiment naloxone (0.5 or 2.0 mg/kg) was injected after five of seven training sessions (12 trials each) to male and female rats. A 6-s delay of reinforcement was inserted between lever retraction and food delivery, slowing acquisition rates and providing the opportunity to test the effects of naloxone throughout a multiple-session task. The low dose retarded acquisition of extended lever touching in both sexes; both doses retarded acquisition of interim lever touching in males. Thus, in some circumstances, post-training naloxone administration may impair learning. The results support the notion that low doses of naloxone may have agonist activity.     

ON THE RELATION OF MANDS AND THE FUNCTION OF DESTRUCTIVE BEHAVIOR

When standard analogue functional analysis procedures produce inconclusive results in children with conversational speech, the child's mands may help to identify the function of destructive behavior. In the current investigation, functional analyses conducted with 2 children who exhibited self-injury, aggression, and property destruction were undifferentiated across conditions. Based on informal observations and school and parental report, an analysis was conducted using mands to help determine the function of the destructive behavior. Using a multielement design, the therapist's compliance with the child's mands occurred either on a fixed-ratio (FR) 1 schedule or contingent on destructive behavior. Destructive behavior occurred at high and consistent levels when reinforcement of mands was contingent on destructive behavior and at near-zero levels when reinforcement of mands occurred on the FR 1 schedule. Based on these results, a second analysis was conducted in which compliance to mands occurred only when the child appropriately requested it (i.e., functional communication training plus extinction) and, for 1 child, compliance with mands was terminated contingent upon destructive behavior (i.e., functional communication training plus response cost). For both children, the rates of destructive behavior decreased markedly. The results suggest that assessing the child's mands may be useful in decreasing destructive behavior when a functional analysis is inconclusive.     

Analysis of response rates during stimulus generalization

In the presence of one click frequency, the presses of two hungry rats on one of two levers were reinforced with food on variable-interval schedules; in the presence of a different click frequency, presses on the other lever were reinforced. In stimulus generalization tests, a variety of click frequencies were presented and reinforcement withheld. The test stimuli were found to exert control over which of the two levers the rats pressed, but not over the rate of pressing the selected lever. The results were interpreted as further evidence that intermediate rates in generalization gradients may be the result of the alternation of several distinct behavior patterns.     

Rats' lever-press durations as psychophysical judgments of time

Hungry rats received food following lever-press durations exceeding a minimum value, which ranged from 0 to 6.4 sec. When no intertrial intervals separated successive presses, modal press durations remained at very short values as the minimum value required for food was increased. This was particularly true immediately after a food presentation. When an 8-sec intertrial interval followed each lever release, modal press durations were always at or beyond the minimum value required for food, and outcome of the preceding press had no effect on press duration. Possible reasons for the effects of intertrial intervals included punishment of short presses, increased delay of reinforcement of short presses, and reduced density of reinforcement. In addition, functions relating discrete-trials lever-press duration to minimum duration required for food were found to be qualitatively and quantitatively similar to the power functions recently proposed by Catania (1970) for interresponse time and response latency. This similarity was taken as support for a general psychophysical law of temporal judgments.     

The successive differentiation of a lever displacement response

Maximum displacements of lever presses by rats were recorded under eight successively-smaller reinforcement zones (RZ). The largest RZ included displacements from 3° to 44°; the smallest, from 24° to 29°. As the RZ decreased, displacement distributions reflected a least-effort tendency: distributions peaked at the lower limit of RZ and most non-reinforced presses fell just below the lower limit. Successive distributions (a) differed significantly in shape, (b) showed reduced variability, and (c) indicated more presses and more presses per reinforcement. Prolonged training under the smallest RZ gave no improvement in performance.     

DISCRIMINATION OF VARIABLE SCHEDULES IS CONTROLLED BY INTERRESPONSE TIMES PROXIMAL TO REINFORCEMENT

In Experiment 1, food‐deprived rats responded to one of two schedules that were, with equal probability, associated with a sample lever. One schedule was always variable ratio, while the other schedule, depending on the trial within a session, was: (a) a variable‐interval schedule; (b) a tandem variable‐interval, differential‐reinforcement‐of‐low‐rate schedule; or (c) a tandem variable‐interval, differential‐reinforcement‐of‐high‐rate schedule. Completion of a sample‐lever schedule, which took approximately the same time regardless of schedule, presented two comparison levers, one associated with each sample‐lever schedule. Pressing the comparison lever associated with the schedule just presented produced food, while pressing the other produced a blackout. Conditional‐discrimination accuracy was related to the size of the difference in reinforced interresponse times and those that preceded it (predecessor interresponse times) between the variable‐ratio and other comparison schedules. In Experiment 2, control by predecessor interresponse times was accentuated by requiring rats to discriminate between a variable‐ratio schedule and a tandem schedule that required emission of a sequence of a long, then a short interresponse time in the tandem's terminal schedule. These discrimination data are compatible with the copyist model from Tanno and Silberberg (2012) in which response rates are determined by the succession of interresponse times between reinforcers weighted so that each interresponse time's role in rate determination diminishes exponentially as a function of its distance from reinforcement.     

Reinforcement of human observing behavior by a stimulue correlated with extinction or increased effort

Young men pulled a plunger on mixed and multiple schedules in which periods of variable-interval monetary reinforcement alternated irregularly with periods of extinction (Experiment 1), or in which reinforcement was contingent on different degrees of effort in the two alternating components (Experiment 2). In the baseline conditions, the pair of stimuli correlated with the schedule components could be obtained intermittently by pressing either of two observing keys. In the main conditions, pressing one of the keys continued to produce both discriminative stimuli as appropriate. Pressing the other key produced only the stimulus correlated with variable-interval reinforcement or reduced effort; presses on this key were ineffective during periods of extinction or increased effort. In both experiments, key presses producing both stimuli occurred at higher rates than key presses producing only one, demonstrating enhancement of observing behavior by a stimulus correlated with the less favorable of two contingencies. A control experiment showed that stimulus change alone was not an important factor in the maintenance of the behavior. These findings suggest that negative as well as positive stimuli may play a role in the conditioned reinforcement of human behavior.