Choice of extinction-signalling stimuli by rats

Rats in a two-lever situation were exposed to alternating periods of intermittent reinforcement and extinction. Extinction periods were either unsignalled or were signalled by a response-produced stimulus. The signal was sometimes a stimulus paired with food delivery in the reinforcement periods and sometimes a stimulus that occurred only in extinction periods. Both kinds of signal accelerated extinction relative to the unsignalled condition. When the signal was the stimulus paired with food in reinforcement periods, the rats tended to prefer the lever that gave that signal even though the signal accelerated extinction. There was no comparable effect for the stimulus that occurred only in extinction periods; when this signal was contingent on only one of the two levers, the rats either avoided it (Experiment 3) or were indifferent (Experiment 4). It is concluded that a stimulus can be a “secondary reinforcer” as measured by preference, even though it decreases resistance to extinction; the implications are discussed with reference to formal theories of choice behavior.     

Transfer of conditioned appetitive stimuli to conditioned aversive excitatory and inhibitory stimuli

The transfer of Pavlovian appetitive stimuli to Pavlovian aversive stimuli was examined in three experiments. In Experiment 1, rats received appetitive (Ap) conditioning designed to establish a flashing-light stimulus as either a CS+, CSo, or CS? for food, or to maintain it as a novel stimulus for US-alone subjects. Then, the stimulus was employed as a signal for weak shock in conditioned-emotional-response (CER) training. Both acquisition and extinction results showed that the ApCS+ facilitated and the ApCS? retarded aversive excitatory conditioning relative to the ApCSo and US-alone controls. Experiment 2 replicated the findings of Experiment 1 with both a moderate and a severe shock in CER training. In Experiment 3, different groups received the same appetitive conditioning as before, but to a flashing-light stimulus which was then employed as a signal for no shock in CER training. The ApCS? facilitated and the ApCS+ retarded aversive inhibitory conditioning relative to ApCSo and US-alone controls. Collectively, these findings establish that, in Pavlovian conditioning, transfer of an appetitive CS to an aversive excitor or inhibitor is facilitated by maintaining the initial conditioning contingency.     

Compounding discriminative stimuli controlling free-operant avoidance

The performances of three rats were stabilized on a multiple schedule that maintained responding by a free-operant avoidance schedule during independent presentations of tone and light. The simultaneous absence of these stimuli signalled shock-free periods and controlled response cessation. Subsequently, test sessions were administered consisting of independent presentations of each stimulus and these stimuli compounded (tone-plus-light). During an extinction test, additive summation was observed to the compounded stimuli, i.e., more responses were emitted to the compound than to either tone or light. During a series of 28 maintenance-test sessions in which the shock schedule remained operative, the compounded stimuli produced a generally enhanced response rate and fewer pauses terminating with shock than either single stimulus condition. These results extend the generality of free-operant additive summation to responding maintained by aversive control. In addition, a comparison of the present study with previous experiments reporting additive summation of positively reinforced responding indicates that similar variables—rate and aversive differences between training stimulus conditions—should be considered in accounting for response distributions during stimulus compounding when responding is controlled by either positive or negative contingencies.     

On conditioned reinforcing effects of negative discriminative stimuli

Observing responses by pigeons were studied during sessions in which a food key and an observing key were available continuously. A variable-interval schedule and extinction alternated randomly on the food key. In one condition, food-key pecking during extinction decreased reinforcement frequency during the next variable-interval component, and in the other condition such pecking did not affect reinforcement frequency. Observing responses either changed both keylight colors from white to green (S+) or to red (S−) depending on the condition on the food key, or the observing responses never produced the S+ but produced the S− when extinction was in effect on the food key. Observing responses that produced only S− were maintained only when food-key pecking during extinction decreased reinforcement frequency in the subsequent variable-interval component. The red light conformed to conventional definitions of a negative discriminative stimulus, rendering results counter to previous findings that production of S− alone does not maintain observing. Rather than offering support for an informational account of conditioned reinforcement, the results are discussed in terms of a molar analysis to account for how stimuli acquire response-maintaining properties.     

A test of the reinforcing properties of stimuli correlated with nonreinforcement

The information hypothesis of conditioned reinforcement predicts that a stimulus that “reduces uncertainty” about the outcome of a trial will acquire reinforcing properties, even when the stimulus reliably predicts nonreinforcement. Four pigeons' key pecks produced one of two 5-sec stimuli with 0.50 probability according to a discriminated variable-interval schedule. One stimulus was followed by reinforcement; a second stimulus was followed by blackout. To the same extent, therefore, both stimuli reduced uncertainty about the possibility that food would arrive at the termination of the schedule interval. When a second key in the chamber was lighted, each peck on it could produce the stimulus preceding reinforcement, the stimulus preceding nonreinforcement, a novel stimulus, or no stimulus, across separate conditions. The stimulus preceding food maintained responding at substantial levels on the second, stimulus-producing, key. Such responding was not maintained by other stimuli. These data, replicated when the stimuli were reversed on the variable-interval schedule, do not support the prediction that uncertainty-reducing stimuli are necessarily conditioned reinforcers.     

Devaluation of stimuli contingent on choice: evidence for conditioned reinforcement

Pigeons were presented a concurrent-chains schedule of reinforcement that had terminal links of equal duration. The initial links of the schedule were periodically interrupted by 15-s periods during which an extinction schedule was in effect. The extinction periods were presented on either a response-contingent or a noncontingent basis. Relative response rate for the left alternative decreased when the extinction periods were accompanied by the left terminal-link stimulus. Relative response rate for the right alternative decreased when the extinction periods were accompanied by the right terminal-link stimulus. Relative response rate varied inversely with the frequency of presentation of the extinction periods but was unaffected by presence versus absence of the response contingency in the schedule of extinction-period presentation. Furthermore, relative response rate was unaffected by presentation of extinction periods accompanied by a novel stimulus. When the extinction periods were presented after reinforcement in the left terminal link instead of as interruptions of the initial links, relative response rate for the left alternative was reduced if the postreinforcement extinction period was accompanied by the terminal-link stimulus for the left chain and reduced less if the extinction period was accompanied by the terminal-link stimulus for the right chain. The results demonstrate that the correlation between the terminal-link stimulus and extinction influenced the relative response rate in the initial link.     

Contrast and stimulus generalization following prolonged discrimination training

Different groups of pigeons received discrimination training in which the reinforcement-associated and extinction-associated stimuli were respectively either (a) a line tilt vs a blank key, (b) a blank key vs a line tilt, or (c) two different line tilts. The high response rates that developed to the positive stimulus in all groups during discrimination learning were maintained over 64 sessions of training. After these sessions, all subjects were tested for stimulus generalization along the line-tilt dimension. Gradients of relative (per cent) generalization around the stimulus associated with reinforcement (so-called excitatory gradients) and around the stimulus associated with extinction (so-called inhibitory gradients) were as steep as they typically are after much briefer training periods. These results do not support several of Terrace's predictions on the basis of the hypothesis that emotional responses develop to the stimulus associated with extinction during discrimination learning with errors, but eventually dissipate after extended training.     

Stimulus selection with duration as a relevant cue

To date, research has not explicitly examined how duration competes with other stimulus dimensions for control over responding. The present study investigated some familiar selective mechanisms of stimulus control over key pecking in pigeons, with duration and line tilt as discriminative stimuli in successive discrimination procedures. Specifically, pigeon's key pecking was reinforced with food or extinguished following compound stimuli comprising one of two line orientation stimuli presented for one of two different durations. Traditional experimental designs explored stimulus additivity, overshadowing, blocking, and learned irrelevance. Although stimulus additivity was observed, control by duration was masked by line tilt in extinction testing that followed facilitated acquisition with redundant, relevant cues. In addition, although prior training with duration relevant partially blocked subsequent acquisition of control by line tilt when both stimulus dimensions were relevant, there was a tendency for control by duration to decrease with continued compound training. It was suggested that the greater time required to distinguish duration on a trial—compared to more commonly studied stimulus dimensions, which can be distinguished almost immediately—puts duration at a competitive disadvantage in situations where other relevant dimensions are also available.     

Summation of punishment suppression

In two experiments, eight rats were trained to lever press with food on a variable-interval schedule. Bar pressing produced shock on a variable-interval schedule in the presence of two independently presented stimuli, a light and a tone. Two rats in each experiment received alternative presentations of the light and the tone and were consequently always in the presence of a stimulus that signalled variable-interval punishment. The other two rats in each experiment were treated similarly except that they received periods in which neither light nor tone was present. During these periods, bar pressing was not punished. The two stimuli that signalled punishment were then presented simultaneously to evaluate the effect of stimulus compounding on response suppression. The subjects trained without punishment-free periods did not show summation to the compound stimulus; the subjects trained with punishment-free periods showed summation of suppression. The major difference between the two experiments was the longer mean interval of variable-interval punishment used in the second experiment. This manipulation made the summation effect more resistant to extinction and thus increased its magnitude.     

Discriminative properties of briefly presented stimuli

In Experiment I, pigeons' responses produced food according to a fixed-interval schedule while responses on the key also produced brief stimuli according to a variable-interval schedule. Each brief stimulus reset the fixed interval. Thus, a brief stimulus occurred irregularly but a fixed minimum time separated the occurrence of food from a brief stimulus. Pauses followed brief stimuli and were followed by an accelerated response rate until another brief stimulus or food occurred. In Experiment II, four control procedures were examined. (1) Brief-stimulus presentations were omitted, producing a loss of response patterning. (2) A second-order schedule was studied with fixed-interval components. This schedule produced patterning following brief stimuli similar in kind and degree to that found in Experiment I. (3) A conjoint schedule was arranged in which food was no longer separated from the stimulus by a fixed time; pauses following the stimulus no longer resulted. (4) A brief food reinforcer replaced the brief visual stimulus, resulting in a constant response rate with no pausing following the brief food stimulus. The results suggest that the brief-stimulus effects were due to discriminative functions produced by the fixed temporal relation separating the stimulus from food.     

Compounds of conditioned fear stimuli

A differential conditioning procedure was used with rats to establish different levels of suppression to two different stimuli. One stimulus was paired with shock every time it was presented, while the other stimulus cued shock on only 25% of its presentations. When these two component stimuli were subsequently tested as a simultaneous compound, two different types of component-compound relationships were observed. Some subjects showed greater suppression to the compound than they did to either component stimulus, while other subjects showed an intermediate level of suppression to the compound. The difference in type of component-compound relationship appeared to be related to the degree of stability (over trials) of each subject's reaction to the probabilistic (25%) cue.When a similar discrimination was established based on the intensity of the shock, rather than its probability of occurrence, all subjects showed more suppression to the compound than to its individual component stimuli. These results are discussed in terms of their implications for choice of model for characterizing the interaction of conditioned states.     

A delivery system for olfactory stimuli

This paper describes the design of a new method for controlling and administering olfactory stimuli—namely, thehood system. The hood system involves a stream of vaporized odor (at known concentrations) mixed with odorless air and pumped (at a constant flow rate) into an oxygen therapy hood. It is designed to be used with odorants in solution, such as essential oils, as the olfactory stimulus. The use of oxygen therapy hoods allows for the precise control of a constant concentration of odorized air over time, while allowing subjects to breathe normally. The hood system provides a natural administration of olfactory stimuli and the exact determination of the stimulus concentration. The use of this system will allow experimental conditions to be completely defined and results and replication studies to be accurately interpreted. The hood system is portable, cost effective, and constructed from readily available components. It is proposed that the hood system could be adopted to suit a wide range of olfactory research, particularly that in which the effects of chronic exposure to olfactory stimuli on cognition are examined.     

Observing behavior in squirrel monkeys under a multiple schedule of reinforcement availability

Observing behavior of two squirrel monkeys was examined under a multiple schedule of four components. Lever (observing) responses produced either a stimulus indicating the availability of food or another stimulus indicating food was not available. Key responses in the presence of the food-available stimulus produced food on a continuous reinforcement schedule. In the absence of food-available stimuli, responding on the key had no scheduled consequences. Observing responses produced food-available stimuli according to three different random-interval schedules with mean interstimulus availability times of 1, 2, and 4 min. In the fourth component of the multiple schedule (observing extinction) food-available stimuli never occurred. Each component of the schedule was correlated with a distinctive auditory stimulus. Observing rates decreased with decreasing frequency of the food-available stimulus. Observing rates during extinction continued decreasing when the brief stimulus indicating food unavailability was no longer produced by lever pressing. When the brief stimulus was reinstated response rates increased abruptly.     

Generalization of the disruptive effects of alternative stimuli when combined with target stimuli in extinction

Differential‐reinforcement treatments reduce target problem behavior in the short term but at the expense of making it more persistent long term. Basic and translational research based on behavioral momentum theory suggests that combining features of stimuli governing an alternative response with the stimuli governing target responding could make target responding less persistent. However, changes to the alternative stimulus context when combining alternative and target stimuli could diminish the effectiveness of the alternative stimulus in reducing target responding. In an animal model with pigeons, the present study reinforced responding in the presence of target and alternative stimuli. When combining the alternative and target stimuli during extinction, we altered the alternative stimulus through changes in line orientation. We found that (1) combining alternative and target stimuli in extinction more effectively decreased target responding than presenting the target stimulus on its own; (2) combining these stimuli was more effective in decreasing target responding trained with lower reinforcement rates; and (3) changing the alternative stimulus reduced its effectiveness when it was combined with the target stimulus. Therefore, changing alternative stimuli (e.g., therapist, clinical setting) during behavioral treatments that combine alternative and target stimuli could reduce the effectiveness of those treatments in disrupting problem behavior.     

Conditioned and unconditioned caloric compensation: Evidence for self-regulation of food intake in young children

The purpose of these two experiments was to determine (a) whether young children can be responsive to caloric density cues in regulating their food intake, (b) whether such cues can be associatively conditioned to organoleptic cues in foods, and (c) to obtain evidence regarding which of the many cues available are involved as conditioned stimuli. In Experiment 1 participants were eighteen 3- to 5-year-old children, who were seen for a series of pairs of conditioning trials, followed by extinction test trials. Each trial consisted of a two-part snack: approximately 100 ml of a pudding preload (chocolate or vanilla; high or low caloric density) followed after a delay by ad-lib consumption of snack foods (cookies and crackers). In extinction trials, flavors previously paired with high- or low-caloric density preloads during conditioning were presented in isocaloric intermediate density preloads. Results indicated that 14 of 18 children showed unconditioned caloric compensation on the first pair of conditioning trials; 16 of 18 children showed compensation following the second pair of trials, and 12 of these 16 subjects continued to show this consumption pattern during extinction. Consumption was significantly greater following the low calorie paired flavor than following the high calorie paired flavor during extinction. Experiment 2 (N = 10) replicated these findings, and uncorrelating preload and snack food flavors indicated that flavor cues in the preloads can serve as conditioned stimuli. Children showed both initial responsiveness to caloric density and evidence for associative conditioning of food cues to the physiological consequences of eating. These results provide initial evidence for a mechanism allowing the child to learn to anticipate the caloric consequences of familiar foods and regulate food intake accordingly.     

Aversive CSs suppress lever pressing for food but not the eating of free food

Forty rats received CER acquisition, half “on the baseline” (response lever present), half “off the baseline.” During initial CER extinction, Ss received: (a) normal CER extinction (lever available), (b) free food during CSs only (no lever), (c) free food during non-CSs only (no lever), or (d) free food during both CS and non-CS periods (no lever). While normal extinction Ss were highly suppressed in the presence of the CS, all free food groups readily ate but did not differ in eating latencies. On subsequent CER extinction trials on the baseline, those Ss which received free food during the CSs were no less suppressed than other Ss. These data offer no support for Estes' (1969) reciprocal inhibition explanation of CER and punishment, nor do they support a fear interpretation of CER.     

Discriminated timeout avoidance in pigeons: the roles of added stimuli

Two experiments examined pigeons' postponement of a signaled extinction period, or timeout (TO), from an ongoing schedule of response-dependent food delivery. A concurrent-operant procedure was used in which responses on one (food) key produced food according to a variable-interval schedule and responses on a second (postponement) key delayed the next scheduled TO according to a response-TO (R-TO) interval. A series of response-independent stimulus changes on the food key temporally partitioned the R-TO into three equal segments (S1, S2, and S3). Postponement responses, in addition to postponing TO, also reinstated S1, the stimulus correlated with the greatest temporal distance from TO. In Experiment 1, the R-TO interval was manipulated systematically across blocks of sessions (conditions) at a given ratio of R-TO:TO duration. This R-TO:TO ratio was manipulated across blocks of conditions (phases). Postponement response rates varied inversely with R-TO interval in each phase. Changes in the R-TO:TO ratio did not produce consistent differences except at the 1:10 ratio for some pigeons, where it disrupted postponement responding in some conditions. Most of the postponement responses occurred in the presence of S2 and S3, the stimuli most proximal to TO, whereas most of the food-key responses occurred in S1. In Experiment 2, the R-TO contingencies were systematically manipulated in the presence of the time-correlated stimuli. In one set of conditions, the R-TO contingencies were made either ineffective or less effective in the presence of one or more stimuli. Postponement responses typically shifted to stimuli in the presence of which responses were relatively more effective. Postponement responses decreased markedly when the added stimuli were removed, and then recovered when the stimuli were reinstated. Results from both experiments indicate that the added stimuli in a discriminated TO-avoidance procedure serve predominately discriminative functions, delineating periods during which behavior is maximally effective. The results parallel those obtained in shock-avoidance procedures, providing further evidence that TO functions as an aversive stimulus.     

Assessing potential reinforcement-like effects of brief stimuli unrelated to food reinforcers

It is widely assumed that reinforcers are biologically relevant stimuli, or stimuli that have been associated with biologically relevant stimuli. However, brief, arbitrary stimuli have also been reported to have reinforcement-like effects, despite being unrelated to biologically relevant stimuli like food. The present study explored the potential reinforcement-like effects of brief stimuli across 5 experiments. In Experiments 1 through 4, pigeon subjects responded for food reinforcement and brief stimulus presentations in a 2-component multiple schedule. Neither baseline response rates nor resistance to change during disruption tests were systematically greater in a component with versus without brief stimulus presentations. Increasing the rate and duration of brief stimulus presentations in Experiment 4 did not reveal reinforcement-like effects when compared directly with food. In Experiment 5, pigeons chose between independent terminal links in a concurrent-chains procedure. Across conditions, varying the location, duration, and rate of brief stimulus presentations in the terminal links had no systematic effects on preference. In contrast, varying rates of food reinforcers resulted in large and reliable shifts in preference. Therefore, the present study found no systematic evidence that brief stimuli unrelated to food reliably increase response rates, resistance to change, or preference. These data demonstrate the value of systematic replication, and a behavioral momentum approach to assessing potential reinforcement-like effects.     

The effects of noncontingent delivery of high- and low-preference stimuli on attention-maintained destructive behavior

An adolescent with severe mental retardation and cerebral palsy who displayed attention-maintained destructive behavior was exposed to noncontingent reinforcer delivery (NCR) with either a high-preference or a low-preference stimulus while reinforcement for destructive behavior with attention remained in effect (i.e., NCR without extinction). NCR without extinction was effective only when the high-preference stimulus was available, suggesting that systematic assessment of stimulus quality may enhance the effectiveness of NCR with alternative stimuli.     

Resistance to change and relapse of observing

Four experiments examined relapse of extinguished observing behavior of pigeons using a two-component multiple schedule of observing-response procedures. In both components, unsignaled periods of variable-interval (VI) food reinforcement alternated with extinction and observing responses produced stimuli associated with the availability of the VI schedule (i.e., S+). The components differed in the rate of food arranged (Rich = VI 30 s; Lean = VI 120 s). In Experiment 1, following baseline training, extinction of observing involved removal of both food and S+ deliveries, and reinstatement was examined by presenting either response-independent food or S+ deliveries. In Experiment 2, extinction involved removal of only food deliveries while observing responses continued to produce S+. Reinstatement was examined by delivering food contingent upon the first two food-key responses occurring in the presence of the S+. Experiment 3 assessed ABA renewal of observing by extinguishing food-key and observing responses in the presence of one contextual stimulus (i.e., B) and then returning to the original training context (i.e., A) during continued extinction. Experiment 4 examined resurgence by introducing food reinforcement for an alternative response during extinction, and subsequently removing that alternative source of food. Across experiments, relative resistance to extinction and relapse of observing tended to be greater in the component previously associated with the higher rate of primary reinforcement. Relapse of observing or attending to stimuli associated with primary reinforcement appears to be impacted by frequency of primary reinforcement in a manner similar to responding maintained directly by primary reinforcement.