Conditioned suppression and variable ratio reinforcement

Conditioned suppression is a decrease in response rate during a relatively short duration stimulus that terminates independently of the animal's behavior and coincidentally with a brief unavoidable shock. The degree of conditioned suppression was measured for each of three birds on three variable ratio schedules; that is, the number of responses required for food reinforcement was varied around a mean of 50, 100, or 200. The results indicated a slight and possibly negligible decrease in the degree of suppression as the mean number of responses required on the schedule was increased from 50, to 100, and 200. In general, it was found that all of the variable ratio schedules tested were quite insensitive to the conditioned suppression procedure, although almost complete suppression was obtained on a few occasions. Since the reinforcement was contingent upon the emission of responses, the birds typically displayed a high rate of response during the pre-shock stimulus on all schedules. In addition, the rate during the pre-shock stimulus often changed abruptly independent of the presentation of a reinforcement. As a result of the high rate of response and the abrupt changes in rate, the degree of suppression from trial to trial was quite variable. A clear analysis of an experimental variable on this baseline is thus difficult.     

Conditioned suppression of behavior maintained by intracranial stimulation as a function of stimulation intensity.

Conditioned suppression was demonstrated in two experiments with rats lever pressing on a fixed-ration 1 schedule for lateral hypothalamic intracranaial stimulation (ICS)'n Experiment I, conditioned suppression of responding for low-intensity ICS was obtained with a moderate intensity of foot shock, In Experiment II, low and high intensities of ICS were alternated within the same session and the same animal The suppression that was exhibited with low intensity ICS was minimal or absent with high-intensity stimulation, despite the pairing of foot shock with each warning stimulus. Conditioned suppression was a function of ICS intensity, and was independent of response rates. The inverse relationship between ICS intensity and degree os suppression is consistent with a motivational analysis of conditioned suppression. Previous reports of resistance to suppression of behaviors maintained by ICS may now be attributed to the use of high-intensity stimulation.     

Conditioned reinforcement and choice

In a series of three experiments, rats were exposed to successive schedule components arranged on two levers, in which lever pressing produced a light, and nose-key pressing produced water in 50% of the light periods. When one auditory signal was presented only during those light periods correlated with water on one lever, and a different signal was presented only during those light periods correlated with nonreinforcement on the other lever, the former lever was preferred in choice trials, and higher rates of responding were maintained on the former lever in nonchoice (forced) trials. Thus, the rats preferred a schedule component that included a conditioned reinforcer over one that did not, with the schedules of primary reinforcement and the information value of the signals equated. Preferences were maintained when one or the other of the auditory signals was deleted, but were not established in naive subjects when training began with either the positive or negative signal only. Discriminative control of nose-key pressing by the auditory signals was highly variable across subjects and was not correlated with choice.     

Polydipsia induced by a schedule of brain stimulation reinforcement

Rats pressed a lever for a signaled brain stimulation reinforcement on either fixed-interval or fixed-ratio schedules. Both schedules induced polydipsia, with a median water intake of 22.5 ml (about four times the control level) in 3-hr sessions. The stimulation did not directly elicit drinking or other behaviors. In a second experiment, two out of four rats became polydipsic during extinction following a continuous schedule of brain stimulation reinforcement. The results were compared with previous attempts to induce polydipsia with a schedule of brain stimulation reinforcement and were discussed in terms of polydipsia as a model of human “compulsive” behaviors.     

Conditioned reinforcement and discrimination performance

Pigeons were trained in a three-key chamber to peck one side key in the presence of a vertical line on the center key and to peck the other side key in the presence of a horizontal line. Correct choice responses were reinforced with food according to fixed- and variable-ratio, fixed-interval, and differential-reinforcement-of-long-latency schedules of reinforcement. For each schedule, the birds performed under each of two conditions: (1) each correct choice response produced a brief presentation of stimuli intermittently paired with food, then the next trial; (2) each correct choice response produced an intertrial interval only. For all schedules except one long latency schedule, response rates were higher under the condition of brief stimulus presentation than under the comparable control condition. Presentation of brief magazine stimuli increased choice accuracy. The amount of change in accuracy was correlated with the rate of food presentation. Performance under the schedules with highest food reinforcement rates showed no enhancement; performance under the schedules with the lowest reinforcement rates showed the greatest enhancement.     

Conditioned reinforcement and response strength

Stimuli associated with primary reinforcers appear themselves to acquire the capacity to strengthen behavior. This paper reviews research on the strengthening effects of conditioned reinforcers within the context of contemporary quantitative choice theories and behavioral momentum theory. Based partially on the finding that variations in parameters of conditioned reinforcement appear not to affect response strength as measured by resistance to change, long-standing assertions that conditioned reinforcers do not strengthen behavior in a reinforcement-like fashion are considered. A signposts or means-to-an-end account is explored and appears to provide a plausible alternative interpretation of the effects of stimuli associated with primary reinforcers. Related suggestions that primary reinforcers also might not have their effects via a strengthening process are explored and found to be worthy of serious consideration.     

Conditioned reinforcement by conditional discriminative stimuli.

A concurrent-chains schedule was used to examine how a delay to conditional discriminative stimuli affects conditioned reinforcement strength. Pigeons' key-peck responses in the initial link produced either of two terminal links according to independent variable-interval 30-s schedules. Each terminal link involved an identical successive conditional discrimination and was segmented into three links: a delay interval (green), a color conditional discriminative stimulus (blue or red), and a line conditional discriminative stimulus (vertical or horizontal lines). Food delivery occurred 45 s after entering the terminal link with a probability of .5, but its conditional probability (1.0 or 0) depended on the combination of the color and the line stimuli. One of the color stimuli occurred independently of further responding, 5 s after entry into the right terminal link, but it occurred 35 s after entry into the left terminal link. One of the line stimuli occurred independently of responding 40 s after entry into either terminal link, synchronized with the offset of the color stimulus. The initial-link relative response rate for the right was consistently higher in comparison with a control condition in which the color stimuli occurred 20 s after entry into either terminal link. The preference for the short delay to the color conditional discriminative stimuli suggests the possibility of conditioned reinforcement by information about the relation between the line conditional discriminative stimuli and the outcomes.     

Conditioned reinforcement value and choice.

The delay-reduction hypothesis of conditioned reinforcement states that the reinforcing value of a food-associated stimulus is determined by the delay to primary reinforcement signaled by the onset of the stimulus relative to the average delay to primary reinforcement in the conditioning situation. In contrast, most contemporary models of conditioned reinforcement strength posit that the reinforcing strength of a stimulus is some simple function only of the delay to primary reinforcement in the presence of stimulus. The delay-reduction hypothesis diverges from other conditioned reinforcement models in that it predicts that a fixed-duration food-paired stimulus will have different reinforcing values depending on the frequency of its presentation. In Experiment 1, pigeons' key pecks were reinforced according to concurrent-chains schedules with variable-interval 10-second and variable-interval 20-second terminal-link schedules. The initial-link schedule preceding the shorter terminal link was always variable-interval 60 seconds, and the initial-link schedule requirement preceding the longer terminal link was varied between 1 second and 60 seconds across conditions. In Experiment 2, the initial-link schedule preceding the longer of two terminal links was varied for each of three groups of pigeons. The terminal links of the concurrent chains for the three groups were variable-interval 10 seconds and 20 seconds, variable-interval 10 seconds and 30 seconds, and variable-interval 30 seconds and 50 seconds. In both experiments, preference for the shorter terminal link was either a bitonic function or an inverse function of the initial-link schedule preceding the longer terminal-link schedule. Consistent with the predictions of the delay-reduction hypothesis, the relative values of the terminal-link stimuli changed as a function of the overall frequency of primary reinforcement. Vaughan's (1985) melioration model, which was shown to be formally similar to Squires and Fantino's (1971) delay-reduction model, can be modified so as to predict these results without changing its underlying assumptions.     

Conditioned suppression of free-operant avoidance

The responses of white rats were maintained on an unsignalled free-operant avoidance schedule. Superimposed on the avoidance schedule was a blinking white light followed immediately by response-independent electric shock. Duration of the light stimulus was either 1 or 3 min. Avoidable shock was 1.5 mA; response-independent shock was 7.5 mA. Suppression of responding during the light stimulus (both durations) developed over sessions. Responding immediately following the response-independent light-shock sequence was neither suppressed nor accelerated. The similarity is noted between the present result and findings of “positive conditioned suppression”.     

Conditioned reinforcement and discrimination in second-order schedules

Pigeons were exposed to multiple second-order schedules in which responding on the “main key” was reinforced according to either a variable-interval or fixed-interval schedule by production of a brief stimulus on the “brief-stimulus key”. A response was required to the brief stimulus during its fourth (final) presentation to produce food; responses to the earlier brief stimuli indicated the extent to which the final brief stimulus was discriminated from preceding ones. Main-key response rates were higher in early components of paired brief-stimulus schedules, in which each brief stimulus was the same as that paired with reinforcement, than in comparable unpaired brief-stimulus or tandem schedules. Poor discrimination occurred between paired brief stimuli (Experiment I). When chain stimuli on the main key induced a discrimination between the first two and second two brief stimuli, the response-rate enhancement in the paired brief-stimulus schedule persisted (Experiment II). Rate enhancement diminished when the initial link of the chain included the first three components (Experiment IV). Eliminating the contingency between responding and brief-stimulus production also diminished rate enhancement (Experiment III). The results show that the discriminative and conditioned reinforcing effects of food-paired brief stimuli may be selectively manipulated and suggest that the reinforcing effects are modulated by other reinforcers in the situation.     

Conditioned reinforcement value and resistance to change

Three experiments examined the effects of conditioned reinforcement value and primary reinforcement rate on resistance to change using a multiple schedule of observing-response procedures with pigeons. In the absence of observing responses in both components, unsignaled periods of variable-interval (VI) schedule food reinforcement alternated with extinction. Observing responses in both components intermittently produced 15 s of a stimulus associated with the VI schedule (i.e., S+). In the first experiment, a lower-valued conditioned reinforcer and a higher rate of primary reinforcement were arranged in one component by adding response-independent food deliveries uncorrelated with S+. In the second experiment, one component arranged a lower valued conditioned reinforcer but a higher rate of primary reinforcement by increasing the probability of VI schedule periods relative to extinction periods. In the third experiment, the two observing-response components provided similar rates of primary reinforcement but arranged different valued conditioned reinforcers. Across the three experiments, observing-response rates were typically higher in the component associated with the higher valued conditioned reinforcer. Resistance to change was not affected by conditioned reinforcement value, but was an orderly function of the rate of primary reinforcement obtained in the two components. One interpretation of these results is that S+ value does not affect response strength and that S+ deliveries increase response rates through a mechanism other than reinforcement. Alternatively, because resistance to change depends on the discriminative stimulus-reinforcer relation, the failure of S+ value to impact resistance to change could have resulted from a lack of transfer of S+ value to the broader discriminative context.     

Conditioned reinforcement: Experimental and theoretical issues

The concept of conditioned reinforcement has received decreased attention in learning textbooks over the past decade, in part because of criticisms of its validity by major behavior theorists and in part because its explanatory function in a variety of different conditioning procedures has become uncertain. Critical data from the major procedures that have been used to investigate the concept (second-order schedules, chain schedules, concurrent chains, observing responses, delay-of-reinforcement procedures) are reviewed, along with the major issues of interpretation. Although the role played by conditioned reinforcement in some procedures remains unresolved, the results taken together leave little doubt that the underlying idea of conditioned value is a critical component of behavior theory that is necessary to explain many different types of data. Other processes (marking, bridging) may also operate to produce effects similar to those of conditioned reinforcement, but these clearly cannot explain the full domain of experimental effects ascribed to conditioned reinforcement and should be regarded as complements to the concept rather than theoretical competitors. Examples of practical and theoretical applications of the concept of conditioned reinforcement are also considered.     

Phenylpropanolamine inhibits feeding, but not drinking, induced by hypothalamic stimulation.

In rats bearing lateral hypothalamic electrodes that elicited both feeding and drinking, intraperitoneal injection of the appetite suppressant drug phenylpropanolamine (Propadrine) inhibited only feeding. This occurred whether feeding and drinking were tested simultaneously or separately. Selective inhibition of lateral hypothalamic feeding also followed injection of this drug through lateral, but not medial, hypothalamic electrode cannulas. We conclude that hypothalamically induced feeding is under some of the same pharmacological controls as spontaneous feeding, that this control may be exerted, in part, in or near the lateral hypothalamus, and that the neural systems which induce feeding and drinking during hypothalamic stimulation can be pharmacologically separated.     

Conditioned suppression of an avoidance response

A signal followed by shock was presented at irregular intervals during a free-operant avoidance schedule. The effects of this procedure were studied in terms of the rate of unavoided shock in the presence and absence of the signal and the rate of response before and during the signal. Three shock intensities were employed. Response enhancement as well as response suppression were observed; irrespective of changes in responding, shock rates substantially increased during signalled periods compared to non-signalled periods. Shock rates in non-signalled periods were generally higher than during training.