Choosing between predictable shock schedules: Long- versus short-duration signals

Two experiments assessed the relative aversiveness of different duration preshock signals (5 and 20 seconds) and different duration stimuli identifying shock-free periods. In the first experiment, the responding of 15 of 18 rats was maintained when it produced changes from a predictable-shock condition with a 5-second preshock signal to an identical schedule with a 20-second preshock signal; responding was not maintained when it produced the opposite changes. These results occurred with intershock intervals of both 120 seconds and 240 seconds. The second experiment assessed whether changing to the 20-second schedule was maintained by properties of the preshock signals identifying the shock periods or by properties of the stimuli identifying the shock-free periods. Four subjects were given training with the two signaled schedules in an operant chamber and then later given off-baseline preference tests in a shuttlebox. When given a choice between preshock signals, subjects chose the 5-second signals over the 20-second signals. However, when given a choice between stimuli identifying shock-free periods, subjects chose the stimulus identifying the shorter shock-free periods (i.e., the one previously correlated with the 20-second signals). These findings are discussed within the Rescorla and Wagner model of stimulus compounds and within the context of safety as a contrast phenomenon.     

Choosing schedules of signaled appetitive events over schedules of unsignaled ones

Two experiments were completed allowing albino rats to choose between signaled and unsignaled reward conditions. These experiments examined the effects on preference of (1) response dependent versus response-independent reward and, (2) food pellets versus chocolate milk as the reward. All subjects preferred the signaled condition over the unsignaled condition, whether exposed to response-dependent, or to response-independent delivery of rewards. Preference was controlled most effectively by presenting both the signal itself and the correlated stimulus identifying the signaled condition. The signal presented alone (Extinction 3) controlled preference more effectively than did the stimulus correlated with the signaled condition (Extinction 2). The second experiment showed that the quality of the reinforcer (pellets and chocolate milk) did not affect preference for signaled reward since all subjects preferred the signaled condition at levels comparable to those observed in Experiment 1, with food pellets. These results, along with others, argue against species differences, response-dependency, and reinforcer quality as variables affecting the direction of preference.     

Choice with uncertain outcomes: conditioned reinforcement effects.

Pigeons responded on concurrent chains with equal initial- and terminal-link durations. In all conditions, the terminal links of one chain ended reliably in reinforcement; the terminal links on the alternative chain ended in either food or blackout. In Experiment 1, the terminal-link stimuli were correlated with (signaled) the outcome, and the durations of the initial and terminal links were varied across conditions. Preference did not vary systematically across conditions. In Experiment 2, terminal-link durations were varied under different stimulus conditions. The initial links were variable-interval 80-s schedules. Preference for the reliable alternative was generally higher in unsignaled than in signaled conditions. Preference increased with terminal-link durations only in the unsignaled conditions. There were no consistent differences between conditions with and without a common signal for reinforcement on the two chains. In the first series of conditions in Experiment 3, a single response was required in the initial links, and the stimulus conditions during 50-s terminal links were varied. Preference for the reliable outcome approached 1.0 in unsignaled conditions and was considerably lower (below .50 for 3 of 5 subjects) in signaled conditions. In a final series of signaled conditions with relatively long terminal links, preference varied with duration of the initial links. The results extend previous findings and are discussed in terms of the delay reduction signaled by terminal-link stimuli.     

A temporal parameter influencing choice between signalled and unsignalled shock schedules

The present study investigated whether choice of a signalled variable-time shock schedule over an unsignalled one was influenced by the average intershock interval. Eight rats were given a choice between signalled and unsignalled shock schedules in a series of conditions with average intershock intervals of 510, 270, 150, 90, 60, and 45 sec. Each test condition was preceded by a training-baseline condition, and schedule values were arranged in an ascending (four subjects) or descending (four subjects) order. Choice of the signalled conditions was directly related to the average intershock interval of the variable-time schedule for six of the eight subjects. The per cent of time in the signalled condition was highest when the average intershock interval was 150 sec or longer and lowest when the average intershock interval was 45 sec. The findings were interpreted as being due to changes in the safety features of the signalled schedule, rather than to changes in the average intershock interval per se.     

Effects of signaled versus unsignaled delay of reinforcement on choice

Pigeons chose between 5-s and 15-s delay-of-reinforcement alternatives. The first key peck to satisfy the choice schedule began a delay timer, and food was delivered at the end of the interval. Key pecks during the delay interval were measured, but had no scheduled effect. In Experiment 1, signal conditions and choice schedules were varied across conditions. During unsignaled conditions, no stimulus change signaled the beginning of a delay interval. During differential and nondifferential signal conditions, offset of the choice stimuli and onset of a delay stimulus signaled the beginning of a delay interval. During differential signal conditions, different stimuli were correlated with the 5-s and 15-s delays, whereas the same stimulus appeared during both delay durations during nondifferential signal conditions. Pigeons showed similar, extreme levels of preference for the 5-s delay alternative during unsignaled and differentially signaled conditions. Preference levels were reliably lower with nondifferential signals. Experiment 2 assessed preference with two pairs of unsignaled delays in which the ratio of delays was held constant but the absolute duration was increased fourfold. No effect of absolute duration was found. The results highlight the importance of delayed primary reinforcement effects and challenge models of choice that focus solely on conditioned reinforcement.     

Choice and the dependability of stimuli that predict shock and safety

This study assessed whether choosing a signalled shock condition over an unsignalled one is controlled by a stimulus that predicts the presence of shock (Experiment I), or by a stimulus that predicts the absence of shock (Experiment II). The dependability of these stimuli as predictors of either the presence or the absence of shock was parametrically varied over a wide range, and subjects (rats) were given an option to change from an unsignalled to a signalled condition. In the first experiment, all shocks were preceded by signals; however, the probability of a signal being followed by shock varied from 1.0 to 0.02. The data obtained indicate that the dependability of the signal as a predictor of shock is unimportant. Rats changed to the signalled condition when the signal was completely dependable (all signals followed by shock) and when the dependability of the signal was systematically degraded. In the second experiment, all signals were followed by shock; however, some shocks were not preceded by a signal. The data show that the dependability of a stimulus predicting the absence of shock is important in that, as dependability decreases, changing to the signalled condition also decreases.     

Signal modality and choice between signaled and unsignaled food

Choice between signaled and unsignaled response-independent food schedules was assessed in three experiments using a commitment procedure. In Experiment 1, subjects tested with a 5-s visual signal consistently changed from the signaled to the unsignaled schedule. Changing from the unsignaled to the signaled schedule was observed only occasionally and only at low levels. The same outcome was observed in Experiment 2 with different types of visual signals and with different stimulus combinations identifying the signal period, the signal-absent period, and the unsignaled schedule. In Experiment 3 the visual signal was replaced with an auditory signal for four of the subjects tested in Experiment 2. The subjects then changed from the unsignaled to the signaled schedule or showed a substantial reduction in choice for the unsignaled schedule. The data were assessed using a conditioned-reinforcement interpretation of choice.     

Preference for information about intensity of signaled tailshock

Three experiments are reported which were designed to demonstrate preference for signaled shock and to delineate the roles of the preparatory-response, information-seeking, and signaled-safety hypotheses of this phenomenon. Experiment 1 demonstrated a preference for a signal-tailshock condition over a tailshock-signal condition in a shuttlebox. In Experiment 3, rats were exposed to signaled tailshock of two intensities in both compartments of the shuttlebox, but in one compartment the signals provided information about the intensity of the impending tailshock, whereas in the other compartment the signals did not. The tailshock intensities were .6 and 1.3 mA for one group and .6 and 1.8 mA for a second group. Experiment 2 assured that even the smaller intensity difference was readily discriminable. For Group .6-1.8 a significant preference for the condition in which the signals predicted the intensity of the impending tailshocks was obtained, but no preference for either condition was found for Group .6-1.3. As signaled safety was equated in the two compartments of Experiment 3, the observed preference cannot be accounted for by the signaled-safety hypothesis. Information seeking cannot explain the lack of preference in Group .6-1.3 as the difference between those two intensities was proven discriminable in Experiment 2. Assuming that sufficient differences in shock intensity yield different preparatory responses, the present results are compatible with the preparatory-response hypothesis and suggest that preparatory responding plays a role in producing the preference for signaled shock.     

Dual effects on choice of conditioned reinforcement frequency and conditioned reinforcement value

Pigeons were presented with a concurrent‐chains schedule in which the total time to primary reinforcement was equated for the two alternatives (VI 30 s VI 60 s vs. VI 60 s VI 30 s). In one set of conditions, the terminal links were signaled by the same stimulus, and in another set of conditions they were signaled by different stimuli. Choice was in favor of the shorter terminal link when the terminal links were differentially signaled but in favor of the shorter initial link (and longer terminal link) when the terminal links shared the same stimulus. Preference reversed regularly with reversals of the stimulus condition and was unrelated to the discrimination between the two terminal links during the nondifferential stimulus condition. The present results suggest that the relative value of the terminal‐link stimuli and the relative rate of conditioned reinforcer presentation are important influences on choice behavior, and that models of conditioned reinforcement need to include both factors.     

Modeling signal features of escape response: effects of cessation conditioning in "learned helplessness" paradigm

Six experiments examined the effects of signaling the termination of inescapable shock (cessation conditioning) or shock-free periods (backward conditioning) on later escape deficits in the learned helplessness paradigm, using rats (Sprague-Dawley and Bantin-Kingman). A cessation signal prevented later performance deficits when highly variable inescapable shock durations were used during pretreatment. The inclusion of short minimum intertrial intervals during pretreatment did not alter the benefits of cessation conditioning but eliminated the protection afforded by a safety signal. The beneficial effects of both cessation and backward signals were eliminated when a single stimulus signaled shock termination and a shock-free period. Finally, a combination of cessation and backward signals was found to be most effective in immunizing against the effects of subsequent unsignaled, inescapable shock on later escape performance. These data suggest that cessation conditioning may be crucial to the prophylactic action of an escape response.     

Preference for signaled versus unsignaled reinforcement delay in concurrent-chain schedules

A concurrent-chain schedule was employed to examine pigeons' preferences for signaled versus unsignaled delay of reinforcement in which the delay durations ranged from zero to ten seconds. In general, pigeons preferred signaled delay over unsignaled delay especially when a variable-interval 30-second schedule operated in each initial link; when a variable-interval 90-second schedule operated in each initial link, these preferences tended toward indifference or were attenuated. In addition, prior training seemed to exert partial control over behavior. Responding in the terminal link was higher under signaled delay than unsignaled delay in a majority of the cases. Moreover, response rates under signaled delay remained fairly constant whereas responding under unsignaled delay was initially high, but decreased systematically with delay durations as short as 2.5 seconds. These results are consistent with a number of other studies demonstrating the significant role of a signal for impending positive stimuli.     

Contiguity and conditioned reinforcement in probabilistic choice

In a baseline condition, pigeons chose between an alternative that always provided food following a 30-s delay (100% reinforcement) and an alternative that provided food half of the time and blackout half of the time following 30-s delays (50% reinforcement). The different outcomes were signaled by different-colored keylights. On average, each alternative was chosen approximately equally often, replicating the finding of suboptimal choice in probabilistic reinforcement procedures. The efficacy of the delay stimuli (keylights) as conditioned reinforcers was assessed in other conditions by interposing a 5-s gap (keylights darkened) between the choice response and one or more of the delay stimuli. The strength of conditioned reinforcement was measured by the decrease in choice of an alternative when the alternative contained a gap. Preference for the 50% alternative decreased in conditions in which the gap preceded either all delay stimuli, both delay stimuli for the 50% alternative, or the food stimulus for the 50% alternative, but preference was not consistently affected in conditions in which the gap preceded only the 100% delay stimulus or the blackout stimulus for the 50% alternative. These results support the notion that conditioned reinforcement underlies the finding of suboptimal preference in probabilistic reinforcement procedures, and that the signal for food on the 50% reinforcement alternative functions as a stronger conditioned reinforcer than the signal for food on the 100% reinforcement alternative. In addition, the results fail to provide evidence that the signal for blackout functions as a conditioned punisher.     

Suboptimal choice and initial‐link requirement

Pigeons (n = 14) were trained in a concurrent‐chains suboptimal choice procedure that tested the effect of an increased ratio requirement in the initial links. Fixed‐ratio 1 and 25 conditions were manipulated within subjects in a counterbalanced order. In all conditions, distinct terminal‐link stimuli on a suboptimal alternative signaled either primary reinforcement (20% of the time) or extinction (80% of the time). On an optimal alternative, two distinct terminal‐link stimuli each signaled a 50% chance of primary reinforcement. Preference for the suboptimal alternative was significantly attenuated, and in some birds completely reversed, by the larger response requirement irrespective of condition order. This larger response requirement also generated a notable increase in between‐subject variability. A measure of cumulative choice responding is introduced to mitigate the problems associated with traditional session averages. Ordinal predictions of some current theories of suboptimal choice are also considered in light of the results.     

The preference-for-signalled-shock phenomenon: Signalling shock is reinforcing only if shock is modifiable

Inescapable, unavoidable, 5-sec. shocks were preceded by 5-sec. white-noise signals contingent on previously food-trained (FR 40) bar pressing of 20 rats. Twenty yoked-control subjects receiving the same, but non-contingent, series of signals indicated that the signal was reliably reinforcing only if the shocks were delivered through an alternate, rather than scrambled, floor-grid system. Implications are discussed for previous reports of the preference for signalled shock phenomenon.     

Strict and random alternation in concurrent variable-interval schedules

Six pigeons responded on pairs of concurrent variable-interval schedules with, in different parts, four different arrangements of alternation between schedules. Following a single switching-key response, alternation was either strict or random, and the alternative presented after a switch (the postswitch alternative) was either signaled by the location of the switching key or unsignaled. Generalized-matching analyses showed little difference in behavior among the different alternation arrangements, except the usual finding of lower sensitivity of response allocation than time allocation was eliminated by arranging random alternation. Patterns of interchangeover times were similar for all arrangements except signaled random alternation. Differences in behavior preceding the different postswitch alternatives were found in the signaled random alternation procedure. Preference was biased towards the color of the signaled postswitch alternative and showed increased sensitivity when the postswitch alternative was to be the one with the higher reinforcer rate. Interchangeover times were substantially shorter when the postswitch alternative was signaled to be different from the current alternative than when it was signaled to be the same. However, when separate reinforcer ratios were calculated for the different postswitch alternatives, those effects were eliminated or greatly reduced. We suggest that, although behavior is indeed influenced by the postswitch alternative, the mechanism is indirect. That is, the distributions of reinforcers between alternatives obtained before each postswitch alternative differ when those alternatives are signaled, and those distributions are discriminated, but the same relations between choice and relative reinforcement hold irrespective of which postswitch alternative is signaled.     

Effects of diazepam on schedule-controlled and schedule-induced behavior under signaled and unsignaled shock.

Schedule-controlled lever pressing and schedule-induced licking were studied in rats under a multiple fixed-interval fixed-interval schedule of food reinforcement upon which was superimposed a multiple variable-time variable-time schedule of electric-shock delivery. Shocks were signaled in one component of the multiple schedule and unsignaled in the other. The effects of diazepam upon the suppression of behavior during the signal (conditioned suppression) and during signaled and unsignaled shock (differential suppression) were studied under several shock intensities (Experiment 1) and at increased body weight (Experiment 2). In each study, diazepam led to dose-dependent increases in the rate of pressing and licking during signaled and unsignaled shock, but had little effect on conditioned suppression. the rate-enhancing effects of diazepam depended upon the intensity of shock, nature of the response, and whether or not shocks were signaled. The data was discussed in terms of (1) implications for understanding the effects of signaled and unsignaled shock on behavior, (2) the effects of diazepam on behavior suppressed by response-independent shock, and (3) comparison between operant and schedule-induced behavior.     

A parametric analysis of the relationship between food quantity and rumination

Rumination is the chronic regurgitation, chewing, and reswallowing of previously ingested food. The study reported here, using a parametric design, examined the control of rumination by the quantity of food eaten at meals. The subjects were three profoundly retarded individuals who chronically emitted this behavior. The quantity of food by weight ingested daily was varied in 10-oz steps in both ascending and descending series (data were collected only after breakfasts and lunches). Ruminating decreased when food quantity increased and increased when food quantity decreased. In addition, there was a similar inverse interaction between breakfast food quantity and post-lunch ruminating. The data showed relatively rapid transitions in both frequency and duration at each meal size for all subjects. The data establish a clear functional relation between the quantity of food ingested and ruminating.     

Shock as a signal for shock or no-shock: a feature-negative effect in conditioned suppression.

Rats were trained in conditioned suppression discriminations where shock at the beginning of a trial signaled either shock or no-shock at the end of the trial. In the shock-positive condition, shock at the beginning of a presentation of white noise signaled that noise would end with shock; noise that did not begin with shock did not end with shock. In the shock-negative discrimination, shock at the beginning of noise signaled that noise would not end with shock; presentations of noise that did not begin with shock ended with shock. In shock-random training, shock at the beginning of noise did not reliably signal whether the noise presentation would or would not end with shock. Most subjects in shock-negative training quickly developed a differential pattern of suppression on positive (shock reinforced) trials and no suppression on negative (nonreinforced) trials. The shock-positive discrimination was much more difficult to establish and was not acquired by the majority of the rats. This "feature-negative" effect is a clear exception to the general superiority of feature-positive learning commonly observed in discriminations based on a single distinguishing feature. The results are discussed in terms of Pavlovian stimulus-shock contingencies in the shock-positive and shock-negative paradigms, which appear to favor rapid development of the shock-negative discrimination.     

Evaluation of informational control and preparatory response factors in classical aversive conditioning

Subjects performed in a differential eyelid conditioning paradigm with either airpuff or infraorbital shock as the UCS. The trial series included interpolated UCS-alone presentations, and subjects rated UCS aversiveness on all trials. Ratings of the airpuff, but not the shock UCS, were negatively correlated with the magnitude of anticipatory eyelid CRs, as predicted by preparatory response or law-of-effect models of classical conditioning. However, subjects showed no tendency to rate signaled (CS-UCS) trials as less aversive than unsignaled (UCS-alone) trials, and showed no significant preference for the signaled ucs. These results suggest that the operation of informational control and preparatory response factors is more complex than is assumed by available theories. Also, instrumental shaping and preparatory response mechanisms may not be involved in the acquisition of CRs.     

Variables affecting preference for signaled shock in a symmetrical changeover design

In a signaled-shock situation, the signal's presence serves a warning function (prediction of shock) whereas the signal's absence serves a safety function (prediction of nonshock periods). The relative contribution to preference for signaled shock made by these two potential sources of reinforcement was assessed using a symmetrical changeover procedure in which rats were permitted to control the amount of time spent in one of two mutually exclusive conditions, the signaled and unsignaled states. The stimulus conditions that prevailed in the signaled state differed for several groups of animals. For some groups, the safety function of signals was degraded while the warning function was left intact. For other groups, the warning function was degraded and the safety function was not. The availability of the unaltered feature was then manipulated in both sets of groups to determine its effect on preference for the signaled state. Changeover behavior was strongly related to the availability of the signal's safety function and weakly related to its warning function, indicating that the two variables contribute differentially to preference for signaled shock. Other sources of behavioral control, having little to do with the safety or the warning function of signals, were also demonstrated, suggesting the need for careful control procedures when using changeover behavior as a measure of preference.