The effect of S− on observing behavior

Eight pigeons were run on a one-key, discrete-trials observing procedure. Pecks during a trial produced S⁺ and S^?, colored key lights which signalled whether the trial would end with response-independent grain reinforcement or nonreinforcement. S⁺ and S^? were produced on a VI schedule which began operating at the onset of the trial. In the main experimental condition, only a response preceded by at least 6 sec of nonresponding could produce S^? on nonreinforced trials; any response which satisfied the VI requirement produced S⁺ on reinforced trials. This procedure allowed the birds to produce S⁺ on reinforced trials with or without producing S^? on nonreinforced trials. The subjects learned to produce fewer S^?s over sessions, indicating that S^? had a punishing effect on observing. The results were taken as evidence for the conditioned reinforcement hypothesis of observing and against the uncertainty reduction hypothesis.     

Responding in the squirrel monkey under fixed-ratio schedules of stimulus termination

Key pressing was maintained under a fixed-ratio schedule in which electric shock was scheduled for delivery at a fixed time (t seconds) after each stimulus onset, and every n^th response terminated the stimulus and initiated a timeout from shock. Under this procedure, the higher the rate of responding, the briefer the duration of the stimulus presentation and the lower the frequency of shock delivery. The effects of several schedule parameters were studied to determine whether the maintenance of responding was dependent on an inverse relation between response rate and shock frequency. Shock rate and shock frequency were made independent of response rate by decreasing the value of t to 0.5 second and delivering shock only during the first presentation of the stimulus after a fixed time, including stimulus and timeout durations, had elapsed since the previous shock. The experiments showed that shock frequency and response rate are inversely related when t is of relatively long duration compared to the value of the fixed-ratio parameter, but that a decrease in shock rate or frequency due to a high rate of responding is not necessary for the maintenance of responding under a fixed-ratio schedule of stimulus termination.     

Increased reinforcement when timeout from avoidance includes access to a safe place

Three experiments investigated the reinforcing value of access to a safe place during timeout from an avoidance schedule. Rats were trained on conjoint schedules in which responding both postponed shock on a free-operant avoidance schedule and produced periods of timeout on fixed-ratio schedules. In some conditions, a shelf was inserted into the operant chamber during timeout, enabling subjects to get off the grid floor. The combination of timeout and shelf maintained substantially higher response rates than the baseline avoidance schedule with ratio requirements as high as 90 (Experiment I). Adding the shelf to timeouts in one component of multiple fixed-ratio schedules of timeout resulted in higher response rates in the component where the shelf was included (Experiment II). When timeouts with and without the shelf were arranged on concurrent schedules, the shelf-timeout combination was preferred, even when of shorter duration than timeout alone (Experiment III). In all three experiments, subjects climbed on the shelf, although all shocks were cancelled during timeout periods. The results could not be accounted for solely in terms of the reinforcing properties of changes in shock rates, but required an interpretation that ascribed conditioned reinforcing value to stimuli associated with such changes.     

ASSESSING THE ROLE OF EFFORT REDUCTION IN THE REINFORCING EFFICACY OF TIMEOUT FROM AVOIDANCE

Rats responded on concurrent schedules of shock‐postponement or deletion (avoidance) and timeout from avoidance. In Experiment 1, 3 rats' responses on one lever postponed shocks for 20 s and responses on a second lever produced a 1‐min timeout according to a variable‐interval 45‐s schedule. Across conditions, a warning signal (white noise) was presented 19.5 s, 16 s, 12 s, 8 s, or 4 s before an impending shock. Raising the duration of the warning signal increased both avoidance and timeout response rates. Timeout responding, although positively correlated with avoidance responding, was not correlated with the prevailing shock rate. In Experiment 2, 3 rats' responses on one lever deleted scheduled shocks according to a variable‐cycle 30‐s schedule and responses on a second lever produced a 2‐min timeout as described above. After this baseline condition, the avoidance lever was removed and noncontingent shocks were delivered at intervals yoked to the receipt of shocks in the baseline sessions. Timeout responding decreased when the avoidance lever was removed, even though the shock‐frequency reduction afforded by the timeout remained constant. These results suggest that a key factor in the reinforcing efficacy of timeout is suspension of the requirement to work to avoid shock, rather than the reduction in shock frequency associated with timeout.     

Fear as a discriminative stimulus for an appetitive instrumental response

Two experiments tested the hypothesis that anticipation of shock could be established as a discriminative stimulus for an appetitive instrumental response. In multiphase experiments, bar pressing for food was brought under the discriminative control of intermittent and gradually increasing electric shock. In a second phase, tones were estalished as either a CS+, or CS? for shock. Subsequently, the CSs were introduced on to the operant baseline. Animals trained with shock as the S^D showed an increase in responding to the CS+ and a slight decrease to the CS?. Conversely, animals trained with shock as the S^Δ showed decreased responding to the CS+ and slight increase to the CS?. These findings are seen as supportive of the Discriminative Stimulus hypothesis of learned resistance to punishment.     

Concurrent performances: effect of punishment contingent on the switching response

Pigeons' key-pecking responses were maintained under concurrently available variable-interval schedules of reinforcement. Responses in the presence of two different key-colors were reinforced on two independent and concurrent variable-interval schedules of food reinforcement, each associated with one of the key colors (red or green). Pecks at a second key (changeover key), always white, would alternate the colors on the main key. In Exp. 1 and 2, electric shock of 50 msec duration followed immediately after changeovers. The proportion of responses in the presence of the color associated with the higher frequency of reinforcements per hour was a direct function of shock intensity contingent on changeovers. When both schedules provided equal number of reinforcements per hour, there was no systematic effect of shock intensity on response distribution. In Exp. 3, a timeout period was contingent on changeovers, and response distribution was a function of timeout length.     

Response-produced timeouts under a progressive-ratio schedule with a punished reset option

Three behavioral options were available to food-deprived pigeons: (1) pecking one key resulted in food reinforcement according to a 50-response progressive-ratio schedule, (2) pecking a second key reset the progressive-ratio schedule to the initial progressive-ratio step, and (3) pecking a third key produced a 3-min timeout period. Pecks on the reset key were shocked. Under low and intermediate shock intensities, timeouts were not produced; under high shock levels, timeouts were produced regularly. Timeouts occurred during the initial period of a progressive-ratio step and were more frequent during the longer steps of the progressive-ratio schedule. Response-produced timeouts under these experimental conditions could be interpreted either as an escape from aversive behavioral options or as a low-probability behavior emerging when the food reinforcement schedule exerted weaker control.     

Discriminative functions based on a delay in the reinforcement relation

Pigeons were shown to come under discrimination control when the S^D and S^Δ were temporally separated from reinforcement and non-reinforcement. S^D and S^Δ consisted of distinctive key illuminations presented separately. Responding on an FR 5 in the presence of S^D or S^Δ produced a third stimulus containing a schedule requirement. If this third (or interpolated) stimulus was preceded by S^D, responding in its presence produced reinforcement followed by a time-out (TO). If, on the other hand, the third stimulus was preceded by S^Δ, responding produced TO alone. In this fashion, the same stimulus and the same response requirement were imposed between S^D and the reinforcement as between S^Δ and the TO. In Experiment I, the schedule employed during the interpolated stimulus was FR; in Experiment II, FI. Discrimination reversal was accomplished in both experiments.     

Schedules using noxious stimuli. III. Responding maintained with response-produced electric shocks

Responding was maintained in two squirrel monkeys under several variations of a 10-min fixed-interval schedule of electric shock presentation. The monkeys were first trained under a 2-min variable-interval schedule of food presentation, and then under a concurrent schedule of food presentation and shock presentation. In one monkey, when shocks (12.6 ma) followed each response during the last minute of an 11-min cycle ending with a timeout period, responding was increased during the first 10 min and suppressed during the last minute of each cycle. When the shock schedule was eliminated, both the enhancement and suppression disappeared, and a steady rate of responding was maintained under the variable-interval schedule. When the food schedule was eliminated, the shock schedule maintained a characteristic fixed-interval pattern of responding during the first 10 min, but suppressed responding during the last minute of each cycle. The fixed-interval pattern of responding was maintained when the timeout period was eliminated and when only one shock could occur at the end of the cycle. In the second monkey, responding under the concurrent food and shock schedule was suppressed when responses produced shocks after 3-min. Under an 11-min cycle, responding continued to be maintained at increasing shock intensities. When the food schedule was eliminated, a fixed-interval pattern of responding was maintained under a 10-min schedule of shock presentation (12.6 ma). Whether response-produced electric shocks suppressed responding or maintained responding depended on the schedule of shock presentation.     

Response-dependent and response-independent timeout from an avoidance schedule

While rats were responding in a single-lever apparatus to avoid electric shock, a signal was presented and followed by a 5-min timeout period when all shocks were omitted. For the response-dependent member of each yoked pair, the first response 60 sec after onset of the pre-timeout signal terminated the signal and initiated timeout. The other, yoked animal was exposed to the same sequence except that signal termination and timeout onset were response independent. Under the response-dependent condition, response rates in the presence of the signal increased relative to baseline rates. Rate increases also occurred when timeout was response independent, but were of lesser magnitude and reliability. Subsequent reversal of the yoking arrangement produced stable and equivalent rate increases under both conditions. Other findings were that increased rates in the presence of the signal diminished when timeout was omitted but were maintained for a time on an avoidance-extinction baseline. In general, the results supported the conclusion of previous experiments that timeout from avoidance can serve as a positive reinforcer. The finding that response-independent presentation of timeout produced rate increases, particularly after a history with response-dependent timeout, was interpreted in terms of adventitious reinforcement of previously established behavior.     

The effects of cocaine on behavior maintained by timeout from avoidance.

Rats were trained on concurrent schedules under which responses on one lever postponed shock (avoidance) and responses on the other lever produced brief (2-min) periods of signaled timeout from avoidance. For 6 rats, timeout from avoidance was programmed on a variable-interval 45-s schedule that generally resulted in rates that were lower than those on the avoidance lever. For another 6 rats, timeout was arranged on a variable-ratio 15 schedule that produced higher baseline rates. Cocaine (3 to 40 mg/kg) produced large, dose-dependent increases in behavior maintained by timeout in both groups of rats. Avoidance responding was also generally increased by cocaine, but the increases were of lesser magnitude. Increases in response rates were seen across a broad range of doses on behavior maintained by either interval or ratio schedules, an outcome that was unexpected on the basis of most studies of cocaine on food-maintained behavior. These results were similar to those of previous studies of the effects of amphetamine on behavior maintained by timeout from avoidance and suggest that stimulant drugs affect behavior maintained under a shock-postponement schedule differently than they affect behavior maintained by timeout from avoidance.     

Aversive properties of the negative stimulus in a successive discrimination

Experiment I sought to determine if the stimulus correlated with extinction in a successive discrimination was an aversive stimulus. An escape response provided an index of aversive control. Two groups of pigeons were exposed to a multiple variable-interval 30-sec extinction schedule. For the experimental group, a single peck on a second key produced a timeout during which all lights in the chamber were dark. For the control group, pecks on the second key had no contingency. The rate of responding on the timeout key during extinction for the experimental group was higher than that of the control group during all sessions of discrimination training except the first. In Exp. II, green was correlated with variable interval 30-sec and red was correlated with variable-interval 5-min. Timeouts were obtained from variable-interval 5-min. There were more timeouts from extinction in Exp. I than from variable-interval 5-min in Exp. II. Experiment III showed that not presenting the positive stimulus reduced the number of timeouts from the negative stimulus for the two birds from Exp. I that had the highest rate of timeouts from extinction, but had little effect on the two birds that had the lowest rate of timeouts. These results suggest that in a multiple schedule, the stimulus correlated with extinction, or the lower response rate, functions as a conditioned aversive stimulus. Explanations of the timeout response in terms of extinction produced variability, displaced aggression, and stimulus change, were considered but found inadequate.     

Some notes on time out from reinforcement

Pigeons produced food on a fixed-ratio schedule by pecking at one key, and an S^Δ period by pecking at a second (switching) key. Switching behavior was examined as a function of (a) size of the fixed ratio, (b) whether the S^Δ was of fixed duration or could be determined by the bird, (c) the introduction of a novel food S^D, (d) extinction of food responding, and (e) the stimuli associated with the S^D and S^Δ conditions. No monotonic relationship was obtained between ratio size and switching behavior. Switching behavior was, however, influenced by many variables. The results suggest that an interpretation of switching behavior in terms of its being reinforced by the removal of aversive conditions, is open to considerable question.     

Sequential reacquisition as a function of timeout from avoidance

Rats learned to reacquire four similar three-member response sequences. Each sequence member was associated with a different response lever, and the correct sequence of levers (i.e., 3-1-2, 2-1-3, 1-3-2, and 2-3-1) changed each session. The first two correct responses of each sequence postponed shock for a fixed period of time. The third correct response initiated a signalled timeout from avoidance. Incorrect responses did not affect the shock interval or reset the sequence. The effects of manipulating timeout duration on the sequential reacquisition baseline were investigated. All subjects displayed biphasic reacquisition performances similar to those controlled by food. The phases were characterized by an initial increase in accuracy, which reached a stable level during the latter portion of each session. Timeout duration affected rate of sequence completion and shock density, but not percentage of errors. Rate of sequence completion was fastest with intermediate timeouts (15 to 60 sec), and slowest with extreme durations (1 or 120 sec). Shock densities peaked with extreme durations and were at minimum with intermediate timeout values. The percentage of errors was the same across timeout durations. These data extend the generality of sequential reacquisition as a procedure for studying learning, and demonstrate timeout from avoidance to be a controlling variable.     

Effectiveness of debriefing after a stress-producing deception

This experiment was carried out to determine if after deceiving Ss an E had sufficient credibility to effectively debrief the Ss. Ss in two stress groups were given false information indicating that when a signal light came on they would receive painful electric shocks. Ss in a no-stress group were not told about shocks. After being given the threatening instructions, Ss in one of the stress groups were debriefed concerning the deception (but the “shock electrodes” were not removed) while Ss in the other group were not debriefed. After this the signal light came on for all groups. Arousal was measured by pulse rates and self-reports. The results revealed that (a) the instructions about electric shocks increased the arousal levels of the two stress groups relative to the no-stress group, thus indicating that the deception was effective, and (b) after the signal light came on the stress group which had not been debriefed continued to show a high degree of arousal while the stress group which had been debriefed showed a reduction of arousal to the same low level of arousal as the no-stress group, thus indicating that the process of debriefing had been effective.     

On unpredictability as a causal factor in “learned helplessness”

In two replications, two groups of dogs were exposed to a series of uncontrollable, electric shocks. For one group the shocks were preceded by a tone (i.e., Paired). For the other group the shocks were randomly related to the tones and hence unpredictable (i.e., Random). Each replication also included a third group; in the first it was exposed only to the series of tones (CS-only), while in the second, it was exposed only to a series of shocks (Shocks-only). Then, all dogs were required to learn a discriminative choice escape/avoidance task in which the required response was to lift the correct paw in the presence of each of two visual S^Ds to escape or avoid the shocks [($\text{S}{}_1{}^{\mathrm{D}}\text{?R}{}_1\text{)}\text{(S}{}_2{}^{\mathrm{D}}\text{?R}{}_2$)]. Dogs preexposed to random tones and shocks were least successful in learning the task relative to those groups which experienced either predicted shocks, only the tones, or only the shocks, which in turn did not differ from each other. These results permitted the inference that the proactive interference with choice behavior following random tone CSs and shocks was attributable to a learned irrelevance generalized with respect to CSs.     

The effect of timeout on performance on a variable-interval schedule of electric-shock presentation

Responding was maintained in squirrel monkeys under variable-interval schedules of electric shock presentation when a period of timeout followed each response-dependent shock. Response rate decreased when timeout duration was decreased, and responding ceased when timeout was eliminated. These results indicate that under certain conditions, a shock-free period following each response-produced shock is necessary to maintain responding.     

The Clinical Plus the Experimental Emphasis in Elementary Work in Psychology

Forty-two Ss were run in one experimental and in two control conditions. The Ss recalled a short story immediately after reading it and again 24 hours later. All Ss were monitored on two physiological variables during reading. The experimental group was subjected to a 0.2-second delayed auditory feedback in the middle section of the story. One control group received no special treatment; the other received shock at the point in the story at which the experimental group received delayed auditory feedback. Both the experimental group and the control group that received shock showed marked changes in GSR measurements during the reading of the middle section of the story. The immediate memory of the story was significantly poorer for the experimental group as contrasted with that for each of the control groups. For accuracy of delayed memory, there were no significant differences among the three groups.     

The role of instrumental responding and contiguity of stimuli in the development of infant secondary reinforcement

Ten-month-old infants received contingent pairings of a tone (T+) and food reinforcer. Groups S^r and S^D received the food on an FI 23-sec schedule for target touching, the former group receiving T+ immediately after the response and 1.5 sec prior to food and the latter group receiving T+ at the end of the intertrial interval. Group SC received food reinforcers 1.5 sec after T+ with no response required. A second tone (Tⁿ) was heard by all groups once during each intertrial interval, at randomly determined points. All groups subsequently were given a spatial discrimination task, receiving T+ for one alternative and Tⁿ for the other. Group S^r gave significantly more responses for T+ than for Tⁿ, but neither of the other two groups produced a superiority for T+. Thus, both contiguity with a primary reinforcer and the presence of an operant during training appear to be necessary for a neutral signal to acquire the ability to enhance responding.     

Observing behavior in monkeys (Macaca arctoides): Support for the information hypothesis

A single lever, discrete-trials observing procedure was used with stumptailed monkeys (Macaca arctoides). Lever-presses during a trial produced colored key lights (IS⁺ and IS^?) which signaled whether the trial would end with response-independent food or without food. During the baseline period, both IS⁺ and IS^? were produced on a variable-interval (VI) 15-sec schedule which began operating at the onset of the trial. The two experimental conditions involved a combination of this VI schedule and a DRL schedule. In one of these conditions, only a response that both met the VI requirement and was preceded by at least 6 sec of nonresponding could produce IS^? on nonfood trials, while the schedule for IS⁺ on food trials remained VI 15 sec. In the other experimental condition, the schedules for producing the two stimuli were the reverse. All subjects eventually learned to produce either IS⁺ or IS^? on the combined VI-DRL schedule. These data support an information hypothesis of observing in monkeys and contrast with data from pigeons which support a conditioned reinforcement hypothesis.