Sequential reacquisition as a function of timeout from avoidance

Rats learned to reacquire four similar three-member response sequences. Each sequence member was associated with a different response lever, and the correct sequence of levers (i.e., 3-1-2, 2-1-3, 1-3-2, and 2-3-1) changed each session. The first two correct responses of each sequence postponed shock for a fixed period of time. The third correct response initiated a signalled timeout from avoidance. Incorrect responses did not affect the shock interval or reset the sequence. The effects of manipulating timeout duration on the sequential reacquisition baseline were investigated. All subjects displayed biphasic reacquisition performances similar to those controlled by food. The phases were characterized by an initial increase in accuracy, which reached a stable level during the latter portion of each session. Timeout duration affected rate of sequence completion and shock density, but not percentage of errors. Rate of sequence completion was fastest with intermediate timeouts (15 to 60 sec), and slowest with extreme durations (1 or 120 sec). Shock densities peaked with extreme durations and were at minimum with intermediate timeout values. The percentage of errors was the same across timeout durations. These data extend the generality of sequential reacquisition as a procedure for studying learning, and demonstrate timeout from avoidance to be a controlling variable.     

Repeated acquisition in the analysis of rule-governed behavior

Five children, ranging in age from 3½ years to 5½ years, were taught various four-response chains using conditioned reinforcement. Experiment 1 investigated the effects of presenting “instruction” stimuli—a sequence of lights over the correct response buttons—to assess their role in facilitating the acquisition of a chain of responses. Without the “instruction” stimuli, children made many errors before responses were brought under the control of the programmed contingencies. When confronted with the same contingencies later in the day, these subjects made fewer errors. In contrast, in the presence of the “instruction” stimuli, subjects made virtually no errors. However, when the “instruction” stimuli were discontinued in the subsequent session, all 5 subjects made errors. In Experiment 2, the subjects were taught to verbalize the contingencies during the phase without the “instruction” stimuli. This resulted in errorless performance during the subsequent exposure to the same procedure, but errors nevertheless occurred again during reexposure to the procedure with the “instruction” stimuli discontinued.     

The conditioned reinforcement of repeated acquisition

Three monkeys were trained to emit a chain of three responses on three separate levers in a set of six levers to obtain food. The chain producing food (correct chain) was changed each day. During a trial, a press on any lever produced a feedback stimulus; a press on a correct lever produced an additional distinctive stimulus; the third correct press produced a food pellet. Test sessions in which either the food or the distinctive stimuli were removed were interspersed with baseline sessions. In tests without food presentations, the subjects acquired the correct chain rapidly, with a level of accuracy comparable to baseline. Removing the distintive stimuli for either the first or second member of the correct chain greatly retarded acquisition of that member of the chain. Removing all distinctive stimuli often reduced accuracy throughout the chain to chance level, even though food was presented following each correct chain. These results were interpreted as evidence that the distinctive stimuli presented after correct responses functioned as conditioned reinforcers. Reductions in accuracy following an omitted distinctive stimulus indicated that they were also discriminative stimuli for correct responding in their presence.     

Compounding of discriminative stimuli that maintain responding on separate response levers

In Experiment 1, rats' responses were reinforced on a fixed-interval 30-sec schedule in the presence of either a light or a tone and were not reinforced in their absence. Each stimulus was correlated with its own response lever, with only one lever present during a session. When light and tone were compounded in the presence of the tone-correlated lever, no change in responding occurred. However, when tone was compounded with light in the presence of the light-correlated lever, level of responding was greater than to light alone (response summation). Summation was also found when each stimulus was correlated with the same lever. Next, light and tone were again correlated with separate levers, but both levers were always simultaneously present. Compounding produced both summation and emission of most responses on the light-correlated lever. This prepotency of light was reduced (1) by leaving a houselight on throughout the session; and (2) by correlating each stimulus with a different schedule (either fixed-interval 4.7-sec or fixed-interval 30-sec). With a medium- and high-intensity houselight and with the different reinforcement schedules, similar results were obtained during compounding, regardless of whether compounding occurred in the presence of the light- or tone-correlated lever.     

Duration-reduction of avoidance sessions as negative reinforcement

Five rats were exposed to a shock-postponement procedure in which responses on each of two levers initially had equivalent effects. After an initial training sequence that ensured at least some responding on each lever, an additional consequence was made conjointly operative on the previously less-preferred lever for each animal. Each response on this lever continued to postpone shock, but also reduced the session duration by one minute. The conjoint contingencies were operative until, through session-shortening responses and the passage of time, the session was scheduled to end in two minutes; during the final two minutes the session-shortening contingency was disabled while the shock-postponement contingency continued to be operative on both levers. When responding shifted to a predominance on the session-shortening lever, the conjoint contingency was shifted to the other lever; for four of the five rats this reversal was followed by two additional reversals. Two of the rats' responding showed clear, strong, and unambiguous sensitivity to the session-shortening contingency. The responding of two others was also systematically controlled by that contingency, but the effects were less clearcut. The fifth animal showed an initial shift when session-shortening was introduced, but its subsequent behavior proved insensitive to reversals of procedure. The results clearly indicate a sensitivity of behavior to events on a time scale quite distinct from that of immediate consequences. They also support an interpretation of avoidance sessions, considered in their entirety, as events whose contingent relationship to behavior can affect that behavior—even in the absence of stimuli that delineate those relationships. Finally, these results support an interpretation of aversively based conditioning within a broader context, analogous to the “open versus closed economy” interpretation of appetitively controlled behavior.     

Effects of d-amphetamine, cocaine, and phencyclidine on the acquisition of response sequences with and without stimulus fading.

In each of three components of a multiple schedule, monkeys were required to emit a different sequence of four responses in a predetermined order on four levers. Sequence completions produced food on a fixed-ratio schedule. Errors produced a brief timeout. One component of the multiple schedule was a repeated-acquisition task where the four-response sequence changed each session (learning). The second component of the multiple schedule was also a repeated-acquisition task, but acquisition was supported through the use of a stimulus-fading procedure (faded learning). In a third component of the multiple schedule, the sequence of responses remained the same from session to session (performance). At higher doses, d-amphetamine, cocaine, and phencyclidine decreased the overall rate of responding and increased the percent errors in all three components. At lower doses, however, the three drugs produced selective effects on errors. Errors were increased in the learning component at lower doses than those required to disrupt the behavior in the faded-learning component. The performance component tended to be the least sensitive to disruptive drug effects. The data are consistent with the view that stimulus fading can modulate the effects of drugs on acquisition.     

Response patterns and cardiovascular effects during response sequence acquisition by humans.

The effects of temporal delays imposed between successive responses and of vitamin C administration were examined on the acquisition of response sequences and on cardiovascular reactivity during sequence acquisition. Thirteen adult subjects (6 female, 7 male), in good health, gave written consent prior to participating in 12 weekly 45-min sessions. Points, exchanged for money after each session, were presented when subjects completed 15-response sequences on a touch-sensitive three-response keypad. A position counter increased from 0 to 14 as subjects emitted correct responses in the sequence. Four novel 15-response sequences were presented each session. No delays were imposed between successive responses during the acquisition of one sequence; delays were imposed immediately following each response during the acquisition of a second sequence, thereby delaying response feedback; delays were imposed following feedback during acquisition of a third sequence, resulting in the removal of the stimulus correlated with sequence position; and, as a control condition, delays were imposed following feedback, but stimuli correlated with sequence position were reinstated prior to the next response during acquisition of a fourth sequence. Subjects were exposed to one of two delay durations (0.2 and 0.5 or 0.5 and 1.0 s) each session, and delay durations alternated every session. During Weeks 5 to 8, subjects received 3 grams of vitamin C per day, whereas during Weeks 1 to 4 and 9 to 12, subjects received placebo under single-blind conditions. All subjects acquired the sequences, as evidenced by decreasing percentages of incorrect responses across trials. When temporal delays were imposed between successive responses during sequence acquisition, acquisition efficiency was enhanced. Examination of response latencies suggested that the status of preceding responses (i.e., correct or incorrect) rather than the status of the position counter influenced subsequent responding. Cardiovascular effects were inversely related to the length of the temporal delay. Neither cardiovascular reactivity or sequence acquisition were related to vitamin C administration.     

Rapid acquisition of an auditory localization discrimination by rats

Acquisition of a sound localization discrimination by rats was investigated. Two loudspeakers were located outside an experimental enclosure containing two levers and a dipper feeder. In the same-side condition, responses on the lever nearest the sound-producing speaker were reinforced. Animals in this condition acquired the discrimination rapidly, generally within the first session. In the opposite-side condition, responses on the lever furthest from the sound-producing speaker were reinforced. Acquisition for animals in this condition began below the chance level (50% correct responses) and took on the order of 10 sessions to approach the final, high level. The course of acquisition in both cases appeared to depend upon an initial tendency of rats to respond on the lever nearest the source of sound in this situation. The rise-decay time of the 4-kHz tone burst signal clearly affected the performance level reached. It did not, however, affect the rate at which the discrimination was acquired.     

Response acquisition with delayed reinforcement: a comparison of two-lever procedures.

Groups of 8 experimentally naive rats were exposed during 8-hr sessions to resetting delay procedures in which responses on one lever (the reinforcement lever) produced water after a delay of 8, 16, 32, or 64 s. For rats in one condition, responses on a second (no-consequences) lever had no programmed consequences. For rats in another condition, responses on a second (cancellation) lever during a delay initiated by a response on the reinforcement lever prevented delivery of the scheduled reinforcer; responses on the cancellation lever at other times had no programmed consequences. Under both conditions and at all delays, most subjects emitted more responses on the reinforcement lever than did control rats that never received water emitted on either lever. At 8-s delays, both conditions engendered substantially more responding on the reinforcement lever than on the other lever, and performance closely resembled that of immediate-reinforcement controls. At delays of 16 and 32 s, however, there was clear differential responding on the two levers under the cancellation condition but not under the other condition. When the delay was 64 s, differential responding on the two levers did not occur consistently under either condition. These findings provide strong evidence that the behavior of rats is sensitive to consequences delayed by 8, 16, and 32 s, but only equivocal evidence of such sensitivity to consequences delayed 64 s. They also indicate that acquisition depends, in part, on the measure of performance used to index it.     

The establishment of stimulus control by instructions and by differential reinforcement

A repeated acquisition design was used to study the effects of instructions and differential reinforcement on the performance of complex chains by undergraduates. The chains required responding on a series of keys that corresponded to characters that appeared on a monitor. Each day, subjects performed a new chain in a learning session and later relearned the same chain in a test session. Experiment 1 replicated previous research by showing that instructional stimuli paired with the correct responses in the learning sessions, combined with differential reinforcement in both learning and test sessions, resulted in stimulus control by the characters in each link. Experiment 2 separated the effects of instructional stimuli and differential reinforcement, and showed that stimulus control by the characters could be established solely by differential reinforcement during the test sessions. Experiment 3 showed that when a rule specified the relation between learning and test sessions, some subjects performed accurately in the test sessions without exposure to any differential consequences. This rule apparently altered the stimulus control properties of the characters much as did differential reinforcement during testing. However, compared to differential reinforcement, the rule established stimulus control more quickly.     

The effects of behavioral history on response acquisition with immediate and delayed reinforcement

Effects of prior exposure to the experimental chamber with levers present or absent and variable-time (VT) 60-s water deliveries arranged during one, five, or no 1-hr sessions were examined in rats during a 6-hr response-acquisition session in which presses on one lever produced water delivery immediately or after a 15-s resetting delay, and presses on the other lever canceled scheduled water deliveries. Response acquisition was (a) slower to occur when water deliveries were delayed, (b) most consistent in groups that had received five VT sessions, and (c) impaired by the presence of levers only when there had been five VT sessions and water deliveries were delayed during the acquisition session.     

A choice procedure to assess the aversive effects of drugs in rodents

The goal of this series of experiments was to develop an operant choice procedure to examine rapidly the punishing effects of intravenous drugs in rats. First, the cardiovascular effects of experimenter‐administered intravenous histamine, a known aversive drug, were assessed to determine a biologically active dose range. Next, rats responded on each of two levers with concurrently available fixed‐ratio 1 schedules of food reinforcement. Intravenous histamine was delivered along with food when responses were made on one of the options, and the lever on which both food and histamine were contingent was switched on a regular basis. A dose of 1.0 mg/kg/inj of histamine was effective in moving responding to the alternate lever, whereas saline, 0.1, or 0.3 mg/kg/inj of histamine were not. Histamine injections produced reliable selection of the alternate lever when they were presented on the same lever for three consecutive sessions, but not when they were switched between levers on each session. In addition, histamine produced greater selection of the alternate lever when it was presented with shorter intertrial interval durations. These findings indicate that, with appropriate parameters, the aversive effects of histamine and perhaps other drugs can be established rapidly using a concurrent choice procedure.     

Repeated acquisition of conditional discriminations

A new technique was developed to study the repeated acquisition of conditional discriminations. Using a discrete trial procedure, pigeons were required to learn during each session a different two-member chain of conditional discriminations. Key color and geometric forms were used as stimuli. After the pigeons had reached a steady state of relearning (40 to 60 sessions), the technique was used to investigate variables that have previously been shown to affect the repeated acquisition of response sequences. Various (0 to 90 seconds) durations of timeout for errors were investigated in Experiment I. The stimulus change associated with a timeout, rather than its duration, was found to be the critical variable in acquisition of the discrimination. Extended training on a single chain was found to reduce total errors across sessions in Experiment II. Extended training (three sessions) did not, however, change the pattern of within-session error reduction. In some cases, extended training facilitated acquisition of a partially reversed discrimination. In Experiment III, color rather than chain position was found to control behavior, for three of the four birds, as the second stimulus dimension in the conditional situation. The results of these experiments replicate and extend previous findings concerning some of the variables that affect the repeated acquisition of response sequences.     

Timeout as a reinforcer for errors in a serial position task

After learning to press keys in a predetermined serial position sequence, with timeouts scheduled as a consequence of errors, monkeys developed stereotyped errors. As soon as a new trial started, the animals would make an error. On trial after trial, they pressed the same incorrect key at the first member of the sequence, even though they had previously learned the sequence. First-member errors occurred even when sequences of fully bright keys marking correct choices were presented. When timeout was eliminated as a consequence of one first-member error, subjects switched to an error that did produce the timeout. When all first-member errors failed to produce timeout the monkeys ceased responding. Both prefeeding and reduction in reinforcement density resulted in stereotyped errors occurring earlier in the session.     

Lever attacking and pressing as a function of conditioning and extinguishing a lever-press avoidance response in rats

Six experimental rats were conditioned to press one of two available levers to avoid shock. The levers registered bites as well as presses. For four of these rats, shock was contingent on lever bites when a specified time period had elapsed after the previous shock. An extinction period, in which only periodic noncontingent shocks were presented, followed avoidance training. Six yoked-control rats received the same sequence of shocks as did the corresponding experimental rats in both the conditioning and extinction phases. All six experimental rats repeatedly bit the avoidance lever. Four bit it more than the nonavoidance lever during conditioning, and five bit it more during extinction. Five of the six experimental rats consistently bit the levers many more times during each session than did their respective control rats, suggesting that avoidance conditioning facilitated lever biting. Rates of lever biting and pressing by all of the experimental rats and by some of the control rats were highest immediately following shock throughout both phases. During later portions of the intervals following shock, characteristic effects of conditioning and extinction were observed. This finding suggests that extinction of avoidance behavior by unavoidable shock presentations can be demonstrated more readily when shock-elicited responding is extricated from the data.     

Teaching serial position sequences to monkeys with a delayed matching-to-sample procedure

Comparison was made of two methods for training monkeys to “observe” a two-member serial position sequence by pressing two consecutively lighted keys and then to “report” the sequence by pressing the same two keys in the same order but without the lights. A fading technique involving gradual elimination of brightness cues from “reporting” keys was found more effective than a no-fading procedure in which the cues remained bright during training and then were suddenly removed. Animals that failed to learn to report a new sequence with the no-fading procedure sometimes developed behavior incompatible with that desired. They made repeated and specific errors that prematurely terminated trials of the sequence to-be-learned, even though the correct key was cued by a bright light. They behaved appropriately, however, on succeeding trials of other sequences. Thus, the errors were followed by trials on which reinforcement occurred. Manipulation of this contingency indicated its importance in maintaining the stereotyped error patterns.     

Drug discrimination in rats under concurrent variable-interval variable-interval schedules

Eight rats were trained to discriminate pentobarbital from saline under a concurrent variable-interval (VI) VI schedule, on which responses on the pentobarbital-biased lever after pentobarbital were reinforced under VI 20 s and responses on the saline-biased lever were reinforced under VI 80 s. After saline, the reinforcement contingencies programmed on the two levers were reversed. The rats made 62.3% of their responses on the pentobarbital-biased lever after pentobarbital and 72.2% on the saline-biased lever after saline, both of which are lower than predicted by the matching law. When the schedule was changed to concurrent VI 50 s VI 50 s for test sessions with saline and the training dose of pentobarbital, responding on the pentobarbital-biased lever after the training dose of pentobarbital and on the saline-biased lever after saline became nearly equal, even during the first 2 min of the session, suggesting that the presence or absence of the training drug was exerting minimal control over responding and making the determination of dose-effect relations of drugs difficult to interpret. When the pentobarbital dose-response curve was determined under the concurrent VI 50-s VI 50-s schedule, responding was fairly evenly distributed on both levers for most rats. Therefore, 6 additional rats were trained to respond under a concurrent VI 60-s VI 240-s schedule. Under this schedule, the rats made 62.6% of their responses on the pentobarbital-biased lever after pentobarbital and 73.5% of their responses on the saline-biased lever after saline, which also is lower than the percentages predicted by perfect matching. When the schedule was changed to a concurrent VI 150-s VI 150-s schedule for 5-min test sessions with additional drugs, the presence or absence of pentobarbital continued to control responding in most rats, and it was possible to generate graded dose-response curves for pentobarbital and other drugs using the data from these 5-min sessions. The dose-response curves generated under these conditions were similar to the dose-response curves generated using other reinforcement schedules and other species.     

Bias functions and operating characteristics of rats discriminating auditory stimuli

Rats were trained to discriminate between two bursts of random noise that differed in intensity. In a two-lever, discrete-trial procedure, correct responses were reinforced with brain stimulation, and incorrect responses produced timeout. Responding was studied as a function of the decibel difference between the stimuli, the probabilities of presenting the stimuli, the relative duration of timeout consequent upon the two types of incorrect responses, and the absolute duration of timeout consequent upon incorrect responses. The results showed that the distribution of responses between the two levers depended upon the stimulus probabilities, but were independent of either the absolute or relative durations of timeout. When the stimulus probabilities were varied, the response probabilities did not match the stimulus probabilities; when the relative durations of timeout were varied, the animals did not obtain the maximum rate of reinforcement per unit time. Instead, the animals distributed their responses so as to obtain the maximum number of reinforcements at each level of discrimination. In addition, the level of discrimination increased as a function of the decibel difference between the stimuli.     

Control of responding by the location of an auditory stimulus: role of rise time of the stimulus

The control of responding by the location of tone bursts of 0.2- or 50-msec rise time was investigated in three albino rats. The apparatus consisted of an enclosure with two levers, two loudspeakers (in different locations), and a dipper feeder. The animal was exposed to tone bursts from either one or the other of the two speakers, and the speaker through which the tone bursts were delivered on any particular trial alternated in an irregular manner. Responses on one lever were reinforced with food in the presence of tone bursts from one speaker; responses on the second lever were reinforced with food in the presence of tone bursts from the second speaker. Responding came under the control of the location of 4-kHz tone bursts of 0.2-msec rise time within the first session. At this rise time, animals maintained a stable level of correct responding of greater than 95%. When the rise time was increased to 50 msec the percentage of correct responding fell to an average of 80 to 85%. It was concluded that location of an auditory stimulus is a powerful controller of responding in rats and that the degree of control is dependent upon rise time.     

Responding under chained and tandem fixed-ratio schedules

The role of stimuli in chained fixed-ratio schedules of reinforcement was examined. At various ratio values, responding on schedules consisting of three or five equal components, with a different colored light in each component (“block counter”) was compared with responding on tandem or simple fixed-ratio schedules having the same color present throughout the entire ratio. At all ratio values except the smallest, the chain stimuli resulted in longer pauses after reinforcement. The magnitude of this effect became greater as the size of the ratio was increased. Post-reinforcement pause durations were longer under five-component schedules than under three-component schedules. Running rates in the first component were lower on the chained schedules than on the tandem schedules; on both kinds of schedule, rates were lower in the first component than in the rest of the ratio. When the sequence of stimuli was reversed, the duration of the post-reinforcement pause dropped markedly and the running rate in the initial component increased, but these effects gradually disappeared after the first reversal session. When the final chain stimulus was substituted for the first component stimulus but continued to appear in the final chain component as well, the pause duration dropped and remained at this lower level during subsequent sessions.