Differential Outcomes Effect on Instrumental Serial Feature-Ambiguous Discrimination in Rats

The Psychological Record - Two groups of rats were trained to press the lever in a serial feature-ambiguous (FA) task. A feature stimulus (X) signaled reinforcement during one target stimulus that...     

Signaled reinforcement effects on fixed-interval performance of rats with lever depressing or releasing as a target response

The effect of a diffuse auditory signal filling a brief delay between a response and reinforcement depends upon the nature of the trained target response. Rats in a signaled condition responded slower than rats in an unsignaled condition if the target response was lever release. If the target response was lever depression, however, rats in the signaled condition responded faster than rats in the unsignaled condition at the end of training     

Conditioned suppression of counting behavior in rats

Three rats were trained on a schedule in which a response on lever B was reinforced only if it was preceded by a minimum number of consecutive responses on lever A. The minimum requirement was 27 A responses for Rat 1, and 20 A responses for Rats 2 and 3. The schedule maintained high rates of responding on lever A, and a slow, spaced pattern of responding on lever B. The mean number of consecutive responses on lever A was slightly greater than the minimum required. The effect of superimposing on this behavior a stimulus that ended with an unavoidable shock was the suppression of responding on both levers during the pre-shock stimulus. Responses on lever A were more suppressed, and the proportion of relatively short response runs on lever A during the pre-shock stimulus increased. With all three rats, the mean number of consecutive responses on lever A during the pre-shock stimulus decreased to a value below the minimum requirement for reinforcement of the subsequent B response.     

Resurgence and downshifts in alternative reinforcement rate

Resurgence refers to an increase in a previously suppressed target behavior with a relative worsening of conditions for a more recently reinforced alternative behavior. This experiment examined the relation between resurgence and the magnitude of a reduction in the rate of reinforcement for the alternative behavior. Groups of both male and female rats initially pressed a target lever for food on a variable-interval (VI) 30-s schedule. In a second phase, responding to the target lever was extinguished for all groups and pressing an alternative lever was reinforced on a VI 10-s schedule. Next, the rate of reinforcement for alternative behavior was reduced differentially across groups by arranging extinction, VI 80-s, VI 40-s, VI 20-s, or continued VI 10-s reinforcement. Target responding increased as an exponential function of the magnitude of the reduction in alternative reinforcement rates. With the exception that males appeared to show higher rates of target responding in baseline and higher rates of alternative responding in other phases, the overall pattern of responding across phases was not meaningfully different between sexes. The pattern of both target and alternative response rates across sessions and phases was well described quantitatively by the Resurgence as Choice in Context model.     

Further evaluation of a nonsequential approach to studying operant renewal

Basic‐laboratory assessments of renewal may inform clinical efforts to maintain reduction of severe destructive behavior when clients transition between contexts. The contextual changes arranged during standard renewal procedures, however, do not necessarily align with those that clients experience during outpatient therapy. More specifically, clients transition between clinical (associated with extinction for target behavior) and home/community (associated with reinforcement for target behavior) contexts during outpatient treatment. Standard renewal assessments do not incorporate these contextual alternations during treatment. The present experiment aimed to directly compare renewal of rats' lever pressing following a standard (“sequential”) ABA renewal procedure (i.e., baseline in Context A, extinction in Context B, renewal test in Context A) and a “nonsequential” renewal assessment wherein treatment consisted of daily alternation between Context A (associated with reinforcement for lever pressing) and Context B (associated with extinction). Lever pressing renewed to a greater extent for rats in the Nonsequential group than for rats in the Sequential group, suggesting the contextual changes that clients experience during outpatient treatment for severe destructive behavior may be a variable that is important to consider in translational research on renewal. Potential implications of these findings for basic and clinical research on renewal are discussed.     

Resurgence and alternative‐reinforcer magnitude

Resurgence is defined as an increase in the frequency of a previously reinforced target response when an alternative source of reinforcement is suspended. Despite an extensive body of research examining factors that affect resurgence, the effects of alternative‐reinforcer magnitude have not been examined. Thus, the present experiments aimed to fill this gap in the literature. In Experiment 1, rats pressed levers for single‐pellet reinforcers during Phase 1. In Phase 2, target‐lever pressing was extinguished, and alternative‐lever pressing produced either five‐pellet, one‐pellet, or no alternative reinforcement. In Phase 3, alternative reinforcement was suspended to test for resurgence. Five‐pellet alternative reinforcement produced faster elimination and greater resurgence of target‐lever pressing than one‐pellet alternative reinforcement. In Experiment 2, effects of decreasing alternative‐reinforcer magnitude on resurgence were examined. Rats pressed levers and pulled chains for six‐pellet reinforcers during Phases 1 and 2, respectively. In Phase 3, alternative reinforcement was decreased to three pellets for one group, one pellet for a second group, and suspended altogether for a third group. Shifting from six‐pellet to one‐pellet alternative reinforcement produced as much resurgence as suspending alternative reinforcement altogether, while shifting from six pellets to three pellets did not produce resurgence. These results suggest that alternative‐reinforcer magnitude has effects on elimination and resurgence of target behavior that are similar to those of alternative‐reinforcer rate. Thus, both suppression of target behavior during alternative reinforcement and resurgence when conditions of alternative reinforcement are altered may be related to variables that affect the value of the alternative‐reinforcement source.     

Behaviour of humans in concurrent schedules programmed on spatially separated operanda

Six human subjects responded on either of two levers for monetary reinforcement in a continuous choice situation. Responses on one lever (lever A) were reinforced at different frequencies specified by five variable-interval schedules. Reinforcements for responding on the other lever (lever B) were delivered according to a variable-interval schedule of standard reinforcement frequency. Results indicated that absolute response rate on lever A increased, while absolute response rate on lever B decreased as a function of reinforcement frequency for lever A. In terms of preference, the data conformed closely to Baum's (1974) generalized matching equation. Performances departing from perfect matching were obtained in four cases, but these deviations were not systematic either in the direction of overmatching or undermatching.     

Differential acquisition of lever pressing in inbred and outbred mice: comparison of one-lever and two-lever procedures and correlation with differences in locomotor activity

Recent progress in mouse genetics has led to an increased interest in developing procedures for assessing mouse behavior, but relatively few of the behavioral procedures developed involve positively reinforced operant behavior. When operant methods are used, nose poking, not lever pressing, is the target response. In the current study differential acquisition of milk-reinforced lever pressing was observed in five inbred strains (C57BL/6J, DBA/2J, 129X1/SvJ, C3H/HeJ, and BALB/cJ) and one outbred stock (CD-1) of mice. Regardless of whether one or two levers (an "operative" and "inoperative" lever) were in the operant chamber, a concomitant variable-time fixed-ratio schedule of milk reinforcement established lever pressing in the majority of mice within two 120-min sessions. Substantial differences in lever pressing were observed across mice and between procedures. Adding an inoperative lever retarded acquisition in C57BL/6J, DBA/2J, 129X1/SvJ, and C3H/HeJ mice, but not in CD-1 and BALB/cJ mice. Locomotor activity was positively correlated with number of lever presses in both procedures. Analyses of durations of the subcomponents (e.g., time to move from hopper to lever) of operant behavior revealed further differences among the six types of mice. Together, the data suggest that appetitively reinforced lever pressing can be acquired rapidly in mice and that a combination of procedural, behavioral, and genetic variables contributes to this acquisition.     

Concurrent performances: a baseline for the study of conditioned anxiety

Three rats were trained to lever press on concurrent random interval 2-min random interval 2-min schedules of milk reinforcement. With a 5-sec changeover delay, relative response rate matched the relative reinforcement duration associated with each lever. A stimulus, during which unavoidable shocks occurred at random intervals, was superimposed on this concurrent baseline, and shifts in preference were found. However, data from this procedure were ambiguous, apparently confounded by shock-elicited response bursts. Termination of the shocks during the stimulus resulted in a rapid recovery of matching, which was preceded by a brief facilitation of responding on the less-preferred lever. The procedure was then changed to a conventional conditioned anxiety paradigm with a variable duration pre-shock stimulus. A marked shift in relative response rate towards the preferred lever was found in all three rats; that is, responding on the preferred lever was far less suppressed during the pre-shock stimulus than responding on the less-preferred lever.     

A chamber for separating visual and physical prey access from predators

A chamber is described that provides automated separation of various phases of depredation dynamics. Two solenoid actuated doors control visual and physical access to prey items. A lever operandum is programmable to require differential response outputs by the predator. Search lever responses open one door and allow detection which may then be followed by pursuit lever responses and ultimate access for capture. The chamber is suitable for such medium sized predators as foxes and coyotes.     

The extent of processing of noise elements during selective encoding from visual displays

Circular visual displays of 12 elements, consisting of A, H, M, and U, were presented to S. She responded with a lever movement in one direction if a given letter designated by a bar indicator was a member of the set A-U and in the opposite direction if it was from the other set. The principal experimental variables were the SOA by which the indicator preceded the display and whether the target letter was flanked by letters of the same or opposite set. The results indicated that the interference produced by noise letters is primarily on the response as opposed to the processing side. Also, there is a limit to the precision of selective attention in the visual field. The interference of opposite-set letters was inversely proportional to their distance from the target.     

Stimuli previously associated with reinforcement mitigate resurgence

Resurgence refers to the recurrence of an extinguished target behavior following subsequent suspension of alternative reinforcement. Delivery of reinforcers during extinction of alternative behavior has been shown to mitigate resurgence. The present experiment aimed to determine whether delivering stimuli associated with reinforcers during resurgence testing similarly mitigates resurgence. Three groups of rats pressed target levers for food according to variable‐interval 15‐s schedules during Phase 1. In Phase 2, lever pressing was extinguished, and an alternative nose‐poke response produced alternative reinforcement according to a variable‐interval 15‐s schedule. Food reinforcement was always associated with illumination of the food aperture and an audible click from the pellet dispenser during Phases 1 and 2. Phase 3 treatments differed between groups. For one group, nose poking continued to produce food and food‐correlated stimuli. Both of these consequences were suspended for a second group. Finally, nose poking produced food‐correlated stimuli but not food for a third group. Target‐lever pressing resurged in the group that received no consequences and in the group that received only food‐correlated stimuli for nose poking. Resurgence, however, was smaller for the group that received food‐correlated stimuli than for the group that received no consequences for nose poking. Target‐lever pressing did not increase between phases in the group that continued to receive food and associated stimuli. Thus, delivery of stimuli associated with food reinforcement after suspension of food reduced but did not eliminate resurgence of extinguished lever pressing. These findings contribute to potential methodologies for preventing relapse of extinguished problem behavior in clinical settings.     

A direct comparison of four methods for eliminating a response

Thirty-two rats pressed one lever (lever A) on a VI 30-sec schedule of food reinforcement and were then shifted to one of four procedures for eliminating the lever A response: extinction, differential reinforcement of other behavior, reinforcement of a different response (pole pushing), and reinforcement of a similar response (pressing lever B). Effectiveness of a response-elimination procedure was measured by (1) how quickly lever A response rate fell to a low level when the procedure was in effect, (2) how much lever A responding recovered when the procedure was discontinued, and (3) how resistant lever A responding was to reinstatement when the VI reinforcement schedule was reimposed. No one method was superior by all three measures. Extinction produced the most variable behavior, while differential reinforcement of other behavior produced the least. Reinforcing alternative behavior produced the greatest recovery in the original lever A response when the response-elimination procedure was discontinued.     

Behavioral effects of delayed timeouts from reinforcement

Timeouts are sometimes used in applied settings to reduce target responses, and in some circumstances delays are unavoidably imposed between the onset of a timeout and the offset of the response that produces it. The present study examined the effects of signaled and unsignaled timeouts in rats exposed to concurrent fixed‐ratio 1 fixed‐ratio 1 schedules of food delivery, where each response on one lever, the location of which changed across conditions, produced both food and a delayed 10‐s timeout. Delays of 0 to 38 s were examined. Delayed timeouts often, but not always, substantially reduced the number of responses emitted on the lever that produced timeouts relative to the number emitted on the lever that did not produce timeouts. In general, greater sensitivity was observed to delayed timeouts when they were signaled. These results demonstrate that delayed timeouts, like other delayed consequences, can affect behavior, albeit less strongly than immediate consequences.     

Discrimination between reinforced action patterns in the rat

Laboratory rats were rewarded for face-washing, rearing, or scratching by being given the opportunity to press retractable levers for food reward. Yoked control animals received the same number of lever presentations and food rewards, but did not have to face-wash, rear, or scratch to obtain the levers. The experimental animals showed increases in number of bouts of reinforced target behavior above control levels, and the total amount of time spent face-washing increased when a 1.5-sec criterion for reinforced bout length was introduced. The activities in this experiment were made to serve a discriminative as well as an instrumental function, since the cue to tell the rat which lever to press for reward when the levers were presented was the activity that the rat had engaged in to obtain lever presentation. In two separate experiments high levels of discrimination between behaviors were obtained. Discrimination was worse following scratching than after other actions, and scratching also showed relatively poor instrumental conditioning. The relationship between Pavlovian and instrumental conditioning processes in this situation is discussed.     

Rats’ acquisition of the ephemeral reward task

The ephemeral reward task provides a subject with a choice between two alternatives A and B. If it chooses alternative A, reinforcement follows and the trial is over. If it chooses alternative B, reinforcement follows but the subject can also respond to alternative A which is followed by a second reinforcement. Thus, it would be optimal to choose alternative B. Surprisingly, Salwiczek et al. (PLoS One 7:e49068, 2012. doi: 10.1371/journal.pone.00490682012) reported that adult fish (cleaner wrasse) mastered this task within 100 trials, whereas monkeys and apes had great difficulty with it. The authors attributed the species differences to ecological differences in the species foraging experiences. However, Pepperberg and Hartsfield (J Comp Psychol 128:298–306, 2014) found that parrots too learned this task easily. We have found that with a similar task pigeons are not able to learn to choose optimally within 400 trials (Zentall et al. in J Comp Psychol 130:138–144, 2016). In Experiment 1 of the present study, we found that rats did not learn to choose optimally in 840 trials; however, in Experiment 2 we added a prior commitment to the initial choice by increasing delay to reinforcement for the choice response from a single lever press to the first lever press after 20 s (FI20 s). In a comparable amount of training to Experiment 1, the rats learned to choose optimally. Although the use of a prior commitment increases the delay to reinforcement, it appears to reduce impulsive responding which in turn leads to optimal choice.     

To press or not to press? Differential receptor expression and response to novelty in rats learning an operant response for reward

Learning to perform instrumental tasks is an ability of all animals. In a population of rats, not all individuals will acquire an operant response for reward. We hypothesized that there could be a genetic explanation for differences between High Consumers (those that acquired the lever press response) and Low Consumers (lever press response is low). Additionally, we proposed that this genetic difference could produce measurable changes in response to novelty. Wistar rats were trained to lever press for a 0.2% saccharin reward and on the 10th day they were placed in a novel open field for 30 min to record locomotor activity. The prefrontal cortex and hippocampus were dissected and qPCR was used to measure mRNA expression. A significant difference (p=.048; 2-way ANOVA) in gene expression was observed between Low and High Consumers. A principal component analysis (PCA), to cluster which genes represent this difference, identified 4 genes; 5-HT2A and mGlu1 in the hippocampus and AMPA GluR1 and adrenergic alpha2A in the prefrontal cortex. Response to a novel open field also differed since Low Consumers displayed a higher Total Distance in comparison to High Consumers. Additionally, Low Consumers could be subdivided into Low-Lever (with lever press response only when water deprived) and Low-Non-Lever (lever press response is low throughout training). PCA with this subdivision identified an additional nine genes differing within the divisions; NMDA NR2B and GABAAalpha3 in the prefrontal cortex and adrenergic alpha2B and alpha2A, AMPA GluR1, GluR2 and GluR3, 5-HT1B and GABAAalpha5 in the hippocampus.     

Discriminative properties of shock-correlated reinforcement in an operant context

A two-choice discrete operant procedure was devised for the study of shock-correlated reinforcement effects in rats. In the presence of one auditory stimulus, responding on one response lever was reinforced with food; with another auditory stimulus, responding on a second lever was reinforced. It was found that discrimination performance of one group, relative to appropriate control groups, was facilitated when electric shock was correlated with reinforcement on one lever and not on the other. Further, relative discrimination levels were found to be higher on the lever correlated with the shock than on the alternate lever. The significance of the results for operant within-S studies and for a mediational theory of shock-correlated reinforcement was discussed.     

Lever attacking by rats during free-operant avoidance

Rats pressed a lever to avoid shock on a free-operant avoidance schedule. Some subjects were also exposed to extinction in which the response-shock contingency was eliminated while the shock-shock contingency remained in effect. A specially constructed lever was used that registered not only presses, but also biting attacks on the lever. Throughout various phases of the study, shocks often elicited lever biting as well as post-shock responding. The results suggested that shock-elicited attacks that are forceful enough to activate the operandum might account for some of the responding that occurs in experiments on free-operant avoidance behavior. In particular, shock-elicited operandum attacking might account for post-shock response bursting during free-operant avoidance and the extreme persistence of responding sometimes noted when shocks are delivered during the extinction of avoidance behavior. To the extent that this is true, these phenomena should not be characterized as operant behavior in interpreting the results of experiments on free operant avoidance.     

Resurgence of goal-directed actions and habits

This study investigated how goal-directed and habitual behaviors recover after extinction within the context of the resurgence effect, a form of relapse induced by the removal or worsening of alternative reinforcement. Rats were trained to press a target lever with one reinforcer (O1) for either minimal (4) or extended (16) sessions. An extinction test after the completion of O1 devaluation confirmed that minimal and extended training formed goal-directed and habitual behaviors, respectively. Then, pressing an alternative lever was reinforced with a second reinforcer (O2) while the target response was placed on extinction. When O2 was discontinued, the minimally trained target response resurged with goal-directed status as in the extinction test. However, the extinguished habitual behavior in the extensively trained rats did not recover as a habit but instead with goal-directed status, possibly due to the context specificity of habits or the introduction of a new response–reinforcer contingency. The critical finding that reinforcer devaluation consistently led to less resurgence regardless of the amount of acquisition training provides a clinical implication that coupling differential-reinforcement-of-alternative-behavior (DRA) treatments with the devaluation of the associated reinforcer of problematic behavior could effectively diminish its recurrence.