Maintained generalization testing of conditioned suppression

This study concerns the use of a multiple stimulus discrimination procedure for producing data on the generalization of conditioned suppression. Four rats were maintained on a variable interval schedule of milk reinforcement in the presence of five stimuli varying in auditory click rate. When response rates were stable, electric shock was regularly paired with the termination of one of the click stimuli. For two rats the shock was paired with the slowest click rate, and for two rats shock was paired with the fastest click rate. The VI schedule remained in effect. Plots of the relative rates of response to each of the five stimuli yielded concave gradients for both animals suppressed at the slowest click rate, and flat gradients with a sharp drop at the warning stimulus for both animals suppressed at the fastest click rate. When the warning stimuli were reversed for both pairs of subjects, both gradient forms were reproduced. The present procedure was contrasted with procedures used by other investigators.     

Acquisition and extinction of signaled avoidance as a function of intermittent reinforcement

Signaled, shuttle-box avoidance responding in female rats of the Fischer₃₄₄ strain was examined as a function of four separate contingencies of intermittent reinforcement. In Experiment 1, when avoidance responses during acquisition were reinforced 25% of the time with prompt CS termination, animals responded equally often during acquisition and significantly more often during extinction than animals who received such reinforcement on a 100% schedule. Similar results were found under a trace procedure in Experiment 2 when avoidance responses were reinforced 25% of the time with informational feedback stimuli. In contrast, during Experiment 3, when animals were shocked on only 25% of the trials on which they failed to respond, the level of avoidance responding during both acquisition and extinction was significantly less than it was when animals were shocked on a 100% schedule. Comparable results were found in Experiment 4 when avoidance responses during acquisition averted shock on only 25% of the trials. Thus, intermittent reinforcement contingencies involving response-contingent feedback stimuli and shock have differential effects on avoidance responding during both acquisition and extinction trials under the signaled avoidance procedure.     

Persistent behavior maintained by unavoidable shocks

Squirrel monkeys were trained on a multiple schedule in which 10-min periods on a continuous shock avoidance schedule, indicated by a yellow light, alternated with 10-min periods on a 1.5-min variable interval schedule of food reinforcement (VI 1.5). A white light indicated that VI 1.5 was in effect, except for the middle 2 min of the period on VI 1.5, in which a blue light appeared and terminated with the delivery of a 0.5-sec unavoidable shock. Stable response rates developed in the avoidance and VI 1.5 components. However, the highest response rates occurred in the blue, preshock stimulus. A series of experiments showed that responding in the blue stimulus persisted even when responding had been extinguished on both the VI schedule of food reinforcement and the shock avoidance schedule. Responding in the blue stimulus ceased when the blue stimulus terminated without shock or when it terminated with a response-contingent shock. Each time responding ceased, it was restored by terminating the blue stimulus with an unavoidable shock. When the blue stimulus was on throughout each session and unavoidable shocks were delivered at regular 10-min intervals, responding was well maintained. These results show that in monkeys that have been trained on a continuous avoidance schedule, unavoidable shocks can maintain responding even under conditions where responses have no programmed consequences.     

Rats' performance on variable-interval schedules with a linear feedback loop between response rate and reinforcement rate

Three experiments investigated whether rats are sensitive to the molar properties of a variable-interval (VI) schedule with a positive relation between response rate and reinforcement rate (i.e., a VI+ schedule). In Experiment 1, rats responded faster on a variable ratio (VR) schedule than on a VI+ schedule with an equivalent feedback function. Reinforced interresponse times (IRTs) were shorter on the VR as compared to the VI+ schedule. In Experiments 2 and 3, there was no systematic difference in response rates maintained by a VI+ schedule and a VI schedule yoked in terms of reinforcement rate. This was found both when the yoking procedure was between-subject (Experiment 2) and within-subject (Experiment 3). Mean reinforced IRTs were similar on both the VI+ and yoked VI schedules, but these values were more variable on the VI+ schedule. These results provided no evidence that rats are sensitive to the feedback function relating response rate to reinforcement rate on a VI+ schedule.     

Reinforcement of spaced responding in a simultaneous discrimination

Pigeons were exposed to three stimuli simultaneously with responses reinforced according to differential-reinforcement-of-low-rate schedules. Responses to one stimulus (the positive stimulus) that were spaced appropriately resulted in food presentation. The variables manipulated were the time parameter of the schedule (5, 10, 20, 30 sec) and the consequences of responding to the other two stimuli (the negative stimuli). The percentage of the total responses that occurred to each stimulus was independent of the schedule value but was dependent on the consequences of responding to the negative stimuli. If responses to both reset the schedule timer, responding was confined largely to the positive stimulus. If responses to neither had scheduled effects, the birds were more likely to respond to those stimuli. Responding to one negative stimulus could be selectively attenuated by having responses to that stimulus alone reset the timer. With the schedule time value held constant, the absolute rate of responding to the positive stimulus was either stable or decreased with maintained exposure; it did not change as a function of increases or decreases in responding to the negative stimuli. Rather than interacting and affecting each other, responses to the three stimuli were controlled independently by their relation to reinforcement. There was no evidence that responses to the negative stimuli mediated the spacing of responses to the positive stimulus.     

Quasireinforcement: Control of behavior by second-order interval schedules

Pigeons were trained on a variable-interval 66-sec schedule of reinforcement that was segmented into either fixed- or variable-interval 10-sec components. Three-second access to food followed some components according to the overall VI 66-sec schedule, but 3-sec periods of nonreinforcement followed the other components. With both FI 10-sec and VI 10-sec segments, overall response rates were generally higher when the completion of unreinforced segments was signaled by a red key (never paired with food) than when it was unsignaled. Response rates during the red-key periods dropped to zero. Brief presentations of the red key engendered the distinctive (FI or VI) patterns of responding which would be expected if each segment were followed by food. These data demonstrate behavioral control by brief stimuli which are not paired with primary reinforcement and show that such control may develop even when the sequences of behavior required to produce food or brief stimuli are variable in duration.     

Signal Detection Analysis of Choice Behavior and Aging

The primary research question was whether older adults respond to the contingencies of reinforcement such that they allocate their behavior as predicted by the reinforcement contingencies and discriminative stimuli programmed by the experimenters. Six subjects, ranging in age from 62 to 74 years, participated in 15 experimental sessions. Subjects were instructed to press the F1 key when they saw a white circle, and to press the F12 key when they saw a red letter A. Responses on the F1 key were reinforced on a variable interval (VI) 30-s schedule (i.e., a VI schedule of reinforcement with 11 arithmetically spaced intervals with a mean of 30 s); F12 key responses were reinforced on a VI 60-s schedule (i.e., a VI schedule of reinforcement with 11 arithmetically spaced intervals with a mean of 60 s). Reinforcers included monetary units (10 cents) and verbal praise. Results indicated positive effects of stimulus control of behavior. Collective false alarms and miss rates represented only 0.05% of the total responses. Also, two thirds of the subjects allocated behavior consistent with the second hypothesis that a denser reinforcement schedule (i.e., the VI 30-s) would cause subjects to respond more quickly to this condition. Implications of the present results for clinical studies of Alzheimer's Disease and dementia, as well as behavior modification strategies with aging individuals, are discussed.     

Predator videos and electric shock function as punishers for zebrafish (Danio rerio)

Zebrafish (Danio rerio) are a promising animal model for studying the effects of gene–environment interactions on behavior. Two experiments were conducted to assess punishment effects of presenting predator videos (Indian leaf fish; Nandus nandus) and electric shock on operant approach responses in zebrafish. In Experiment 1, the predator video and shock stimuli were presented upon a response maintained by a single variable‐interval schedule of food reinforcement in different groups of fish. In Experiment 2, the predator video and shock stimuli were presented upon one of two response alternatives maintain by concurrently available variable‐interval schedules of food reinforcement in different groups of fish. Responding decreased when the predator video and shock stimuli were presented relative to their absence in both experiments. Moreover, responding on an unpunished alternative did not reliably decrease in Experiment 2. These results indicate that the decrease in responding resulted from the punishment contingency rather than from elicited species‐specific defense responses or conditioned avoidance. Thus, the predator video and electric shock functioned as punishers of operant behavior for zebrafish. Identifying punishers for this species could lead to research on how gene–environment interactions influence individual differences in sensitivity to punishment.     

Instrumental performance on negative schedules

Three experiments examined the performance of rats pressing a lever for food reinforcement on a schedule in which high rates of response resulted in lowered rates of reinforcement (i.e. a schedule with a negative component). In Experiment 1, rats responded on a variable interval (VI) schedule with a conjoint component such that every 30 responses a reinforcement programmed by the VI schedule was cancelled. These subjects generally emitted a lower response rate than rats responding on a VI schedule yoked to the former subjects with respect to the delivery of reinforcement, although response rate differences were sometimes not large. Similar response-rate effects were obtained in Experiment 2 using a within-subject yoking procedure. In Experiment 3, reinforced interresponse times were matched on negative and VI schedules yoked in terms of reinforcement rate, and the response rate emitted in these conditions were similar. These results give support to theories of instrumental conditioning that stress the strengthening and shaping properties of reinforcement.     

Observing responses and uncertainty reduction. II The effect of varying the probability of reinforcement

The observing response paradigm was used to assess the reinforcing properties of discriminative stimuli by allowing animals either to work for food in the presence of a neutral stimulus or to first make an observing response by pressing a lever. On a progressive ratio schedule this resulted in the appearance of easily discriminable stimuli which marked positive and negative trials. The uncertainty associated with the imposed neutral stimulus was varied by manipulating the proportion of positive and negative trials in the session. The animals switched most often when the probability of reinforcement was low, less often when it was intermediate, and least often when it was high. This was not consistent with an uncertainty reduction hypothesis but could be explained if the uncertainty had to do with response rather than outcome.     

Dynamics of time matching: Arousal makes better seem worse

Matching of time allocation across alternatives in proportion to relative reinforcement rates is a ubiquitous finding in the animal-learning literature on choice. The dynamics of the underlying mechanism, however, remain poorly understood. A recent finding by Belke (1992) profoundly challenges scalar expectancy theory (SET; Gibbon et al., 1988) and other accounts of matching in concurrent variable interval (VI) schedules. He studied concurrent probe tests of stimuli associated with equal VIs but trained in alternative concurrent pairs. In training, one was preferred and the other not. Unreinforced probes revealed a strong preference for the alternative preferred in training. An experiment is reported replicating this result and showing that it is not due to generalization of preference levels from training. When the probe is between the two preferred training stimuli, the richer schedule is unpreferred. A SET account of these results is presented which implicates two processes in time allocation: (1) the choice between alternatives based on memory for delays to reinforcement, and (2) the times at which such choices are made. The former process is sensitive to reinforcement scheduling; the latter is sensitive to arousal levels induced by overall reinforcement rates in training.     

Free-operant performance on variable interval schedules with a linear feedback loop: no evidence for molar sensitivities in rats

Four experiments examined rats' sensitivity to molar and molecular factors on instrumental schedules of reinforcement. Rats were exposed to a variable interval schedule with a positive feedback loop (VI+), such that faster responding led to a shorter interreinforcement interval. In Experiments 1 and 2, rats responded faster on a variable response (VR) schedule than on either a VI schedule matched for reinforcement rate or a VI+ schedule matched for the feedback function. In Experiment 3, rats responded no differently on a VI schedule than they did on a VI+ schedule with equated rates of reinforcement. In Experiment 4, rats responded faster on a VI+ schedule with an interresponse time requirement yoked to that experienced on a VR schedule, than on a VI+ schedule with the same feedback function as the VR schedule. Taken together these results suggest that rats are more sensitive to the molecular than the molar properties of the schedules.     

Hearing test for a profoundly retarded human subject

A profoundly retarded 28-yr.-old female was trained to avoid an aversive but harmless shock to the foot by withdrawing the foot upon presentation of a visual cue. She was later unable to learn to avoid the shock consistently upon presentation of an auditory cue, confirming the ward staff's contention that she had a hearing disability. The audiometric technique using negative reinforcement bridges the problems of (1) difficulty in finding positive reinforcers for patients of low functioning and (2) satiation which may result from the continued use of positive reinforcers. The use of aversive stimuli raises ethical concerns. The growing trend in research is that aversive stimuli are permissible for individuals for whom positive techniques have not been effective and when used by trained professionals under careful review.     

Behavioural contrast and autoshaping

This experiment was designed to investigate the importance of autoshaping to a signal for reinforcement in the production of behavioural contrast. Two groups of pigeons were given discrimination training on a mult VI-EXT schedule: the stimuli present in the two schedule components shared common attributes, but were distinguished by the presence or absence of a visual feature. For one group (the feature positive group) the feature signalled the availability of reinforcement. For the other group (the feature negative group) the feature signalled nonrein-forcement, and for this group there was no stimulus element which unambiguously signalled reinforcement. The feature positive group showed a higher response rate during the VI component of the mult VI-EXT schedule than the feature negative group. This finding was interpreted as support for the autoshaping explanation of behavioural contrast. The results differed from those of Jenkins and Sainsbury (1969, 1970) in that both the feature positive and the feature negative groups showed discrimination learning.     

Behavioral and dimensional contrast in rats.

Rats pressed a nose key for brain stimulation reinforcement presented on a fixed-interval schedule. Stimuli were drawn at random from a continuum of 12 white noise intensities in the range 62-95 decibels, spaced in 3 decibel steps. Experiment 1 varied the number of stimuli and the reinforcement contingencies associated with them. In Condition I (baseline) all stimuli signaled reinforcement; in Conditions II and III stimuli from one half of the continuum signaled reinforcement and those from the other half, extinction. However, in Condition II the 6 stimuli from the middle of the continuum were omitted. Experiment 2 held constant the number of stimuli and varied their spacing. In Condition I, each of 6 sounds signaled reinforcement. In Conditions II and II, three stimuli from one half of the continuum signaled reinforcement and three from the other half, extinction. However, in Condition II the stimuli were near the extremes of the continuum (Stimuli 1, 3, 4, 9, 10, 12). Condition III replaced Stimulus 3 with Stimulus 6 and Stimulus 10 with Stimulus 7. Behavioral contrast was seen in an increase over baseline in response rate to the stimuli associated with the constant schedule component when the variable component was changed to extinction. Dimensional contrast was seen in a further elevation of rate to intermediate positive stimulus values when stimuli were added to the border region between positive and negative values.     

Human sensitivity to reinforcement feedback functions

Two experiments investigated human sensitivity to the temporally extended aspects of reinforcement schedules. Experiment 1 investigated human sensitivity to the extended and local aspects of three reinforcement schedules: variable ratio (VR), variable interval (VI), and variable-interval-plus-linear-feedback (VI+) schedules. Experiment 2 investigated this sensitivity on two reinforcement schedules: VI and VI+ schedules. In both experiments, there was evidence of sensitivity to the temporally extended aspects of the schedule: There were differences between the response rate on the VI+ and yoked-VI schedules, but no statistical difference in rates of response between the VR and VI+ schedules. The VI+ versus VI difference was much more pronounced when a lower response force needed to depress a lever was used. These results suggest that human subjects do show some sensitivity to temporally extended aspects of schedules of reinforcement.     

Acquisition of cup drinking using previously refused foods as positive and negative reinforcement

We used previously refused foods as positive and negative reinforcement in the acquisition of cup drinking. Cup drinking increased with positive and negative reinforcement, both alone and in combination (without escape extinction), indicating that treatment of food refusal can establish some foods as appetitive stimuli whereas others remain aversive.     

Conditioned acceleration and conditioned suppression in pigeons

Two experiments were performed to investigate the effects on pigeons' keypecking behavior of stimuli that signal different kinds of aversive events: time-out from positive reinforcement, electric shock, loud noise, and loud tone. Behavior maintained by a variable-interval schedule of reinforcement was suppressed by a stimulus before shock, was accelerated by a stimulus before time-out from positive reinforcement, and was unchanged by a stimulus before loud noise or a stimulus before loud tone. Conditioned acceleration with time-out from positive reinforcement and conditioned suppression with shock were obtained regardless of whether a response contingent or response-independent procedure was employed.     

Effects of fixed-time shocks and brief stimuli on food-maintained behavior of rats

When a fixed-time schedule of shocks was presented to rats lever pressing for food on a random-interval schedule, a pattern of behavior developed with a high rate of pressing after shock declining to near zero before the next shock was delivered. Once this pattern had stabilized, one-quarter of the shocks were replaced with brief auditory stimuli (tones) in a random sequence. Tone maintained behavior similar to shock, although tone was never paired with shock. Both tone and shocks elicited responding when presented at various times as probe stimuli, and responding was usually totally suppressed if neither stimulus occurred at the beginning of the fixed-time interval. When other stimuli were paired with tone and shock, only those paired with tone gained discriminative control and elicited responding. These findings suggest that stimuli that signal a shock-free, or safe, period will maintain the pattern of behavior generated by shock on a fixed-time schedule. There is a parallel between this phenomenon and the control of behavior on second-order schedules of positive reinforcement with nonpaired brief stimuli.     

Solution of the intermediate size problem by pigeons

Pigeons learned to respond to the middle-sized member (S^D) of a set of three simultaneously presented stimuli with responses to the S^D reinforced on a VI 1 schedule. They were then tested for several days with other sets of three stimuli. One procedure presented reinforcements on a VI 1 schedule during the test independent of the stimulus chosen when a reinforcement was programmed. The tests were also given under extinction conditions. With the testing carried out with extinction, preference consistently was for the test stimulus most similar in physical size to the S^D. However, when the tests were with reinforcement, random responding resulted. Another effect of testing with reinforcement was an increase in incorrect responding with the training set. Such a test procedure was unsatisfactory for determining the effective aspect of the S^D. The conclusion, based on the data of the extinction series, was that pigeons learned the intermediate size problem on the basis of the discrimination of absolute stimulus properties.