Reinforcer frequency and restricted stimulus control.

Stimulus control was evaluated in 3 individuals with moderate to severe mental retardation by delayed identity matching-to-sample procedures that presented either one or two discrete forms as sample stimuli on each trial. On pretests, accuracy scores on one-sample trials were uniformly high. On two-sample trials, the correct stimulus (i.e., the one that subsequently appeared in the comparison array) varied unpredictably, and accuracy scores were substantially lower, suggesting that both sample stimuli did not exert stimulus control on every trial. Subjects were then given training sessions with the one-sample task and with a new set of four stimuli. For two of the stimuli, correct matching responses were followed by reinforcers on a variable-ratio schedule that led to a high reinforcer rate. For the other two stimuli, correct responses were followed by reinforcers on a variable-ratio schedule that led to a substantially lower reinforcer rate. Results on two-sample tests that followed showed that (a) on trials in which comparison arrays consisted of one high reinforcer-rate and one low reinforcer-rate stimulus, subjects most often selected the high-rate stimulus; and (b) on trials in which the comparison arrays were either two high reinforcer-rate stimuli or two low reinforcer-rate stimuli and the samples were one high reinforcer- and one low reinforcer-rate stimulus, accuracy was higher on trials with the high-rate comparisons. These results indicate that the frequency of stimulus control by high reinforcer-rate samples was greater than that by low reinforcer-rate samples. Following more training with the one-sample task and reversed reinforcement schedules for all stimuli, the differences in stimulus control frequencies on two-sample tests also reversed. These results demonstrate experimental control by reinforcement contingencies of which of two sample stimuli controlled selections in the two-sample task. The procedures and results may prove to be relevant for understanding restricted stimulus control and stimulus overselectivity.     

Observing behavior and atypically restricted stimulus control

Restricted stimulus control refers to discrimination learning with atypical limitations in the range of controlling stimuli or stimulus features. In the study reported here, 4 normally capable individuals and 10 individuals with intellectual disabilities (ID) performed two-sample delayed matching to sample. Sample-stimulus observing was recorded with an eye-tracking apparatus. High accuracy scores indicated stimulus control by both sample stimuli for the 4 nondisabled participants and 4 participants with ID, and eye tracking data showed reliable observing of all stimuli. Intermediate accuracy scores indicated restricted stimulus control for the remaining 6 participants. Their eye-tracking data showed that errors were related to failures to observe sample stimuli and relatively brief observing durations. Five of these participants were then given interventions designed to improve observing behavior. For 4 participants, the interventions resulted initially in elimination of observing failures, increased observing durations, and increased accuracy. For 2 of these participants, contingencies sufficient to maintain adequate observing were not always sufficient to maintain high accuracy; subsequent procedure modifications restored it, however. For the 5th participant, initial improvements in observing were not accompanied by improved accuracy, an apparent instance of observing without attending; accuracy improved only after an additional intervention that imposed contingencies on observing behavior. Thus, interventions that control observing behavior seem necessary but may not always be sufficient for the remediation of restricted stimulus control.     

Integrating functional neuroimaging and human operant research: brain activation correlated with presentation of discriminative stimuli

Results of numerous human imaging studies and nonhuman neurophysiological studies on "reward" highlight a role for frontal, striatal, and thalamic regions in operant learning. By integrating operant and functional neuroimaging methodologies, the present investigation examined brain activation to two types of discriminative stimuli correlated with different contingencies. Prior to neuroimaging, 10 adult human subjects completed operant discrimination training in which money was delivered following button pressing (press-money contingency) in the presence of one set of discriminative stimuli, and termination of trials followed not responding (no response-next trial contingency) in the presence of a second set of discriminative stimuli. After operant training, subjects were instructed to memorize a third set of control stimuli unassociated with contingencies. Several hours after training, functional magnetic resonance imaging was performed while subjects viewed discriminative and control stimuli that were presented individually for 1,500 ms per trial, with stimulus presentations occurring, on average, every 6 s. Activation was found in frontal and striatal brain regions to both sets of discriminative stimuli relative to control stimuli. In addition, exploratory analyses highlighted activation differences between discriminative stimuli. The results demonstrate the utility of coupling operant and imaging technologies for investigating the neural substrates of operant learning in humans.     

Reinforcement of human observing behavior by a stimulue correlated with extinction or increased effort

Young men pulled a plunger on mixed and multiple schedules in which periods of variable-interval monetary reinforcement alternated irregularly with periods of extinction (Experiment 1), or in which reinforcement was contingent on different degrees of effort in the two alternating components (Experiment 2). In the baseline conditions, the pair of stimuli correlated with the schedule components could be obtained intermittently by pressing either of two observing keys. In the main conditions, pressing one of the keys continued to produce both discriminative stimuli as appropriate. Pressing the other key produced only the stimulus correlated with variable-interval reinforcement or reduced effort; presses on this key were ineffective during periods of extinction or increased effort. In both experiments, key presses producing both stimuli occurred at higher rates than key presses producing only one, demonstrating enhancement of observing behavior by a stimulus correlated with the less favorable of two contingencies. A control experiment showed that stimulus change alone was not an important factor in the maintenance of the behavior. These findings suggest that negative as well as positive stimuli may play a role in the conditioned reinforcement of human behavior.     

Stimulus control and compounding with ambient odor as a discriminative stimulus on a free-operant baseline

Previous experiments have demonstrated that the simultaneous presentation of independently established discriminative stimuli can control rates of operant responding substantially higher than the rates occasioned by the individual stimuli. This "additive summation" phenomenon has been shown with a variety of different reinforcers (e.g., food, water, shock avoidance, cocaine, and heroin). Discriminative stimuli previously used in such studies have been limited to the visual and auditory sensory modalities. The present experiment sought to (1) establish stimulus control on a free-operant baseline with an ambient olfactory discriminative stimulus, (2) compare olfactory control to that produced with an auditory discriminative stimulus, and (3) determine whether compounding independently established olfactory and auditory discriminative stimuli produces additive summation. Rats lever pressed for food on a variable-interval schedule in the presence of either a tone or an odor, with comparable control developed to each stimulus. In the absence of these stimuli responding was not reinforced. During stimulus compounding tests, the tone-plus-odor compound occasioned more than double the responses occasioned by either the tone or odor presented individually. Thus, the current study (1) established stimulus control with an ambient olfactory discriminative stimulus in a traditional free-operant setting and (2) extended the generality of stimulus-compounding effects by demonstrating additive summation when olfactory and auditory discriminative stimuli were presented simultaneously.     

Observational learning of two visual discriminations by pigeons: a within-subjects design.

Pigeon's observational learning of successive visual discrimination was studied using within-subject comparisons of data from three experimental conditions. Two pairs of discriminative stimuli were used; each bird was exposed to two of the three experimental conditions, with different pairs of stimuli used in a given bird's two conditions. In one condition, observers were exposed to visual discriminative stimuli only. In a second condition, subjects were exposed to a randomly alternating sequence of two stimuli where the one that would subsequently be used as S+ was paired with the operation of the grain magazine. In a third experimental condition, subjects were exposed to the performance of a conspecific in the operant discrimination procedure. After exposures to conspecific performances, there was facilitation of discriminative learning, relative to that which followed exposures to stimulus and reinforcement sequences or exposures to stimulus sequences alone. Exposure to stimulus and food-delivery sequences enhanced performance relative to exposure to stimulus sequences alone. The differential effects of these three types of exposure were not attributable to order effects or to task difficulty; rather, they clearly were due to the type of exposure.     

High-probability stimulus control topographies with delayed S+ onset in a simultaneous discrimination procedure

Experimenters and teachers use discrimination learning procedures to encourage reliable attending to stimulus differences defined as relevant for their purposes. Put another way, the goal of discrimination training is to establish high-probability stimulus control topographies that are coherent with experimenter or teacher specifications. The present research was conducted to investigate a novel procedure for encouraging stimulus control topography coherence. Participants were 13 adolescents with severe intellectual handicaps. During an initial Condition A, all were exposed to a simultaneous discrimination procedure. Participants could select a form alternating with a black field (S+) or an identical form that did not alternate (S-). Accuracy scores were typically low, and there was little evidence of coherent stimulus control topographies. Subsequently, the procedure was changed. During Condition B, every trial initially presented two identical nonalternating S- forms (Trial State 1). If the participant made no selection for 5 s, one of the forms began to alternate with the black field, and he or she could make the S+/S- discrimination (Trial State 2). Selections during Trial State I prolonged the delay to Trial State 2 until there had been no response for 5 s. During Condition B, S+/S- discrimination accuracy scores improved rapidly and markedly for most participants. Reinstating Condition A often resulted in diminished accuracy scores. This study thus (a) demonstrated a novel procedure for encouraging stimulus control topography coherence and (b) provided support for the interpretation that intermediate accuracy scores may be due to different topographies of stimulus control that co-occur in the same discriminative baseline.     

The nature of control by spoken words over visual stimulus selection.

Eight retarded adolescents were trained to select one (a trained S+) of two visual stimuli in response to a spoken word (a trained word). Two different visual stimuli alternated randomly as the S-. To determine if the spoken work was merely a temporal discriminative stimulus for when to respond, or if it also specified which visual stimulus to select, the subjects were given intermittent presentations of untrained (novel) spoken words. All subjects consistently selected the trained S+ in response to the trained spoken word and selected the previous S- in response to the untrained spoken words. It was hypothesized that the subjects were responding away from the trained S+ in response to untrained spoken words, and control by untrained spoken words would not be observed when the trained S+ was not present. The two visual S- stimuli selected on trials of untrained spoken words were presented simultaneously. The untrained spoken words presented on these trials no longer controlled stimulus selections for seven subjects. The results supported the hypothesis that previous control by spoken words was due to responding away from the trained S+ in response to untrained spoken words.     

Relations among equivalence, naming, and conflicting baseline control

Three studies were conducted with different groups of 6 students each to explore the effects of training class-inconsistent relations and naming on demonstrations of emergent arbitrary stimulus relations. In all studies, two three-member equivalence classes of Greek symbols (A1B1C1 and A2B2C2) emerged as a result of training in conditional discriminations. Two new symbols were introduced (X and Y), and additional conditional discriminations were trained, whereby X was designated as the positive discriminative stimulus (S+) and Y was designated as the negative discriminative stimulus (S-) for A1 and B2. Conversely, Y was designated as the S+ and X as the S- for B1 and A2. This introduced conflicting sources of control within and between classes. In Study 1, subjects were not provided with names for the stimuli. In Study 2, the experimenter provided common names for the stimuli within each class. In Study 3, the subjects were required to use the common names during conditional discrimination training and test-trial blocks. In all experiments, equivalence responding with respect to the original classes was disrupted for some subjects subsequent to learning the new relations. Furthermore, in Studies 2 and 3, there were frequent examples of noncorrespondence between observed (listener or speaker) naming patterns and derived relations. These results support the view that demonstrations of equivalence are subject to control from a variety of sources rather than being fundamentally dependent on naming.     

Contextual conditional discriminations

Three experiments used a discriminated operant procedure to study conditional discrimination learning in rats. The first experiment showed that rats were capable of learning a biconditional discrimination in which two contexts served as conditional cues signalling the reinforcement contingencies associated with two discriminative stimuli. The discrimination was learned equally well when one discriminative stimulus signalled food, the other its absence, and when one stimulus signalled food, the other extinction plus mild footshock.

In Experiment 2 it was shown that prior training on such a conditional discrimination enhanced the subsequent context specificity of simple conditioning relative to control groups of animals for whom the prior training had not been conditional. Experiment 3 showed that a reversal of the significance of one pair of discriminative stimuli produced no spontaneous reversal in performance to a second, target, pair.

The pattern of results is best accounted for by an analysis of contextual conditional discrimination learning in terms of stimulus configurations and offers no support for the notion that rats may learn a general conditional rule or set.     

Reversibility of single-incentive selective associations.

Rats were trained to press a lever in the presence of a tone-light compound stimulus and not to press in its absence. In each of two experiments, schedules were designed to make the compound a conditioned punisher for one group and a conditioned reinforcer for the other. In Experiment 1, one group's responding produced food in the presence of the compound but not in its absence. The other group's responding terminated the compound stimulus, and food was presented only in its absence. When tone and light were later presented separately, light controlled more responding than did tone in the former group, but tone gained substantial control in the latter. The same effects were also observed within subjects when the training schedules were switched over groups. In Experiment 2, two groups avoided shock in the presence of the compound stimulus. In the absence of the compound, one group was not shocked, and the other received both response-independent and response-produced shock. When tone and light were presented separately, the former group's responding was mainly controlled by tone, but the latter group's responding was almost exclusively controlled by light. These effects were also observed within subjects when the training schedules were switched over groups. Thus, these single-incentive selective association effects (appetitive in Experiment 1 and aversive in Experiment 2) were completely reversible. The schedules in which the compound should have been a conditioned reinforcer consistently produced visual control, and auditory control increased when the compound should have become a conditioned punisher. Currently accepted accounts of selective associations based on affinities between shock and auditory stimuli and between food and visual stimuli (i.e., stimulus-reinforcer interactions) do not adequately address these results. The contingencies of reinforcement most recently associated with the compound and with its absence, rather than the nature of the reinforcer, determined whether auditory or visual stimulus control developed.     

Delayed Matching to Sample With Outcome-Specific Contingencies in Mentally Retarded Humans

Four mentally retarded human subjects performed delayed identity matching-to-sample tasks under conditions of outcome-specific and nonspecific reinforcement contingencies. With outcome-specific contingencies, experimentally defined correct selections of different discriminative stimuli were perfectly correlated with one of two different consequences (e.g., different foods); with nonspecific contingencies, correct selections were equally likely to be followed by either consequence. Previous studies with pigeons have shown consistently higher accuracy on delayed-matching tasks with outcome-specific contingencies. Results of this study failed to replicate that finding. For three subjects, accuracy scores were similar in specific and nonspecific conditions at fixed delays ranging from 0 s to 5 s. For the other subject, delay duration varied within each session, and accuracy scores improved across sessions regardless of outcome condition.

     

Analysis of discriminative control by social behavioral stimuli

Visual discriminative control of the behavior of one rat by the behavior of another was studied in a two-compartment chamber. Each rat's compartment had a food cup and two response keys arranged vertically next to the clear partition that separated the two rats. Illumination of the leader's key lights signaled a “search” period when a response by the leader on the unsignaled and randomly selected correct key for that trial illuminated the follower's keys. Then, a response by the follower on the corresponding key was reinforced, or a response on the incorrect key terminated the trial without reinforcement. Accuracy of following the leader increased to 85% within 15 sessions. Blocking the view of the leader reduced accuracy but not to chance levels. Apparent control by visual behavioral stimuli was also affected by auditory stimuli and a correction procedure. When white noise eliminated auditory cues, social learning was not acquired as fast nor as completely. A reductionistic position holds that behavioral stimuli are the same as nonsocial stimuli; however, that does not mean that they do not require any separate treatment. Behavioral stimuli are usually more variable than nonsocial stimuli, and further study is required to disentangle behavioral and nonsocial contributions to the stimulus control of social interactions.     

The conditioned reinforcement of repeated acquisition

Three monkeys were trained to emit a chain of three responses on three separate levers in a set of six levers to obtain food. The chain producing food (correct chain) was changed each day. During a trial, a press on any lever produced a feedback stimulus; a press on a correct lever produced an additional distinctive stimulus; the third correct press produced a food pellet. Test sessions in which either the food or the distinctive stimuli were removed were interspersed with baseline sessions. In tests without food presentations, the subjects acquired the correct chain rapidly, with a level of accuracy comparable to baseline. Removing the distintive stimuli for either the first or second member of the correct chain greatly retarded acquisition of that member of the chain. Removing all distinctive stimuli often reduced accuracy throughout the chain to chance level, even though food was presented following each correct chain. These results were interpreted as evidence that the distinctive stimuli presented after correct responses functioned as conditioned reinforcers. Reductions in accuracy following an omitted distinctive stimulus indicated that they were also discriminative stimuli for correct responding in their presence.     

The nature of standard control in children's matching-to-sample

In Experiment I, six preschool-aged children were given matching-to-sample training with two figures in which they were required to choose one of two comparison stimuli that was identical in shape to the standard stimulus. Following this training, they were given intermittent test trials in which a novel stimulus figure was substituted for the previously correct comparison stimulus. Five of the six subjects consistently chose the substituted stimulus during test trials. Experiment II replicated the findings of Experiment I with three other preschool-aged children. Experiment II also provided controls for the possibility that the subjects of Experiment I were selecting the substituted stimulus because of its novelty. The investigators concluded that eight of the nine subjects were exhibiting the type of control described by Berryman, Cumming, Cohen, and Johnson (1965) as S-delta responding.     

Autoshaping with common and distinctive stimulus elements, compact and dispersed arrays

Four groups of pigeons were trained with a standard autoshaping procedure in which a brief fixed-duration interval always followed by a grain delivery alternated with a longer variable-duration interval never associated with grain delivery. One of two stimuli was always presented during each interval. One of them contained three black dots and a black star on a green background; the other contained four black dots on a green background. The four elements of each stimulus were arranged in a more compact array for two groups and in a more dispersed array for the other two groups. Which of the two stimuli preceded grain delivery was counterbalanced within each pair of groups. The speed of occurrence of the first autoshaped peck was not affected by whether the stimulus containing the distinctive star element preceded grain delivery, but autoshaping was faster when the stimulus arrays were compact than when they were dispersed. During 560 response-independent training trials that followed the first autoshaped peck, this pattern reversed; both discriminative control over responding and the relative frequency of pecking the stimulus that preceded grain delivery were greater for the two groups where this stimulus contained the discriminative element than for the two groups where it contained only common elements. During subsequent testing with stimuli containing only a single element each, the distinctive feature was responded to proportionately more often by the two groups for which it had been an element of the stimulus preceding grain delivery than by the two groups for which it had been an element of the stimulus complex that never was associated with grain delivery. These data add further support to the hypothesis that the initial occurrence of autoshaped responding and its subsequent maintenance are not affected by the same variables. They also suggest that automaintenance is as sensitive as response-dependent training to the presence or absence of a distinctive stimulus element among several common elements and that this sensitivity appears to be independent of the specific method used for presenting the stimuli during automaintenance.     

Free-operant forgetting: Delayed stimulus control of multiple-schedule performance

Pigeons' responses were reinforced in two components of several multiple variable-interval variable-interval schedules of food reinforcement. In one component, the key was illuminated green for 15 seconds and white for 45 seconds. In the other component, the key was illuminated red for 15 seconds and white for 45 seconds. Values for the exponent of the power functions relating response ratios to reinforcement ratios were higher in the presence of the discriminative stimuli (green or red) than in the presence of white. Sensitivity of response ratios to changes in reinforcement ratios provided an index of the extent to which responding was under delayed stimulus control by prior discriminative stimuli.     

Stimulus properties of conspecific behavior

Two experiments identified the conditions in which the behavior of one bird acquired discriminative control of the behavior of a second bird. The schedule-controlled behaviors of the “stimulus” bird were differentially correlated with the components of a multiple schedule according to which the pecking of an “experimental” bird produced food. In Experiment 1, three pairs of pigeons acquired a successive discrimination and two reversals with the conspecific stimuli. Experiment 2 included a control condition in which no systematic relationship existed between the conspecific stimuli and the component schedules. While differential responding during the components of the multiple schedule was again found when the conspecific stimuli were available, differential responding did not occur in the control condition. Test conditions included in the experiments indicated that (a) the differential responding was not dependent on the discriminative properties of reinforcement, (b) the pecking of the stimulus and experimental birds was temporally interrelated, (c) the visual conspecific stimuli were critical to the maintenance of the discrimination, and (d) the observed stimulus control immediately generalized to an unfamiliar conspecific.     

Stimulus control and generalization of point-loss punishment with humans

Two experiments demonstrated stimulus control and generalization of conditioned punishment with humans. In both studies, responses first were reinforced with points exchangeable for money on a variable-interval schedule in the presence of one line length (S(D)). Next, a second line length was introduced, and point loss followed every response in the presence of that line (S(D)p). In the final training condition, points were deducted at session end. Response rate was lower in the presence of the S(D)p despite equal rates of points for money in the presence of both stimuli. In generalization testing for Experiment 1, the two lines were included in a 10-line continuum; S(D)p fell in the middle and the trained SD was at one end. Lines were presented randomly, and point delivery and loss contingencies were as in training but with points available in the presence of all lines. For all subjects, response rates were lowest around S(D)p and increased towards the SD end of the continuum. Because testing included only one or two lines beyond S(D), this pattern did not rule out S(D) generalization. Thus, in Experiment 2, stimuli beyond S(D) were added to generalization tests. Response rates did not decrease as a function of distance from S(D), clarifying the demonstration of punishment generalization.     

Stimulus control of habits: Evidence for both stimulus specificity and devaluation insensitivity in a dual-response task

Goal-directed and habitual actions are clearly defined by their associative relations. Whereas goal-directed control can be confirmed via tests of outcome devaluation and contingency-degradation sensitivity, a comparable criterion for positively detecting habits has not been established. To confirm habitual responding, a test of control by the stimulus–response association is required while also ruling out goal-directed control. Here we describe an approach to developing such a test in rats using two discriminative stimuli that set the occasion for two different responses that then earn the same outcome. Performance was insensitive to outcome devaluation and showed stimulus–response specificity, indicative of stimulus-controlled behavior. The reliance of stimulus–response associations was further supported by a lack of sensitivity during the single extinction test session used here. These results demonstrate that two concurrently trained responses can come under habitual control when they share a common outcome. By reducing the ability of one stimulus to signal its corresponding response–outcome association, we found evidence for goal-directed control that can be dissociated from habits. Overall, these experiments provide evidence that tests assessing specific stimulus–response associations can be used to investigate habits.