Variation, repetition, and choice

Experiment 1 investigated the controlling properties of variability contingencies on choice between repeated and variable responding. Pigeons were exposed to concurrent-chains schedules with two alternatives. In the REPEAT alternative, reinforcers in the terminal link depended on a single sequence of four responses. In the VARY alternative, a response sequence in the terminal link was reinforced only if it differed from the n previous sequences (lag criterion). The REPEAT contingency generated low, constant levels of sequence variation whereas the VARY contingency produced levels of sequence variation that increased with the lag criterion. Preference for the REPEAT alternative tended to increase directly with the degree of variation required for reinforcement. Experiment 2 examined the potential confounding effects in Experiment 1 of immediacy of reinforcement by yoking the interreinforcer intervals in the REPEAT alternative to those in the VARY alternative. Again, preference for REPEAT was a function of the lag criterion. Choice between varying and repeating behavior is discussed with respect to obtained behavioral variability, probability of reinforcement, delay of reinforcement, and switching within a sequence.     

Behavioral variability as avoidance behavior

This study aimed to investigate whether variable patterns of responses can be acquired and maintained by negative reinforcement under an avoidance contingency. Six male Wistar rats were exposed to sessions in which behavioral variability was reinforced according to a Lag contingency: Sequences of three responses on two levers had to differ from one, two or three previous sequences for shocks to be avoided (Lag 1, Lag 2 and Lag 3, respectively). Performance under the Lag conditions was compared with performance on a Yoke condition in which the animals received the same reinforcement frequency and distribution as in the Lag condition but behavioral variability was not required. The results showed that most of the subjects varied their sequences under the Lag contingencies, avoiding shocks with relatively high probability (≥ 0.7). Under the Yoke procedure, responding continued to occur with high probability, but the behavioral variability decreased. These results suggest that behavioral variability can be negatively reinforced.     

Reinforced variability and operant learning

Reinforcement of variability may help to explain operant learning. Three groups of rats were reinforced, in different phases, whenever the following target sequences of left (L) and right (R) lever presses occurred: LR, RLL, LLR, RRLR, RLLRL, and in Experiment 2, LLRRL. One group (variability [VAR]) was concurrently reinforced once per minute for sequence variations, a second group also once per minute but independently of variations, that is, for any sequences (ANY), and a control group (CON) received no additional reinforcers. The 3 groups learned the easiest targets equally. For the most difficult targets, CON animals' responding extinguished whereas both VAR and ANY responded at high rates. Only the VAR animals learned, however. Thus, concurrent reinforcers--contingent on variability or not--helped to maintain responding when difficult sequences were reinforced, but learning those sequences depended on reinforcement of variations.     

Effect of signaled reinforcement on response variability

Three experiments examined the effect of signaling reinforcement on rats' lever pressing on contingencies that reinforced variable responding to extend the exploration of signaled reinforcement to a schedule that has previously not been examined in this respect. In Experiment 1, rats responding on a lag-8 variability schedule with signaled reinforcement displayed greater levels of variability (U values) than rats on the same schedule lacking a reinforcement signal. In Experiment 2, rats responding on a differential reinforcement of least frequent responses schedule also displayed greater operant variability with a signal for reinforcement compared with rats without a reinforcement signal. In Experiment 3, a reinforcement signal decreased the variability of a response sequence when there was no variability requirement. These results offer empirical corroboration that operant variability responds to manipulations in the same manner as do other forms of operant response and that a reinforcement signal facilitates the emission of the required operant.     

Reinforcement rate and force proportional reinforcement

The research examines the consequences of making two reinforcement contingencies simultaneously available for the same bar-pressing response. Specifically, two experiments are described which concern the effects of reinforcement rate upon force proportional reinforcement (FPR). In Experiment 1, reinforcement rate was constrained by combining differential reinforcement of low rate with a force-proportional reinforcement contingency. The group of rats exposed to this regimen eventually showed a significant increase in response force compared to control groups. Experimental vs control response rates were not significantly different. In Experiment 2, reinforcement rate was affected through the expedient of relating the probability of obtaining a single pellet to the response force. The group for which the probability-force relationship was discontinuous came to exert higher forces than either a group for which the relationship was fairly continuous or an ordinary variable ratio control group. Response rate was significantly higher for the control group.     

Reinforcement contingencies and signal detection.

Pigeons were trained to discriminate temporal stimuli in a discrete-trial signal-detection procedure. Pecks to one side key were reinforced intermittently after exposure to one duration, and pecks to the other side key were reinforced intermittently after exposure to a different duration. In Experiment I, the allocation of reinforcers was varied systematically for correct responses and for errors, using a procedure that controlled the obtained numbers of reinforcers. When reinforcers were allocated symmetrically, the level of discrimination decreased as the proportion of reinforcers for errors increased. When reinforcers were allocated asymmetrically, the decrease in discrimination was less systematic. Bias toward one or the other side key roughly matched the ratio of reinforcers obtained by pecks at those keys, independent of the level of discrimination. In Experiment II, the overall rate of reinforcement for correct responses was varied both within and between experimental conditions. The level of discrimination was positively related to the overall rate of reinforcement. The discrimination data of both experiments were interpreted in relation to the contingencies of reinforcement and nonreinforcement, characterized by the average difference in reinforcement probability for correct responses and errors.     

Persistence and relapse of reinforced behavioral variability

The present study examined persistence and relapse of reinforced behavioral variability in pigeons. Pigeons emitted four‐response sequences across two keys. Sequences produced food according to a lag schedule, in which a response sequence was followed by food if it differed from a certain number of previous sequences. In Experiment 1, food was delivered for sequences that satisfied a lag schedule in both components of a multiple schedule. When reinforcement was removed for one component (i.e., extinction), levels of behavioral variability decreased for only that component. In Experiment 2, food was delivered for sequences satisfying a lag schedule in one component of a multiple schedule. In the other component, food was delivered at the same rate, but without the lag variability requirement (i.e., yoked). Following extinction, levels of behavioral variability returned to baseline for both components after response‐independent food delivery (i.e., reinstatement). In Experiment 3, one group of pigeons responded on a lag variability schedule, and the other group responded on a lag repetition schedule. For both groups, levels of behavioral variability increased when alternative reinforcement was suspended (i.e., resurgence). In each experiment, we observed some evidence for extinction‐induced response variability and for variability as an operant dimension of behavior.     

Effects of response variability on the sensitivity of rule-governed behavior

Two experiments examined the relation between response variability and sensitivity to changes in reinforcement contingencies. In Experiment 1, two groups of college students were provided complete instructions regarding a button-pressing task; the instructions stated “press the button 40 times for each point” (exchangeable for money). Two additional groups received incomplete instructions that omitted the pattern of responding required for reinforcement under the same schedule. Sensitivity was tested in one completely instructed and one incompletely instructed group after responding had met a stability criterion, and for the remaining two groups after a short exposure to the original schedule. The three groups of subjects whose responding was completely instructed or who had met the stability criterion showed little variability at the moment of change in the reinforcement schedule. The responding of these three groups also was insensitive to the contingency change. Incompletely instructed short-exposure responding was more variable at the moment of schedule change and was sensitive to the new contingency in four of six cases. In Experiment 2, completely and incompletely instructed responding first met a stability criterion. This was followed by a test that showed no sensitivity to a contingency change. A strategic instruction was then presented that stated variable responding would work best. Five of 6 subjects showed increased variability after this instruction, and all 6 showed sensitivity to contingency change. The findings are discussed from a selectionist perspective that describes response acquisition as a process of variation, selection, and maintenance. From this perspective, sensitivity to contingency changes is described as a function of variables that produce response variability.     

Group versus individual reinforcement contingencies within the context of group study conditions

We investigated the effects of two teaching variables on students' Spanish vocabulary quiz performance: (a) group study and (b) individual versus group contingencies. In Experiment 1, we compared students' quiz scores under conditions in which students either studied independently and received no programmed reinforcement or studied in groups and received individual rewards for high scores. The results showed that, on average, the group-study individual-reward condition produced superior quiz scores. In Experiment 2, we compared individual (i.e., the superior condition in Experiment 1) and group contingencies within the context of the group study condition. On average across the class, group contingencies produced performance superior to individual contingencies. In both studies, however, benefits for the classes as a whole were mitigated by effects on individual students. These results extend the literature on the effects of group-based instructional activities and reinforcement contingencies. Educators who choose such procedures may encounter conflicting findings depending on whether they examine results at the group or individual level.     

Acquisition and extinction of signaled avoidance as a function of intermittent reinforcement

Signaled, shuttle-box avoidance responding in female rats of the Fischer₃₄₄ strain was examined as a function of four separate contingencies of intermittent reinforcement. In Experiment 1, when avoidance responses during acquisition were reinforced 25% of the time with prompt CS termination, animals responded equally often during acquisition and significantly more often during extinction than animals who received such reinforcement on a 100% schedule. Similar results were found under a trace procedure in Experiment 2 when avoidance responses were reinforced 25% of the time with informational feedback stimuli. In contrast, during Experiment 3, when animals were shocked on only 25% of the trials on which they failed to respond, the level of avoidance responding during both acquisition and extinction was significantly less than it was when animals were shocked on a 100% schedule. Comparable results were found in Experiment 4 when avoidance responses during acquisition averted shock on only 25% of the trials. Thus, intermittent reinforcement contingencies involving response-contingent feedback stimuli and shock have differential effects on avoidance responding during both acquisition and extinction trials under the signaled avoidance procedure.     

Choice as a function of local versus molar reinforcement contingencies.

Rats were trained on a discrete-trial probability learning task. In Experiment 1, the molar reinforcement probabilities for the two response alternatives were equal, and the local contingencies of reinforcement differentially reinforced a win-stay, lose-shift response pattern. The win-stay portion was learned substantially more easily and appeared from the outset of training, suggesting that its occurrence did not depend upon discrimination of the local contingencies but rather only upon simple strengthening effects of individual reinforcements. Control by both types of local contingencies decreased with increases in the intertrial interval, although some control remained with intertrial intervals as long as 30 s. In Experiment 2, the local contingencies always favored win-shift and lose-shift response patterns but were asymmetrical for the two responses, causing the molar reinforcement rates for the two responses to differ. Some learning of the alternation pattern occurred with short intertrial intervals, although win-stay behavior occurred for some subjects. The local reinforcement contingencies were discriminated poorly with longer intertrial intervals. In the absence of control by the local contingencies, choice proportion was determined by the molar contingencies, as indicated by high exponent values for the generalized matching law with long intertrial intervals, and lower values with short intertrial intervals. The results show that when molar contingencies of reinforcement and local contingencies are in opposition, both may have independent roles. Control by molar contingencies cannot generally be explained by local contingencies.     

Baseline response rates affect resistance to change

The effect of response rates on resistance to change, measured as resistance to extinction, was examined in two experiments. In Experiment 1, responding in transition from a variable‐ratio schedule and its yoked‐interval counterpart to extinction was compared with pigeons. Following training on a multiple variable‐ratio yoked‐interval schedule of reinforcement, in which response rates were higher in the former component, reinforcement was removed from both components during a single extended extinction session. Resistance to extinction in the yoked‐interval component was always either greater or equal to that in the variable‐ratio component. In Experiment 2, resistance to extinction was compared for two groups of rats that exhibited either high or low response rates when maintained on identical variable‐interval schedules. Resistance to extinction was greater for the lower‐response‐rate group. These results suggest that baseline response rate can contribute to resistance to change. Such effects, however, can only be revealed when baseline response rate and reinforcement rate are disentangled (Experiments 1 and 2) from the more usual circumstance where the two covary. Furthermore, they are more cleanly revealed when the programmed contingencies controlling high and low response rates are identical, as in Experiment 2.     

EFFECTS OF DIRECT,INTERMITTENT, AND VICARIOUS REINFORCEMENT PROCEDURES ON THE DEVELOPMENT AND MAINTENANCE OF INSTRUCTION-FOLLOWING BEHAVIORS IN A GROUP OF YOUNG CHILDREN1

A group of young children (mean age: 2.5 yr) were instructed to follow different requests by a teacher in a day-care setting. Experiment I verified that mean group instruction following was low (10%) despite the opportunity for “observational learning”, i.e., the group of 12 children could watch a nonreinforced adult comply with the teacher's request. In Experiment II, when positive consequences were provided contingent on the adult model's behavior, mean group instruction following was relatively unaffected (14%). When direct reinforcement was given to four peer models, each for several sessions, the individual performances of three of the four peer models was elevated (from 50% to 80%); however, the mean performance of the remaining nonreinforced children (N = 7) was only moderately affected (21%). When reinforcement contingencies were again changed, so that each group member was provided direct, but intermittent reinforcement, mean group performance increased substantially to levels of over 70%. Once instruction following was high, presentation of reinforcement only to one peer model sufficed to maintain performance whereas earlier, this same vicarious reinforcement procedure had failed to establish group compliance. The maintenance of instruction-following behavior when reinforcement was applied solely to one child was interpreted mainly in terms of a high resistance to extinction following a history of intermittent reinforcement rather than a “vicarious”- or “self”-reinforcement mechanism. Finally, removal and re-introduction of group intermittent reinforcement, respectively, lowered performance (to levels of 40%) and elevated (to levels of 65%) the group's performance.     

Operant conditioning of autogrooming in vervet monkeys: Cercopithecus aethiops

Vervet monkeys received food reinforcement contingent on autogrooming. Experiment 1 reinforced grooming on a schedule of increasing intermittency and grooming increased in frequency and duration; with only pauses reinforced, grooming decreased in frequency and duration. Experiment 2 demonstrated differentiation of operant autogrooming; in each session a different single form of grooming was reinforced (for example, grooming the tail only), and that form increased in frequency while other forms became less frequent. In Experiment 3 scratching was succesfully conditioned with a method that selectively reinforced variety in behavior; reinforcement was contingent on a shift in scratching form. In Experiment 4, with no contingencies on grooming, a prefood stimulus did not increase autogrooming whether or not grooming had previously resulted in contingent reinforcement. The form of conditioned autogrooming resembled the form of unconditioned autogrooming. The discussion suggests how reinforcement principles can account for changes in the topography of operant behavior.     

Parametric effects of reinforcement frequency, amount of reinforcement, and required response force on sheltered workshop behavior

Three experiments involving parametric manipulation of reinforcement contingencies were performed with retardates in an automated Sheltered Workshop token economy. Experiment I showed that with amount of reinforcement held constant, work rates were positively related to reinforcement rates on fixed-interval schedules and inversely related to reinforcement rates on fixed-ratio schedules. Experiment II demonstrated an interaction between frequency of ratio reinforcement and torque required to complete a work unit: work rates were positively related to reinforcement rates when required response force was high and negatively related to reinforcement rates when required response force was low. Experiment III revealed that, with reinforcement frequency held constant, there was in inverse relationship between amount of reinforcement and work rate.     

Local patterns in human operant behavior and a behaving model to interrelate animal and human performances

College students pressed buttons for points delivered according to molecular concurrent reinforcement contingencies similar to those used in previous experiments with animal subjects. Relative frequency of reinforcement and the relative and absolute durations of two reinforced patterns of button pressing were experimentally varied. Qualitative effects of all three variables resembled those obtained previously with animals. Computer simulation of a cognitive processing model described these qualitative effects. A similar model was used previously to describe corresponding behavior in pigeons. Therefore, similar cognitive processing may underlie the local temporal patterning of animal and human operant behavior maintained by this concurrent reinforcement contingency.     

Operant variability: evidence,functions, and theory

Although responses are sometimes easy to predict, at other times responding seems highly variable, unpredictable, or even random. The inability to predict is generally attributed to ignorance of controlling variables, but this article is a review of research showing that the highest levels of behavioral variability may result from identifiable reinforcers contingent on such variability. That is, variability is an operant. Discriminative stimuli and reinforcers control it, resulting in low or high variability, depending on the contingencies. Schedule-of-reinforcement effects are orderly, and choosing to vary or repeat is lawfully governed by relative reinforcement frequencies. The operant nature of variability has important implications. For example, learning, exploring, creating, and problem solving may partly depend on it. Abnormal levels of variability, including those found in psychopathologies such as autism, depression, and attention deficit hyperactivity disorder, may be modified through reinforcement. Operant variability may also help to explain some of the unique attributes of voluntary action.     

Response rate, reinforcement frequency, and conditioned suppression

In the first of two experiments, periods of noise were terminated with unavoidable shock to 36 rats. The rats' continuously reinforced responding was later completely suppressed during the noise when it was introduced without shock. The rats were then assigned to nine experimental groups. Each group was exposed to different paced variable-interval schedules of reinforcement, which independently controlled response rate and reinforcement frequency. Periods of the noise were periodically superimposed on these schedules, and loss of response suppression was studied. Differences between the groups were assessed statistically. The second experiment used a steady-state design. Six rats were exposed to paced schedules which generated two alternating response rates but gave constant reinforcement frequencies, and six rats to schedules which maintained the same response rates throughout, but in which the reinforcement frequency was alternately high and low. Response suppression was studied during a pre-shock stimulus superimposed on each rat's two behavioral baselines. Both experiments suggest that (1) conditioned suppression is affected by rate of operant responding, high rates being most suppressed, and (2) the frequency of reinforcements obtained also has an effect, most suppression occurring when frequency is low.     

Delayed reinforcement as an indiscriminable contingency in verbal/nonverbal correspondence training

We investigated the programming of generalization and maintenance of correspondence between verbal and nonverbal behavior in a preschool setting. Four children participated in a series of multiple-baseline designs. In Experiment 1, delayed reinforcement of verbal behavior effectively controlled maintenance of correspondence with previously trained responses and also resulted in generalization of correspondence to one untrained response. As the latter effect was limited, Experiment 2 was a further assessment of the effects of delayed reinforcement of generalization of correspondence to untrained responses, and consistent generalization was shown. Experiment 2 also showed that generalization, if lost, could be recovered through use of "booster training," in which the original contingencies were reinstated for a brief period. Experiment 3 provided replications, with two additional children, of the effects of delayed reinforcement on maintenance of correspondence. Results are discussed in terms of using delayed reinforcement as an indiscriminable contingency.     

The effects of early handling on the partial reinforcement extinction effect and the partial punishment effect in male and female rats

Handled (Day 1-22) and non-handled infantile Wistar rats were tested in maturity for the partial reinforcement extinction effect (PREE) and the partial punishment effect (PPE). In Experiments 1 and 2, mature male and female rats were trained to run in an alley for food reward on a 1-trial/day schedule. In the PREE paradigm (Experiment 1), the partially reinforced group (PRF) received reinforcement on a quasi-random 50% schedule, while the continuously reinforced group (CRF) received reinforcement on every trial. In the test stage, both groups were given extinction training. In the PPE paradigm (Experiment 2), the partially punished (PP) group received, together with continuous reinforcement, shocks on a quasirandom 50% schedule, while the continuously reinforced group was reinforced on every trial. In test, all animals were given both reinforcement and shock on every trial. In Experiment 1, PREE—i.e. increased resistance to extinction in the PRF as compared to the CRF group—was more pronounced in the handled animals. More specifically, no PREE was obtained in the non-handled males, and in the non-handled females the PREE was reduced compared to the handled females. The results of Experiment 2 revealed no effect of handling or sex on PPE, that is, increased resistance to punishment in the PP group as compared to the CRF group was evident in all four conditions. In Experiment 3, handled and non-handled male rats were tested for the PREE using a multi-trial procedure in an operant chamber. PREE was obtained in the handled but not in the non-handled animals. The implications of these results for the differential effects of handling on male and female rats and the distinction between the PREE and PPE paradigms are discussed.