The effect of contingency upon the appetitive conditioning of free-operant behavior

The problem of maintaining independence between response rates and reinforcement probabilities when determining the effect of varying the response-reinforcement contingency upon free-operant behavior was solved by programming local reinforcement probabilities for response and no response on a second-by-second basis. Fifty-seven rats were trained to lever-press on schedules of water reinforcement involving different values of contingency. All rats were first trained on a high positive contingency and then shifted to less positive, zero, or negative contingencies. Under these conditions, rate of lever-pressing declined appropriately when the contingency between response and reinforcement decreased or was made negative. The decline in rate produced by a zero contingency cannot be attributed to extinction, since the probability of reinforcement given the occurrence of a response was the same as for the positive contingency from which the shift to zero was made. That is, there was no change in the opportunity for response-reinforcement contiguity. It was concluded that the technique of programming local reinforcement probabilities offers promise for more critical examinations of the effects of contingency upon free-operant behavior.     

Response acquisition with delayed reinforcement

Discrete responses of experimentally naive, food-deprived White Carneaux pigeons (key pecks) or Sprague-Dawley rats (bar or omnidirectional lever presses) initiated unsignaled delay periods that terminated with food delivery. Each subject first was trained to eat from the food source, but no attempt was made to shape or to otherwise train the response. In both species, the response developed and was maintained. Control procedures excluded the simple passage of time, response elicitation or induction by food presentation, type of operandum, food delivery device location, and adventitious immediate reinforcement of responding as the basis for the effects. Results revealed that neither training nor immediate reinforcement is necessary to establish new behavior. The conditions that give rise to both the first and second response are discussed, and the results are related to other studies of the delay of reinforcement and to explanations of behavior based on contingency or correlation and contiguity.     

Marking, memory and superstition in the pigeon

Learning is generally poor if reinforcement is delayed, but it improves substantially if a brief stimulus is presented immediately after the response to be learned. The marking hypothesis suggests that the unexpected stimulus triggers a backward memory search, which effectively marks the preceding response in memory, making it more likely that it will be recalled when food is presented. In the present study, pigeons were occasionally reinforced after a 10-sec delay for pecking a split key. Reinforcement was presented regardless of which side was pecked, but for one group a marker followed a peck to the left half of the key during the delay preceding food, and for the other group a peck to the right. On non food trials these contingencies were reversed. Subjects developed a significant preference for the side marked on food trials, despite the absence of any contingency between responding to this side and food. In addition to providing further support for the marking hypothesis, these results favour theories of reinforcement emphasizing contiguity rather than contingency. Contiguity, however, needs to be interpreted within a memory framework. What is crucial is the contiguity of events within working memory, rather than in the real world.     

Equivalence relations and the reinforcement contingency

Where do equivalence relations come from? One possible answer is that they arise directly from the reinforcement contingency. That is to say, a reinforcement contingency produces two types of outcome: (a) 2‐, 3‐, 4‐, 5‐, or n‐term units of analysis that are known, respectively, as operant reinforcement, simple discrimination, conditional discrimination, second‐order conditional discrimination, and so on; and (b) equivalence relations that consist of ordered pairs of all positive elements that participate in the contingency. This conception of the origin of equivalence relations leads to a number of new and verifiable ways of conceptualizing equivalence relations and, more generally, the stimulus control of operant behavior. The theory is also capable of experimental disproof.     

Stimuli, reinforcers, and behavior: an integration

We propose that a fundamental unit of behavior is the concurrent discriminated operant, and we discuss in detail a quantitative model of the concurrent three-term contingency that is based on the notion that an animal's behavior is controlled to differing extents by both stimulus—behavior and behavior—reinforcer relations. We show how this model can describe performance in a variety of experimental procedures: conditional discrimination and matching to sample, both with and without reinforcement for responses that are traditionally identified as errors; conditional discrimination with more than two stimuli and choice alternatives; delayed matching to sample and delayed reinforcement in matching to sample; second-order and complex conditional discrimination; and multiple and concurrent schedules. Although the model is incomplete in its coverage, and may be incorrect, we believe that this conceptual approach will bear fruit in the development of behavior theory.     

Complex form of instrumental behavior of dogs

A complex form of higher nervous activity, conditional reflex transswitching or switching, was elaborated in four dogs under conditions of unrestrained movements, freedom of reinforcement choice (food or water), place of reinforcement determined by situation factors as well as independent switching on of conditional stimuli. It was shown that motivated goal-directed behavior of the animals was determined by activation of forward and backward connections. The chains of instrumental conditioned reflexes forming in the final analysis the complex behavior are developed according to the “trialand-error” principle. In the instrumental conditioned reflexes there are two constantly coexisting types of mutually complementing conditioned connections of the signal with reinforcement—direct and indirect. In experiments with unrestrained conditions of animals and independent regulation of the experiment, one of the frequently encountered physiological mechanisms of generalization appears to be efferent and afferent generalization.     

Reinforcement contingencies and social reinforcement: some reciprocal relations between basic and applied research.

Reinforcement contingencies and social reinforcement are ubiquitous phenomena in applied behavior analysis. This discussion paper is divided into two sections. In the first section, reinforcement contingencies are discussed in terms of the necessary and sufficient conditions for reinforcement effects. Response‐stimulus dependencies, conditional probabilities, and contiguity are discussed as possible mechanisms of, and arrangements for, reinforcement effects. In the second section, social reinforcement is discussed in terms of its functional subtypes and reinforcement context effects. Two underlying themes run throughout the discussion: (a) Applied research would benefit from a greater understanding of existing basic research, and (b) basic research could be designed to specifically address some of the issues about reinforcement that are central to effective application.     

Contiguity and the outcome density bias in action-outcome contingency judgements

In cause-outcome contingency judgement tasks, judgements often reflect the actual contingency but are also influenced by the overall probability of the outcome, P(O). Action-outcome instrumental learning tasks can foster a pattern in which judgements of positive contingencies become less positive as P(O) increases. Variable contiguity between the action and the outcome may produce this bias. Experiment 1 recorded judgements of positive contingencies that were largely uninfluenced by P(O) using an immediate contiguity procedure. Experiment 2 directly compared variable versus constant contiguity. The predicted interaction between contiguity and P(O) was observed for positive contingencies. These results stress the sensitivity of the causal learning mechanism to temporal contiguity.     

An evaluation of contingency strength and response suppression

Typically, functional analyses of severe problem behavior have been conducted in two ways: (a) The target response is reinforced immediately after it occurs, or (b) the target response is reinforced on some schedule thought to mimic a naturally occurring schedule. We evaluated the effects of contingency strength in reducing levels of problem behavior with 2 participants who had been diagnosed with developmental disabilities. Results showed that under a neutral contingency, one in which the probability of reinforcement for aggression was equal to the probability of reinforcement for the nonoccurrence of aggression, rates of aggression were suppressed to low levels for both participants.     

Contiguity and contingency in the acquisition and maintenance of an operant

Three experiments compared the effects of nondifferential and differential reinforcement of response location on a circular dimension. Rats were required to operate a vertical joystick to produce food. When food was delivered immediately after responses, but independent of response location, the spatial concentration of responding was low and no progressive changes were observed. Traditional and percentile schedules of differential reinforcement for response location produced highly reliable acquisition of spatially concentrated responding. Once concentrated responding had been established, nondifferential reinforcement was sufficient to maintain it in some subjects. Since only the differential reinforcement schedules established a contingency with respect to response location, it was concluded that this relationship was necessary for acquisition, but that response-reinforcer contiguity may be sufficient for maintenance. This conclusion is consistent with the view that operant conditioning is a contiguity-based process, but that contingencies are required to produce reliable contiguity between reinforcers and particular responses.     

The correlation-based law of effect

It is commonly understood that the interactions between an organism and its environment constitute a feedback system. This implies that instrumental behavior should be viewed as a continuous exchange between the organism and the environment. It follows that orderly relations between behavior and environment should emerge at the level of aggregate flow in time, rather than momentary events. These notions require a simple, but fundamental, change in the law of effect: from a law based on contiguity of events to a law based on correlation between events. Much recent research and argument favors such a change. If the correlation-based law of effect is accepted, it favors measures and units of analysis that transcend momentary events, extending through time. One can measure all consequences on a common scale, called value. One can define a unit of analysis called the behavioral situation, which circumscribes a set of values. These concepts allow redefinition of reinforcement and punishment, and clarification of their relation to discriminative stimuli.     

Interresponse-time shaping by variable-interval-like interresponse-time reinforcement contingencies

The interresponse-time reinforcement contingencies and distributions of interreinforcement intervals characteristic of certain variable-interval schedules were mimicked by reinforcing each key peck with a probability equal to the duration of the interresponse time it terminated, divided by the scheduled mean interreinforcement interval. The interresponse-time reinforcement contingency was then eliminated by basing the probability of reinforcement on the fifth interresponse time preceding the key peck. Even though distributions of interreinforcement intervals were unaffected by this manipulation, response rates consistently increased. A second experiment replicated this effect and showed it to combine additively with that of mean reinforcement rate. These results provide strong support for the contention that current analyses of variable-interval response rates that ignore the inherent interresponse-time reinforcement contingency may be seriously in error.     

Intersections of behavior analysis with cognitive models of contingency detection

Bower and Watson have offered, respectively, a logical hypothesis-testing model and a conditional probability model of contingency detection by young infants. Although each could represent cognitive processes concomitant with operant learning, empirical support for these models is sparse. The limitations of each model are discussed, and suggestions are made for a more parsimonious approach by focusing on the areas of overlap between the two.     

Precurrent contingencies: Behavior reinforced by altering reinforcement probability for other behavior

The present study explored the effects of a precurrent contingency in which one (precurrent) activity increased the reinforcement probability for another (current) activity. Four human subjects responded on a two-key computer mouse. Each right-key press was reinforced (points exchangeable for money) with .02 probability. In one condition (no precurrent contingency), pressing the left key had no scheduled consequence; in another condition (precurrent contingency), pressing the left key increased the reinforcement probability for right-key responding to .08 for 15 s. Initial exposure to the precurrent contingency resulted in acquisition of precurrent left-key responding for 3 subjects, but for the 4th subject a special contingency was required. Right-key responding occurred at a high stable rate across the conditions. Changeovers to left-key responding dropped to near zero when the precurrent contingency was absent and were maintained at enhanced levels when the precurrent contingency was present. Contacts with the left key consisted of short response runs. Right-key responses were more frequently emitted within 15 s of a left-key response when the precurrent contingency was present, an efficient adaptation to the contingency. Continued research on precurrent behavior may produce insights into complex phenomena such as autoclitics and self-control.     

Behavioral control by the response–reinforcer correlation

Using a discrete‐trials procedure, two experiments examined the effects of response–reinforcer correlations on responding while controlling molecular variables that operated at the moment of reinforcer delivery (e.g., response–reinforcer temporal contiguity, interresponse times preceding reinforcement). Each trial consisted of three successive components: Response, Timeout, and Reinforcement, with the duration of each component held constant. The correlation between the number of responses in the Response component and reinforcer deliveries in the Reinforcement component was varied. In the Positive‐correlation condition, a larger number of responses in the Response component programmed a higher reinforcement rate (Experiment 1) or a shorter time to reinforcement (Experiment 2) in the Reinforcement component. Although programmed in this way, the actual reinforcer delivery was dependent on, and occurred immediately after, a response in the Reinforcement component. In the Zero‐correlation condition, the programmed rates of reinforcement (Experiment 1) or the times to reinforcement (Experiment 2) in the Reinforcement component of each trial were yoked to those in the preceding Positive‐correlation condition. Responding in the Response component was higher in the Positive‐ than in the Zero‐correlation condition, without systematic changes in molecular variables. The results suggest that the response–reinforcer correlation can be a controlling variable of behavior.     

FUNCTIONAL ANALYSIS OF INAPPROPRIATE SOCIAL INTERACTIONS IN STUDENTS WITH ASPERGER'S SYNDROME

We analyzed the inappropriate social interactions of 3 students with Asperger's syndrome whose behavior was maintained by social positive reinforcement. We tested whether inappropriate social behavior was sensitive to social positive reinforcement contingencies and whether such contingencies could be reversed to increase the probability of socially appropriate responding. Our results show that social positive reinforcers can be identified for inappropriate social interactions and that appropriate social behaviors can be sensitive to reinforcement contingency reversals.     

Serial conditioning as a function of stimulus, response, and temporal dependencies

Six experiments were used to examine the effects of explicit response, stimulus, and temporal dependencies on responding in an interfood interval. The first two experiments demonstrated that 10-segment 60-s interfood clocks controlled similar distributions of key pecking in pigeons regardless of whether response–reinforcement contiguity was required, allowed, or precluded. The third and fourth experiments found that in the absence of an explicit response–reinforcement dependency, systematic explicit stimuli in an interfood interval were sufficient to establish and maintain the characteristic distribution of key pecking and that an interval without an explicit clock failed to establish or maintain key pecking. The last two experiments demonstrated that the interfood interval need not be of fixed length, and that a simple correlation of stimuli with increments from either a minimum to a maximum imminency or probability of food presentation controlled behavior in a similar manner. Successively higher rates generally occurred to successively later stimuli in the upper half of the range.     

Associative retrieval processes in free recall

I present a new method for analyzing associative processes in free recall. While previous research has emphasized the prominence of semantic organization, the present method illustrates the importance of association by contiguity. This is done by examining conditional response probabilities in the output sequence. For a given item recalled, I examine the probability and latency that it follows an item from a nearby or distant input position. These conditional probabilities and latencies, plotted as a function of the lag between studied items, reveal several regularities about output order in free recall. First, subjects tend to recall items more often and more rapidly from adjacent input positions than from remote input positions. Second, subjects are about twice as likely to recall adjacent pairs in the forward than in the backward direction and are significantly faster in doing so. These effects are observed at all positions in the output sequence. The asymmetry effect is theoretically significant because, in cued recall, nearly symmetric retrieval is found at all serial positions (Kahana, 1995; Murdock, 1962). An attempt is made to fit the search of associative memory model (Raaijmakers & Shiffrin, 1980, 1981) with and without symmetric interitem associations to these data. Other models of free recall are also discussed.     

Saying and doing: A contingency-space analysis

Correspondences between verbal responding (saying) and nonverbal responding (doing) may be organized in terms of the classes of verbal/nonverbal relations into which particular instances of verbal/nonverbal response sequences can enter. Contingency spaces, which display relations among events in terms of the probability of one event given or not given another, have been useful in analyses of nonverbal behavior. We derive a taxonomy of verbal/nonverbal behavior relations from a contingency space that takes into account two conditional probabilities: the probability of a nonverbal response given a verbal response and that probability given the absence of the verbal response. For example, positive correspondence may be said to exist as a response class when the probability of doing is high given saying but is otherwise low. Criteria for other generalized classes, including negative correspondence, follow from this analysis.