Behavioral contrast and response independent reinforcement

Four pigeons received pre-training that included presentation of the reinforcer independently of behavior and then baseline training on a variable-interval schedule of reinforcement. With the introduction of a multiple schedule, in which the first stimulus was associated with a response contingent and a second stimulus with a response independent, 1-min variable-interval schedule, a reduction in response rate was obtained in the second component, which was not accompanied by a behavioral contrast effect in the first component. A further three pigeons were given the same pre-training and baseline training before the introduction of an otherwise identical multiple schedule, in which no reinforcement occured in the second component. Behavioral contrast was obtained from all three subjects. The results indicated that under conditions of constant reinforcement density a reduction in responding is not a sufficient condition for the occurrence of behavioral contrast.     

Behavioral contrast: Pavlovian effects and anticipatory contrast.

Two sources of behavioral contrast have been identified previously: Pavlovian stimulus-reinforcer relations and component sequence effects (anticipatory contrast). This study sought to isolate these sources of control procedurally in a four-ply multiple schedule composed of two fixed two-component sequences. Different cues were associated with the first component of each sequence, and contrast effects were studied in these target components. In Experiment 1, differential cuing of Component 2 between sequences and availability of reinforcement during target components were varied across three groups of pigeons; the stimulus-reinforcer relation between target-component cues and schedule of reinforcement in Component 2 was varied within subjects. Control by the Pavlovian relation was demonstrated under all conditions, and anticipatory contrast was not observed. In Experiment 2, target-component duration was systematically varied in the three groups of Experiment 1. Control by the Pavlovian relation was reliably obtained only when target-component behavior was unreinforced, and diminished with increases in component duration. Anticipatory contrast emerged in the two groups for which target-component reinforcement was available. These and other data indicate that Pavlovian effects in multiple schedules may be obscured when the requisite conditions for anticipatory contrast are present.     

Neural events in the reinforcement contingency

When neural events are analyzed as stimuli and responses, functional relations among them and among overt stimuli and responses can be unveiled. The integration of neuroscience and the experimental analysis of behavior is beginning to provide empirical evidence of involvement of neural events in the three-term contingency relating discriminative stimuli, responses, and consequences. This paper is aimed at highlighting exemplar instances in the development of this issue. It has long been known that the electrical stimulation of certain cerebral areas can have a reinforcing function. Extraordinary technological advances in recent years show that neural activity can be selected by consequences. For example, the activity of in vitro isolated neurons that receive dopamine as a reinforcer functions as a cellular analogue of operant conditioning. The in vivo activity of populations of neurons of rats and monkeys can be recorded on an instant-to-instant basis and can then be used to move mechanical arms or track a target as a function of consequences. Neural stimulation acts as a discriminative stimulus for operant responses that are in turn maintained by neural consequences. Together with investigations on the molecular basis of classical conditioning, those studies are examples of possibilities that are being created for the study of behavior-environment interactions within the organism. More important, they show that, as an element in the three-term contingency, neural activity follows the same laws as other events.     

Rethinking reinforcement: allocation, induction, and contingency

The concept of reinforcement is at least incomplete and almost certainly incorrect. An alternative way of organizing our understanding of behavior may be built around three concepts: allocation, induction, and correlation. Allocation is the measure of behavior and captures the centrality of choice: All behavior entails choice and consists of choice. Allocation changes as a result of induction and correlation. The term induction covers phenomena such as adjunctive, interim, and terminal behavior-behavior induced in a situation by occurrence of food or another Phylogenetically Important Event (PIE) in that situation. Induction resembles stimulus control in that no one-to-one relation exists between induced behavior and the inducing event. If one allowed that some stimulus control were the result of phylogeny, then induction and stimulus control would be identical, and a PIE would resemble a discriminative stimulus. Much evidence supports the idea that a PIE induces all PIE-related activities. Research also supports the idea that stimuli correlated with PIEs become PIE-related conditional inducers. Contingencies create correlations between "operant" activity (e.g., lever pressing) and PIEs (e.g., food). Once an activity has become PIE-related, the PIE induces it along with other PIE-related activities. Contingencies also constrain possible performances. These constraints specify feedback functions, which explain phenomena such as the higher response rates on ratio schedules in comparison with interval schedules. Allocations that include a lot of operant activity are "selected" only in the sense that they generate more frequent occurrence of the PIE within the constraints of the situation; contingency and induction do the "selecting."     

Effects of a DRL contingency added to a fixed-interval reinforcement schedule

Following 30 days of reinforcement for the bar press response of two white rats on 30-sec fixed-interval (FI), a DRL component was added so that a minimal interresponse time (IRT) for the reinforced response, in addition to the FI variable, was necessary for reinforcement. Marked control over response rate by the superimposed DRL requirement was demonstrated by an inverse hyperbolic function as the DRL component was increased from 1 to 24 sec within the constant 30-sec FI interval. Interresponse time and post-reinforcement (post-S^R) “break” distributions taken at one experimental point (DRL = 24 sec) suggested that a more precise temporal discrimination was initiated by an S^R than by a response, since the relative frequency of a sequence of two reinforced responses appeared greater than that of a sequence of a non-reinforced response followed by a reinforced one. This latter finding was confirmed with new animals in a follow-up experiment employing a conventional 24-sec DRL schedule.     

Parametric manipulation of interresponse-time contingency independent of reinforcement rate

Pecking of pigeons was reinforced under a modified interval-percentile procedure that allowed independent manipulation of overall reinforcement rate and the degree to which reinforcement depended on interresponse-time duration. Increasing the contingency, as measured by the phi coefficient, between reinforcement and long interresponse times while controlling the overall rate of reinforcement systematically increased the frequency of those interresponse times and decreased response rate under both of the reinforcement rates studied. Increasing reinforcement rate also generally increased response rate, particularly under weaker interresponse-time contingencies. Random-interval schedules with comparable reinforcement rates generated response rates and interresponse-time distributions similar to those obtained with moderate-to-high interresponse-time reinforcement contingencies. These results suggest that interresponse-time reinforcement contingencies inherent in random-interval and constant-probability variable-interval schedules exercise substantial control over responding independent of overall reinforcement rate effects. The interresponse-time reinforcement contingencies inherent in these schedules may actually mask the effects of overall reinforcement rate; thus differences in response rate as a function of reinforcement rate when interresponse-time reinforcement is eliminated may be underestimated.     

From beliefs to behavior: the mediating role of hostile response selection in predicting aggression

This study evaluated the complementary roles that aggressive normative beliefs and hostile response selections play in predicting adolescents' aggressive behavior. Self‐, peer‐, and teacher‐reported data were collected from 2003 young adolescents from a variety of ethnic backgrounds (Black, Latino, Asian, White) during the fall of their sixth grade year to test the hypothesis that adolescents' hostile response selections mediate the association between their normative beliefs and aggressive reputations among their peers and teachers. Structural equation modeling analyses supported our hypothesis, and the model was found to be consistent across both gender and ethnicity. The results suggest that even though gender and ethnic differences in average levels of aggressive cognitions and behavior exist, the general process linking cognitions and behaviors is the same for all young adolescents. Aggr. Behav. 00:1–20, 2005. © 2005 Wiley‐Liss, Inc.     

Absence of anticipatory contrast in rats trained on multiple schedules.

Rats were trained on three- and four-component multiple schedules in which two of the components were correlated with identical reinforcement schedules that remained unchanged throughout training. These target components differed in terms of whether their respective following schedules were either higher or lower in value. Unlike corresponding experiments previously reported with pigeons, higher response rates occurred in the target component followed by a higher valued schedule than in the target component followed by the lower valued schedule. Overall contrast effects occurred independently of these sequential effects, but were inconsistent across subjects. The results suggest that the effects of a following schedule of reinforcement are opposite for pigeons and rats, and that one reason previous studies have often failed to show contrast effects with rats is that the effects of the following schedule in rats are in competition with contrast dynamics.     

The role of contingency in classical conditioning

The assumption that classical conditioning depends on a contingent relation between the conditioned stimulus (CS) and the unconditioned stimulus (US), which was proposed some decades ago as an alternative to the traditional contiguity assumption, still is widely accepted as an empirical generalization, if no longer as a theoretical postulate. The first support for the contingency assumption was provided by experiments in which occasional CS-US pairings produced no response to the CS in random training--i.e., training in which the probability of the US was the same in the presence and absence of the CS. Those early experiments, the results of which too often are taken at face value, are reconsidered along with various later experiments that show conditioning, both of the CS and its context, in random training. The evidence suggests that CS-US contingency is neither necessary nor sufficient for conditioning and that the concept has long outlived any usefulness it may once have had in the analysis of conditioning.     

Use of a lag differential reinforcement contingency to increase varied selections of classroom activities

The present study evaluated the effects of a lag differential reinforcement contingency on 2 students' activity selections using reversal designs. Results showed that the lag contingency was responsible for promoting increased novel selections, engagement in diverse activities, and greater progress with respect to programmed academic activities.     

The response-stimulus contingency and reinforcement learning as a context for considering two non-behavior-analytic views of contingency learning

This paper introduces a special section on the contingency. Bower and Watson were invited to present their views of contingency learning in human infants from outside the context of behavior analysis, and Cigales, Marr, and Lattal and Shahan provided commentaries that point out some of the more interesting and controversial aspects of those views from a behavior-analytic perspective. The debate turns on how to conceptualize the response-stimulus contingency of operant learning. The present paper introduces the contingency concept and contingency detection by subjects, as well as research practices in behavior analysis, in a context in which the dependency between infant responding and the presentation of environmental consequences may be disrupted through procedures in which ordinarily consequent events occur before the response or in its absence. These points can relate to and serve as an introduction to the Bower and Watson papers on infant contingency learning as well as to the three commentaries that follow.     

Successive discrimination training with equated reinforcement frequencies: failure to obtain behavioral contrast

In two experiments, pigeons were trained on two-component multiple schedules in which responding in one component (S₁) was always maintained by a variable-interval schedule. In Experiment I, low response rates were reinforced in the second (S₂) component for six master subjects. This schedule was adjusted to equate reinforcement frequencies in the two components. These subjects were compared to yoked partners, for which reinforcement in the S₂ component was made available on a variable-interval schedule whose value was determined by the master subjects. A similar procedure was used in Experiment II, where the S₂ schedule for master subjects made reinforcers contingent on the absence of responding. No evidence was found in either experiment for a behavioral contrast effect in the S₁ component attributable to response reduction in the S₂ component. A reliable contrast effect was obtained from a group of pigeons given extinction conditions in the S₂ component, which was compared to a group maintained throughout on a multiple variable-interval schedule. The results suggest that previous indications of behavioral contrast in similar situations were probably caused by uneven reinforcement distributions or reflect uncontrolled fluctuations in response rates.     

Order effects in contingency learning: The role of task complexity

Dennis and Ahn (2001) found that during contingency learning, initial evidence influences causal judgments more than does later evidence (a primacy effect), whereas López, Shanks, Almaraz, and Fernández (1998) found the opposite (a recency effect). We propose that in contingency learning, people use initial evidence to develop an anchoring hypothesis that tends to be underadjusted by later evidence, resulting in a primacy effect. Thus, factors interfering with initial hypothesis development, such as simultaneously learning too many contingencies, as in López et al., would reduce the primacy effect. Experiment 1 showed a primacy effect with learning contingencies involving only one outcome but no primacy effect with two outcomes. Experiment 2 demonstrated that the magnitude of the primacy effect correlated with participants' verbal working memory capacity. It is concluded that a critical moderator for exhibition of the primacy effect is task complexity, presumably because it interferes with initial hypothesis development.     

Interactions in multiple schedules: the role of the stimulus-reinforcer contingency

In Experiments I and II, pigeons were exposed to single-key multiple schedules of response-independent and -dependent food presentation. Components were correlated with different keylights. When the rate of food presentation in the first component exceeded that in the second component, the local rate of key pecking was relatively high at onset of the first component. Overall rate in that component varied inversely with component duration and the rate of food presentation in the second component. When responding was maintained in the second component, the local rate of key pecking was relatively low at onset of that component. Overall rate in the second component varied directly with component duration and the rate of food presentation in that component. In Experiment III, pigeons were exposed to a two-key multiple schedule. Pecks on a constantly illuminated key produced food. Components were correlated with the color of a second key on which pecks had no scheduled consequences. The effects of component duration and rate of food presentation under the single-key response-dependent schedule were synthesized by combining response rates on each concurrently available key under the two-key procedure. The results support an account of multiple-schedule interactions in terms of the joint influence on responding of stimulus-reinforcer and response-reinforcer contingencies.     

From the psychiatric hospital to the community: integrating conditional release and contingency management

Psychiatric hospital recidivism has been and continues to be a persistent problem in treating individuals with chronic mental illness. Conditional release, a form of involuntary outpatient commitment, has been suggested as one possible solution. Guided by therapeutic jurisprudence, this article presents a proposal about conditional release that would maximize convergence of social values and would be empirically testable. Specifically, a scientifically validated treatment intervention for individuals with chronic mental illness, contingency management, is integrated with conditional release. From this proposal, a number of empirical hypotheses and legal questions about discharging psychiatric patients are generated and discussed.     

Negative anticipatory contrast and preference conditioning: flavor cues support preference conditioning, and environmental cues support contrast.

In 2 experiments, access to a 0.15% saccharin solution was followed on alternating days by access to a 32% sucrose solution and the same saccharin solution. In Experiment 1, rats increased both intake of and preference for a flavored saccharin solution that predicted sucrose, but neither effect was found using a predictive odor cue alone. Experiment 2 replicated the predictive flavor results but showed suppression of saccharin intake when environmental cues predicted sucrose. When both flavor and environment predicted sucrose, saccharin intake did not change, but preference for the predictive flavor increased. Discriminative taste cues appear to facilitate the development of preference conditioning, but environmental cues favor negative anticipatory contrast effects. Also, preference conditioning and contrast may develop concurrently and compete for expression.     

Judgement of act-outcome contingency: The role of selective attribution

In the first experiment subjects were presented with a number of sets of trials on each of which they could perform a particular action and observe the occurrence of an outcome in the context of a video game. The contingency between the action and outcome was varied across the different sets of trials. When required to judge the effectiveness of the action in controlling the outcome during a set of trials, subjects assigned positive ratings for a positive contingency and negative ratings for a negative contingency. Furthermore, the magnitude of the ratings was related systematically to the strength of the actual contingency. With a fixed probability of an outcome given the action, judgements of positive contingencies decreased as the likelihood that the outcome would occur without the action was raised. Correspondingly, the absolute value of ratings of negative contingencies was increased both by an increment in the probability of the outcome in the absence of the action and by a decrement in the probability of the outcome following the action. A systematic bias was observed, however, in that positive judgements were given under a non-contingent relationship when the outcome frequency was relatively high. However, this bias could be reduced by giving extended exposure to the non-contingent schedule (Experiment 2).

This pattern of contingency judgements can be explained if it is assumed that a process of selective attribution operates, whereby people are less likely to attribute an outcome to some potential target cause if another effective cause is present. Experiments 2 and 3 demonstrated the operation of this process by showing that initially establishing another agent as an effective cause of the outcome subsequently reduced or blocked the extent to which the subjects attributed the outcome to the action.

Finally, we argue that the pattern and bias in contingency judgements based upon interactions with a causal process can be explained in terms of contemporary conditioning models of associative learning.     

Successive, simultaneous, and anticipatory contrast in the consumption of saccharin solutions

Contrast effects were obtained in rats in the consumption of saccharin solutions in three different paradigms. Degree of negative contrast varied as a function of concentration disparity, but not equally in the three procedures. Successive negative contrast occurred following shifts from 0.15% to either 0.075% or 0.05% saccharin but did not occur following shifts to 0.10% or 0.125% saccharin. Some degree of simultaneous contrast was obtained with all four concentration disparities. Anticipatory contrast, where the intake of the first substance is suppressed by a more preferred second substance, occurred only in the case of the 0.05%-0.15% difference in concentrations. It was suggested that the several contrast paradigms engage somewhat different psychological processes differentially involving emotional, sensory, and associative mechanisms, but all lead to behavior based on relative value. A modification of Toates's (1981) incentive model of ingestive behavior was suggested to incorporate relativity effects based on both associative and nonassociative factors in the consumption of both nutritive and nonnutritive substances.