Reinforcement Probability and Delay as Determinants of Human Impulsiveness

In behavior theory, “impulsiveness” refers to the choice of an immediate, small reinforcer over a delayed, large reinforcer. Such behavior generally is attributed to a reduction in the value of the large reinforcer as a function of the duration of delay. In contrast, social learning theorists have suggested that human impulsiveness can result from a lowered “expectancy” (subjective probability) of reinforcement. Effects of probability and delay were assessed by asking adults to make repeated choices between reinforcement schedules in which the reinforcers were slides of entertainment figures. An immediate, 5-s reinforcer was consistently chosen over an immediate, 40-s reinforcer if the probability of receiving the large reinforcer had previously been low (.20), implying that impulsiveness can occur without time-based discounting. However, reinforcement delay was also influential: Choice between a certain, small reinforcer and an uncertain, large reinforcer varied according to which reinforcer was immediate and which delayed.

     

Effects of Response Probability on Response Force in Simple RT

Response force (RF) was measured in a simple reaction time (RT) experiment varying response uncertainty by cuing the probability of the response on each trial. In all cases, RF decreased as response probability increased. The dependence of RF on response probability was insensitive to foreperiod length and to the use of loud auditory response signals, although the dependence of RT on response probability was sensitive to both of these manipulations. In combination with previous findings, these results provide evidence that RT and RF can be dissociated. We describe an extension of Naatanen's readiness model that can account for the effects of response probability on RF and RT. According to this model, the distance between motor activation and a threshold for action is relatively large when subjects are unprepared, and a large increment is needed to exceed this threshold, resulting in slow but foreceful responses. A possible neurophysiological implementation of this model is suggested.     

Unpacking Response Contingent Positive Reinforcement: Reward Probability,but Not Environmental Suppressors,Prospectively Predicts Depressive Symptoms via Behavioral Activation

Behavioral models of depression conceptualize the development and maintenance of depressive symptoms as occurring within a system of response-contingent positive reinforcement (RCPR). Such models propose a negative cycle in which a decrease in positive reinforcement of adaptive, approach-oriented behaviors (i.e., RCPR) results in reduced behavioral activation and increased depressive symptoms, which in turn feeds back into further decreases in positive reinforcement and maintains the cycle. Previous research indicates that RCPR has two lower-order constructs, reward probability and environmental suppressors. The present study evaluated a hypothesized mediational model in which the paths from each of the two domains of RCPR to depressive symptoms are mediated by behavioral activation. The sample consisted of 150 college students (78.0% women), ranging in age from 18 to 47 years (M = 20.81, SD = 3.96), who endorsed moderate to severe depressive symptoms at a baseline (time 1) evaluation. Findings from a path analysis provide support for the mediated path between reward probability at time 1 and depressive symptoms at time 3, via behavioral activation at time 2. In contrast, environmental suppressors at time 1 demonstrated a direct impact on depressive symptoms at time 3, and this relationship was not mediated by behavioral activation at time 2. These findings provide a partial test of behavioral frameworks and provide evidence of separate pathways between domains of RCPR and depressive symptoms.     

On Setting Response Criteria for Calibrated Subjective Probability Estimates

Framing the issue of subjective probability calibration in signal-detection-theory terms, this paper first proves a theorem regarding the placement of well-calibrated response criteria and then develops an algorithm guaranteed to find such criteria, should they exist. Application of this algorithm to tasks varying in difficulty and number of response categories shows that perfect calibration is easiest to attain under median difficulty levels (d' approximately 1.4) and is practically or theoretically impossible to attain when the task is either very hard (d' approximately 0.5) or very easy (d' approximately 10). Implications for calibration research, including the hard-easy effect, are discussed. Copyright 2001 Academic Press.     

Variable-Interval Schedule Control Following Response Acquisition with Delayed Reinforcement

The Psychological Record - Four experimentally naive White Carneau pigeons acquired a key-peck response without specific response shaping or other training when such responding was reinforced...     

Sequences of Fixed-Ratio Schedules: The Relative Effects of Reinforcer Magnitude and Ratio Size in the Second Fixed Ratio

The Psychological Record - Four pigeons were given a choice between two sequences of fixed-ratio schedules. The second fixed ratio was a fixed-ratio 5 followed by 2-s access to grain in one...     

Reinforcement Delay Fading During Differential Reinforcement of Communication: The Effects of Signals on Response Maintenance

Signals during delays to reinforcement may lessen reductions in responding that typically occur when there is a delay between a response and its reinforcer. Sparse applied research has been devoted to understanding the conditions under which responding may be maintained when delays to reinforcement are introduced. We evaluated the extent to which providing signals during delay fading affected responding in the context of differential reinforcement of communication responses. Three individuals were exposed to gradually increasing signaled and unsignaled reinforcement delays in multiple‐schedule and/or withdrawal designs. Results for 2 of 3 participants suggested that (a) the presence of signals facilitated response maintenance under delayed reinforcement and (b) coordinated basic and applied research may advance both conceptual understanding and clinical outcomes of delayed reinforcement.     

Chained schedules of avoidance: Reinforcement within and by avoidance situations

Four rats were exposed to chained schedules with variable-cycle avoidance in both links. Responding in the initial link cancelled shocks scheduled once per minute and, according to a conjoint fixed-ratio schedule, produced a terminal link where scheduled shock rates varied from 0 to 8 shocks per minute in different conditions of the experiment. Response rates in the terminal link increased as a function of the scheduled shock rate. Response rates in the initial link, on the other hand, decreased as a function of the shock rate actually received (rather than scheduled) in the terminal link. While consistent with other studies of aversive control, these results differ from those obtained in chained schedules of positive reinforcement in that increases in reinforcement within the terminal link of the chain did not systematically increase the reinforcing value of that link.     

The Modulation of Operant Variation by the Probability, Magnitude, and Delay of Reinforcement

Recent studies have demonstrated that the expectation of reward delivery has an inverse relationship with operant behavioral variation (e.g., Stahlman, Roberts, & Blaisdell, 2010). Research thus far has largely focused on one aspect of reinforcement – the likelihood of food delivery. In two experiments with pigeons, we examined the effect of two other aspects of reinforcement: the magnitude of the reward and the temporal delay between the operant response and outcome delivery. In the first experiment, we found that a large reward magnitude resulted in reduced spatiotemporal variation in pigeons’ pecking behavior. In the second experiment, we found that a 4-s delay between response-dependent trial termination and reward delivery increased variation in behavior. These results indicate that multiple dimensions of the reinforcer modulate operant response variation.     

The Role of Emotions in Reinforcement: Response Selection in Humans

The Psychological Record - Historically, researchers have never quite been able to agree as to the role of emotions, if any, when behavior is selected by its consequences. A brief review of...     

Stimulus Fading and Response Elaboration in Differential Reinforcement for Alternative Behavior

A hallmark of applied behavior analysis is the development of function‐based interventions for problem behavior. A widely recommended function‐based intervention is differential reinforcement of alternative behavior (DRA), in which reinforcement is contingent upon socially acceptable alternatives to problem behavior (e.g., teaching communication skills). Typically, DRA is introduced under rich schedules of reinforcement. Although effective for initiating behavior change, rich schedules are often impractical in the natural setting. In this study, we evaluated the extent to which a stimulus fading program could be employed to elaborate alternative behavior (mands) in two individuals diagnosed with an autism spectrum disorder. For both participants, problem behavior was reduced substantially upon implementation of the DRA procedure. Further, problem behavior rates remained low and mand rates decreased to more practical levels as the DRA behavioral requirements increased during the fading program. The fading approach demonstrated in this paper may be a useful component of intervention packages for clinicians. Copyright © 2015 John Wiley & Sons, Ltd.     

Reinforcement contingencies maintaining collateral responding under a DRL schedule

Two-key conjunctive schedules were studied with one key (food key) under a differential-reinforcement-of-low-rate 20-sec schedule, while the consequences of responding on another key (collateral key) were varied. When food depended not only upon a food-key interresponse time in excess of 20 sec, but also upon the occurrence of one or more collateral-key responses during the food-key interresponse time, the rate of collateral-key responding was low and food-key interresponse times rarely exceeded 20 sec. When collateral-key responses could produce a discriminative stimulus correlated with the availability of food under the DRL schedule, the discriminative stimulus functioned as a conditioned reinforcer to maintain higher rates of collateral-key responding, and the spacing of food-key responses increased. If the occurrence of the discriminative stimulus was independent of collateral-key responses, the rate of collateral-key responding was again low, but the spacing of food-key responses was still controlled by the discriminative stimulus. Both the conditioned reinforcer and the explicit reinforcement contingency could maintain collateral-key responding, but the adventitious correlation between collateral-key responses and the delivery of food could not maintain very much collateral-key responding. The pattern of responding on the food-key was determined to a much greater extent by the correlation between the discriminative stimulus and the delivery of food than by the pattern of responding on the collateral key.     

Performance under Competitive and Self-Competitive Fixed-Interval Schedules of Reinforcement

The Psychological Record - Participants completed blocks of trials under simple (i.e., work-alone), competitive, and self-competitive fixed-interval 20-s schedules of reinforcement. In general,...     

Context Conditioning and Free operant Acquisition under Delayed Reinforcement

Three experiments examined the effect of context conditioning on the acquisition of freeoperant lever pressing by hungry rats when the presentation of the food reinforcer was delayed for 32 sec. The first study replicated the preexposure effect reported by Dickinson, Watt, and Griffiths (1992): Exposure to the contextual cues with the lever withdrawn prior to each instrumental training session enhanced acquisition, an effect that was attenuated by the presentation of non-contingent reinforcement during the preexposure periods. Signalling the non-contingent reinforcers during the preexposure periods with a brief auditory stimulus enhanced acquisition in a second study, suggesting that the non-contingent reinforcement interferes with acquisition through context conditioning. The final study confirmed this conclusion using a within-subject procedure in which pressing different levers was reinforced in two contexts, one of which was also associated with non-contingent reinforcers.     

Effects Of Histories Of Differential Reinforcement Of Response Rate On Variable-interval Responding

Three pigeons were exposed first to multiple differential-reinforcement-of-high-rate and differential-reinforcement-of-low-rate schedules that were correlated with green and red keys, respectively, and then were shifted to a variable-interval schedule arranged on a white key. In subsequent test sessions, the variable-interval schedule continued to operate, but green and red keys replaced the white key in alternate sessions. In Part 1 of the experiment, the variable-interval schedule correlated with the white key was introduced immediately after the multiple-schedule condition, and the test condition began 15 days later. This sequence was repeated twice, with a reversal of the correlation of the key colors with the components of the multiple schedule at the start of each new cycle. Part 2 added a 6-month break between the multiple-schedule history and the white-key variable-interval schedule followed by test sessions. The procedure was then repeated with a reversal of the correlation between key colors and multiple-schedule components. In the test sessions of Part 1, all pigeons consistently responded faster in the presence of the key color most recently correlated with the differential-reinforcement-of-high-rate contingency than during the color most recently correlated with the differential-reinforcement-of-low-rate contingency. Similar but smaller effects were observed in Part 2. The effects of the reversals in these two parts of the experiment showed that only the most recent contingency exerted an influence on subsequent responding. The data suggest that this effect of the most recent history continues to operate on behavior under current contingencies even after a long lapse of time.     

被试作答的偶然性对IRT能力估计的影响研究

被试能力参数估计是项目反应理论应用研究最重要的技术之一。本文在理想的测验情境下,研究被试作答的偶然性对被试能力值估计的影响。研究设计了被试作答的两种偶然性情况：一是偶然做对了一道项目难度高于其能力值的试题,二是偶然做错了一道或几道项目难度低于其能力值的试题．然后分别探讨了这两种情况下对被试的能力估计所带来的影响,并且就如何消除这些偶然性所带来的影响提出了相应的方法。     

Response selection modulates visual search within and across dimensions

In feature search tasks, uncertainty about the dimension on which targets differ from the nontargets hampers search performance relative to a situation in which this dimension is known in advance. Typically, these cross-dimensional costs are associated with less efficient guidance of attention to the target. In the present study, participants either had to perform a feature search task or had to perform a nonsearch task, that is, respond to a target presented without nontargets. The target varied either in one dimension or across dimensions. The results showed similar effects both in search and nonsearch conditions: Preknowledge of the target dimension gave shorter response times than when the dimension was unknown. Similar results were found using a trial-by-trial cueing. It is concluded that effects that typically have been attributed to early top-down modulation of attentional guidance may represent effects that occur later in processing.