Differential reinforcement of interresponse times with controlled probability of reinforcement per response

Pigeons were presented food after interresponse times (IRTs) longer or shorter than a fixed percentage of their most recent IRTs. This procedure controlled probability of reinforcement per response while still allowing different classes of IRTs to be reinforced differentially. Support was found for IRT-reinforcement theory in that response rates were determined by the degree and direction of differential reinforcement of IRTs, but were relatively independent of probability of reinforcement per response and of the length of the control system's IRT memory. Stimulus control of these differential response rates was also demonstrated.     

Timeout from positive reinforcement following persistent, high-rate behavior in retardates

Brief isolation from a group situation was found to suppress persistent, high-rate misbehavior in two extremely withdrawn children, even though no positive reinforcement for other behaviors was systematically administered. Changes in a variety of behaviors, including looking, touching, speaking, responding, and other non-punished misbehaviors, were observed when isolation timeout was administered contingent on only one misbehavior of each child.     

Differential reinforcement and signal detection.

Reinforcement was introduced for responses normally treated as errors in signal-detection procedures. The first experiment used a standard two-response discrete-trial procedure with no reinforcement for errors. Results showed that rats altered their response biases but maintained constant sensitivity to visual signals when reinforcement probabilities varied, and that their sensitivity depended on the physical difference between signals, in accordance with the predictions of signal-detection theory. Experiment II, with rats, and Experiment III, with pigeons, demonstrated that sensitivity decreased in this procedure when reinforcement was scheduled for errors with the signals held constant, despite independence of overall number of reinforcers and sensitivity. Experiment IV, with rats, replicated the decrease in sensitivity in a continuous procedure employing only one response. The decrements in sensitivity were similar across Experiments II, III, and IV, and accorded well with earlier research. Thus, contrary to a fundamental assumption of signal-detection theory, estimates of sensitivity are not always invariant with respect to the outcomes of responding, but depend on relative reinforcement of correct responses.     

The reinforcement of least-frequent interresponse times

A new schedule of reinforcement was used to maintain key-pecking by pigeons. The schedule reinforced only pecks terminating interresponse times which occurred least often relative to the exponential distribution of interresponse times to be expected from an ideal random generator. Two schedule parameters were varied: (1) the rate constant of the controlling exponential distribution and (2) the probability that a response would be reinforced, given that it met the interresponse-time contingency. Response rate changed quickly and markedly with changes in the rate constant; it changed only slightly with a fourfold change in the reinforcement probability. The schedule produced stable rates and high intra- and inter-subject reliability, yet interresponse time distributions were approximately exponential. Such local interresponse time variability in the context of good overall control suggests that the schedule may be used to generate stable, predictable, yet sensitive baseline rates. Implications for the measurement of rate are discussed.     

The reinforcement of short interresponse times

Five contingencies were superimposed successively on a variable-interval schedule of reinforcement. In each of the resulting conditions, a different short, interresponse time was reinforced and an interresponse-time distribution was obtained from each of three pigeons. The lower bound of the reinforced interresponse times ranged from 0.3 to 2.4 sec. The resulting distributions were combined, according to a rationale based upon concurrent operants, induction, and a property of variable-interval schedules, to describe the interresponse-time distributions from a variable-interval schedule.     

The concurrent reinforcement of two interresponse times: absolute rate of reinforcement

Three pigeons obtained food on a one-key schedule of reinforcement for two concurrent, discriminated interresponse times. The overall rate of reinforcement was determined by a family of variable-interval schedules and by a continuous reinforcement schedule. The average frequency of reinforcement varied from 1.1 to 300 reinforcements per hour; the relative frequency of reinforcement for each of the two interresponse times was 0.5 throughout the experiment. The number of responses per minute increased sharply as the number of reinforcements per hour increased from 1 to 20. Beyond 30 reinforcements per hour, the curve was approximately flat, although it sometimes decreased slightly at the highest reinforcement rates. The relative frequency of the shorter interresponse time also increased sharply as the number of reinforcements per hour increased from 1 to 20. The asymptote of the relative frequency function approximately equalled the relative reciprocal of the length of the shorter interresponse time for reinforcement rates greater than 30 or 40 reinforcements per hour. This approximation was obscured by the response-rate function.     

Differential reinforcement of vocal duration

The effects of differential reinforcement of vocal duration were examined in a series of experiments in which each of 28 subjects (Ss) emitted a vowel whenever a light was flashed. In the first phase of each experiment, a penny was dispensed after each of 20 responses. In the second and subsequent phases, only those responses whose durations exceeded a criterion were reinforced; when 10 successive reinforcements were presented, one phase was terminated and the next begun. The criterion for reinforcement in each phase was determined by a different schedule in each of six experiments; it ranged from 80 to 120 per cent of the mean duration of the 10 terminal responses in the prior phase. Differential reinforcement effected a large and systematic change in the duration of vocal responses as long as the responses selected for reinforcement had a sufficiently high probability of occurrence. This requirement was formulated as the difference between the criterion duration and the mean duration of the terminal responses in the prior phase, divided by their standard deviation. This statistic, named the shaping index, was correlated with the number of responses emitted before each phase was terminated. It was found to be large whenever the shaping process failed. Many Ss failed to tact the reinforcement contingency despite marked changes in their vocal behavior and extensive probing by a questionnaire, administered at the end of each session.     

Manual reaction times and error rates in stutterers.

In this study 18 stutterers and 18 nonstutterers were presented trials on which they should press a button as fast as possible, intermixed with trials which required no responding. Stutterers had slightly faster reaction times but also made slightly more errors, that is, they tended to press the button when they should not have done so. As neither difference was significant, it was concluded that stutterers did not differ from normal speakers in manual reaction speed, nor did they choose a different speed-accuracy trade-off criterion for the given task.     

Contrast as seen in visual search reaction times.

Three pigeons searched arrays of alphabetic letters displayed on computer monitors. On each trial, either an A or an E appeared, and the reaction time and accuracy with which the bird pecked at this target were measured. In each block of trials, each target (A or E) was displayed alone, or together with a number of distractor letters (2 or 18) that varied in their similarity to the target. During a baseline series of sessions, responses to the A and to the E each yielded food reinforcement on 10% of the trials. In the next series of sessions, reinforcement continued at 10% for A, but rose to 30% for E. In a final series, these reinforcement conditions were reversed. As expected, reaction times increased with target-distractor similarity and (for similar distractors) with the number of distractors. Increased reinforcement of E had no effect on reaction times to E, but produced a very consistent increase in reaction times to A; the average increase was constant across the various display conditions. Reversal of the differential reinforcement conditions reversed this contrast effect. Analysis of the reaction time distributions indicated that increased reinforcement to E decreased the momentary probability of response to A by a constant amount, regardless of display conditions. These results are discussed in relation to theories of contrast, memory, and of the search image.     

Response rates track the history of reinforcement times

When conditioning involves a consistent temporal relationship between the conditioned stimulus (CS) and unconditioned stimulus (US), the expression of conditioned responses within a trial peaks at the usual time of the US relative to the CS. Here we examine the temporal profile of responses during conditioning with variable CS-US intervals. We conditioned stimuli with either uniformly distributed or exponentially distributed random CS-US intervals. In the former case, the frequency of each CS-US interval within a specified range is uniform but the momentary probability of the US (the hazard function) increases as time elapses during the trial; with the latter distribution, short CS-US intervals are more frequent than longer intervals, but the momentary probability of the US is constant across time within the trial. We report that, in a magazine approach paradigm, rats' response rates remained stable as time elapses during the CS when the CS-US intervals were uniformly distributed, whereas their response rates declined when the CS-US intervals were exponentially distributed. In other words, the profile of responding during the CS matched the frequency distribution of the US times, not the momentary probability of the US during the CS. These results are inconsistent with real-time associative models, which predict that associative strength tracks the momentary probability of the US, but may provide support for timing models of conditioning in which conditioned responding is tied to remembered times of reinforcement.     

Development of verbal control over bizarre gestures of retardates through imitative and nonimitative reinforcement procedures

Two retardates, manifesting hand gestures and minimal instructional control, were trained by imitative reinforcement procedures to imitate a response that was in contrast to gesturing. Next, with the contrast response continuing to be imitatively reinforced, gesturing was reduced by nonimitative reinforcement procedures; while providing facial and gesture cues, the adult said, "Do not do this". Imitative and nonimitative procedures were found to have the same effects on the contrast response as on the gesturing response, such that imitative procedures increased both responses, whereas nonimitative procedures decreased both. Nonperformance of gesturing was further maintained when (1) explicit verbal directions for nongesturing were superimposed upon the demonstrational-facial-verbal cues as these collective stimuli were faded out and (2) food reinforcers for nongesturing were gradually removed while social consequences continued to be administered.     

The reinforcement of four interresponse times in a two-alternative situation

Pigeons pecked for food in a two-key procedure. A concurrent variable-interval variable-interval schedule of reinforcement for two classes of interresponse times was arranged on each key. A visual stimulus set the occasion for potential reinforcement of the four operant classes: shorter and longer interresponse times on left and right keys. In Exp. I, the relative frequency of respones on a key equalled the relative frequency of reinforcement on that key. In Exp. II, the relative frequency of an interresponse time equalled the relative reciprocal of its length. In Exp. III, the relative frequency of an interresponse time was a monotonically increasing function of its relative frequency of reinforcement. These functions relating the relative frequency of an interresponse time to its relative length and to its relative frequency of reinforcement were the same as if there had been no second key. Also, the distribution of responses between keys was independent of the relative frequency of an interresponse time on either key. Experiment IV replicated Exp. I except that choices between keys were controlled by a stimulus that signalled the availability of reinforcement on the right key. A comparison of Exp. I and IV suggested that the relative frequency of an interresponse time on one key generally was independent of behavior on the other key, but that the number of responses per minute on a key did depend on behavior on the other key.     

Warned reaction times of sociopaths

A warned reaction time (RT) task was employed with eight male sociopaths and eight normal male Ss. A warning light appeared at a variable interval preceding a light to which the S responded with a key press. All Ss received both a regular and an irregular series with warning intervals of 1, 2, 4, 8, and 16 sec. In the regular series, blocks of intervals were presented in an ascending order. In the irregular series each interval followed every other interval equally often. Sociopaths obtained reliably slower RTs than did control Ss, showing a constant decrement across all intervals in the two kinds of series.     

Macroscopic thermodynamics of reaction times

I present a new interpretation of reaction time (RT) data from behavioural experiments. From a physical perspective, the entropy of the RT distribution–the temporal entropy–provides a model-free estimate of the amount of processing performed by the cognitive system. This new measure shifts the focus from the conventional interpretation of RTs being either long or short, into their distribution being more or less complex in terms of entropy. I introduce the formulation of the theory, followed by an empirical test using a large database of human RTs in lexical processing tasks. Using the measure, I obtain estimates of the processing loads to individual stimuli (i.e., words), as well as estimates for the overall rate at which the system processes information in these tasks. The relation between the temporal entropy and the RTs can be captured by a simple linear equation. I argue that this equation constitutes the equivalent of a ‘phase diagram’ of a task, providing indications about the different mechanisms that are at play in it, and locating critical points signalling the transitions between these different mechanisms. The results suggest an adaptive system that adjusts its operational processing speed to the demands of each individual stimulus. This finding is in contradiction with a generalization of Hick’s Law positing a relatively constant processing speed within an experimental context.