Conditioned reinforcement value and resistance to change

Three experiments examined the effects of conditioned reinforcement value and primary reinforcement rate on resistance to change using a multiple schedule of observing-response procedures with pigeons. In the absence of observing responses in both components, unsignaled periods of variable-interval (VI) schedule food reinforcement alternated with extinction. Observing responses in both components intermittently produced 15 s of a stimulus associated with the VI schedule (i.e., S+). In the first experiment, a lower-valued conditioned reinforcer and a higher rate of primary reinforcement were arranged in one component by adding response-independent food deliveries uncorrelated with S+. In the second experiment, one component arranged a lower valued conditioned reinforcer but a higher rate of primary reinforcement by increasing the probability of VI schedule periods relative to extinction periods. In the third experiment, the two observing-response components provided similar rates of primary reinforcement but arranged different valued conditioned reinforcers. Across the three experiments, observing-response rates were typically higher in the component associated with the higher valued conditioned reinforcer. Resistance to change was not affected by conditioned reinforcement value, but was an orderly function of the rate of primary reinforcement obtained in the two components. One interpretation of these results is that S+ value does not affect response strength and that S+ deliveries increase response rates through a mechanism other than reinforcement. Alternatively, because resistance to change depends on the discriminative stimulus-reinforcer relation, the failure of S+ value to impact resistance to change could have resulted from a lack of transfer of S+ value to the broader discriminative context.     

Investigating Behavioral Dynamics With A Fixed-time Extinction Schedule And Linear Analysis

This paper describes the behavioral adaptation observed for 16 pigeons responding to a step transition in the reinforcement rate in a repeated-trial design. Within each trial, following exposure for a fixed period to a variable-interval schedule, there was an unsignaled change in the schedule to extinction. The step transition allowed an experimental test of the applicability of a linear analysis to steady-state dynamic behavior. The computations required for this test yielded, as an intermediate result, transfer functions for each of the 16 birds from 1 mHz to 256 mHz. The transfer functions obtained show greater responsiveness to lower frequencies (i.e., longer time-scale structures in the reinforcement schedule); hence, the pigeons have the characteristics of a low-pass filter. The outcome of the test is that some predictability of the pigeons' future behavior is possible.     

Aversive aspects of a schedule of positive reinforcement

Six male White Carneaux pigeons were trained to peck at one of two keys to obtain food on several fixed-ratio schedules of reinforcement. Concurrently, the first response on a second key could, I—change the conditions of visual stimulation and remove the food reinforcement contingency, II—change the conditions of stimulation and have no effect upon the reinforcement contingency, or III—do nothing. The second response on the stimulus change key always restored baseline conditions. When second-key responses produced a stimulus change, the number of such responses was a function of the ratio value on the first key. Typically, second-key responses occurred before the start of fixed-ratio runs. The duration of stimulus change periods was an exponential function of the number of responses required for reinforcement when the possibility for reinforcement was not disturbed by periods of stimulus change (Condition II).     

Determination of a behavioral transfer function: White-noise analysis of session-to-session response-ratio dynamics on concurrent VI VI schedules

Six pigeons were exposed to concurrent variable-interval schedules in which the programmed reinforcer ratios changed from session to session according to a pseudorandom binary sequence. This procedure corresponded to the stochastic identification paradigm (“white-noise experiment”) of systems theory and enabled the relation between log response ratios in the current session and log reinforcer ratios in all previous sessions to be determined. Such dynamic relations are called linear transfer functions. Both nonparametric and parametric representations of these, in the form of “impulse-response functions,” were determined for each bird. The session-to-session response ratios resulting from the session-to-session pseudorandom binary variations in reinforcer ratios were well predicted by the impulse-response functions identified for each pigeon. The impulse-response functions were well fitted by a second-order dynamic model involving only two parameters: a time constant and a gain. The mean time constant was 0.67 sessions, implying that the effects of abrupt changes in log reinforcer ratios should be 96% complete within about five sessions. The mean gain was 0.53, which was surprisingly low inasmuch as it should equal the sensitivity to reinforcement ratio observed under steady-state conditions. The same six pigeons were subjected to a similar experiment 10 months following the first. Despite individual differences in impulse-response functions between birds within each experiment, the impulse-response functions determined from the two experiments were essentially the same.     

Extinction-induced aggression

Pigeons were conditioned to peck a response key under a procedure that alternated periods of food reinforcement with periods of extinction. The pigeons attacked a nearby pigeon at the onset of extinction. Some also attacked a stuffed model of a pigeon. The duration of attack was an inverse function of the time since the last food reinforcement and a direct function of the number of reinforcements. The pigeons attacked after the last food delivery whether or not the conditioned pecking response was required and whether or not the extinction period was signaled. The food had to be eaten; the mere sight and sound of food being delivered did not produce attack. Prior satiation reduced attack. The phenomenon was not attributable to a past history of competition between pigeons since socially deprived pigeons also attacked. Superstitious reinforcement of attack was not found to be a factor. The results indicated that the transition from food reinforcement to extinction was an aversive event that produced aggression.     

Assessment and treatment of low-rate high-intensity problem behavior.

Functional analysis of low‐rate aggression was conducted during extended observation periods and showed behavior to be maintained by positive reinforcement. Treatment consisted of variable‐momentary differential reinforcement of other behavior and was successful in reducing problem behavior throughout these extended observation periods.     

Short-term and long-term effects of reinforcers on choice

The relation between molar and molecular aspects of time allocation was studied in pigeons on concurrent variable-time variable-time schedules of reinforcement. Fifteen-minute reinforcer-free periods were inserted in the middle of every third session. Generalized molar matching of time ratios to reinforcer ratios was observed during concurrent reinforcement. Contrary to melioration theory, preference was unchanged during the reinforcer-free periods as well as in extinction. In addition to this long-term effect of reinforcement, short-term effects were observed: Reinforcers increased the duration of the stays during which they were delivered but had little consistent effect either on the immediately following stay in the same schedule or on the immediately following stay in the alternative schedule. Thus, an orderly effect of reinforcer delivery on molecular aspects of time allocation was observed, but because of its short-term nature, this effect cannot account for the matching observed at the molar level.     

Fixed-ratio punishment by timeout of concurrent variable-interval behavior

Pigeons' responding was maintained by two concurrently available variable-interval reinforcement schedules. A fixed-ratio punishment schedule of timeout periods from the concurrent reinforcement schedules was arranged for responding during one of the variable-interval schedules. The greater the probability of a timeout after a response on the punished variable-interval schedule (the smaller the fixed ratio that produced timeout), the greater the decline in the relative punished response rates. Relative reinforcement rates remained invariant when relative response rates declined. Both behavioral contrast and induction effects were observed on the unpunished variable-interval schedule as a function of timeout punishment of the other schedule.     

EFFECTS AND SIDE EFFECTS OF DRO AS TREATMENT FOR SELF-INJURIOUS BEHAVIOR

A three-part controlled case study is presented in which severe and longstanding self-injurious behavior exhibited by a 9-year-old-boy was treated successfully with differential reinforcement of other behavior. In Phase 1, an experimental analysis demonstrated that the boy's scratching was not maintained by environmental contingencies; instead, it appeared that the self-injurious behavior was a stereotypic (automatically reinforced) response. In Phase 2, the effects of an escalating differential-reinforcement-of-other-behavior schedule mediated through token reinforcement (pennies) were evaluated in a reversal design. Results showed that differential-reinforcement-of-other-behavior eliminated self-injurious behavior very quickly and for periods of time as long as 30 min. A noteworthy side effect observed during Phase 2 was the occurrence of crying behavior following the nondelivery of reinforcement. In Phase 3, the token program was gradually extended in 30-min increments throughout the day. Additionally, results of a brief multielement manipulation showed that the effects of token reinforcement were superior to those of a more easily administered differential reinforcement of other behavior based on social reinforcement, which differed little from baseline.     

Psychophysics of key-peck duration in the pigeon

The duration of the pigeon's key peck was differentially reinforced in either a trials or a free-operant procedure. Mean emitted peck duration was a power function of the duration required for food delivery to occur. The exponents of the power function differed considerably from those observed in earlier research involving longer duration responses in pigeons and other species. The coefficients of variation also did not correspond with those of the earlier research on other responses, nor did consideration of the durations actually reinforced resolve the differences. Duration was neither a function of response rate nor of intermittency of reinforcement. Key-peck duration was changed in an orderly way by differential reinforcement. However, it appeared to be more strongly determined by its duration in the absence of differential reinforcement than were longer duration responses.     

A component analysis of "stereotypy as reinforcement" for alternative behavior

Results from several studies have suggested that the opportunity to engage in stereotypic behavior may function as reinforcement for alternative, more socially desirable behaviors. However, the procedural components of this intervention include several distinct operations whose effects have not been analyzed separately. While measuring the occurrence of stereotypy and an alternative behavior (manipulation of leisure materials), we exposed 3 participants to three or four components of a "stereotypy as reinforcement" contingency: (a) continuous access to materials, (b) prompts to manipulate materials, (c) restricted access to stereotypy (i.e., response blocking), and (d) access to stereotypy contingent on manipulating the materials. Continuous access to materials and prompting (a and b) produced negligible results. Restriction of stereotypy (c) produced a large increase in the alternative behavior of 2 participants, suggesting that response restriction per se may occasion alternative behavior. However, contingent access to stereotypy (d) was necessary to increase the 3rd participant's object manipulation; this finding provided some support for the use of stereotypy as reinforcement for alternative behavior. Finally, when transfer of the effects of intervention was assessed during periods in which active intervention components were withdrawn, the alternative behavior was maintained for 1 participant.     

Response strength in extreme multiple schedules

Four pigeons were trained in a series of two-component multiple schedules. Reinforcers were scheduled with random-interval schedules. The ratio of arranged reinforcer rates in the two components was varied over 4 log units, a much wider range than previously studied. When performance appeared stable, prefeeding tests were conducted to assess resistance to change. Contrary to the generalized matching law, logarithms of response ratios in the two components were not a linear function of log reinforcer ratios, implying a failure of parameter invariance. Over a 2 log unit range, the function appeared linear and indicated undermatching, but in conditions with more extreme reinforcer ratios, approximate matching was observed. A model suggested by McLean (1991), originally for local contrast, predicts these changes in sensitivity to reinforcer ratios somewhat better than models by Herrnstein (1970) and by Williams and Wixted (1986). Prefeeding tests of resistance to change were conducted at each reinforcer ratio, and relative resistance to change was also a nonlinear function of log reinforcer ratios, again contrary to conclusions from previous work. Instead, the function suggests that resistance to change in a component may be determined partly by the rate of reinforcement and partly by the ratio of reinforcers to responses.     

Human, free-operant avoidance of “time out” from monetary reinforcement

To assess the aversive effects of withdrawing monetary reinforcement, human subjects were exposed to a free-operant avoidance procedure in which periods of no reinforcement occurred if the subject failed to respond, and each response postponed withdrawal of reinforcement. Avoidance behavior was developed either through specific instructions about the consequence of responding or through preliminary escape-avoidance training. In all cases, rates of response were found to be a positively accelerated function of decreases in the duration by which responding postponed reinforcement withdrawal. The findings with respect to the function relating avoidance behavior to the interval of postponement were viewed as similar to those obtained when shock is used as the aversive event in free-operant avoidance conditioning.     

ARTIFACTUAL EFFECTS OF SENSORY-INTEGRATIVE THERAPY ON SELF-INJURIOUS BEHAVIOR

Three individuals who exhibited self-injurious behavior (SIB) were exposed to sensory-integrative therapy. Prior to treatment, a functional analysis baseline was conducted to identify the motivational features of their SIB. One subject's SIB appeared to be an attention-getting response (maintained by positive reinforcement), which varied subsequently as a function of attention being either withheld or provided noncontingently during sensory-integration sessions. The 2nd subject displayed a pattern of responding suggestive of stereotypic SIB (maintained by automatic reinforcement), which paradoxically increased during sensory-integration sessions. The 3rd subject's SIB appeared to function as an escape response (maintained by negative reinforcement), and his behavior during sensory-integration sessions was similar to that observed during baseline sessions in which demands were not present. The SIB of all 3 subjects later was reduced when behavioral interventions were applied. The data presented raise questions about the active components of sensory-integrative therapy and the functional types of SIB for which it might be appropriate.     

The transfer of specific and general consequential functions through simple and conditional equivalence relations

The purpose of this study was to examine the transfer of consequential (reinforcement and punishment) functions through equivalence relations. In Experiment 1, 9 subjects acquired three three-member equivalence classes through matching-to-sample training using arbitrary visual forms. Comparison stimuli were then given conditioned reinforcement or punishment functions by pairing them with verbal feedback during a sorting task. For 8 of the 9 subjects, trained consequential functions transferred through their respective equivalence classes without additional training. In Experiment 2, transfer of function was initially tested before equivalence testing per se. Three of 4 subjects showed the transfer without a formal equivalence test. In Experiment 3, 3 subjects were given training that gave rise to six new three-member conditional equivalence classes. For 2 of the subjects, the same stimulus could have either a reinforcement or punishment function on the basis of contextual cues that defined its class membership. Experiment 4 assessed whether equivalence training had established general or specific consequential functions primarily by adding novel stimuli in the transfer test. Subjects treated even novel feedback stimuli in the transfer test as consequences, but the direction of consequential effects depended upon the transfer of specific consequential functions through equivalence relations.     

Temporal discrimination and a free-operant psychophysical procedure.

Pigeons were presented a series of keylight time periods (separated by blackouts) during which two response keys were lit, one by blue light and the other either by orange or green. Blue-key responses changed the color on the other key. Orange-key responses sometimes produced food during the first half of a time period; green-key responses sometimes produced food during the second half. In three experiments, the probability of a green-key response increased as a function of elapsed time. Experiment 1 compared performance when the duration of the keylight periods was varied across a wide range. Discrimination of performance was similar across the range of durations. Experiment 2 varied both relative reinforcement rate and the local reinforcement rate for orange-key and green-key responses. These manipulations produced changes in response bias but not discrimination sensitivity. Experiment 3 varied the local temporal placement of reinforcers within time periods and demonstrated that choice behavior was affected by differential reinforcement at different points during the time periods. The results were consistent with previous research on duration discrimination that used psychophysical trials procedures.     

Repeated measurements of reinforcement effects on gradients of stimulus control

Two experiments studied the effects of reinforcement schedules on generalization gradients. In Exp. 1, after pigeons' responding to a vertical line was reinforced, the pigeons were tested with 10 lines differing in orientation. Reconditioning and the redetermination of generalization gradients were repeated from 8 to 11 times with the schedule of reinforcement varied in the reconditioning phase. Stable gradients could not be observed because the successive reconditionings and tests steepened the gradients and reduced responding. Experiment 2 over-came these effects by first training the birds to respond to all of the stimuli. Then, brief periods of reinforced responding to the stimulus correlated with reinforcement alternated with the presentation of the 10 lines in extinction. The development of stimulus control was studied eight times with each bird, twice with each of four schedules of reinforcement. Gradients were similar each time a schedule was imposed; the degree of control by the stimulus correlated with reinforcement varied with particular schedules. Behavioral contrast occurred when periods of reinforcement and extinction alternated and was more durable with fixed-interval, variable-interval, and variable-ratio schedules than with fixed-ratio or differential-reinforcement-of-low-rate schedules.     

Response rate and sensitivity to the molar feedback function relating response and reinforcement rate on VI+ schedules of reinforcement

In 5 experiments, the author examined rats' sensitivity to the molar feedback function relating response rate to reinforcement rate on schedules of reinforcement. These studies demonstrated that, at lower rates of responding, rats' performance on variable ratio (VR), variable interval (VI), and variable interval with linear feedback loop (VI+) schedules was determined largely by reinforcement of interresponse times; response rates were faster on VR than on both VI and VI+ schedules. In contrast, when procedures were adopted to maintain high rates of response, rats showed sensitivity to the molar characteristics of the schedules; they responded as fast on a VI+ schedule as on a VR schedule and faster on both of these schedules than on a yoked VI schedule. When the variance of response rate was manipulated, this factor was noted as an important element in determining sensitivity to the molar characteristics of the schedule.