Combinations of response-dependent and response-independent schedule-correlated stimulus presentation in an observing procedure

Pigeons pecked a response key on a variable-interval (VI) schedule, in which responses produced food every 40 s, on average. These VI periods, or components, alternated in irregular fashion with extinction components in which food was unavailable. Pecks on a second (observing) key briefly produced exteroceptive stimuli (houselight flashes) correlated with the component schedule currently in effect. Across conditions within a phase, the dependency between observing and presentation of the stimuli was decreased systematically while the density of stimulus presentation was held constant. Across phases, the proportion of session time spent in the VI component was adjusted from 0.5 to 0.25, and then to 0.75. Results indicate that rate of observing decreased as the dependency between responses and stimulus presentations was decreased. Further, discriminative control by the schedule-correlated stimuli was systematically weakened as dependency was decreased. Increasing the proportion of session time spent in VI decreased the rate of observing. This effect was additive with the manipulation of the dependency between observing and presentation of the stimuli. Overall, these results show that conditioned reinforcers function similarly to unconditioned reinforcers with respect to response-consequence dependencies, and that stimulus control is enhanced under conditions in which the relevant stimuli are produced by an organism's behavior.     

Maintenance of behavior controlling the duration of discriminative stimuli

The maintenance of a response controlling the duration of positive and negative discriminative stimuli in a multiple schedule was examined with respect to the possible consequences of the response: none, escape from the negative stimulus, production of the positive stimulus, and initiation of the reinforcement schedule associated with the latter. The last two seemed to be the major factors in producing and maintaining the response. Escape from the negative stimulus maintained it in most subjects, but only at a much lower level.     

Effects of varying stimulus disparity and the reinforcer ratio in concurrent-schedule and signal-detection procedures.

The present study measured the effects of stimulus and reinforcer variations on pigeons' behavior in two different choice procedures. Two intensities of white light were presented as the stimuli on the main key in a switching-key concurrent schedule and as the sample stimuli in a signal-detection procedure. Under both procedures, the scheduled rate of reinforcement was varied across conditions to produce various ratios of obtained reinforcement. These ratios were obtained for seven pairs of light intensities. In the concurrent schedules, the effects of reinforcer-ratio variations were positively correlated with the physical disparity between the two light intensities. In the signal-detection procedure, changes in the reinforcer ratio produced greater effects on performance when stimulus disparity was very low or very high compared to those found at intermediate levels of stimulus disparity. This discrepancy creates a dilemma for existing behavioral models of signal-detection performance.     

Effects of chlorpromazine on fixed-ratio responding: modification by fixed-interval discriminative stimuli.

Effects of chlorpromazine (1 to 100 mg/kg) were assessed on two pigeons' responding under various modifications of a multiple schedule of food delivery. During a fixed-interval component, the first response after 5 min produced food; during the subsequent, fixed-ratio component, the 30th response produced food. Modifications of the schedule entailed changes in stimulus conditions imposed during the fixed-ratio component that did not systematically alter characteristics of performance under non-drug conditions. In the first phase of the experiment, distinctive visual stimuli were correlated with each schedule component (conventional multiple schedule); chlorpromazine produced small decreases in fixed-ratio responding (20% at 30 mg/kg). When each response during the fixed-ratio component produced the stimulus correlated with the fixed-interval schedule (fixed-interval discriminative stimulus) for 1.2 s, effects of chlorpromazine were not different from those under the conventional multiple schedule. Chlorpromazine produced greater decreases in fixed-ratio responding (55% at 30 mg/kg) when either the first response of each fixed ratio changed the stimulus correlated with the fixed-ratio schedule to the fixed-interval discriminative stimulus for the remainder of the fixed-ratio component, or when the fixed-interval discriminative stimulus was presented independently of responding according to a matched temporal sequence. When the fixed-interval discriminative stimulus was present continuously during the fixed-ratio component (mixed schedule), chlorpromazine produced even more substantial decreases in fixed-ratio responding (greater than 80% at 30 mg/kg). Effects of chlorpromazine on fixed-interval responding were also modified by the schedules of fixed-interval discriminative stimulus presentation. The effects of chlorpromazine were a joint function of the stimuli prevailing during the multiple schedule and the degree to which responding influenced these stimuli.     

Separating the reinforcing and discriminative properties of brief-stimulus presentations in second-order schedules.

Pigeons' responses were maintained under multiple schedules to study properties of briefly presented stimuli. Responses in one component produced food according to a second-order schedule with fixed-interval components in which food or a brief stimulus occurred with equal probability. In the second component responses produced only the brief stimulus under a fixed-ratio schedule. Under various conditions the brief stimulus in the first component was (a) paired with food, (b) not paired with food, (c) partially omitted, or (d) scheduled simultaneously with the second-order schedule under an independent variable-interval schedule. Paired and nonpaired brief stimuli maintained similar response patterning in the second-order schedule. However, only paired stimuli maintained responses in the second component. The data suggest that nonpaired brief stimuli engender response patterning in second-order schedules as a result of their discriminative properties. When the stimulus is paired with food, these discriminative properties sometime mask a reinforcement effect, and no change in response patterning is observed. When the discriminative properties of the brief stimulus are absent, the reinforcing effects of pairing the brief stimulus with food may be observed.     

Choice between response units: The rate constancy model

In a conjoint schedule, reinforcement is available simultaneously on two or more schedules for the same response. The present experiments provided food for key pecking on both a random-interval and a differential-reinforcement-of-low-rate (DRL) schedule. Experiment 1 involved ordinary DRL schedules; Experiment 2 added an external stimulus to indicate when the required interresponse time had elapsed. In both experiments, the potential reinforcer frequency from each component was varied by means of a second-order fixed-ratio schedule, and the DRL time parameter was changed as well. Response rates were described by a model stating that time allocation to each component matches the relative frequency of reinforcement for that component. When spending time in a given component, the subject is assumed to respond at the rate characteristic of baseline performance. This model appeared preferable to the absolute-rate version of the matching law. The model was shown to be applicable to multiple-response concurrent schedules as well as to conjoint schedules, and it described some of the necessary conditions for response matching, undermatching, and bias. In addition, the pigeons did not optimize reinforcer frequency.     

The acquisition of observing.

Pigeons were exposed to stimuli correlated with the presence or absence of a variable-interval 60-second schedule of reinforcement only while they depressed a crossbar or "perch." In the first experiment, the stimuli were different tilts of a line displayed on the key. When the difference in brightness between the line and the background (salience) was maximal, seven of eight birds acquired the discrimination, but when the difference was reduced by 50%, only one succeeded. In the second experiment, wavelength of chamber illumination served as the relevant dimension. Neither experiment showed a large effect attributable to the magnitude of the difference (disparity) between the positive and the negative stimulus. Individual differences in time spent observing were positively correlated with level of discrimination in the presence of the stimuli. All birds produced the positive stimulus for a greater proportion of the available time than they did the negative stimulus. This may be the mechanism that provides selective reinforcement of observing. Finally, the formation of a discrimination was analyzed in terms of changes in the proportion of time spent in contact with the discriminative stimuli.     

Concurrent schedules: reinforcer magnitude effects

Five pigeons were trained on pairs of concurrent variable-interval schedules in a switching-key procedure. The arranged overall rate of reinforcement was constant in all conditions, and the reinforcer-magnitude ratios obtained from the two alternatives were varied over five levels. Each condition remained in effect for 65 sessions and the last 50 sessions of data from each condition were analyzed. At a molar level of analysis, preference was described well by a version of the generalized matching law, consistent with previous reports. More local analyses showed that recently obtained reinforcers had small measurable effects on current preference, with the most recently obtained reinforcer having a substantially larger effect. Larger reinforcers resulted in larger and longer preference pulses, and a small preference was maintained for the larger-magnitude alternative even after long inter-reinforcer intervals. These results are consistent with the notion that the variables controlling choice have both short- and long-term effects. Moreover, they suggest that control by reinforcer magnitude is exerted in a manner similar to control by reinforcer frequency. Lower sensitivities when reinforcer magnitude is varied are likely to be due to equal frequencies of different sized preference pulses, whereas higher sensitivities when reinforcer rates are varied might result from changes in the frequencies of different sized preference pulses.     

Human concurrent performances: The effects of experience, instructions, and schedule-correlated stimuli

In order to investigate the effects of separate exposures to single schedules, of unique schedule-correlated stimuli, and of independency-informing instructions upon choice under concurrent variable-interval schedules, 28 human subjects were divided into eight groups. Each subject was exposed to a baseline procedure, an experimental procedure, and a return to the baseline procedure. Different combinations of these three manipulations were applied to the different groups only during the experimental phase (except for the independency-informing instructions, which were given to half of the groups at the start of training). For the group of subjects exposed to the combination of all three manipulations, the logarithms of the ratios of response frequencies tended to be linearly related to the logarithms of the ratios of reinforcement frequencies during the experimental phase. These orderly effects were not obtained with subjects in the other groups. The results suggest that human choice was well described by the generalized matching law when the three manipulations were simultaneously in effect, and that unreported differences in the use of these three procedural variables might partly account for contradictions between results in previous studies of human concurrent performance.     

Duration and rate of reinforcement as determinants of concurrent responding

The duration and frequency of food presentation were varied in concurrent variable-interval variable-interval schedules of reinforcement. In the first experiment, in which pigeons were exposed to a succession of eight different schedules, neither relative duration nor relative frequency of reinforcement had as great an effect on response distribution as they have when they are manipulated separately. These results supported those previously reported by Todorov (1973) and Schneider (1973). In a second experiment, each of seven pigeons was exposed to only one concurrent schedule in which the frequency and/or duration of reinforcement differed on the two keys. Under these conditions, each pigeon's relative rate of response closely matched the relative total access to food that each schedule provided. This result suggests that previous failures to obtain matching may be due to factors such as an insufficient length of exposure to each schedule or to the pigeons' repeated exposure to different concurrent schedules.     

Response rate and changeover performance on concurrent variable-interval schedules

Six pigeons were exposed to variable-interval schedules arranged on one, two, three, and four response keys. The reinforcement rate was also varied across conditions. Numbers of responses, the time spent responding, the number of reinforcements, and the number of changeovers between keys were recorded. Response rates on each key were an increasing function of reinforcement rate on that key and a decreasing function of the reinforcement rate on other keys. Response and time-allocation ratios under-matched ratios of obtained reinforcements. Three sets of equations were developed to express changeover rate as a function of response rate, time allocation, and reinforcement rate respectively. These functions were then applied to a broad range of experiments in the literature in order to test their generality. Further expressions were developed to account for changeover rates reported in experiments where changeover delays were varied.     

Choice, time allocation, and response rate during stimulus generalization

Six pigeons were trained to discriminate between two noise intensities using a procedure that assessed choice, time allocation, and response rate simultaneously and independently. Responses on the left or right key (R1 or R2) were respectively correct in the presence of two different intensities, S1 and S2. After a correct response, reinforcement became available for pecks on the center key. Reinforcement density for R1¦S1 relative to R2¦S2 was varied across experimental conditions. Generalization tests followed extensive training at each condition. As a function of stimulus intensity, proportions of initial choices of R2, of time spent in R2-initiated components, and of center-key responses emitted in R2-initiated components all yielded sigmoidal gradients of similar slope, which shifted slightly in location when relative reinforcement density changed. Changeovers were maximal where initial choice proportions approximated 0.5. Gradients relating the absolute number of center-key responses to stimulus intensity were also roughly sigmoidal, but were more sensitive to changes in reinforcement density. Gradients of momentary response rate also depended on reinforcement density. During training, large but transitory shifts in choice responding occurred when reinforcement density changed, while differences in momentary response rate developed slowly, suggesting separate control of choice and response rate by the contingencies of reinforcement.     

Reinforcement magnitude and the inhibiting effect of reinforcement

In a two-key concurrent variable-interval schedule (using pigeons), if the reinforcement frequency for one response is held constant while that for the other is increased, the rate of response on the constant key decreases. The immediate reinforcement for key pecking can usually be conceptualized as the change from a condition in which the key light is on and the food hopper light is off to one in which the key light is off and the hopper light is on. The prechange condition is associated with a delay to food of one-half the average interreinforcement interval in effect during this condition. The postchange condition is associated with a delay to food of about .5 seconds. The programming of additional reinforcement results in a decrease in the delay to food associated with the prechange stimulus condition, and thus a decrease in the value of the improvement that results from the change. This would appear to be analogous to a decrease in the amount of reinforcement, and thus sufficient explanation for the decrease in the rate of the response.     

Revisiting the role of bad news in maintaining human observing behavior

Results from studies of observing responses have suggested that stimuli maintain observing owing to their special relationship to primary reinforcement (the conditioned-reinforcement hypothesis), and not because they predict the availability and nonavailability of reinforcement (the information hypothesis). The present article first reviews a study that challenges that conclusion and then reports a series of five brief experiments that provide further support for the conditioned-reinforcement view. In Experiments 1 through 3, participants preferred occasional good news (a stimulus correlated with reinforcement) or no news (a stimulus uncorrelated with reinforcement) to occasional bad news (a stimulus negatively correlated with reinforcement). In Experiment 4 bad news was preferred to no news when the absence of stimulus change following a response to the bad-news option was reliably associated with good news. When this association was weakened in Experiment 5 the results were intermediate. The results support the conclusion that information is reinforcing only when it is positive or useful. As required by the conditioned-reinforcement hypothesis, useless information does not maintain observing.     

The role of discriminative stimuli in concurrent performances

Key pecking in pigeons was examined under concurrent and parallel arrangements of two independent and simultaneously available variable-interval schedules. Pecks on the changeover key alternated the schedule of reinforcement for responses on the main key. Under concurrent schedules, discriminative stimuli were paired with the reinforcement schedule arranged in each component and changeover responses also alternated these stimuli. Under parallel schedules, changeover responses alternated the effective reinforcement schedule, but did not change the discriminative stimulus. On concurrent procedures, changeover response rate was inversely related to the difference in reinforcement rate between the two components, whereas on parallel schedules no consistent relationship was found. With both schedules, absolute response and reinforcement rates were positively related, although for a given set of reinforcement frequencies, rates were often higher on the concurrent schedules. On concurrent schedules, relative response rates and relative times were equal to relative reinforcement rates. On parallel schedules these ratios were positively related, but response and time ratios were much smaller than were obtained with comparable concurrent schedules. This inequality was most pronounced when absolute reinforcement frequencies were lowest.     

Stimulus effects on concurrent performance in transition

Six experimentally naive pigeons were exposed to concurrent variable-interval variable-interval schedules in a three-key procedure in which food reinforcement followed pecks on the side keys and pecks on the center key served as changeover responses. In Phase 1, 3 birds were exposed to 20 combinations of five variable-interval values, with each variable-interval value consistently associated with a different color on the side keys. Another 3 pigeons were exposed to the same 20 conditions, but with a more standard procedure that used a nondifferential discriminative stimulus on the two side keys throughout all conditions. In Phase 2, the differential and nondifferential stimulus conditions were reversed for each pigeon. Each condition lasted for one 5-hr session and one subsequent 1-hr session. In the last 14 conditions of each phase, the presence of differential discriminative stimuli decreased the time necessary for differential responding to develop and increased the sensitivity of behavior to reinforcement distribution in the 1st hr of training; during the last hours of training in each condition, however, the effects of the differential discriminative stimuli could not be distinguished from the effects of reinforcement distribution per se. These results show the importance of studying transitions in behavior as well as final performance. They may also be relevant to discrepancies in the results of previous experiments that have used nonhuman and human subjects.     

Conditioned reinforcement as a function of duration of stimulus

Pigeons were provided with three keys. Pecking the center key produced grain on a schedule that alternated at unpredictable times between a variable-interval component and extinction. On concurrent variable-interval schedules, pecking either side key produced a stimulus associated with the variable-interval component on the center key provided that said schedule was currently in effect. The independent variable was the length of time this stimulus remained on the keys. Pecking one side key produced the stimulus for 27 seconds, whereas the duration produced by pecking the other key varied for successive blocks of sessions. For the first four birds, the values tested were 3, 9, 27, and 81 seconds. For the second group, numbering three birds, the values tested were 1, 3, 9, and 27 seconds. The dependent variable was the proportion of total side key pecks that occurred on the variable key. For all birds, the function was positive in slope and negative in acceleration. This finding supports a formulation that ascribes the maintenance of observing responses in a normal setting to the fact that the subject exposes itself to the positive discriminative stimulus for a longer mean duration than it does to the negative stimulus.     

Alternative reinforcement effects on fixed-interval performance

Pigeons' key pecks were reinforced with food on a fixed-interval schedule. Food also was available at variable time periods either independently of responding or for not key pecking (a differential-reinforcement-of-other-behavior schedule). The latter condition arranged reinforcement following the first pause of t seconds after it became available according to a variable-time schedule. This schedule allowed separation of the effects of pause requirements ≤ five-seconds and reinforcement frequency. The time spent pausing increased as the duration of the pause required for reinforcement increased from 0 to 30 seconds and as the frequency of reinforcement for pausing increased from 0 to 2 reinforcers per minute. Key pecking was more evenly distributed within each fixed interval with shorter required pauses and with more frequent reinforcement for pausing. The results complement those obtained with other concurrent schedules in which the same operant response was reinforced in both components.     

Preference for fixed-interval schedules: effects of unequal initial links

Six homing pigeons were trained on concurrent chain schedules in which the terminal links were fixed-interval schedules of 5 sec or 15 sec. One initial-link schedule was always VI 27-sec; the other was varied over conditions from VI 27-sec to VI 181-sec. Preference measured in the initial links varied as a joint function of the initial- and terminal-link schedules. When the initial links were varied with constant, but unequal, terminal links, the slope of the function relating the logarithm of the initial-link response ratio to the logarithm of the terminal-link entry ratio differed from that obtained with equal terminal links. This result indicates that biases attributable to the terminal-link schedules were not constant. The rate of change of preference, or degree of undermatching, in the initial links depended on whether the shorter initial link led to the shorter or the longer terminal link. These results raise the question of whether bias and undermatching in concurrent schedule performance are independent measures.     

The effects of chlorpromazine and imipramine on rate and stimulus control of matching to sample.

Pigeons were trained to perform simultaneous, two-color matching to sample under a multiple fixed-ratio fixed-interval schedule of food presentation. The sequence terminating with a peck on the matching key (a "match") was treated as a unit, analogous to a single key peck in conventional schedules. Except for intermittent reinforcement of matches, no consequent stimulus distinguished matches from mismatches (sequences terminating with pecks on the nonmatching key). The pattern of matches during nondrug sessions resembled that of simpler operants maintained by similar schedules. Matches increased in rate toward the end of both components; mismatch rates increased more slowly. Imipramine increased the rate of mismatches, disrupted schedule patterning, and lowered accuracy in a dose-dependent fashion. Chlorpromazine lowered the overall rate of matches but affected schedule patterns and accuracy less than imipramine. The types of errors during drug sessions were not systematically related to the types of errors that appeared during nondrug sessions. Stimulus control was evaluated for each of the four possible color configurations and was found to be by the entire configuration of colors, not simply by the color of the sample.