A discriminated rapid-acquisition laboratory procedure for human continuous choice

Previous continuous choice laboratory procedures for human participants are either prohibitively time-intensive or result in inadequate fits of the generalized matching law (GML). We developed a rapid-acquisition laboratory procedure (Procedure for Rapidly Establishing Steady-State Behavior, or PRESS-B) for studying human continuous choice that reduces participant burden and produces data that is well-described by the GML. To test the procedure, 27 human participants were exposed to 9 independent concurrent random-interval random-interval reinforcement schedules over the course of a single, 37-min session. Fits of the GML to the participants' data accounted for large proportions of variance (median R²: 0.94), with parameter estimates that were similar to those previously found in human continuous choice studies [median a: 0.67; median log(b): -0.02]. In summary, PRESS-B generates human continuous choice behavior in the laboratory that conforms to the GML with limited experimental duration.     

Within-subject testing of the signaled-reinforcement effect on operant responding as measured by response rate and resistance to change

Response rates under random-interval schedules are lower when a brief (500 ms) signal accompanies reinforcement than when there is no signal. The present study examined this signaled-reinforcement effect and its relation to resistance to change. In Experiment 1, rats responded on a multiple random-interval 60-s random-interval 60-s schedule, with signaled reinforcement in only one component. Response resistance to alternative reinforcement, prefeeding, and extinction was compared between these components. Lower response rates, and greater resistance to change, occurred in the component with the reinforcement signal. In Experiment 2, response rates and resistance to change were compared after training on a multiple random-interval 60-s random-interval 60-s schedule in which reinforcer delivery was unsignaled in one component and a response-produced uncorrelated stimulus was presented in the other component. Higher response rates and greater resistance to change occurred with the uncorrelated stimulus. These results highlight the significance of considering the effects of an uncorrelated signal when used as a control condition, and challenge accounts of resistance to change that depend solely on reinforcer rate.     

Effect of schizotypy on responding maintained by free-operant schedules of reinforcement

Deviations from otherwise reliable patterns of responding in human performance on schedules of reinforcement could be associated with personality characteristics, such as psychometrically-measured schizotypy. The present study examined differences between high and low scorers on four schizotypy subscales (unusual experiences, cognitive disorganisation, introvertive anhedonia and impulsive non-conformity) on a random-ratio (RR) and random-interval (RI) schedule of reinforcement. Results showed a higher rate of responding on the RR than RI schedule, consistent with the differences in performance normally observed between these schedules. However, these differences were dependant on whether or not participants scored high or low on the schizotypy subscales, as well as the specific schedule parameters between the schedules. Specifically, high scorers on the unusual experiences (UE) subscale of the O-LIFE(B) were the only group in Experiment 1 not to show a difference in schedule performance, suggesting high scores on this subscale are linked to altered RR and RI performance. Experiment 2 explored this finding further using different schedule parameters, and confirmed that high scorers in UE did not differ in response rates between the two schedules.     

Choice between response units: The rate constancy model

In a conjoint schedule, reinforcement is available simultaneously on two or more schedules for the same response. The present experiments provided food for key pecking on both a random-interval and a differential-reinforcement-of-low-rate (DRL) schedule. Experiment 1 involved ordinary DRL schedules; Experiment 2 added an external stimulus to indicate when the required interresponse time had elapsed. In both experiments, the potential reinforcer frequency from each component was varied by means of a second-order fixed-ratio schedule, and the DRL time parameter was changed as well. Response rates were described by a model stating that time allocation to each component matches the relative frequency of reinforcement for that component. When spending time in a given component, the subject is assumed to respond at the rate characteristic of baseline performance. This model appeared preferable to the absolute-rate version of the matching law. The model was shown to be applicable to multiple-response concurrent schedules as well as to conjoint schedules, and it described some of the necessary conditions for response matching, undermatching, and bias. In addition, the pigeons did not optimize reinforcer frequency.     

Relationship between contingency awareness and human performance on random ratio and random interval schedules

In three experiments, human participants pressed the space bar on a computer keyboard to earn points on random-ratio (RR) and random-interval (RI) schedules of reinforcement. Verbalized contingency awareness (CA) for each schedule was measured after the entire task (Experiments 1 and 2), or after each RR–RI trial (Experiment 3). In all three experiments, an ability to accurately verbalize the appropriate contingency for each schedule was associated with typical schedule responding. In addition, lagged correlational analysis of verbalized contingency awareness and schedule performance was undertaken in Experiment 3. Results of these correlations initially suggested that CA scores were better predictors of typical schedule performance on the following trial than typical schedule performance was of CA scores on the following trial; however, further analysis revealed no significant differences between the alternate lagged correlation types in each pair. Findings, therefore, suggest that operant performance on RR and RI schedules is associated with participant-derived rule governed behavior, and results are discussed in terms of the introduction of a new paradigm for measuring this association, and the potential for further research to manipulate these parameters further in order to investigate causality in this respect.     

Concurrent random-interval schedules and the matching law

In Experiment I, a group of eight pigeons performed on concurrent random-interval schedules constructed by holding probability equal and varying cycle time to produce ratios of reinforcer densities of 1:1, 3:1, and 5:1 for key pecking. Schedules for a second group of seven were constructed with equal cycle times and unequal probabilities. Both groups deviated from simple matching, but the two forms of the schedules appeared to produce no consistent patterns of deviation. The data were found to be consistent with those obtained in concurrent variable-interval situations. The parameters of the matching equation in the form of Y=k X^a were estimated; the value of k was unity and a was 0.84. In Experiment II, six pigeons were exposed to two conc RI RI schedules in which one component increasingly approximated an FI schedule. The value of k was not 1.0. Concurrent RI RI schedules were shown to represent a continuum from conc FI VI to conc VI VI schedules. The use of the exponential equation in testing “matching laws” suggests that a<1 will continue to be observed, and this will set limits on the form of new laws and the assumed or rational values of the component variables in these laws.     

Variable-ratio versus variable-interval schedules: response rate, resistance to change, and preference

Two experiments asked whether resistance to change depended on variable-ratio as opposed to variable-interval contingencies of reinforcement and the different response rates they establish. In Experiment 1, pigeons were trained on multiple random-ratio random-interval schedules with equated reinforcer rates. Baseline response rates were disrupted by intercomponent food, extinction, and prefeeding. Resistance to change relative to baseline was greater in the interval component, and the difference was correlated with the extent to which baseline response rates were higher in the ratio component. In Experiment 2, pigeons were trained on multiple variable-ratio variable-interval schedules in one half of each session and on concurrent chains in the other half in which the terminal links corresponded to the multiple-schedule components. The schedules were varied over six conditions, including two with equated reinforcer rates. In concurrent chains, preference strongly overmatched the ratio of obtained reinforcer rates. In multiple schedules, relative resistance to response-independent food during intercomponent intervals, extinction, and intercomponent food plus extinction depended on the ratio of obtained reinforcer rates but was less sensitive than was preference. When reinforcer rates were similar, both preference and relative resistance were greater for the variable-interval schedule, and the differences were correlated with the extent to which baseline response rates were higher on the variable-ratio schedule, confirming the results of Experiment 1. These results demonstrate that resistance to change and preference depend in part on response rate as well as obtained reinforcer rate, and challenge the independence of resistance to change and preference with respect to response rate proposed by behavioral momentum theory.     

Schedules using noxious stimuli. I. Multiple fixed-ratio and fixed-interval termination of schedule complexes

The termination of a schedule complex, comprising a stimulus in the presence of which brief presentations of electric shocks are scheduled, is a reinforcer. Conditions were studied under which schedule-controlled patterns of responding characteristic of fixed-interval, fixed-ratio, and multiple fixed-interval fixed-ratio schedules can be maintained in the squirrel monkey by terminating a schedule complex. The schedule of shock presentation was a critical determinant of the patterns of responding, especially under fixed-interval schedules of termination. The rates and patterns of responding under various schedules of termination of schedule complexes were generally akin to those maintained under comparable schedules of food presentation. The findings suggest a general similarity in the dynamic aspects of performances under schedules of schedule-complex termination and comparable schedules of food presentation. The schedule of reinforcement is more important than the nature of the reinforcer in the control of behavior.     

Choice and the rate of punishment in concurrent schedules

Rats' responses on two levers were reinforced according to independent random-interval 1.5-min food schedules. In addition, both lever presses were intermittently punished according to several concurrent random-interval random-interval shock schedules. For the left, the scheduled rate of punishment was kept constant according to a random-interval 6-min schedule. For the right, the rate of punishment varied. As the frequency of punishment for the right lever press increased, its rate decreased. The rate of the left punished lever press increased, however, even though its scheduled reinforcement rate and punishment rate remained unchanged.     

Response strength in extreme multiple schedules

Four pigeons were trained in a series of two-component multiple schedules. Reinforcers were scheduled with random-interval schedules. The ratio of arranged reinforcer rates in the two components was varied over 4 log units, a much wider range than previously studied. When performance appeared stable, prefeeding tests were conducted to assess resistance to change. Contrary to the generalized matching law, logarithms of response ratios in the two components were not a linear function of log reinforcer ratios, implying a failure of parameter invariance. Over a 2 log unit range, the function appeared linear and indicated undermatching, but in conditions with more extreme reinforcer ratios, approximate matching was observed. A model suggested by McLean (1991), originally for local contrast, predicts these changes in sensitivity to reinforcer ratios somewhat better than models by Herrnstein (1970) and by Williams and Wixted (1986). Prefeeding tests of resistance to change were conducted at each reinforcer ratio, and relative resistance to change was also a nonlinear function of log reinforcer ratios, again contrary to conclusions from previous work. Instead, the function suggests that resistance to change in a component may be determined partly by the rate of reinforcement and partly by the ratio of reinforcers to responses.     

When to Provide Knowledge of Results During Motor Learning: Scheduling Effects

On what basis should sources of information feedback be scheduled when de- livered to someone learning a motor skill? In two experiments, we examined the influence of several knowledge of results (KR) schedules on the acquisi- tion and retention of a multisegment movement timing task. In these experi- ments a blocked KR schedule was detrimental to both skill acquisition and retention, likely because the delivery of KR for one particular information source over repeated trials focused the learner on only that part of the action, rather than learning the whole action. In Experiment 1, we found that a ran- dom schedule produced better acquisition and retention performance than the blocked schedule. In Experiment 2, we attempted to optimize a schedule by delivering KR for the segment that had been performed most poorly. Al- though this performance-sensitive (P-S) manipulation resulted in better learning than under a blocked schedule, it was no better than a quasi-random, yoked control condition. These findings are discussed in terms of the nature of the action planning activities of the learner and how schedules of KR affect learner strategies.     

Response induction during the acquisition and maintenance of lever pressing with delayed reinforcement

The acquisition of lever pressing by rats and the occurrence of unreinforced presses at a location different from that of the reinforced response were studied using different delays of reinforcement. An experimental chamber containing seven identical adjoining levers was used. Only presses on the central (operative) lever produced food pellets. Groups of 3 rats were exposed to one of seven different tandem random-interval (RI) fixed-time (FT) schedules. The average RI duration was the complement of the FT duration such that their sum yielded a nominal 32-s interreinforcement interval on average. Response rate on the operative lever decreased as the FT value was lengthened. The spatial distribution of responses on the seven levers converged on the operative lever when the FT was 0 or 2 s and spread across the seven levers as the FT value was lengthened to 16 or 32 s. Presses on the seven levers were infrequent during the FT schedule. Both operative- and inoperative-lever pressing intertwined in repetitive patterns that were consistent within subjects but differed between subjects. These findings suggest that reinforcer delay determined the response-induction gradient.     

Time and rate measures in choice transitions

Three experiments with pigeons studied the relation between time and rate measures of behavior under conditions of changing preference. Experiment 1 studied a concurrent chain schedule with random-interval initial links and fixed-interval terminal links; Experiment 2 studied a multiple chained random-interval fixed-interval schedule; and Experiment 3 studied simple concurrent random-interval random-interval schedules. In Experiment 1, and to a lesser extent in the other two experiments, session-average initial-link wait-time differences were linearly related to session-average response-rate differences. In Experiment 1, and to a lesser extent in Experiment 3, ratios of session-average initial-link wait times and response rates were related by a power function. The weaker relations between wait and response measures in Experiment 2 appear to be due to the absence of competition between responses. In Experiments 1 and 2, initial-link changes lagged behind terminal-link changes. These findings may have implications for the relations between fixed- and variable-interval procedures and suggest that more attention should be paid to temporal measures in studies of free-operant choice.     

Negative reinforcement and choice in humans

Participants chose between reinforcement schedules differing in delay and/or duration of noise offset. In Experiment 1 it was found that (1) immediate reinforcement was preferred to delayed reinforcement when amounts (durations) of reinforcement were equal; (2) a relatively large reinforcer was preferred to a smaller one when both reinforcers were obtained immediately; and (3) preference for an immediate, small reinforcer versus a delayed, large reinforcer increased as the delay preceding the large reinforcer increased, a sign of “impulsivity”. In Experiment 2, the schedules differed in amount or delay and equal intervals were added either to the constant parameter or the varied parameter. A shift from virtually exclusive preference to indifference occurred in the latter case but not the former, a result supporting a model of self-control that assumes that the value of a schedule depends on the ratio of amount and delay, and that choice between schedules depends on the ratio of these values.     

Varying reinforcement magnitude on interval schedules

In Experiment 1, rats were trained on either a random-interval or a variable-interval 60-sec schedule of reinforcement, and reinforcement magnitude was varied across conditions between one and four pellets. Although the two schedules maintained different patterns of behaviour, patterns and rates of responding were not systematically affected by the variation in reinforcement magnitude. In Experiment 2, a regulated probability interval schedule that generated similar rates of reinforcement to those of the schedules of Experiment 1 was used, with the pattern of behaviour generated resembling that typical of a random-interval schedule. Changing reinforcement magnitude again produced few systematic changes in behaviour. In Experiment 3, a variable-ratio schedule was used within a procedure that otherwise resembled that of Experiments 1 and 2. Increasing the reinforcement magnitude now decreased the rates of responding, and examination of the patterns of responding showed that this came about because rates of responding were higher early in the interreinforcer interval in the one-pellet condition. These experiments demonstrate the insensitivity of behaviour under interval schedules to changes in reinforcement magnitude and suggest the operation of mechanisms different from those engaged by ratio schedules and discretetrial learning procedures.     

The Probability of Small Schedule Values and Preference for Random-Interval Schedules

Preference for working on variable schedules and temporal discrimination were simultaneously examined in two experiments using a discrete-trial, concurrent-chains arrangement with fixed interval (FI) and random interval (RI) terminal links. The random schedule was generated by first sampling a probability distribution after the programmed delay to reinforcement on the FI schedule had elapsed, and thus the RI never produced a component schedule value shorter than the FI and maintained a rate of reinforcement half that of the FI. Despite these features, the FI was not strongly preferred. The probability of obtaining the smallest programmed delay to reinforcement on the RI schedule was manipulated in Experiment 1, and the interaction of this probability and initial link length was examined in Experiment 2. As the probability of obtaining small values in the RI increased, preference for the schedule increased while the discriminated time of reinforcer availability in the terminal link decreased. Both of these effects were attenuated by lengthening the initial links. The results support the view that in addition to the delay to reinforcement, the probability of obtaining a short delay is an important choice-affecting variable that likely contributes to the robust preferences for variable, as opposed to fixed, schedules of reinforcement.     

Choice dynamics in concurrent ratio schedules of reinforcement

The generalized matching law predicts performance on concurrent schedules when variable-interval schedules are programmed but is trivially applicable when independent ratio schedules are used. Responding usually is exclusive to the schedule with the lowest response requirement. Determining a method to program concurrent ratio schedules such that matching analyses can be usefully employed would extend the generality of matching research and lead to new avenues of research. In the present experiments, ratio schedules were programmed dependently such that responses to either of the two options progressed the requirement on both schedules. Responding is not exclusive because the probability of reinforcement increases on both schedules as responses are allocated to either schedule. In Experiment 1, performance on concurrent variable-ratio schedules was assessed, and reinforcer ratios were varied across conditions to investigate changes in sensitivity. Additionally, the length of a changeover delay was manipulated. In Experiment 2, performance was compared under concurrently available, dependently programmed variable-ratio and fixed-ratio schedules. Performance was well described by the generalized matching law. Increases in the changeover delay decreased sensitivity, whereas sensitivity was higher when variable-ratio schedules were employed, compared with fixed-ratio schedules. Concurrent ratio schedules can be a viable approach to studying functional differences between ratio and interval schedules.     

Human operant performance: Sensitivity and pseudosensitivity to contingencies

Undergraduates' button presses occasionally produced points exchangeable for money. Left and right buttons were initially correlated with multiple random-ratio (RR) and random-interval (RI) components, respectively. During interruptions of the multiple schedule, students filled out sentence-completion guess sheets describing the schedules, and points were contingent upon the accuracy of guesses. To test for sensitivity to schedule contingencies, schedule components were repeatedly reversed between the two buttons. Pressing rates were consistently higher in ratio than in interval components even when feedback for guesses was discontinued, demonstrating sensitivity to the difference between ratio and interval contingencies. The question was whether this sensitivity was based directly on the contingencies or whether it was rule-governed. For two students, when multiple RR RI schedules were changed to multiple RI RI schedules, rates became low in both components of the multiple RI RI schedule; however, subsequent prevention of point deliveries for the first few responses in any component produced high rates in that component. For a third student, response rates became higher in the RI component that provided the lower rate of reinforcement. In each case, performance was inconsistent with typical effects of the respective schedules with nonhuman organisms; it was therefore plausible to conclude that the apparently contingency-governed performances were instead rule-governed.     

Arousal, changeover responses, and preference in concurrent schedules

Pigeons were trained on multiple schedules that provided concurrent reinforcement in each of two components. In Experiment 1, one component consisted of a variable-interval (VI) 40-s schedule presented with a VI 20-s schedule, and the other a VI 40-s schedule presented with a VI 80-s schedule. After extended training, probe tests measured preference between the stimuli associated with the two 40-s schedules. Probe tests replicated the results of Belke (1992) that showed preference for the 40-s schedule that had been paired with the 80-s schedule. In a second condition, the overall reinforcer rate provided by the two components was equated by adding a signaled VI schedule to the component with the lower reinforcer rate. Probe results were unchanged. In Experiment 2, pigeons were trained on alternating concurrent VI 30-s VI 60-s schedules. One schedule provided 2-s access to food and the other provided 6-s access. The larger reinforcer magnitude produced higher response rates and was preferred on probe trials. Rate of changeover responding, however, did not differ as a function of reinforcer magnitude. The present results demonstrate that preference on probe trials is not a simple reflection of the pattern of changeover behavior established during training.     

Some temporal parameters of non-contingent reinforcement

In each baseline session, pigeons were exposed to a multiple schedule in which each of five distinctive stimuli was correlated with a different frequency of reinforcement. In one component, responses were reinforced with a probability of 0.10 (random-ratio schedule); in the other four components, responses were reinforced with different scheduled temporal frequencies averaging 30 to 240 sec between reinforcements (random-interval schedules). For periods lasting 30 sessions, contingent reinforcement was discontinued and reinforcement was presented independent of responding at irregular intervals averaging 30, 60, or 120 sec, while the sequence of stimuli continued. After each such period, the baseline was reinstated for 30 sessions. The data indicated that: (1) The rate of responding in the presence of all stimuli decreased as exposure to the non-contingent reinforcement procedure was prolonged, at all the frequencies of reinforcement employed; (2) The rate under the random-ratio schedule declined faster than the rates under all the random-interval schedules, presumably because the decrease in reinforcement frequency under this stimulus condition was greatest; (3) The decline in rates of responding under the stimuli correlated with the random-interval schedules tended to be greatest for the stimuli paired with the lowest frequencies of reinforcement.