Methodological improvements to a Procedure for Rapidly Establishing Steady-State Behavior

We performed three experiments to improve the quality and retention of data obtained from a Procedure for Rapidly Establishing Steady-State Behavior (PRESS-B; Klapes et al., 2020). In Experiment 1, 120 participants worked on nine concurrent random-interval random-interval (conc RI RI) schedules and were assigned to four conditions of varying changeover delay (COD) length. The 0.5-s COD condition group exhibited the fewest instances of exclusive reinforcer acquisition. Importantly, this group did not differ in generalized matching law (GML) fit quality from the other groups. In Experiment 2, 60 participants worked on nine conc RI RI schedules with a wider range of scheduled reinforcement rate ratios than was used in Experiment 1. Participants showed dramatic reductions in exclusive reinforcer acquisition. Experiment 3 entailed a replication of Experiment 2 wherein blackout periods were implemented between the schedule presentations and each schedule remained in operation until at least one reinforcer was acquired on each alternative. GML fit quality was slightly more consistent in Experiment 3 than in the previous experiments. Thus, these results suggest that future PRESS-B studies should implement a shorter COD, a wider and richer scheduled reinforcement rate ratio range, and brief blackouts between schedule presentations for optimal data quality and retention.     

Concurrent random-interval schedules and the matching law

In Experiment I, a group of eight pigeons performed on concurrent random-interval schedules constructed by holding probability equal and varying cycle time to produce ratios of reinforcer densities of 1:1, 3:1, and 5:1 for key pecking. Schedules for a second group of seven were constructed with equal cycle times and unequal probabilities. Both groups deviated from simple matching, but the two forms of the schedules appeared to produce no consistent patterns of deviation. The data were found to be consistent with those obtained in concurrent variable-interval situations. The parameters of the matching equation in the form of Y=k X^a were estimated; the value of k was unity and a was 0.84. In Experiment II, six pigeons were exposed to two conc RI RI schedules in which one component increasingly approximated an FI schedule. The value of k was not 1.0. Concurrent RI RI schedules were shown to represent a continuum from conc FI VI to conc VI VI schedules. The use of the exponential equation in testing “matching laws” suggests that a<1 will continue to be observed, and this will set limits on the form of new laws and the assumed or rational values of the component variables in these laws.     

Choice between response units: The rate constancy model

In a conjoint schedule, reinforcement is available simultaneously on two or more schedules for the same response. The present experiments provided food for key pecking on both a random-interval and a differential-reinforcement-of-low-rate (DRL) schedule. Experiment 1 involved ordinary DRL schedules; Experiment 2 added an external stimulus to indicate when the required interresponse time had elapsed. In both experiments, the potential reinforcer frequency from each component was varied by means of a second-order fixed-ratio schedule, and the DRL time parameter was changed as well. Response rates were described by a model stating that time allocation to each component matches the relative frequency of reinforcement for that component. When spending time in a given component, the subject is assumed to respond at the rate characteristic of baseline performance. This model appeared preferable to the absolute-rate version of the matching law. The model was shown to be applicable to multiple-response concurrent schedules as well as to conjoint schedules, and it described some of the necessary conditions for response matching, undermatching, and bias. In addition, the pigeons did not optimize reinforcer frequency.     

Income and choice between different goods

In Experiment 1, 3 rats chose between two simultaneously operating variable-interval schedules, one of which provided saccharin water and the other, food. In five conditions, the absolute (and equal) reinforcement rates provided by the pair of equal-valued schedules were manipulated in the range of 36 to 240 per hour. Experiment 2 was identical to Experiment 1 except that these schedules operated successively, arranged by requiring the rat to stand on the side of the chamber correlated with each schedule. Food/saccharin choice ratios were inversely related to reinforcement rate in both experiments, although this effect was stronger in Experiment 2. When delivery rates were high, preference for food over saccharin often reversed as the session progressed. The results were interpretable in terms of economic accounts of choice (e.g., the minimum-needs hypothesis), as well as in terms of traditional psychological accounts (e.g., matching theory).     

Asymmetry of reinforcement and punishment in human choice

The hypothesis that a penny lost is valued more highly than a penny earned was tested in human choice. Five participants clicked a computer mouse under concurrent variable-interval schedules of monetary reinforcement. In the no-punishment condition, the schedules arranged monetary gain. In the punishment conditions, a schedule of monetary loss was superimposed on one response alternative. Deviations from generalized matching using the free parameters c (sensitivity to reinforcement) and log k (bias) were compared in the no-punishment and punishment conditions. The no-punishment conditions yielded values of log k that approximated zero for all participants, indicating no bias. In the punishment condition, values of log k deviated substantially from zero, revealing a 3-fold bias toward the unpunished alternative. Moreover, the c parameters were substantially smaller in punished conditions. The values for bias and sensitivity under punishment did not change significantly when the measure of net reinforcers (gains minus losses) was applied to the analysis. These results mean that punishment reduced the sensitivity of behavior to reinforcement and biased performance toward the unpunished alternative. We concluded that a single punisher subtracted more value than a single reinforcer added, indicating an asymmetry in the law of effect.     

The symmetrical law of effect and the matching relation in choice behavior

In a concurrent-chains procedure, pigeons chose between outcomes that differed in the rate of response-independent delivery of food and electric shocks. The application of functional measurement techniques confirmed the matching relation—between choice and rate of reinforcement value—for two of three pigeons. Scale values of the outcomes were extracted for the two birds that conformed to matching, and the value of a single occurrence of shock per minute—in terms of negative food units—was estimated. A second experiment with concurrent chains provided a test of these parameter estimates. The close correspondence between predicted and obtained choice behavior found in Experiment II indicated that the estimates of outcome value were indeed reliable. Both experiments together support the contention that the effects on choice behavior of positive and aversive stimuli appear to be equal, though opposite in sign.     

Rat choice in rapidly changing concurrent schedules

In two experiments, experimentally naïve rats were trained in concurrent variable‐interval schedules in which the reinforcer ratios changed daily according to a pseudorandom binary sequence. In Experiment 1, relative response rates showed clear sensitivity to current‐session reinforcer ratios, but not to previous sessions' reinforcer ratios. Within sessions, sensitivity to the current session's reinforcement rates increased steadily, and by session end, response ratios approached matching to the current‐session reinforcer ratios. Across sessions, sensitivity to the current session's reinforcer ratio decreased with continued exposure to the pseudorandom binary sequence, contrary to expectations based on previous studies demonstrating learning sets. Using a second group of naïve rats, Experiment 2 replicated the main results from Experiment 1 and showed that although there were increases over sessions in both changeover rate and response rate during the changeover delay, neither could explain the accompanying reductions in sensitivity. We consider the role of reinforcement history, showing that our results can be simulated using two separate representations, one local and one nonlocal, but a more complex approach will be needed to bring together these results and other history effects such as learning sets and spontaneous recovery.     

A model for residence time in concurrent variable interval performance

A component-functions model of choice behavior is proposed for performance on interdependent concurrent variable-interval (VI) variable-interval schedules based on the product of two component functions, one that enhances behavior and one that reduces behavior. The model is the solution to the symmetrical pair of differential equations describing behavioral changes with respect to two categories of reinforcers: enhancing and reducing, or excitatory and inhibitory. The model describes residence time in interdependent concurrent VI VI schedules constructed from arithmetic and exponential distributions. The model describes the data reported by Alsop and Elliffe (1988) and Elliffe and Alsop (1996) with a variance accounted for of 87% compared to 64% accounted for by the Davison and Hunter (1976) model and 42% by Herrnstein's (1970) hyperbola. The model can explain matching, undermatching, and overmatching in the same subject under different procedures and has the potential to be extended to performance on concurrent schedules with more than two alternatives, multiple schedules, and single schedules. Thus, it can be considered as an alternative to Herrnstein's quantitative law of effect.     

The logistics of choice

The generalized matching law (GML) is reconstructed as a logistic regression equation that privileges no particular value of the sensitivity parameter, a. That value will often approach 1 due to the feedback that drives switching that is intrinsic to most concurrent schedules. A model of that feedback reproduced some features of concurrent data. The GML is a law only in the strained sense that any equation that maps data is a law. The machine under the hood of matching is in all likelihood the very law that was displaced by the Matching Law. It is now time to return the Law of Effect to centrality in our science.     

A brief computer-based assessment of reinforcer dimensions affecting choice

In an extension of Neef, Shade, and Miller (1994), we used a brief computer-based assessment of differential responsiveness to reinforcer rate, quality, delay, and response effort in affecting the choices of 11 participants. The assessment involved successive presentations of two concurrent sets of math problems, each set associated with competing reinforcer or response dimensions in a counterbalanced fashion. The results showed that the reinforcer and response dimensions differentially affected choice, with time-allocation patterns varying across students.     

Responses and pauses: discrimination and a choice catastrophe.

Pigeons produced a stimulus change either by responding or by not responding for a specified time period (by pausing). They then had to choose between two responses to obtain food. One choice was correct if the first component had been completed by a response; the other was correct if the component had been completed by a pause. The pigeons usually chose correctly, thereby indicating that they used their own prior behavior as a discriminative stimulus. Fixed pause requirements did not produce equal first component completions by a response and by a pause. To obtain equality, the pause requirement was titrated as a function of current performance. Titration resulted in equal completions and also produced accurate discrimination. In addition to showing that pigeons discriminated whether they had responded or paused, the data displayed and discontinuous functions predicted by catastrophe theory. Another procedure used forced choice rather than titration to produce equal completions by pausing and responding and also showed accurate discrimination of behavior.     

Stimulus control of respondent and operant key pecking: A single key procedure

Pigeons' responses to a uniformly illuminated response key were either reinforced on a variable-interval one-minute schedule of reinforcement or extinguished for one-minute periods. When 1.5 second signals were presented at the beginning of each component, so as to differentially predict reinforcement, the pigeons pecked at the signals, at rates higher than rates during the remainder of the component. When the brief signals were not differentially predictive of reinforcement, pecking in their presence decreased to near zero levels. Similar results were obtained with signals based upon colors and upon line orientations. Changes in rates of (unreinforced) pecking occurred during the signal whether pigeons responded differentially during the remainder of the component or not. Experiment II demonstrated that the presence of the signal correlated with extinction was not necessary for pecking to develop at the signal which preceded the component in which responding was intermittently reinforced. The experiments demonstrated a clear dissociation of respondent control from operant control of a response. In addition, operant behavior was shown to be relatively insensitive to differing rates of reinforcement, as compared to the sensitivity of respondent behavior to differing rates of reinforcement produced by the very same operant behavior.     

A quantitative analysis of extreme choice

Six homing pigeons were trained on a variety of concurrent variable-interval schedules in a switching-key procedure. Unlike previous work, reinforcer ratios of up to 160 to 1 and concurrent extinction variable-interval schedules were arranged in order to investigate choice when reinforcer-frequency outcomes were extremely different. The data obtained over 11 conditions were initially analyzed according to the generalized matching law, which fitted the data well. The generalized matching law was then fitted only to conditions in which the reinforcer ratios were between 1 to 10 and 10 to 1. The deviations of choice measures from the other four more extreme reinforcer-ratio conditions were significantly more towards equal choice than predicted by this second generalized matching fit. A contingency-discriminability model, which predicts such deviations, described the data more effectively than did the generalized matching law, and also correctly predicted the maintenance of responding on both alternatives when one was associated with extinction.     

Zebrafish choice behavior is sensitive to reinforcer rate,immediacy, and magnitude ratios

Behavioral flexibility has, in part, been defined by choice behavior changing as a function of changes in reinforcer payoffs. We examined whether the generalized matching law quantitatively described changes in choice behavior in zebrafish when relative reinforcer rates, delays/immediacy, and magnitudes changed between two alternatives across conditions. Choice was sensitive to each of the three reinforcer properties. Sensitivity estimates to changes in relative reinforcer rates were greater when 2 variable-interval schedules were arranged independently between alternatives (Experiment 1a) than when a single schedule pseudorandomly arranged reinforcers between alternatives (Experiment 1b). Sensitivity estimates for changes in relative reinforcer immediacy (Experiment 2) and magnitude (Experiment 3) were similar but lower than estimates for reinforcer rates. These differences in sensitivity estimates are consistent with studies examining other species, suggesting flexibility in zebrafish choice behavior in the face of changes in payoff as described by the generalized matching law.     

Concurrent reinforcement schedules for problem behavior and appropriate behavior: experimental applications of the matching law

This study evaluated how children who exhibited functionally equivalent problem and appropriate behavior allocate responding to experimentally arranged reinforcer rates. Relative reinforcer rates were arranged on concurrent variable-interval schedules and effects on relative response rates were interpreted using the generalized matching equation. Results showed that relative rates of responding approximated relative rates of reinforcement. Finally, interventions for problem behavior were evaluated and differential reinforcement of alternative behavior and extinction procedures were implemented to increase appropriate behavior and decrease problem behavior. Practical considerations for the application of the generalized matching equation specific to severe problem behavior are discussed, including difficulties associated with defining a reinforced response, and obtaining steady state responding in clinical settings.     

The general matching law describes choice on concurrent variable-interval schedules of wheel-running reinforcement.

Six male Wistar rats were exposed to concurrent variable-interval schedules of wheel-running reinforcement. The reinforcer associated with each alternative was the opportunity to run for 15 s, and the duration of the changeover delay was 1 s. Results suggested that time allocation was more sensitive to relative reinforcement rate than was response allocation. For time allocation, the mean slopes and intercepts were 0.82 and 0.008, respectively. In contrast, for response allocation, mean slopes and intercepts were 0.60 and 0.03, respectively. Correction for low response rates and high rates of changing over, however, increased slopes for response allocation to about equal those for time allocation. The results of the present study suggest that the two-operant form of the matching law can be extended to wheel-running reinforcement. 'I'he effects of a low overall response rate, a short Changeover delay, and long postreinforcement pausing on the assessment of matching in the present study are discussed.     

Nonstable concurrent choice in pigeons

Six pigeons were trained on concurrent variable-interval schedules in which the arranged reinforcer ratios changed from session to session according to a 31-step pseudorandom binary sequence. This procedure allows a quantitative analysis of the degree to which performance in an experimental session is affected by conditions in previous sessions. Two experiments were carried out. In each, the size of the reinforcer ratios arranged between the two concurrent schedules was varied between 31-step conditions. In Experiment 1, the concurrent schedules were arranged independently, and in Experiment 2 they were arranged nonindependently. An extended form of the generalized matching law described the relative contribution of past and present events to present-session behavior. Total performance in sessions was mostly determined by the reinforcer ratio in that session and partially by reinforcers that had been obtained in previous sessions. However, the initial exposure to the random sequence produced a lower sensitivity to current-session reinforcers but no difference in overall sensitivity to reinforcement. There was no evidence that the size of the reinforcer ratios available on the concurrent schedules affected either overall sensitivity to reinforcement or the sensitivity to reinforcement in the current session. There was also no evidence of any different performance between independent and nonindependent scheduling. Because of these invariances, this experiment validates the use of the pseudorandom sequence for the fast determination of sensitivity to reinforcement.     

Optimization and the matching law as accounts of instrumental behavior

The interaction between instrumental behavior and environment can be conveniently described at a molar level as a feedback system. Two different possible theories, the matching law and optimization, differ primarily in the reference criterion they suggest for the system. Both offer accounts of most of the known phenomena of performance on concurrent and single variable-interval and variable-ratio schedules. The matching law appears stronger in describing concurrent performances, whereas optimization appears stronger in describing performance on single schedules.