The matching law in and within groups of rats

In each of the two experiments, a group of five rats lived in a complex maze containing four small single-lever operant chambers. In two of these chambers, food was available on variable-interval schedules of reinforcement. In Experiment I, nine combinations of variable intervals were used, and the aggregate lever-pressing rates (by the five rats together) were studied. The log ratio of the rates in the two chambers was linearly related to the log ratio of the reinforcement rates in them; this is an instance of Herrnstein's matching law, as generalized by Baum. Summing over the two food chambers, food consumption decreased, and response output increased, as the time required to earn each pellet increased. In Experiment II, the behavior of individual rats was observed by time-sampling on selected days, while different variable-interval schedules were arranged in the two chambers where food was available. Individual lever-pressing rates for the rats were obtained, and their median bore the same “matching” relationship to the reinforcement rates as the group aggregate in Experiment I. There were differences between the rats in their distribution of time and responses between the two food chambers; these differences were correlated with differences in the proportions of reinforcements the rats obtained from each chamber.     

Matching, maximizing, and hill-climbing

In simple situations, animals consistently choose the better of two alternatives. On concurrent variable-interval variable-interval and variable-interval variable-ratio schedules, they approximately match aggregate choice and reinforcement ratios. The matching law attempts to explain the latter result but does not address the former. Hill-climbing rules such as momentary maximizing can account for both. We show that momentary maximizing constrains molar choice to approximate matching; that molar choice covaries with pigeons' momentary-maximizing estimate; and that the “generalized matching law” follows from almost any hill-climbing rule.     

Performances on ratio and interval schedules of reinforcement: Data and theory

Two differences between ratio and interval performance are well known: (a) Higher rates occur on ratio schedules, and (b) ratio schedules are unable to maintain responding at low rates of reinforcement (ratio “strain”). A third phenomenon, a downturn in response rate at the highest rates of reinforcement, is well documented for ratio schedules and is predicted for interval schedules. Pigeons were exposed to multiple variable-ratio variable-interval schedules in which the intervals generated in the variable-ratio component were programmed in the variable-interval component, thereby “yoking” or approximately matching reinforcement in the two components. The full range of ratio performances was studied, from strained to continuous reinforcement. In addition to the expected phenomena, a new phenomenon was observed: an upturn in variable-interval response rate in the midrange of rates of reinforcement that brought response rates on the two schedules to equality before the downturn at the highest rates of reinforcement. When the average response rate was corrected by eliminating pausing after reinforcement, the downturn in response rate vanished, leaving a strictly monotonic performance curve. This apparent functional independence of the postreinforcement pause and the qualitative shift in response implied by the upturn in variable-interval response rate suggest that theoretical accounts will require thinking of behavior as partitioned among at least three categories, and probably four: postreinforcement activity, other unprogrammed activity, ratio-typical operant behavior, and interval-typical operant behavior.     

Concurrent reinforcement schedules for problem behavior and appropriate behavior: experimental applications of the matching law

This study evaluated how children who exhibited functionally equivalent problem and appropriate behavior allocate responding to experimentally arranged reinforcer rates. Relative reinforcer rates were arranged on concurrent variable-interval schedules and effects on relative response rates were interpreted using the generalized matching equation. Results showed that relative rates of responding approximated relative rates of reinforcement. Finally, interventions for problem behavior were evaluated and differential reinforcement of alternative behavior and extinction procedures were implemented to increase appropriate behavior and decrease problem behavior. Practical considerations for the application of the generalized matching equation specific to severe problem behavior are discussed, including difficulties associated with defining a reinforced response, and obtaining steady state responding in clinical settings.     

Temporal proximity and reinforcement sensitivity in multiple schedules

Distributions of reinforcers between two components of multiple variable-interval schedules were varied over a number of conditions. Sensitivity to reinforcement, measured by the exponent of the power function relating ratios of responses in the two components to ratios of reinforcers obtained in the components, did not differ between conditions with 15-s or 60-s component durations. The failure to demonstrate the “short-component effect,” where sensitivity is high for short components, was consistent with reanalysis of previous data. With 60-s components, sensitivity to reinforcement decreased systematically with time since component alternation, and was higher in the first 15-s subinterval of the 60-s component than for the component whose total duration was 15 s. Varying component duration and sampling behavior at different times since component transition may not be equivalent ways of examining the effects of average temporal distance between components.     

The role of contingencies and "principles of behavioral variation" in pigeons' pecking

Staddon and Simmelhag's proposal that behavior is produced by “principles of behavioral variation” instead of contingencies of reinforcement was tested in two experiments. In the first experiment pigeons were exposed to either a fixed-interval schedule of response-contingent reinforcement, an autoshaping schedule of stimulus-contingent reinforcement, or a fixed-time schedule of noncontingent reinforcement. Pigeons exposed to contingent reinforcement came to peck more rapidly than those exposed to noncontingent reinforcement. Staddon and Simmelhag's “principles of behavioral variation” included the proposal that patterns (interim and terminal) were a function of momentary probability of reinforcement. In the second experiment pigeons were exposed to either a fixed-time or a random-time schedule of noncontingent reinforcement. Pecking showed a constant frequency of occurrence over postfood time on the random-time schedule. Most behavior showed patterns on the fixed-time schedule that differed in overall shape (i.e., interim versus terminal) from those shown on the random-time schedule. It was concluded that both the momentary probability of reinforcement and postfood time can affect patterning.     

Concurrent variable-interval variable-ratio schedules can provide only weak evidence for matching

Herrnstein and Heyman (1979) showed that when pigeons' pecking is reinforced on concurrent variable-interval variable-ratio schedules, (1) their behavior ratios match the ratio of the schedules' reinforcer frequencies, and (2) there is more responding on the variable interval. Since maximizing the reinforcement rate would require responding more on the variable ratio, these results were presented as establishing the primacy of matching over maximizing. In the present report, different ratios of behavior were simulated on a computer to see how they would affect reinforcement rates on these concurrent schedules. Over a wide range of experimenter-specified choice ratios, matching obtained — a result suggesting that changes in choice allocation produced changes in reinforcer frequencies that correspond to the matching outcome. Matching also occurred at arbitrarily selected choice ratios when reinforcement rates were algebraically determined by each schedule's reinforcement-feedback function. Additionally, three birds were exposed to concurrent variable-interval variable-ratio schedules contingent on key pecking in which hopper durations were varied in some conditions to produce experimenter-specified choice ratios. Matching generally obtained between choice ratios and reinforcer-frequency ratios at these different choice ratios. By suggesting that reinforcer frequencies track choice on this procedure, instead of vice versa, this outcome questions whether matching-as-outcome was due to matching-as-process in the Herrnstein and Heyman study.     

WITHIN‐SESSION TRANSITIONS IN CHOICE: A STRUCTURAL AND QUANTITATIVE ANALYSIS

The present study used within‐session transitions between two concurrent schedules to evaluate choice in transition. Eight female Long‐Evans rats were trained to respond under concurrent schedules of reinforcement during experimental sessions that lasted 22 hr. The generalized matching equation was used to model steady‐state behavior at the end of each session, while transitional behavior that emerged following the change in reinforcement schedules was modeled using a logistic equation. The generalized matching and logistic equations were appropriate models for behavior generated during single‐session transitions. A local analysis of behavior on the two response alternatives during acquisition was used to determine the source of preference as revealed in response ratios. The number of “low‐response” visits, those containing three to five responses, remained stable. Preference ratios largely reflected a sharp increase in the number of visits with long response bouts on the rich alternative and a decrease in the number of such visits to the leaner alternative.     

Concurrent fixed-interval variable-ratio schedules and the matching relation

Five rats responded under concurrent fixed-interval variable-ratio schedules of food reinforcement. Fixed-interval values ranged from 50-seconds to 300-seconds and variable-ratio values ranged from 30 to 360; a five-second changeover delay was in effect throughout the experiment. The relations between reinforcement ratios obtained from the two schedules and the ratios of responses and time spent on the schedules were described by Baum's (1974) generalized matching equation. All subjects undermatched both response and time ratios to reinforcement ratios, and all subjects displayed systematic bias in favor of the variable-ratio schedules. Response ratios undermatched reinforcement ratios less than did time ratios, but response ratios produced greater bias than did time ratios for every subject and for the group as a whole. Local rates of responding were generally higher on the variable-ratio than on the fixed-interval schedules. When responding was maintained by both schedules, a period of no responding on either schedule immediately after fixed-interval reinforcement typically was followed by high-rate responding on the variable-ratio schedule. At short fixed-interval values, when a changeover to the fixed-interval schedule was made, responding usually continued until fixed-interval reinforcement was obtained; at longer values, a changeover back to the variable-ratio schedule usually occurred when fixed-interval reinforcement was not forthcoming within a few seconds, and responding then alternated between the two schedules every few seconds until fixed-interval reinforcement finally was obtained.     

Behavioral stereotypy and the generalized matching equation

The development of behavioral stereotypy is a common result of exposure to both response-dependent and response-independent reinforcement procedures. The generalized matching equation and two dynamic versions of that equation, which take into account the time differential between reinforcements and their effect on behavior, predict this outcome of many procedures involving reinforcement. Following from the assumption that distinct response topographies, distinct response sequences, or orientations to distinct stimuli can be treated in the equations as distinct classes of behavior, the equations predict that-at least for matching and undermatching-the behavior class that is most biased relative to other behavior classes of the same type will tend to predominate to the exclusion or near exclusion of those behavior classes.     

Quantification of rats' behavior during reinforcement periods

What is treated as a single unit of reinforcement often involves what could be called a reinforcement period during which two or more acts of ingestion may occur, and each of these may have associated with it a series of responses, some reflexive, some learned, that lead up to ingestion. Food-tray presentation to a pigeon is an example of such a “reinforcement period.” In order to quantify this behavior, a continuous-reinforcement schedule was used as the reinforcement period and was chained to a fixed-ratio schedule. Both fixed-ratio size and reinforcement-period duration were manipulated. Rats were used as subjects, food as reinforcement, and a lever press as the operant. Major findings included (a) a rapid decline in response rates during the first 15 to 20 seconds of the reinforcement periods, and (b) a strong positive relationship between these response rates and the size of the fixed ratio. Also revealed was a short scallop not normally found in fixed-ratio response patterns, whose length was a function of fixed-ratio size and reinforcement-period duration. It is suggested that rapidly fluctuating excitatory processes can account for many of these findings and that such processes are functionally significant in terms of behavioral compensation.     

USING MICROSOFT OFFICE EXCEL® 2007 TO CONDUCT GENERALIZED MATCHING ANALYSES

The generalized matching equation is a robust and empirically supported means of analyzing relations between reinforcement and behavior. Unfortunately, no simple task analysis is available to behavior analysts interested in using the matching equation to evaluate data in clinical or applied settings. This technical article presents a task analysis for the use of Microsoft Excel to analyze and plot the generalized matching equation. Using a data‐based case example and a step‐by‐step guide for completing the analysis, these instructions are intended to promote the use of quantitative analyses by researchers with little to no experience in quantitative analyses or the matching law.     

Undermatching and overmatching as deviations from the matching law

A model of performance under concurrent variable-interval reinforcement schedules that takes as its starting point the hypothetical “burst” structure of operant responding is presented. Undermatching and overmatching are derived from two separate, and opposing, tendencies. The first is a tendency to allocate a certain proportion of response bursts randomly to a response alternative without regard for the rate of reinforcement it provides, others being allocated according to the simple matching law. This produces undermatching. The second is a tendency to prolong response bursts that have a high probability of initiation relative to those for which initiation probability is lower. This process produces overmatching. A model embodying both tendencies predicts (1) that undermatching will be more common than overmatching, (2) that overmatching, when it occurs, will tend to be of limited extent. Both predictions are consistent with available data. The model thus accounts for undermatching and overmatching deviations from the matching law in terms of additional processes added on to behavior allocation obeying the simple matching relation. Such a model thus enables processes that have been hypothesized to underlie matching, such as some type of reinforcement rate or probability optimization, to remain as explanatory mechanisms even though the simple matching law may not generally be obeyed.     

The law of effect and avoidance: a quantitative relationship between response rate and shock-frequency reduction

Two experiments were conducted to investigate the quantitative relationship between response rate and reinforcement frequency in single and multiple variable-interval avoidance schedules. Responses cancelled delivery of shocks that were scheduled by variable-interval schedules. When shock-frequency reduction was taken as the measure of reinforcement, the relationship between response rate and reinforcement frequency on single variable-interval avoidance schedules was accurately described by Herrnstein's (1970) equation for responding on single variable-interval schedules of positive reinforcement. On multiple variable-interval avoidance schedules with brief components, asymptotic relative response rate matched relative shock-frequency reduction. The results suggest that many interactions between response rates and shock-frequency reduction in avoidance can be understood within the framework of the generalized matching relation, as applied by Herrnstein (1970) to positive reinforcement.     

The identities hidden in the matching laws, and their uses

Various theoretical equations have been proposed to predict response rate as a function of the rate of reinforcement. If both the rate and probability of reinforcement are considered, a simple identity, defining equation, or "law" holds. This identity places algebraic constraints on the allowable forms of our mathematical models and can help identify the referents for certain empirical or theoretical coefficients. This identity can be applied to both single and compound schedules of reinforcement, absolute and relative measures, and to local, global and overall rates and probabilities. The rate matching equations of Hernstein and Catania appear to have been approximations to, and to have been evolving toward, one form of this algebraic identity. Estimates of the bias and sensitivity terms in the generalized ratio and logarithmic matching models are here held to be averaging artifacts arising from fitting procedures applied to models that violate or conceal the underlying identities.     

QUANTIFICATION OF ETHANOL'S ANTIPUNISHMENT EFFECT IN HUMANS USING THE GENERALIZED MATCHING EQUATION

Increases in rates of punished behavior by the administration of anxiolytic drugs (called antipunishment effects) are well established in animals but not humans. The present study examined antipunishment effects of ethanol in humans using a choice procedure. The behavior of 5 participants was placed under six concurrent variable‐interval schedules of monetary reinforcement. In three of the six concurrent schedules, punishment, in the form of monetary loss, was superimposed on one alternative. Data were analyzed according to the generalized matching equation which distinguishes between bias (allocation of behavior beyond what matching to relative reinforcer densities would predict) and sensitivity to reinforcement (how well behavior tracks relative reinforcer densities). In addition, participants completed a pencil‐tapping test. Under placebo punishment conditions, all participants demonstrated low response rates and a bias against the alternative associated with punishment, despite a resultant loss of available reinforcers. Bias against the punished alternative was dose‐dependently reduced in participants shown to be most sensitive to ethanol (0.6, 1.2, and 1.8 g/kg) in measures of overall responding and on the pencil‐tapping test. No ethanol‐induced change in bias was noted when punishment was not imposed. Sensitivity to reinforcement also decreased for participants shown to be sensitive to ethanol. In addition to extending antipunishment effects to humans, these results also show that antipunishment effects can be quantified via the matching equation.     

Pigeons'' spatial memory: II. Acquisition of delayed matching of key location and transfer to new locations

Five hungry pigeons first received delayed matching of key location training. Trials began with a “ready” stimulus (brief operation of the grain feeder). Then one (randomly chosen) of a set of four keys from a three-by-three matrix was lit briefly as the sample. After a short delay (retention interval), the sample key was lit again along with one of the other eight keys. A peck at the key that had served as the sample produced grain reinforcement, whereas a peck to the other key produced only the intertrial interval. After delayed matching of key location was learned, the remaining five key locations were introduced as samples. Four of the five birds performed at considerably above-chance levels on the novel sample trials during the first as well as subsequent sessions. These results suggest that pigeons sometimes learn the single rule—“choose the location that matches the sample.” The relevance of these results to the issue of whether pigeons learn a generalized matching rule (i.e., a concept of “sameness”) is discussed.     

On Herrnstein's equation and related forms

In 1970, Herrnstein proposed a simple equation to describe the relation between response and reinforcement rates on interval schedules. Its empirical basis is firm, but its theoretical foundation is still uncertain. Two approaches to the derivation of Herrnstein's equation are discussed. It can be derived as the equilibrium solution to a process model equivalent to familiar linear-operator learning models. Modifications of this approach yield competing power-function formulations. The equation can also be derived from the assumption that response strength is proportional to reinforcement rate, given that there is a ceiling on response rate. The proportional relation can, in turn, be derived from a threshold assumption equivalent to Shimp's “momentary maximizing”. This derivation implies that the two parameters of Herrnstein's equation should be correlated, and may explain its special utility in application to internal schedules.     

Choice, matching, and human behavior: A review of the literature

This review concerns human performance on concurrent schedules of reinforcement. Studies indicate that humans match relative behavior to relative rate of reinforcement. Herrnstein's proportional matching equation describes human performance but most studies do not evaluate the equation at the individual level. Baum's generalized matching equation has received strong support with humans as subjects. This equation permits the investigation of sources of deviation from ideal matching and a few studies have suggested variables which control such deviations in humans. While problems with instructional control are raised, the overall findings support the matching law as a principle of human choice.