Rats' performance on variable-interval schedules with a linear feedback loop between response rate and reinforcement rate

Three experiments investigated whether rats are sensitive to the molar properties of a variable-interval (VI) schedule with a positive relation between response rate and reinforcement rate (i.e., a VI+ schedule). In Experiment 1, rats responded faster on a variable ratio (VR) schedule than on a VI+ schedule with an equivalent feedback function. Reinforced interresponse times (IRTs) were shorter on the VR as compared to the VI+ schedule. In Experiments 2 and 3, there was no systematic difference in response rates maintained by a VI+ schedule and a VI schedule yoked in terms of reinforcement rate. This was found both when the yoking procedure was between-subject (Experiment 2) and within-subject (Experiment 3). Mean reinforced IRTs were similar on both the VI+ and yoked VI schedules, but these values were more variable on the VI+ schedule. These results provided no evidence that rats are sensitive to the feedback function relating response rate to reinforcement rate on a VI+ schedule.     

Accuracy of performance on a matching-to-sample procedure under interval schedules

Correct matches on a matching-to-sample procedure were reinforced under fixed-interval, chained fixed-interval, and fixed-interval schedules with exteroceptive stimulus changes correlated with time since the last reinforcer (an added clock). For all four pigeons, accuracy changed within the fixed-interval and fixed-interval schedules with added clock, decreasing from the beginning of the interval to some point in the middle. The performance then became increasingly more accurate until the end of the interval. Under the chained schedules, accuracy also changed within the components. During the initial component, accuracy decreased from the beginning of the fixed interval to some point in the middle or at the end. During the middle component, the performance usually remained at an intermediate level of accuracy. During the terminal component, the initially inaccurate performance became increasingly more accurate throughout the interval. Systematic relationships between response rate and per cent error showed that all four pigeons performed most accurately at high rates. The accuracy of the performance at low rates was also quite high. These relationships held for all three types of schedules through an eight-fold variation in scheduled interreinforcement time.     

Delayed auditory feedback and rhythmic tapping: evidence for a critical interval shift

Delayed auditory feedback (DAF) impairs performance in speech, music, and tapping, with maximal impairment in speech occurring at a delay of about 200 msec. This critical interval has played a central role in many explanations of the DAF effect, including both closed-loop feedback explanations and alternative proposals. We investigated the nature of the critical interval in rhythmic tapping--in particular, whether the critical interval has a constant value or is dependent on performance rate. Three experiments in which a synchronization-continuation paradigm was used consistently showed that the critical interval shifted with different tapping rates, with maximal impairment occurring when the delay approximately equaled the tapping rate. We address the implications of these results for theories of DAF.     

Human performance on negative slope schedules of points exchangeable for money: a failure of molar maximization

Panel pressing was generated and maintained in 5 adult humans by schedules of points exchangeable for money. Following exposure to a variable-interval 30-s schedule and to a linear variable-interval 30-s schedule (which permitted points to accumulate in an unseen "store" in the absence of responding), subjects were exposed to a series of conditions with a point-subtraction contingency arranged conjointly with the linear variable-interval schedule. Specifically, points were added to the store according to the linear-variable interval 30-s schedule and were subtracted from the store according to a ratio schedule. Ratio value varied across conditions and was determined individually for each subject such that the subtraction contingency would result in an approximately 50% reduction in the rate of point delivery. Conditions that included the subtraction contingency were termed negative slope schedules because the feedback functions were negatively sloped across all response rates greater than the inverse of the variable-interval schedule, in this case, two per minute. Overall response rates varied inversely with the subtraction ratio, indicating sensitivity to the negative slope conditions, but were in excess of that required by accounts based on strict maximization of overall reinforcement rate. Performance was also not well described by a matching-based account. Detailed analyses of response patterning revealed a consistent two-state pattern in which bursts of high-rate responding alternated with periods of prolonged pausing, perhaps reflecting the joint influence of local and overall reinforcement rates.     

Performance in concurrent interval schedules: a systematic replication

Five pigeons were trained on a variety of concurrent interval schedules that arranged reinforcements at either fixed or variable times after the last reinforcement. Two measures were obtained: the number of responses on each schedule, and the time spent responding on each schedule. Ratios of response rates on the two schedules did not equal ratios of reinforcement rates when both schedules were variable nor when one was variable and the other fixed. Ratios of times spent responding approximately equalled ratios of reinforcement rates when both schedules were variable, but did not do so when one was fixed.     

Effects of variations in local reinforcement rate on local response rate in variable interval schedules

Rats trained to lever press for sucrose were exposed to variable-interval schedules in which (i) the probability of reinforcement in each unit of time was a constant, (ii) the probability was high in the first ten seconds after reinforcement and low thereafter, (iii) the probability was low for ten seconds and high thereafter, (iv) the probability increased with time since reinforcement, or (v) the probability was initially zero and then increased with time since reinforcement. All schedules generated similar overall reinforcement rates. A peak in local response rate occurred several seconds after reinforcement under those schedules where reinforcement rate at this time was moderate or high ([i], [ii], and [iv]). Later in the inter-reinforcement interval, local response rate was roughly constant under those schedules with a constant local reinforcement rate ([i], [ii], and [iii]), but increased steadily when local reinforcement rate increased with time since reinforcement ([iv] and [v]). Postreinforcement pauses occurred on all schedules, but were much longer when local reinforcement rate was very low in the ten seconds after reinforcement ([iii]). The interresponse time distribution was highly correlated with the distribution of reinforced interresponse times, and the distribution of postreinforcement pauses was highly correlated with the distribution of reinforced postreinforcement pauses on some schedules. However, there was no direct evidence that these correlations resulted from selective reinforcement of classes of interresponse times and pauses.     

A model for residence time in concurrent variable interval performance

A component-functions model of choice behavior is proposed for performance on interdependent concurrent variable-interval (VI) variable-interval schedules based on the product of two component functions, one that enhances behavior and one that reduces behavior. The model is the solution to the symmetrical pair of differential equations describing behavioral changes with respect to two categories of reinforcers: enhancing and reducing, or excitatory and inhibitory. The model describes residence time in interdependent concurrent VI VI schedules constructed from arithmetic and exponential distributions. The model describes the data reported by Alsop and Elliffe (1988) and Elliffe and Alsop (1996) with a variance accounted for of 87% compared to 64% accounted for by the Davison and Hunter (1976) model and 42% by Herrnstein's (1970) hyperbola. The model can explain matching, undermatching, and overmatching in the same subject under different procedures and has the potential to be extended to performance on concurrent schedules with more than two alternatives, multiple schedules, and single schedules. Thus, it can be considered as an alternative to Herrnstein's quantitative law of effect.     

Momentary maximizing in concurrent schedules with a minimum interchangeover interval

Eight pigeons were trained on concurrent variable-interval variable-interval schedules with a minimum interchangeover time programmed as a consequence of changeovers. In Experiment 1 the reinforcement schedules remained constant while the minimum interchangeover time varied from 0 to 200 s. Relative response rates and relative time deviated from relative reinforcement rates toward indifference with long minimum interchangeover times. In Experiment 2 different reinforcement ratios were scheduled in successive experimental conditions with the minimum interchangeover time constant at 0, 2, 10, or 120 s. The exponent of the generalized matching equation was close to 1.0 when the minimum interchangeover time was 0 s (the typical procedure for concurrent schedules without a changeover delay) and decreased as that duration was increased. The data support the momentary maximizing theory and contradict molar maximizing theories and the melioration theory.     

On the expression and experience of surprise: no evidence for facial feedback, but evidence for a reverse self-inference effect

Three studies examined the effects of experimentally manipulated surprise expressions on the experience of surprise. Surprise was induced by a sudden, unannounced change of the stimulus presentation during a computerized task. Facial expression was manipulated by leading participants to adopt an expression akin to surprise, or by forcing them to look up steeply to a monitor. The expression manipulations had no intensifying effect on the experience of surprise, whereas manipulations of unexpectedness and mental load had strong effects. In addition, mental load was found to affect beliefs about facial expression, suggesting that the participants used their feelings of surprise to infer their probable facial displays. Path analyses supported this reverse self-inference hypothesis.     

Hazard function effects on promoting self-control in variable interval time-based interventions in rats

The present experiments investigated properties of time-based interventions used to increase self-control. Rats received impulsive-choice assessments before and after interventions that consisted of different distributions of delays to reinforcement. In Experiment 1, rats received an intervention with an increasing hazard function where delays were more evenly distributed, a decreasing hazard function where delays were mostly short, or a constant hazard function where delays were exponentially distributed. Surprisingly, rats that received the decreasing hazard function made the most self-controlled choices. Response rates during intervention trials showed that rats anticipated reinforcement based on the shape of the distributions they received. In Experiment 2, rats received an intervention with a decreasing hazard function with a steep slope or a shallow slope. Both time-based interventions increased self-control and produced similar response-rate patterns, indicating that the slope of the decreasing hazard function may not play a strong role in intervention efficacy. While this research aligns with previous literature showing that time-based interventions improved self-control, exposure to short delays produced the biggest improvements. Ultimately, exposure to short delays may increase the subjective value of the larger–later choice while occasional long delays may promote the ability to wait, which may have important implications for translational applications.     

Auditory feedback and memory for music performance: Sound evidence for an encoding effect

Research on the effects of context and task on learning and memory has included approaches that emphasize processes during learning (e.g., Craik & Tulving, 1975) and approaches that emphasize a match of conditions during learning with conditions during a later test of memory (e.g., Morris, Bransford, & Franks, 1977; Proteau, 1992; Tulving & Thomson, 1973). We investigated the effects of auditory context on learning and retrieval in three experiments on memorized music performance (a form of serial recall). Auditory feedback (presence or absence) was manipulated while pianists learned musical pieces from notation and when they later played the pieces from memory. Auditory feedback during learning significantly improved later recall. However, auditory feedback at test did not significantly affect recall, nor was there an interaction between conditions at learning and test. Auditory feedback in music performance appears to be a contextual factor that affects learning but is relatively independent of retrieval conditions.     

Choice behavior in transition: development of preference with ratio and interval schedules.

In Experiment 1, the choice responses of 8 pigeons were observed during 50 periods of transition. Each condition began with equal probabilities of reinforcement on 2 response keys and switched to unequal probabilities. With the ratio of the 2 probabilities held constant, preference for the higher probability developed more rapidly when the 2 probabilities were high than when they were low. In Experiment 2, each condition began with 2 equal variable-interval schedules, but later 1 key delivered 60%, 75%, or 90% of the reinforcers. The rate of approach to asymptotic performance was roughly the same with all 3 reinforcement percentages. These and previous results pose difficulties for some well-known models of acquisition, but the results are well described by a simple model that states that the strength of each response is independently increased by reinforcement and decreased by nonreinforcement.     

Response acquisition under targeted percentile schedules: a continuing quandary for molar models of operant behavior.

The number of responses rats made in a "run" of consecutive left-lever presses, prior to a trial-ending right-lever press, was differentiated using a targeted percentile procedure. Under the nondifferential baseline, reinforcement was provided with a probability of .33 at the end of a trial, irrespective of the run on that trial. Most of the 30 subjects made short runs under these conditions, with the mean for the group around three. A targeted percentile schedule was next used to differentiate run length around the target value of 12. The current run was reinforced if it was nearer the target than 67% of those runs in the last 24 trials that were on the same side of the target as the current run. Programming reinforcement in this way held overall reinforcement probability per trial constant at .33 while providing reinforcement differentially with respect to runs more closely approximating the target of 12. The mean run for the group under this procedure increased to approximately 10. Runs approaching the target length were acquired even though differentiated responding produced the same probability of reinforcement per trial, decreased the probability of reinforcement per response, did not increase overall reinforcement rate, and generally substantially reduced it (i.e., in only a few instances did response rate increase sufficiently to compensate for the increase in the number of responses per trial). Models of behavior predicated solely on molar reinforcement contingencies all predict that runs should remain short throughout this experiment, because such runs promote both the most frequent reinforcement and the greatest reinforcement per press. To the contrary, 29 of 30 subjects emitted runs in the vicinity of the target, driving down reinforcement rate while greatly increasing the number of presses per pellet. These results illustrate the powerful effects of local reinforcement contingencies in changing behavior, and in doing so underscore a need for more dynamic quantitative formulations of operant behavior to supplement or supplant the currently prevalent static ones.     

Undermatching: a reappraisal of performance on concurrent variable-interval schedules of reinforcement

The extant data for pigeons' performance on concurrent variable-interval schedules were examined in detail. Least-squares lines relating relative pecks and time to the corresponding relative reinforcements were obtained for four studies. The between-study group slopes for time and pecks and five of seven within-study group slopes from individual studies were less than 1.00. This suggested the generality that pigeons respond less to the richer reinforcement schedule than predicted by matching. For pecks, a nonparametric test for distribution of points also supported this concept of undermatching (to the richer reinforcement schedule). In addition, using mean squared error as the criterion, a cubic curve fit the peck proportion data better than any line or other polynomial. This indicates that the relation between peck and reinforcement proportions may be nonlinear.     

Preference for simple interval schedules of reinforcement in concurrent chains: Effects of segmentation ratio

A concurrent-chains procedure was used to examine pigeons' preferences between segmented and unsegmented terminal-link schedules of reinforcement. During the initial link, a pair of independent, concurrent variable-interval 60-s schedules was in effect. In the terminal link, reinforcement was provided by a chain fixed-interval fixed-interval schedule on one key and by a simple fixed-interval schedule with an equal interreinforcement interval in the other. The relative duration between the first and second components (segmentation ratio) in the terminal-link chained schedule was systematically varied while the terminal-link duration was kept constant at either 15 s or 30 s in two sets of conditions. With few exceptions, the simple schedule was preferred to the chained schedule. Furthermore, this preference was inversely related to the size of the segmentation ratio in the segmented schedule. When the segmentation ratio was smaller than 1:1, preference was more extreme for a 30-s condition than for a 15-s condition. However, preference decreased more rapidly in conditions with the longer terminal-link duration when the ratio increased. Taken together, these results were consistent with previous findings concerning the effect of the terminal-link duration on choice between segmented and unsegmented schedules. In addition, the data suggested that segmentation ratio in a segmented schedule constitutes another potent factor influencing preference for the unsegmented schedule.     

Techniques for establishing schedules with wheel running as reinforcement in rats.

In three experiments, access to wheel running was contingent on lever pressing. In each experiment, the duration of access to running was reduced gradually to 4, 5, or 6 s, and the schedule parameters were expanded gradually. The sessions lasted 2 hr. In Experiment 1, a fixed-ratio 20 schedule controlled a typical break-and-run pattern of lever pressing that was maintained throughout the session for 3 rats. In Experiment 2, a fixed-interval schedule of 6 min maintained lever pressing throughout the session for 3 rats, and for 1 rat, the rate of lever pressing was positively accelerated between reinforcements. In Experiment 3, a variable-ratio schedule of 20 or 35 was in effect and maintained lever pressing at a very stable pace throughout the session for 2 of 3 rats; for 1 rat, lever pressing was maintained at an irregular rate. When the session duration was extended to successive 24-hr periods, with food and water accessible in Experiment 3, lever pressing settled into a periodic pattern occurring at a high rate at approximately the same time each day. In each experiment, the rats that developed the highest local rates of running during wheel access also maintained the most stable and highest rates of lever pressing.     

Multiple-interval timing in rats: Performance on two-valued mixed fixed-interval schedules

Three experiments studied timing in rats on 2-valued mixed-fixed-interval schedules, with equally probable components, Fixed-Interval S and Fixed-Interval L (FI S and FI L, respectively). When the L:S ratio was greater than 4, 2 distinct response peaks appeared close to FI S and FI L, and data could be well fitted by the sum of 2 Gaussian curves. When the L:S ratio was less than 4, only 1 response peak was usually visible, but nonlinear regression often identified separate sources of behavioral control, by FI S and FI L, although control by FI L dominated. Data were used to test ideas derived from scalar expectancy theory, the behavioral theory of timing, and learning to time.     

Effects of cocaine on performance under fixed-interval schedules with a small tandem ratio requirement

Daily administration of cocaine often results in the development of tolerance to its effects on responding maintained by fixed-ratio schedules. Such effects have been observed to be greater when the ratio value is small, whereas less or no tolerance has been observed at large ratio values. Similar schedule-parameter-dependent tolerance, however, has not been observed with fixed-interval schedules arranging comparable interreinforcement intervals. This experiment examined the possibility that differences in rate and temporal patterning between the two types of schedule are responsible for the differences in observed patterns of tolerance. Five pigeons were trained to key peck on a three-component multiple (tandem fixed-interval fixed-ratio) schedule. The interval values were 10, 30, and 120 s; the tandem ratio was held constant at five responses. Performance appeared more like that observed under fixed-ratio schedules than fixed-interval schedules. Effects of various doses of cocaine given weekly were then determined for each pigeon. A dose that reduced responding was administered prior to each session for 50 days. A reassessment of effects of the range of doses revealed tolerance. The degree of tolerance was similar across components of the multiple schedule. Next, the saline vehicle was administered prior to each session for 50 days to assess the persistence of tolerance. Tolerance diminished in all subjects. Overall, the results suggested that schedule-parameter-dependent tolerance does not depend on the temporal pattern of responding engendered by fixed-ratio schedules.     

Organizational media choice in performance feedback: a multifaceted approach

This study was designed to examine media choice in performance feedback, an organizational communication process that has been overlooked in the literature. Specifically, we adopted a multifaceted approach and examined how contextual variables in feedback, variables identified from media choice theories, and individual differences in media perception influenced selection of a medium to communicate performance information. A policy‐capturing survey was administered to 178 full‐time employees in Hong Kong. Participants read feedback scenarios and provided media preference ratings of face‐to‐face, telephone, electronic mail, and written communication. Results showed that the three facets jointly influenced media choice. Given that communication quality is partly determined by the medium selected, the present work sheds light on the understanding of initiation process in performance feedback.