Reductions in shock frequency and response effort as factors in reinforcement by timeout from avoidance

Rats' presses on one lever canceled shocks programmed after variable cycles, while presses on a second lever occasionally produced a 2-min timout during which the shock-delection schedule was suspended and its correlated stimuli removed. These concurrent schedules of avoidance and timeout were embedded in a multiple schedule whose components differed, within and across conditions, in terms of the programmed shock rate associated with the shock-deletion schedule. Analyses based on the generalized matching law suggest that the reduction in the response requirement correlated with termination of the avoidance schedule was a more important factor in the reinforcing effectiveness of timeout than was shock-frequency reduction, at least in 2 of 3 rats. After training in each condition, responding on the timeout lever was extinguished by withholding timeouts in both components over seven sessions. Resistance to extinction varied directly with the rates of both shock-frequency reduction and avoidance-response reduction experienced during training. Although reduction in response effort appeared to dominate shock-frequency reduction in the maintenance of responding, neither factor had a clear advantage in predicting the course of extinction.     

Human fixed-interval performance with concurrently programmed schedules: A parametric analysis

Young adults pressed a lever for points, exchangeable for money, programmed on concurrent schedules in which one component was a fixed-interval and the other component either a fixed-ratio (Experiment 1) or a differential-reinforcement-of-low-rate (Experiment 2). Two general patterns of fixed-interval responding, postreinforcement pause or constant rate, occurred in both experiments as a function of the parameter values of each component. Also patterns of interaction between the component schedules developed, in which responding or point delivery on one component appeared to be discriminative for responding on the other component. Once a pattern of responding was established, it tended to persist when the parameter values of the schedule were changed. On many schedules, subjects with an experimental history responded differently than did naive subjects, although certain schedule values were resistant to the history effects. The role of verbal strategies in mediating history effects was discussed.     

A parametric evaluation of the hedonic and motoric effects of drugs: pimozide and amphetamine

This study uses a curve-fitting approach to evaluate the effects of drugs on reinforced responding in rats. The subjects obtained reinforcement according to a series of five different variable-interval schedules (a five-component multiple schedule). For each rat, pimozide, a neuroleptic, decreased response rate, and the decrease was associated with (1) a decrease in the estimated asymptotic response rate and (2) an increase in the rate of reinforcement necessary for half-asymptotic responding. That is, pimozide decreased the proportion of responding maintained by a given rate of reinforcement. In contrast, intermediate doses of amphetamine increased response rate and increased the proportion of responding maintained by a given rate of reinforcement. It was proposed that the response rate asymptote indexes motor capacity, and the rate of reinforcement necessary for half-asymptotic responding indexes reinforcement efficacy; accordingly, pimozide decreased motor capacity and reinforcement strength and amphetamine increased reinforcement strength.     

Control rate of response or reinforcement and amphetamine's effect on behavior.

The roles of control response rate and reinforcement frequency in producing amphetamine's effect on operant behavior were evaluated independently in rats. Two multiple schedules were arranged in which one variable, either response rate or reinforcement frequency, was held constant and the other variable manipulated. A multiple differential-reinforcement-of-low-rate seven-second yoked variable-interval schedule was used to equate reinforcement frequencies at different control response rates between multiple-schedule components. Amphetamine increased responding under the variable-interval component. In contrast, amphetamine decreased responding equivalently between components of a multiple random-ratio schedule that produced similar control response rates at different reinforcement frequencies. The results provide experimental support to the rate-dependency principle that control rate of responding is an important determinant of amphetamine's effect on operant behavior.     

Fixed-interval performance: The dynamics of behavior and the interval length

Postreinforcement pauses from successive intervals under various fixed-interval schedules (ranging from 15 seconds to 480 seconds in length) were subjected to lag-1 autocorrelation analysis. Results from both rats and pigeons suggested that there was a consistent tendency for pause values in successive intervals to be weakly positively related. This tendency did not appear to change systematically with interval length and was exhibited both when the reinforcer magnitude was constant and when it was variable at different interval values. The findings do not support suggestions that the dynamic properties of performance under fixed-interval schedules vary systematically with interval length, and are in the opposite direction from some previous findings suggesting that measures of behavior (such as post-reinforcement pause length or number of responses) in successive intervals are inversely related.     

Effects of variations in local reinforcement rate on local response rate in variable interval schedules

Rats trained to lever press for sucrose were exposed to variable-interval schedules in which (i) the probability of reinforcement in each unit of time was a constant, (ii) the probability was high in the first ten seconds after reinforcement and low thereafter, (iii) the probability was low for ten seconds and high thereafter, (iv) the probability increased with time since reinforcement, or (v) the probability was initially zero and then increased with time since reinforcement. All schedules generated similar overall reinforcement rates. A peak in local response rate occurred several seconds after reinforcement under those schedules where reinforcement rate at this time was moderate or high ([i], [ii], and [iv]). Later in the inter-reinforcement interval, local response rate was roughly constant under those schedules with a constant local reinforcement rate ([i], [ii], and [iii]), but increased steadily when local reinforcement rate increased with time since reinforcement ([iv] and [v]). Postreinforcement pauses occurred on all schedules, but were much longer when local reinforcement rate was very low in the ten seconds after reinforcement ([iii]). The interresponse time distribution was highly correlated with the distribution of reinforced interresponse times, and the distribution of postreinforcement pauses was highly correlated with the distribution of reinforced postreinforcement pauses on some schedules. However, there was no direct evidence that these correlations resulted from selective reinforcement of classes of interresponse times and pauses.     

Averaging Effects In The Study Of Fixed-ratio Response Patterns

Three rats were exposed to multiple fixed-ratio schedules in which large and small ratios alternated in an irregular order. Over a series of training phases, one ratio was held constant as the second ratio was increased to higher values. On average, postreinforcement pauses increased in duration as the ratio size was increased. Pausing was controlled by the size of the upcoming ratio; the previous ratio had smaller and less consistent effects. However, more detailed consideration of the aggregated data indicated that the pause distributions were positively skewed and that changes in average performances were more a consequence of increased skew rather than shifts of the entire distributions. Moreover, the distributions of pauses from condition to condition overlapped, and brief pauses were common even at the highest ratios. These results demonstrated that depictions of pausing based on aggregated data can be misleading without corresponding information about variations in the distribution on which the averages are based.     

Falsification of matching theory: changes in the asymptote of Herrnstein''s hyperbola as a function of water deprivation.

Five rats pressed levers on variable-interval schedules of water reinforcement at various levels of water deprivation. In one phase of the experiment, three deprivation conditions that replicated conditions in Heyman and Monaghan (1987) were arranged, along with three less extreme deprivation conditions. In a second phase, water deprivation was arranged so that subjects were exposed to a greater range of access to water per day. Herrnstein's hyperbola described the rats' response-rate data well. The y asymptote, k, of the hyperbola appeared roughly constant over the conditions that replicated those of Heyman and Monaghan, but decreased markedly when less extreme deprivation conditions were included. In addition, k varied systematically when the second method of arranging deprivation was used. These results falsify a strong form of matching theory and confirm predictions made by linear system theory.     

Behavioral economics and within-session changes in responding

Pigeons and rats responded on fixed-ratio schedules with requirements ranging from 5 to 120 responses. Consistent with past results from several schedules and procedures, responding usually changed systematically within experimental sessions. The within-session changes were usually larger and were less symmetrical around the middle of the session for schedules that provided higher, rather than lower, rates of reinforcement. These results suggest that similar variables contribute to within-session changes in responding under different schedules. When an economic demand function was fit to the data, the intensity and elasticity of demand for food and the percentage of the variance accounted for decreased within sessions, although the trend for elasticity did not reach statistical significance for pigeons. These results suggest that relatively short sessions should be used to study economic demand in open economies and that demand may differ at different times in a session and in sessions of different lengths. Within-session changes in intensity, but not necessarily elasticity, of demand are consistent with behavioral economic theories.     

Extinction of responding maintained by timeout from avoidance

The resistance to extinction of lever pressing maintained by timeout from avoidance was examined. Rats were trained under a concurrent schedule in which responses on one lever postponed shock on a free-operant avoidance (Sidman) schedule (response-shock interval = 30 s) and responses on another lever produced 2 min of signaled timeout from avoidance on a variable-ratio 15 schedule. Following extended training (106 to 363 2-hr sessions), two experiments were conducted. In Experiment 1 two different methods of extinction were compared. In one session, all shocks were omitted, and there was some weakening of avoidance but little change in timeout responding. In another session, responding on the timeout lever was ineffective, and under these conditions timeout responding showed rapid extinction. The within-session patterns produced by extinction manipulations were different than the effects of drugs such as morphine, which also reduces timeout responding. In Experiment 2 shock was omitted for many consecutive sessions. Response rates on the avoidance lever declined relatively rapidly, with noticeable reductions within 5 to 10 sessions. Extinction of the timeout lever response was much slower than extinction of avoidance in all 4 rats, and 2 rats continued responding at baseline levels for more than 20 extinction sessions. These results show that lever pressing maintained by negative reinforcement can be highly resistant to extinction. The persistence of responding on the timeout lever after avoidance extinction is not readily explained by current theories.