INFLUENCES OF RESPONSE RATE AND DISTRIBUTION ON THE CALCULATION OF INTEROBSERVER RELIABILITY SCORES

We examined the effects of several variations in response rate on the calculation of total, interval, exact‐agreement, and proportional reliability indices. Trained observers recorded computer‐generated data that appeared on a computer screen. In Study 1, target responses occurred at low, moderate, and high rates during separate sessions so that reliability results based on the four calculations could be compared across a range of values. Total reliability was uniformly high, interval reliability was spuriously high for high‐rate responding, proportional reliability was somewhat lower for high‐rate responding, and exact‐agreement reliability was the lowest of the measures, especially for high‐rate responding. In Study 2, we examined the separate effects of response rate per se, bursting, and end‐of‐interval responding. Response rate and bursting had little effect on reliability scores; however, the distribution of some responses at the end of intervals decreased interval reliability somewhat, proportional reliability noticeably, and exact‐agreement reliability markedly.     

USING CALIBRATION AND INTEROBSERVER AGREEMENT ALGORITHMS TO ASSESS THE ACCURACY AND PRECISION OF DATA FROM ELECTRONIC AND PEN‐AND‐PAPER CONTINUOUS RECORDING METHODS

Often it is assumed that electronic recording by observers necessarily provides better quality data than pen‐and‐paper methods. Fifteen novice observers recorded rates of responding from 10 role‐played video samples using one of three continuous recording input formats: keyboard (laptop), touchscreen (personal digital assistants), or pen‐and‐paper. We evaluated the quality of the observers' data compared with criterion records using calibration and interobserver agreement algorithms. Results of the calibration analysis revealed that observers in the touchscreen group produced the most consistently accurate and precise data, the keyboard group observers showed wide variation in precision and accuracy, and the pen‐and‐paper group observers were significantly less precise than the touchscreen group. We conclude that although electronic recording has the potential to be as accurate as, and more precise than, pen‐and‐paper methods, this is far from guaranteed. Analyses of observers' errors advise recommendations for improving data accuracy and precision when using each method. Copyright © 2014 John Wiley & Sons, Ltd.     

CONTINUOUS RECORDING AND INTEROBSERVER AGREEMENT ALGORITHMS REPORTED IN THE JOURNAL OF APPLIED BEHAVIOR ANALYSIS (1995–2005)

We reviewed all research articles in 10 recent volumes of the Journal of Applied Behavior Analysis (JABA): Vol. 28(3), 1995, through Vol. 38(2), 2005. Continuous recording was used in the majority (55%) of the 168 articles reporting data on free‐operant human behaviors. Three methods for reporting interobserver agreement (exact agreement, block‐by‐block agreement, and time‐window analysis) were employed in more than 10 of the articles that reported continuous recording. Having identified these currently popular agreement computation algorithms, we explain them to assist researchers, software writers, and other consumers of JABA articles.     

A comparison of frequency, interval, and time-sampling methods of data collection

Data representing high, medium, and low response rates in constant and nonconstant patterns were generated by electromechanical equipment to determine whether the same data collected by time-sampling, interval recording, and frequency recording would be represented similarly by each method. Results indicated: (1) that time-sampling provided an extremely inaccurate estimate of responding, and (2) that interval recording accurately represented responding of low and medium rates, but grossly underestimated high-rate responding.     

Rates of responding measured by continuous recording in applied behavioral research

There is no published source that can inform behavioral measurement research about typical response rates measured by continuous recording in applied behavior analysis. The present study was designed to determine the distribution of data on rate of responding reported across a recent 10‐volume (1998–2007) sample of research articles from the Journal of Applied Behavior Analysis. Rate of responding was measured for 20 553 datapoints in 360 sets of data (one participant in one graph) presented in 60 research articles. The majority of datapoints showed response rates of less than 1.0 response per minute, with 28.3% at zero. The highest rate reported was 104.5 responses per minute. The maximum response rate in datasets was more than 1.0 response per minute in 96.7% of datasets, indicating that continuous recording has been employed in research in which a relatively high rate of responding was recorded at some time during the study. The data from this report will assist researchers to determine empirically the representativeness of response rates in studies of measurement in applied behavior analysis. Copyright © 2010 John Wiley & Sons, Ltd.     

Factors influencing responding under multiple schedules of conditioned and unconditioned reinforcement

Two experiments examined pigeons' responses under multiple schedules of conditioned and unconditioned reinforcement. In one component, responses produced food according to a fixed-interval schedule; in a second component, responses produced brief stimuli according to a fixed-ratio schedule. When brief-stimulus presentations were paired with food in the first component, rates in the second component were usually higher than 10 responses per minute. When pairing in the first component was eliminated, responding continued to be maintained in the second component. Elimination of food presentation from the first component substantially decreased responding in the second component, even though the brief stimulus had not been paired with food. Experiment II demonstrated that response rate was affected by the duration of both the second component and the brief stimulus. The results suggest that three conditions are important in maintaining responding with brief-stimulus presentations: (1) pairing the brief stimulus, at least initially, with food, (2) maintaining unconditioned reinforcement in one component, and (3) employing optimal brief-stimulus and component durations.     

Temporal properties of responding during stimuli that precede response-independent food

Pigeons' keypecks were reinforced with grain on the average of once per minute by schedules that maintained low response rates and by schedules that maintained high response rates. During these schedules, a fixed-duration conditioned stimulus (CS) ranging from 7.5 to 120 sec in duration across conditions terminated with response-independent food. Response rates during the CS were inversely related to CS duration. The rates and the temporal patterns of responding during the shortest CS were similar whether the ongoing schedule maintained high response rates or low response rates. As CS duration increased, the rate and pattern of responding during the CS converged on the rate and pattern of responding maintained by the baseline schedule. These data indicate that changes in responding during stimuli that signal response-independent reinforcement are not homogeneous throughout the CS; that response measures, such as “suppression ratios”, which presume homogeneity may mislead us; and that conditioned suppression and conditioned enhancement may be better talked about in terms of species-specific approach and avoidance than in terms of emotional states.     

DIFFERENTIALLY REINFORCING LOW RATES OF MISBEHAVIOR WITH NORMAL ELEMENTARY SCHOOL CHILDREN1

Effective nonpunitive procedures for reducing counterproductive classroom behaviors are of potential benefit to both students and teachers. A recent strategy for dealing with this class of problem behaviors involves the reinforcement of acceptably low levels of such behavior. The laboratory version of this procedure, called differential reinforcement of low rates of responding (or DRL), provides for a reinforcer to be delivered contingent upon a response that is separated from the last preceding response by a minimum amount of time. To make this procedure more amenable to classroom use, the present authors have modified it so that a reinforcer is delivered if fewer than a specified number of responses occur within a preset time interval (Deitz and Repp, Journal of Applied Behavior Analysis, 1973, 6 , 457–463). Previous studies using this procedure have found it effective in reducing and maintaining low rates of targeted behaviors. However, these effects have been demonstrated with groups of subjects and/or individuals from dependent populations. The present study investigated use of this modified DRL procedure with individual students in normal elementary classrooms. In the first of three studies, “talk-outs” of an 11-yr-old fifth-grade male were reduced when nonexchangeable gold stars were made contingent on two or fewer responses per session. During baseline sessions, an average of 4.45 talkouts were observed per 45-min session. Average responding subsequently fell to 1.83 when the modified DRL contingency was applied, increased to 7.60 during a reversal phase, and dropped again to an average of 1.20 when the contingency was reapplied. In the second study, out-of-seat behavior of a 12-yr-old sixth-grade female was reduced when gold stars were made contingent on two or fewer responses per 45-min class period. Baseline responding averaged 6.10 responses per session. When the contingency was applied, average responding fell to 0.16. During the reversal period, responding increased to an average of 6.00 and fell again, after the contingency was re-introduced to an average of 0.40. In the third study, a reduction in both talking-out and out-of-seat behaviors of another 11-yr-old fifth-grade male was demonstrated with a multiple-baseline design. Using different lengths of baselines, gold stars were made contingent first on a low rate of out-of-seat behavior, and then on a low rate of talk-outs. Out-of-seat responding fell from a baseline average of 7.50 to a treatment average of 1.14. Talk-outs went from a baseline average of 4.66 to a treatment average of 1.14. In all three studies, the modified DRL procedure proved effective with the children and was manageable by the classroom teacher. For the students, nonexchangeable conditioned reinforcers (stars) were sufficient to maintain lowered rates of inappropriate behavior with the modified DRL schedule; there was no need for an elaborate token economy, a process that in many cases may be only a form of behavioral “overkill”. As in other studies investigating DRL schedules, students were not informed of their accumulation of responses; the differential effects of providing or withholding this feedback need to be investigated. Overall, these studies add single-subject replication with normal children to the literature on modified DRL procedures.     

Within-session changes in key and lever pressing for water during several multiple variable-interval schedules

Rats pressed keys or levers for water reinforcers delivered by several multiple variable-interval schedules. The programmed rate of reinforcement varied from 15 to 240 reinforcers per hour in different conditions. Responding usually increased and then decreased within experimental sessions. As for food reinforcers, the within-session changes in both lever and key pressing were smaller, peaked later, and were more symmetrical around the middle of the session for lower than for higher rates of reinforcement. When schedules provided high rates of reinforcement, some quantitative differences appeared in the within-session changes for lever and key pressing and for food and water. These results imply that basically similar factors produce within-session changes in responding for lever and key pressing and for food and water. The nature of the reinforcer and the choice of response can also influence the quantitative properties of within-session changes at high rates of reinforcement. Finally, the results show that the application of Herrnstein's (1970) equation to rates of responding averaged over the session requires careful consideration.     

EFFECTS OF PREFERENCE AND REINFORCER VARIATION ON WITHIN-SESSION PATTERNS OF RESPONDING

We examined correspondence between preference assessment outcome and within‐session patterns of responding in one subject with autism. Responding maintained by a single highly preferred item resulted in a greater total number of responses, a slower decline in within‐session response rates, and a greater proportion of short interresponse times compared to responding maintained by varied moderately preferred (MP) stimuli. Presenting varied MP stimuli within the same session produced greater levels and more sustained responding than presenting those same stimuli individually.     

Recording supervision: Educational,therapeutic, and enhances the supervisory working alliance?

Aims: To explore the impact of listening to an audio‐recording of their latest supervision session on supervisees. Method: Participants recorded a supervision session, listened to it and were interviewed about this experience of listening to it within seven days of listening to the recording. The phenomenological interviews informed by the researcher's knowledge were analysed using grounded theory. Participants: Fifteen participants with 0–20 years’ experience, representing person‐centred, CBT, psychodynamic and integrative approaches, and qualifications ranging from student to doctorate formed the theoretical sample. Findings: Listening to the audio‐recording allowed participants to re‐experience their thoughts and emotions. Participants recalled and re‐experienced events which had been forgotten, unnoticed or (arguably) repressed. This re‐experiencing was usually less emotive than in real‐time; but suppressed emotions could be more intense. As participants integrated both these noticed and unnoticed events, they were able to accept what had been unacceptable, gaining self‐awareness, which was potentially therapeutic. Noticing what had been unnoticed permitted insights into many aspects of therapy and supervision, which was more than just ‘reflection‐on‐action’. Conclusion: Listening to an audio‐recording of the latest supervision may be educational for any supervisee; it may be therapeutic by facilitating challenge of maladaptive beliefs and behaviours; and it also seems to strengthen the supervisory working alliance.     

Effects of response rate, reinforcement frequency, and the duration of a stimulus preceding response-independent food

Food-reinforced key pecking in the pigeon was maintained under a four-component multiple schedule. In two components, responding was maintained at high rates under a random-ratio schedule. In the other two components, responding was maintained at low rates under a schedule that specified a minimum interresponse time. For both high and low response rates, one of the schedule components was associated with a high reinforcement frequency and the other components with a lower reinforcement frequency. During performance under these schedules, a stimulus terminated by access to response-independent food was periodically presented. The duration of this pre-food stimulus was 5, 30, 60, or 120 sec. Changes in rate of key pecking during the pre-food stimulus were systematically related to baseline response rate and the duration of the stimulus. Both high and low response rates were increased during the 5-sec stimulus. At longer stimulus durations, low response rates were unaffected and high response rates were decreased during the stimulus. For two of three pigeons, high response rates maintained under a lower frequency of reinforcement tended to be decreased more than high response rates maintained under a higher reinforcement frequency. In general, the magnitude of decrease in high response rates was inversely related to the duration of the pre-food stimulus.     

Effects of cocaine on fixed-interval responding reinforced by the opportunity to run

Rate-dependent drug effects have been observed for operant responding maintained by food, water, heat, light onset, electrical brain stimulation, shock-stimulus termination, and shock presentation. The present study sought to determine if the effects of cocaine on lever pressing maintained by the opportunity to run could also be described as rate dependent. Seven male Wistar rats were trained to respond on levers for the opportunity to run in a wheel. The schedule of reinforcement was fixed-interval 60 s, and the reinforcing consequence was the opportunity to run for 60 s. On this schedule, overall rates of responding were low, usually below six presses per minute, and pauses frequently exceeded the 60-s interval. Despite these differences, an overall scalloped pattern of lever pressing was evident for each rat. Doses of 1, 2, 4, 8, and 16 mg/kg cocaine were administered 10 min prior to a session. Only at the 16 mg/kg dose did the responding of the majority of rats change in a manner suggestive of a rate-dependent drug effect. Specifically, lower response rates at the beginning of the intervals increased and higher rates at the end of the intervals decreased, as indicated by the fact that slopes from the regression of drug rates on control rates decreased. These data provide tentative support for the generalization of rate-dependent effects to operant responding maintained by wheel running. Differences in the baseline performance maintained by wheel running compared to those for food and water point to the need for further experimentation before this effect can be firmly established.     

Contingency tracking during unsignaled delayed reinforcement: Effects of delay duration and d‐amphetamine

In two experiments, the role of the response–reinforcer relation in maintaining low‐rate responding under unsignaled delay conditions was investigated. In both experiments pecking by pigeons on one response key, denoted the relevant key, was reinforced under an unsignaled delay‐of‐reinforcement procedure (defined as tandem variable‐interval (VI) differential‐reinforcement‐of‐other behavior [DRO] schedule). Responding on a second key, denoted the irrelevant key, had no programmed consequences. Between sessions, the location of the relevant key varied (after one, two, or three sessions) pseudorandomly. In Experiment 1, the delay (DRO) duration was manipulated parametrically. Overall, proportional relevant‐key response rates (relevant‐key response rates / [relevant‐key response rates + irrelevant key response rates]) increased across 3‐session sequences in which the relevant key remained in the same location and decreased as the DRO duration was changed systematically (2, 5, and 10 s). In Experiment 2, acute administration of d‐amphetamine increased proportional relevant‐key response rates during 1‐day sequences for only the DRO 5‐s duration, and results over 3‐day sequences, once a discrimination had already been established, were inconsistent. Results support that the response–reinforcer relation is the primary determinant of responding, and such discriminations are relatively resistant to disruption or potentiation by behaviorally active doses of d‐amphetamine.     

The automatic recording of aggression in pigeons: A comparison of two methods with live targets

Three pigeons were exposed to a fixed-time response independent food-delivery schedule and a live target pigeon restrained in a holder mounted on a spring and microswitch assembly. This common method of recording aggression was compared with a photocell system, and both were evaluated by observation of video-tape recordings. Dependent variables included the number of interfood intervals with an attack, attacks per minute, and attack duration. The photocell proved more reliable than the microswitch and correlated highly with observations of both the number of interfood intervals with an attack for three subjects and attack duration for two. Neither apparatus provided accurate information about the rate of attacks. The microswitch apparatus was not sensitive to changes in the subject’s attack topography, while both recording devices were susceptible to activation by responses in the attacking pigeon other than discrete pecks or physical blows. In view of these findings, attacks per minute may not be an appropriate measure of aggression in studies using such devices.     

Reduction of shock duration as negative reinforcement in free-operant avoidance

Rats were trained on a free-operant procedure in which shock duration was controlled by responses within a limited range of interresponse times. Shocks of 1.6-mA intensity occurred randomly with average density of 10 shocks per minute. As long as interresponse times were 15 seconds or less, any shocks received were at the briefer of two durations (.3 second). Whenever interresponse times exceeded 15 seconds, any shocks received were at the longer duration (1.0 second). For six of eight animals, avoidance responding developed quickly and reached levels of better than 90%. Four yoked animals stopped responding within the first few sessions. Shock duration reduction without change in shock probability or intensity was sufficient for the acquisition and maintenance of avoidance responding.     

Comparison of drug effects on fixed-ratio performance and chain performance maintained under a second-order fixed-ratio schedule.

In one component of a multiple schedule, pigeons were required to complete the same four-response chain each session by responding sequentially on three identically lighted keys in the presence of four successively presented colors (chain performance). Food presentation occurred after five completions of the chain (i.e., after 20 correct responses). Errors, such as responding on the center or right key when the left was designated correct, produced a brief timeout but did not reset the chain. In the other component, responding on a single key (lighted white) was maintained by food presentation under a fixed-ratio 20 schedule. In general, phencyclidine and d-amphetamine produced dose-dependent decreases in the overall response rates in both components. With pentobarbital, overall rate in each component generally increased at intermediate doses and decreased at higher doses. All three drugs produced dose-dependent disruptive effects on chain-performance accuracy. Phencyclidine and pentobarbital increased percent errors at doses that had little or no rate-decreasing effects, whereas d-amphetamine generally increased percent errors only at doses that substantially decreased overall rate. At high doses, all three drugs produced greater disruption of chain performance than of fixed-ratio performance, as indicated by a slower return to control responding, although the effects of d-amphetamine were less selective than those of phencyclidine or pentobarbital.     

A review of the observational data-collection and reliability procedures reported in The Journal of Applied Behavior Analysis

The research published in the Journal of Applied Behavior Analysis (1968 to 1975) was surveyed for three basic elements: data-collection methods, reliability procedures, and reliability scores. Three-quarters of the studies reported observational data. Most of these studies' observational methods were variations of event recording, trial scoring, interval recording, or time-sample recording. Almost all studies reported assessment of observer reliability, usually total or point-by-point percentage agreement scores. About half the agreement scores were consistently above 90%. Less than one-quarter of the studies reported that reliability was assessed at least once per condition.