A comparison of performance feedback procedures on teachers' treatment implementation integrity and students' inappropriate behavior in special education classrooms

This study examined the extent to which treatment integrity of 4 special education teachers was affected by goal setting, performance feedback regarding student or teacher performance, and a meeting cancellation contingency. Teachers were trained to implement function-based treatment packages to address student problem behavior. In one condition, teachers set a goal for student behavior and received daily written feedback about student performance. In a second condition, teachers received daily written feedback about student performance as well as their own accuracy in implementing the intervention and would be able to avoid meeting with a consultant to practice missed steps by implementing the intervention with 100% integrity. This latter package increased treatment integrity the most above baseline levels. Higher levels of treatment integrity were significantly correlated with lower levels of student problem behavior for 3 of the 4 teacher-student dyads. Three of the 4 teachers also rated both feedback procedures as highly acceptable. Implications for increasing and maintaining treatment integrity by teachers via a consultation model are discussed.     

Effects of immediate performance feedback on implementation of behavior support plans

Research has focused on increasing the treatment integrity of school-based interventions by utilizing performance feedback. The purpose of this study was to extend this literature by increasing special education teachers' treatment integrity for implementing antecedent and consequence procedures in an ongoing behavior support plan. A multiple baseline across teacher-student dyads (for two classrooms) design was used to evaluate the effects of performance feedback on the percentage of antecedent and consequence components implemented correctly during 1-hr observation sessions. Performance feedback was provided every other week for 8 to 22 weeks after a stable or decreasing trend in the percentage of antecedent or consequence components implemented correctly. Results suggested that performance feedback increased the treatment integrity of antecedent components for 4 of 5 teachers and consequence components for all 5 teachers. These results were maintained following feedback for all teachers across antecedent and consequence components. Teachers rated performance feedback favorably with respect to the purpose, procedures, and outcome, as indicated by a social validity rating measure.     

Enhancing Treatment Integrity Maintenance Through Fading with Indiscriminable Contingencies

School psychologists are often asked to develop treatment to remediate students’ academic skills or social behavior problems. When teachers implement treatment recommendations with high levels of treatment integrity, students benefit. Treatment integrity has been enhanced by use of direct training, performance feedback, and negative reinforcement procedures. Still, maintenance of treatment integrity has received limited systematic inquiry. This study investigated maintenance of treatment integrity over time by utilizing a systemic fading procedure with indiscriminable contingencies. The results showed that the teachers continued to implement treatment protocols at a high level after performance feedback was faded. Students also fared better when treatment integrity was high.     

Effects of video modeling on treatment integrity of behavioral interventions

We examined the effects of individualized video modeling on the accurate implementation of behavioral interventions using a multiple baseline design across 3 teachers. During video modeling, treatment integrity improved above baseline levels; however, teacher performance remained variable. The addition of verbal performance feedback increased treatment integrity to 100% for all participants, and performance was maintained 1 week later. Teachers found video modeling to be more socially acceptable with performance feedback than alone, but rated both positively.     

IMPROVING INTERVENTION INTEGRITY OF DIRECT‐SERVICE PRACTITIONERS THROUGH PERFORMANCE FEEDBACK AND PROBLEM SOLVING CONSULTATION

The purpose of this study was to improve intervention integrity of behavior support plan implementation by teachers at a specialized school. During the first phase of a multiple baseline design, performance feedback was given to the teachers during (correction) and immediately following (praise) classroom observations. In a second phase, the teachers received problem solving consultation that addressed behavior support plan misapplication and respective procedural modifications. Only one of the three teachers demonstrated improved intervention integrity with performance feedback, whereas problem solving consultation produced uniform improvement. We discuss these findings and implications for intervention integrity assessment and research. Copyright © 2014 John Wiley & Sons, Ltd.     

Behavioral skills training in scent detection research: Interactions between trainer and animal behavior

Animal trainers working in scent detection programs are responsible for arranging training contingencies as well as for observing and recording animal behavior. We provided behavioral skills training (BST) to animal trainers working with scent detection rats to improve the treatment integrity of scent‐detection research sessions. We evaluated the trainers' behavior at baseline and during the sequential introduction of each component of BST (instructions, modeling, and feedback). We observed incremental improvements in treatment integrity with the introduction of each BST component. Posttraining probes revealed that these improvements were sustained at least 3 weeks post‐BST. As the trainers' behavior was modified during BST, we observed decrements in measures of rat performance. We discuss the nature of these interactions and their implications for the use of BST in scent detection research and in situations in which intervention with one party produces concomitant effects on the behavior of another.     

Effects of Varying Levels of Treatment Integrity on Toddler Aggressive Behavior

Abstract

Effects of different levels of treatment integrity for a time-out intervention in relation to the aggressive behavior of an 18-month-old were investigated. The time-out intervention was composed of four steps and treatment integrity was systematically varied at 50%, 25%, 75%, and 100%. The results suggest that the 75% and 100% treatment integrity conditions were most effective for decreasing aggressive behavior. Further, there were only minor differences in aggressive behavior between the 25% and 50% treatment integrity conditions. Discussion focuses on the need for assessing optimal and minimum levels of treatment integrity for producing behavior change and the implications for classroom-based interventions.     

Discrete‐trial training performance of behavior interventionists with autism spectrum disorder: A systematic replication and extension

Further research is needed on strategies to improve employment outcomes for individuals with autism spectrum disorder (ASD). We replicated and extended prior research by examining the acquisition, maintenance, and generalization of discrete‐trial training (DTT) performance of adults with ASD who were interested in careers as behavior technicians. Three participants received training on how to implement DTT with children and we assessed the maintenance and generalization of DTT performance in the absence of feedback. Results indicated that procedural integrity generalized across children and targets and maintained for 6 to 17 session days without feedback. Results have implications for the successful employment of individuals with ASD as behavior technicians and for future research on strategies to enhance their performance on the job.     

AN EVALUATION OF SELF‐MONITORING TO IMPROVE SWIMMING PERFORMANCE

This study evaluated the effects of self‐monitoring by swimmers to improve their performance and assessed the effects of reactivity to observer presence on their performance. Seven public high school swim team members self‐monitored their behavior on dry erase boards to increase the number of assigned laps completed at practice. Reactivity to observer presence was assessed as a confederate, recorded the number of laps completed during observer absent conditions. The percentage of assigned laps completed increased during the self‐monitoring phases. The self‐monitoring and feedback phase further increased the percentage of assigned laps completed and increased the participants' mean level of reporting accuracy. The effects from the reactivity assessment were mixed across participants. Further research is needed to examine reactivity effects in sport settings. Copyright © 2014 John Wiley & Sons, Ltd.     

Teacher Use Of Interventions In General Education Settings: Measurement And Analysis Of The Independent Variable

This study examined the integrity with which 4 general education teachers implemented an intervention designed to improve the academic performance of elementary school students. Treatment integrity was measured daily using permanent products. The results showed that the 4 teachers markedly increased the integrity of the delivered treatment when they were provided with performance feedback. The results suggest that academic performance improved for the group of students as integrity improved.     

Using performance feedback to enhance implementation fidelity of the problem-solving team process

Implementation integrity is a potentially critical issue for problem-solving teams (PST) and most response-to-intervention models. The current study hypothesized that providing performance feedback, which has consistently been shown to increase implementation integrity, to PSTs would enhance the procedural integrity of the process. The PSTs for three elementary schools were provided performance feedback with a 20-item checklist created from the literature. A multiple-baseline design across schools revealed an immediate change in level after providing performance feedback. The resulting percentages of non-overlapping data were 90.9%, 90.0%, and 100%. However, PSTs still did not monitor student progress, assess the effectiveness of the intervention, or measure the integrity with which the intervention was implemented even after receiving feedback. Thus, providing performance feedback could be a method to increase the fidelity with which critical components of data-based problem-solving are implemented, but these data suggest the need for additional research.     

Effect of video self‐monitoring on procedural integrity

A video self‐monitoring treatment package was used to evaluate the procedural integrity of staff implementing behavioral guidelines for one child with autism. Staff members with low procedural integrity scores were asked to participate. The treatment package incorporated an implementation video which reviewed the relevant behavioral guidelines and a procedural integrity monitoring system. Each participant was taught to score a pre‐training video, which highlighted components of implementing behavioral guidelines, with a procedural integrity tool denoting correct implementation of the procedures. The participant then scored their own baseline video using a similar tool constructed to highlight the target student's behavior plan. Participant scores were compared to experimenter scores and verbal feedback was given by the experimenter if the scores varied. Results showed increased procedural integrity implementing the behavior guidelines from baseline to 100% implementation for two out of three participants. Copyright © 2010 John Wiley & Sons, Ltd.     

Teacher Implementation of Trial-Based Functional Analysis and Differential Reinforcement of Alternative Behavior for Students with Challenging Behavior

The purpose of this study was to examine the effects of a training package on three middle school special education teachers’ accurate implementation of trial-based functional analysis (TBFA) and differential reinforcement of alternative behavior (DRA) with their students with autism spectrum disorders or emotional and behavioral disorders in the classroom setting. We also examined the effects of the DRA on six students’ challenging and replacement behaviors. Using a multiple probe across teacher participants design, we demonstrated that all three teachers implemented TBFA during TBFA training with performance feedback conditions and DRA during DRA training with performance feedback conditions with high procedural fidelity. Further, two of three teachers generalized high procedural fidelity with a new student. Finally, results indicated a reduction in all six students’ challenging behavior and an increase in their replacement behavior after the teachers received training and performance feedback on DRA implementation. Limitations and implications for research and practice are discussed.     

Effects of public feedback during RTI team meetings on teacher implementation integrity and student academic performance

As a result of the new revision of IDEA (2004), models of early intervention and response-to-intervention (RTI) have received a great deal of attention in the literature. Although various tiered models have been described in detail, one aspect of RTI that has received little empirical attention is the need to ensure integrity of intervention as part of a team decision making process. One method that has good support for improving treatment integrity is performance feedback (PFB); however, the utilization of PFB within a team context has received very little attention in the literature. Experiment 1 evaluated the effect of PFB delivered within the context of an RTI team on treatment integrity after fidelity had fallen to unacceptable levels. Experiment 2 evaluated the effects of PFB on the maintenance of treatment integrity prior to integrity fall off. Results indicate that PFB can be used as an efficient means of improving or maintaining treatment integrity when applied within a team model.     

Reactivity to a laboratory stressor among individuals with bipolar I disorder in full or partial remission

Strong links have been documented between life events and the course of bipolar disorder. Laboratory studies of stress provide an opportunity to understand the mechanisms involved in reactivity to stressors, but few such studies have been done in the bipolar field. In the current study, 28 people with bipolar I disorder in full or partial remission and 40 people with no history of a mood disorder were randomly assigned to 1 of 3 conditions involving different levels of failure feedback on a concept formation task. Confidence, affective reactions, and performance on a subsequent anagram task were assessed. Results provide tentative support for reactivity to the stressor among the bipolar group, although reactivity was limited to impaired anagram performance.     

EFFECTS OF TREATMENT INTEGRITY FAILURES DURING DIFFERENTIAL REINFORCEMENT OF ALTERNATIVE BEHAVIOR: A TRANSLATIONAL MODEL

Differential reinforcement of alternative behavior (DRA) is used frequently as a treatment for problem behavior. Previous studies on treatment integrity failures during DRA suggest that the intervention is robust, but research has not yet investigated the effects of different types of integrity failures. We examined the effects of two types of integrity failures on DRA, starting with a human operant procedure and extending the results to children with disabilities in a school setting. Human operant results (Experiment 1) showed that conditions involving reinforcement for problem behavior were more detrimental than failing to reinforce appropriate behavior alone, and that condition order affected the results. Experiments 2 and 3 replicated the effects of combined errors and sequence effects during actual treatment implementation.     

Does Treatment Integrity Matter? A Preliminary Investigation of Instructional Implementation and Mathematics Performance

This study examined the impact of three levels of treatment integrity on students' responding on mathematics tasks. Instruction was provided separately for addition and subtraction to six second-grade students who were referred due to poor performance with these operations. The treatment consisted of a computer delivered delayed prompt to use a counting strategy to solve problems, accuracy feedback, and intermittent presentation of animated praise sequences. Instruction was presented via computerized instruction to assure precise delivery of the varying levels of treatment integrity. The study examined the delivery of prompts for all, two-thirds, and one-third of instructional trials. Continuous delivery of instructional prompts was the most effective treatment or one of the most effective treatments for all participants. Lesser levels of prompt implementation were associated with poorer outcomes for the majority of the students. The implications of these findings for continuing research regarding the impact of reduced treatment integrity on student outcomes are discussed.     

Modifying Students' Classroom Behaviors Using an Electronic Daily Behavior Report Card

This study examined the effects of e-mailed daily behavior report cards (DBRC) on students' disruptive classroom behaviors. Additionally, teacher acceptability of e-mailed DBRC as an intervention was assessed. Participants included 46 elementary students (37 males and 9 females), that were assigned to one of three conditions; delayed treatment control, e-mailed DBRC, and e-mailed DBRC with performance feedback. Student behaviors were measured by direct observations and teacher ratings. Results suggest that e-mailed DBRC can decrease students' disruptive behaviors, that e-mailed DBRC with performance feedback did not produce superior outcomes, and that e-mailed DBRC was perceived as acceptable.     

Effects of Feedback Complexity on Dynamic Decision Making

Prior research shows people suffer from misperceptions of feedback, generating systematic dysfunctional behavior in the presence of dynamic complexity - settings with multiple feedback loops, time delays, and nonlinearities. However, prior work has not adequately mapped the effect of these elements of complexity on performance. We report an experiment where subjects managed an inventory in the face of stochastic sales, a classic dynamic decision task. We vary the time delays and strength of the feedback loops to explore the impact of these elements of dynamic complexity on behavior. Subjects faced financial incentives and had opportunities to learn. Yet performance was significantly worse than optimal across all conditions. Subjects outperformed a naive "do-nothing" rule in the simple conditions, but performance deteriorated dramatically with increasing time delays and feedback effects, and most were outperformed by the do-nothing rule in the complex conditions. Regression analysis of subjects′ decisions showed most ignored the supply line of pending production and undercontrolled the system. Undercontrol increased significantly with growing time delays and feedback strength, showing subjects were insufficiently adaptive despite perfect knowledge of system structure and parameters. Subjects′ understanding of complex feedback settings declines as delays between cause and effect increase and as actions have stronger side effects. Few indications were found of active experimentation or learning: the need to control seemed to override the ability to learn.     

Effects of treatment‐integrity failures on a response‐cost procedure

Effects of incorrect or partial implementation (poor treatment integrity) on response cost are largely unknown. We evaluated reduced treatment integrity during response cost on rates of 2 concurrently available responses. College students earned points by clicking on either a black circle or a red circle on a computer screen. Experiment 1 compared 2 types of treatment‐integrity failures (omission and commission errors) across 2 levels of integrity (20% and 50%). Compared to 100% integrity conditions, omission errors did not suppress responding to the same extent, and commission errors reduced target responding but also decreased rates of alternative behavior. Experiment 2 compared the effects of 20% and 50% omission errors within subjects. Implementation at 50% integrity adequately suppressed responding, but treatment effects were lost at 20% integrity. There may be a critical level at which response cost must be implemented to suppress responding, which has important implications for application.