A comparison of performance feedback procedures on teachers' treatment implementation integrity and students' inappropriate behavior in special education classrooms

This study examined the extent to which treatment integrity of 4 special education teachers was affected by goal setting, performance feedback regarding student or teacher performance, and a meeting cancellation contingency. Teachers were trained to implement function-based treatment packages to address student problem behavior. In one condition, teachers set a goal for student behavior and received daily written feedback about student performance. In a second condition, teachers received daily written feedback about student performance as well as their own accuracy in implementing the intervention and would be able to avoid meeting with a consultant to practice missed steps by implementing the intervention with 100% integrity. This latter package increased treatment integrity the most above baseline levels. Higher levels of treatment integrity were significantly correlated with lower levels of student problem behavior for 3 of the 4 teacher-student dyads. Three of the 4 teachers also rated both feedback procedures as highly acceptable. Implications for increasing and maintaining treatment integrity by teachers via a consultation model are discussed.     

Supporting teacher use of interventions: effects of response dependent performance feedback on teacher implementation of a math intervention

This study examined general education teachers’ implementation of a peer tutoring intervention for five elementary students referred for consultation and intervention due to academic concerns. Treatment integrity was assessed via permanent products produced by the intervention. Following verbal instructions, intervention implementation by four teachers was consistently low or exhibited a downward trend whereas one teacher’s moderate implementation co-occurred with student gains. When classroom training was conducted with four teachers exhibiting low integrity, all teachers improved implementation. After accurately implementing the intervention for three training sessions with classroom training, response dependent performance feedback was provided when teachers independently used the intervention below 100% integrity, as determined by review of permanent products. With response dependent performance feedback, three of the four teachers implemented the intervention at levels above the verbal training performance, whereas implementation for one teacher increased following discussion of an upcoming school team meeting. Student math scores improved during intervention.     

Effects of immediate performance feedback on implementation of behavior support plans

Research has focused on increasing the treatment integrity of school-based interventions by utilizing performance feedback. The purpose of this study was to extend this literature by increasing special education teachers' treatment integrity for implementing antecedent and consequence procedures in an ongoing behavior support plan. A multiple baseline across teacher-student dyads (for two classrooms) design was used to evaluate the effects of performance feedback on the percentage of antecedent and consequence components implemented correctly during 1-hr observation sessions. Performance feedback was provided every other week for 8 to 22 weeks after a stable or decreasing trend in the percentage of antecedent or consequence components implemented correctly. Results suggested that performance feedback increased the treatment integrity of antecedent components for 4 of 5 teachers and consequence components for all 5 teachers. These results were maintained following feedback for all teachers across antecedent and consequence components. Teachers rated performance feedback favorably with respect to the purpose, procedures, and outcome, as indicated by a social validity rating measure.     

Using performance feedback to improve treatment integrity of classwide behavior plans: an investigation of observer reactivity

The current study replicated the positive effects of performance feedback on treatment integrity and extended previous work by examining reactivity using a multiple baseline design with alternating treatments for observer-present and observer-absent conditions on teachers' implementation of a classwide behavior plan. No differences were found between conditions, and treatment integrity improved across all teachers, suggesting that performance feedback, rather than observer reactivity, was responsible for reported behavior changes.     

Tactile Prompting and Weekly Performance Feedback for Increasing Teachers’ Behavior-Specific Praise

The school-based consultation literature includes a variety of empirically supported procedures for increasing teachers’ implementation of classroom management strategies. However, teachers may respond differently to empirically supported consultation procedures. This study used a multiple baseline design across three elementary teachers to test the effects of a tactile prompting and weekly performance feedback consultation procedure for increasing teachers’ behavior-specific praise. Additionally, this study included data for teachers’ rate of reprimands and their classrooms’ appropriately engaged behavior and disruptive behavior. Results indicated that tactile prompting and weekly performance feedback increased behavior-specific praise for all three teachers. However, a functional relation between tactile prompting and weekly performance feedback and changes in teachers’ reprimands and their classrooms’ appropriately engaged and disruptive behavior was not established. Implications for applied practice, and limitations and directions for future research are discussed.     

An exploratory investigation of teachers' intervention planning and perceived implementation barriers

Increasingly teachers are the primary implementer responsible for providing evidence-based interventions to students. However, there is little knowledge regarding the extent to which teachers plan for intervention implementation, receive implementation support, or identify and address implementation barriers. This study explores survey data from over 1200 preschool through grade 12 teachers from 46 public school districts in a Northeastern state. Results indicate that teachers spend significant time engaging in intervention-related behavior and may be a primary source responsible for selecting student interventions. However, the current extent to which they plan for implementation and present levels of implementation support are inadequate to produce high levels of sustained intervention implementation. In addition, almost 60% of implementation barriers reported related to aspects of the intervention itself. Findings from this study provide guidance for future research and preliminary recommendations for ameliorating implementation barriers and proactively supporting treatment integrity in schools.     

Training School Psychologists in Behavior Support Consultation

SUMMARY

Although school consultation began as a means of providing assistance to individual teachers on a voluntary basis, it has evolved into a stand-alone service regulated by state and federal law. In order to meet the demand for increased accountability in the services they provide, school psychologists and other prereferral intervention team members have expressed a need to develop skills in two key areas: (a) selecting interventions that are conceptually relevant and therefore likely to be effective in responding to children's behavior problems, and (b) providing teachers with the resources and support needed to ensure successful plan implementation. In this paper, we describe various strategies that have been shown to be effective at addressing each of these issues by drawing on recent research in school consultation and applied behavior analysis. Considerations surrounding the use of these strategies by school consultants are discussed, as are implications for increasing the effectiveness and accountability of school consultation services.     

Teacher Use Of Interventions In General Education Settings: Measurement And Analysis Of The Independent Variable

This study examined the integrity with which 4 general education teachers implemented an intervention designed to improve the academic performance of elementary school students. Treatment integrity was measured daily using permanent products. The results showed that the 4 teachers markedly increased the integrity of the delivered treatment when they were provided with performance feedback. The results suggest that academic performance improved for the group of students as integrity improved.     

Enhancing Treatment Integrity Maintenance Through Fading with Indiscriminable Contingencies

School psychologists are often asked to develop treatment to remediate students’ academic skills or social behavior problems. When teachers implement treatment recommendations with high levels of treatment integrity, students benefit. Treatment integrity has been enhanced by use of direct training, performance feedback, and negative reinforcement procedures. Still, maintenance of treatment integrity has received limited systematic inquiry. This study investigated maintenance of treatment integrity over time by utilizing a systemic fading procedure with indiscriminable contingencies. The results showed that the teachers continued to implement treatment protocols at a high level after performance feedback was faded. Students also fared better when treatment integrity was high.     

Supporting Integrity of Discrete Trial Teaching via Tiered Consultation: A Pilot Study

Empirically supported consultation procedures for supporting implementation of evidence-based interventions for Autism Spectrum Disorder (ASD), such as performance feedback, may be challenging to implement in public schools. Educators may find performance feedback aversive, and support personnel may find its directiveness incompatible with their professional roles. This pilot study extends recent advances in implementation science, by investigating the influence of tiered consultation on intervention integrity of Discrete Trial Teaching (DTT). A less intensive form of implementation support was examined for its feasibility and effectiveness. Collaborative Student Data Review (CSDR) includes discussion, general feedback, and modeling. Preliminary findings indicate that CSDR is a feasible and acceptable form of implementation support, and that it shows potential as a form of implementation support for DTT.     

Implementing scientific practices through case studies: Examples using home-school interventions and consultation

Integrating scientific methods into school psychological practice is an essential tenet of effective service. In this article, we present two related examples that demonstrate methods by which to conduct high-quality intervention and consultation case studies with integrity. The present case studies utilized home note and consultation-based interventions to improve task completion and accuracy in mathematics for primary grade students who had demonstrated performance difficulties despite adequate intellectual ability and academic skills. Two sets of case studies were conducted. One involved the use of a home-school note implemented by parents and teachers in conjunction with a self-instruction manual. The second set of case studies used the home note and manual instituted in the context of behavioral consultation with parents and teachers. Both interventions (home-note-only and home note with consultation) increased math completion and accuracy in underachievers. Levels of performance were stable and differences between baseline and treatment conditions were statistically significant for consultation students only. Treatment integrity, maintenance of treatment gains, and consumer acceptability were also stronger in the consultation case studies. The implications for practitioners and future research are explored.     

Teleconsultation: The use of technology to improve evidence-based practices in rural communities

Problem-solving consultation in schools has been found to be an effective method of service delivery to support teachers who are struggling to address student social–emotional behavioral (SEB) concerns. Despite its benefits, a number of barriers (e.g., lack of time and limited access to trained professionals) restrict the use of consultation within schools, especially in rural settings. The purpose of the study was to examine the impact of teleconsultation designed to improve behavior support to students living in rural communities. Both student outcomes and teacher perceptions were evaluated. Results indicated that (a) student disruptive behaviors improved through the implementation of an individualized behavior support plan developed through teleconsultation, and (b) teachers found the teleconsultation experience acceptable and feasible. As the demands placed on psychologists and the quality of videoconferencing continue to increase, teleconsultation is becoming a viable option for service delivery within rural school settings.     

Promoting Teachers’ Generalization of Intervention Use Through Goal Setting and Performance Feedback

Consultation is beneficial for increasing teachers’ intervention use for target students. However, little is known about teachers’ generalized intervention use. This study is a systematic replication of Riley-Tillman and Eckert (J Educ Psychol Consult 12:217–241, 2001) and Martens et al. (Sch Psychol Q 12:33–41, 1997) and examines effects of generalization training with goal setting and a feedback note on teachers’ specific labeled praise (SLP) toward target and non-target students. Participants were three teachers who referred students exhibiting disruptive behaviors. Initially, all teachers displayed low levels of SLP toward target and non-target students. Intervention training with the incorporation of goal setting and a daily feedback note increased teachers’ SLP toward target students. SLP toward non-target students also increased, but to a lesser extent. When feedback was withdrawn and generalization to other students was suggested, SLP decreased. Following generalization training with goal setting and a feedback note, all teachers increased SLP toward non-target students. SLP toward target students, however, was low for all teachers. Changes in target students’ disruptive behaviors are also reported. Results suggest that generalization training may be necessary to increase teachers’ SLP toward non-target students, but that additional support for treatment integrity maintenance may also be required. Results are discussed in terms of limitations and directions for future research.     

Increasing intervention implementation in general education following consultation: a comparison of two follow-up strategies

This study examined two strategies for increasing the accuracy with which general education teachers implemented a peer tutoring intervention for reading comprehension. The intervention was implemented for 5 elementary school students who had been referred for consultation services. Initial implementation of the intervention by the teachers was variable, and the data exhibited a downward trend. When consultants held brief daily meetings with the teachers to discuss the intervention, implementation improved for 2 of 5 participants. Four of the teachers implemented the intervention at levels substantially above baseline during the performance feedback condition, whereas implementation for 1 teacher increased following discussion of an upcoming follow-up meeting with the principal. Student reading comprehension scores improved markedly during the peer tutoring intervention. Three students maintained these gains 4 weeks after the intervention ended. The implications of these findings for the maintenance of accurate treatment implementation in applied settings are discussed.     

Enhancing Consultee Problem-Solving Skills in Consultative Interactions

Selected empirical research on consultative problem solving is reviewed. Although there is general support for the effectiveness of consultation in resolving a variety of presenting problems and in improving consultees' skills for solving future problems, there is a need for additional methods to facilitate the process. Direct training of consultees in behavior management, communication, and consultation process skills is advocated as one means of attaining this goal.     

Consultation-based Academic Interventions for Children with ADHD: Effects on Reading and Mathematics Achievement

The purpose of this investigation was to evaluate the relative efficacy of two consultation-based models for designing academic interventions to enhance the educational functioning of children with attention-deficit/hyperactivity disorder (ADHD). Children (N=167) meeting DSM-IV criteria for ADHD were randomly assigned to one of two consultation groups: Individualized Academic Intervention (IAI; interventions designed using a data-based decision-making model that involved ongoing feedback to teachers) and Generic Academic Intervention (GAI; interventions designed based on consultant-teacher collaboration, representing "consultation as usual"). Teachers implemented academic interventions over 15 months. Academic outcomes (e.g., standardized achievement test, and teacher ratings of academic skills) were assessed on four occasions (baseline, 3 months, 12 months, 15 months). Hierarchical linear modeling analyses indicated significant positive growth for 8 of the 14 dependent variables; however, trajectories did not differ significantly across consultation groups. Interventions in the IAI group were delivered with significantly greater integrity; however, groups did not differ with respect to teacher ratings of treatment acceptability. The results of this study provide partial support for the effectiveness of consultation-based academic interventions in enhancing educational functioning in children with ADHD; however, the relative advantages of an individualized model over "consultation as usual" have yet to be established.     

Using performance feedback to enhance implementation fidelity of the problem-solving team process

Implementation integrity is a potentially critical issue for problem-solving teams (PST) and most response-to-intervention models. The current study hypothesized that providing performance feedback, which has consistently been shown to increase implementation integrity, to PSTs would enhance the procedural integrity of the process. The PSTs for three elementary schools were provided performance feedback with a 20-item checklist created from the literature. A multiple-baseline design across schools revealed an immediate change in level after providing performance feedback. The resulting percentages of non-overlapping data were 90.9%, 90.0%, and 100%. However, PSTs still did not monitor student progress, assess the effectiveness of the intervention, or measure the integrity with which the intervention was implemented even after receiving feedback. Thus, providing performance feedback could be a method to increase the fidelity with which critical components of data-based problem-solving are implemented, but these data suggest the need for additional research.     

PREVENTING SCHOOL VANDALISM AND IMPROVING DISCIPLINE: A THREE-YEAR STUDY

Vandalism is a major problem facing educators and taxpayers alike. The present investigation analyzed how vandalism costs and student disruption were related to the implementation of a training and consultation package designed to increase the reinforcing ambience of the school. A positive environment, it was posited, would displace previous events that may have set the occasion for vandalism, with cues to promote productive school performance. Eighteen elementary and junior high schools were involved over a 3-year period. Using a delayed treatment control design, treatment was delivered following either 4 or 13 months of baseline. During treatment, teams of school personnel attended training workshops in behavioral strategies for reducing vandalism and disruption by students in school. Each team also met regularly on its campus to plan and implement programs on a schoolwide basis. To demonstrate that reinforcing procedures were actually implemented and accompanied by change in student performance, these variables were periodically probed throughout the study. Project staff also provided consultation. Vandalism costs decreased significantly (p < .05) more in treatment than control schools, with an average reduction of 78.5% for all project schools. Rates of praise delivered by project teachers and other randomly selected teachers in the school increased significantly (p < .05), and rates of off-task behavior by students decreased significantly (p < .05) following treatment. The staff development model used in this study appeared to be both feasible and economical.     

Teacher implementation of precorrection and praise statements in Head Start classrooms as a component of a program-wide system of positive behavior support

Although research has clearly supported the use of school-wide positive behavior support (PBS) in elementary school settings, data-based research has not been conducted to support program-wide PBS in early childhood settings. The purpose of this study was to specifically support teachers’ use of universal features of program-wide PBS and to determine whether increases in the use of positive teacher behaviors, including precorrection and praise, were functionally related to decreases in students’ problem behavior. Using a multiple baseline design, three teachers were introduced to an intervention to increase their use of precorrections and specific behavioral praise statements. A relationship was established between the three teachers’ use of key features of program-wide PBS and the reduction of students’ problem behavior in a small group setting. However, findings should be viewed as tentative and future research should explore the relative influence of each of the intervention components on preschool students’ behavior.     

Training teachers in behavioral consultation and therapy: An analysis of verbal behaviors

This study evaluated the relative effectiveness of two teacher-training packages under two analogue consultation conditions in a 2 × 2 completely randomized factorial design. The subjects were 56 elementary school teachers in an urban school district in Arizona. The conditions included a training package of classroom behavior modification and consultation, a training package of consultative service procedures, and general multidisciplinary team processes (a nonspecific control). The consultation conditions included one variant in which very specific consultant questions were presented, and a second in which much more general, somewhat vague, consultant elicitors were presented to subjects in problem-identification, problem-analysis, and problem-evaluation behavioral consultation analogue interviews. The effectiveness of these training packages was evaluated in respect to knowledge of behavior modification principles and concepts, and the frequency of specific categories of consultee verbal behaviors. The results indicated that the experimental training package was effective in increasing teachers' knowledge of behavioral procedures and in increasing the frequency of teacher verbalization regarding overt child behaviors and behavioral intervention plans during the problem identification and problem analysis phases of consultation. It was also found that even after training, specific consultant questions were important in eliciting consultee statements related to environmental conditions surrounding behavior. The use of more general consultant verbal behaviors resulted in significantly more vague, unspecified, and irrelevant types of consultee verbalizations. Implications for future research are discussed.