IMPROVING INTERVENTION INTEGRITY OF DIRECT‐SERVICE PRACTITIONERS THROUGH PERFORMANCE FEEDBACK AND PROBLEM SOLVING CONSULTATION

The purpose of this study was to improve intervention integrity of behavior support plan implementation by teachers at a specialized school. During the first phase of a multiple baseline design, performance feedback was given to the teachers during (correction) and immediately following (praise) classroom observations. In a second phase, the teachers received problem solving consultation that addressed behavior support plan misapplication and respective procedural modifications. Only one of the three teachers demonstrated improved intervention integrity with performance feedback, whereas problem solving consultation produced uniform improvement. We discuss these findings and implications for intervention integrity assessment and research. Copyright © 2014 John Wiley & Sons, Ltd.     

Therapist Self-Efficacy in Delivering Cognitive Processing Therapy in a Randomized Controlled Implementation Trial

Efforts to improve the implementation of evidence-based treatments (EBT) have recently made important strides. One such example is understanding the vital role that weekly consultation plays as therapists learn to deliver an EBT. Because mechanism-based research can further support EBT implementation, the present study sought to examine the potential relationship between therapist self-efficacy in relation to treatment fidelity and outcomes.We examined therapist self-efficacy ratings from 80 therapists working with 188 patients. These data were collected as part of a randomized controlled implementation trial testing cognitive processing therapy (CPT). Across post-workshop training conditions, we ran multilevel models to assess (1) changes in therapist self-efficacy, (2) therapist self-efficacy in relation to treatment fidelity, and (3) therapist-self-efficacy in relation to patient PTSD symptom outcomes.We found that therapist self-efficacy significantly improved over the course of 6 months of CPT training. Baseline therapist self-efficacy was differentially associated with client outcomes based on post-workshop training condition. Specifically, therapists with low self-efficacy that did not receive post-workshop consultation tended to have poorer outcomes than therapists with low self-efficacy that received consultation. In the present sample, therapist self-efficacy was not related to treatment fidelity.As this was the first study to examine therapist self-efficacy in the implementation of an evidence-based treatment, our findings suggest that self-efficacy may be an important implementation factor in treatment outcomes and worthy of ongoing research.     

ONGOING CONSULTATION AS A METHOD OF IMPROVING PERFORMANCE OF STAFF MEMBERS IN A GROUP HOME

A model of ongoing consultation was implemented in a community group home for 8 adults with severe and profound mental retardation. Two consultants, highly experienced in working with people with mental retardation and in the procedures used in group homes, taught staff members to use a token reinforcement system, to engage the adults in a variety of activities, and to improve the content and style of the staff members' interactions with the adults. The consultants taught skills to 9 staff members through brief mini-workshops, direct observation of the staff members' use of the skills during regular activities in the group home, and individual verbal feedback regarding a staff member's performance of the skills. Evaluation of the ongoing consultation process by the 2 consultants showed it to be effective in improving the performance of the staff members and in changing the behaviors of the adults who lived in the home. Continued implementation of the process, however, appeared to be necessary for the behavior changes of staff members to be maintained at high levels.     

Tactile Prompting and Weekly Performance Feedback for Increasing Teachers’ Behavior-Specific Praise

The school-based consultation literature includes a variety of empirically supported procedures for increasing teachers’ implementation of classroom management strategies. However, teachers may respond differently to empirically supported consultation procedures. This study used a multiple baseline design across three elementary teachers to test the effects of a tactile prompting and weekly performance feedback consultation procedure for increasing teachers’ behavior-specific praise. Additionally, this study included data for teachers’ rate of reprimands and their classrooms’ appropriately engaged behavior and disruptive behavior. Results indicated that tactile prompting and weekly performance feedback increased behavior-specific praise for all three teachers. However, a functional relation between tactile prompting and weekly performance feedback and changes in teachers’ reprimands and their classrooms’ appropriately engaged and disruptive behavior was not established. Implications for applied practice, and limitations and directions for future research are discussed.     

Increasing intervention implementation in general education following consultation: a comparison of two follow-up strategies

This study examined two strategies for increasing the accuracy with which general education teachers implemented a peer tutoring intervention for reading comprehension. The intervention was implemented for 5 elementary school students who had been referred for consultation services. Initial implementation of the intervention by the teachers was variable, and the data exhibited a downward trend. When consultants held brief daily meetings with the teachers to discuss the intervention, implementation improved for 2 of 5 participants. Four of the teachers implemented the intervention at levels substantially above baseline during the performance feedback condition, whereas implementation for 1 teacher increased following discussion of an upcoming follow-up meeting with the principal. Student reading comprehension scores improved markedly during the peer tutoring intervention. Three students maintained these gains 4 weeks after the intervention ended. The implications of these findings for the maintenance of accurate treatment implementation in applied settings are discussed.     

Discrete‐trial training performance of behavior interventionists with autism spectrum disorder: A systematic replication and extension

Further research is needed on strategies to improve employment outcomes for individuals with autism spectrum disorder (ASD). We replicated and extended prior research by examining the acquisition, maintenance, and generalization of discrete‐trial training (DTT) performance of adults with ASD who were interested in careers as behavior technicians. Three participants received training on how to implement DTT with children and we assessed the maintenance and generalization of DTT performance in the absence of feedback. Results indicated that procedural integrity generalized across children and targets and maintained for 6 to 17 session days without feedback. Results have implications for the successful employment of individuals with ASD as behavior technicians and for future research on strategies to enhance their performance on the job.     

Training School Psychologists in Behavior Support Consultation

SUMMARY

Although school consultation began as a means of providing assistance to individual teachers on a voluntary basis, it has evolved into a stand-alone service regulated by state and federal law. In order to meet the demand for increased accountability in the services they provide, school psychologists and other prereferral intervention team members have expressed a need to develop skills in two key areas: (a) selecting interventions that are conceptually relevant and therefore likely to be effective in responding to children's behavior problems, and (b) providing teachers with the resources and support needed to ensure successful plan implementation. In this paper, we describe various strategies that have been shown to be effective at addressing each of these issues by drawing on recent research in school consultation and applied behavior analysis. Considerations surrounding the use of these strategies by school consultants are discussed, as are implications for increasing the effectiveness and accountability of school consultation services.     

Testing a Theory of Implementation Leadership and Climate Across Autism Evidence-Based Interventions of Varying Complexity

Prominent theories within the field of implementation science contend that organizational leaders can improve providers’ fidelity to evidence-based practices (EBPs) by using focused implementation leadership behaviors that create an organizational climate for EBP implementation. However, this work has been criticized for overreliance on nonspecific, self-report fidelity measures and poor articulation of the boundary conditions that may attenuate leadership and climate’s influence. This study tests the predictions of EBP implementation leadership and climate theory on observed fidelity to three school-based EBPs for autism that vary in complexity: pivotal response training (PRT), discrete trial training (DTT), and visual schedules (VS). Educators in kindergarten to third-grade autism support classrooms in 65 schools assessed their principals’ EBP implementation leadership and school EBP implementation climate prior to the school year. Mid-school year, trained observers rated educator fidelity to all three interventions. Expert raters confirmed PRT was significantly more complex than DTT or VS using the Intervention Complexity Assessment Tool for Systematic Reviews. Linear regression analyses at the school level indicated principals’ increased frequency of EBP implementation leadership predicted a higher school EBP implementation climate, which in turn predicted higher educator fidelity to PRT—however, there was no evidence of a relationship between implementation climate and fidelity to DTT or VS. Comparing principals whose EBP implementation leadership was ±1 SD from the mean, there was a significant indirect association of EBP implementation leadership with PRT fidelity through EBP implementation climate (d = 0.49, 95% CI [0.04, 0.93]). Strategies that target EBP implementation leadership and climate may support fidelity to complex behavioral interventions.     

Examining the Efficacy of School-Based Consultation: Recommendations for Improving Outcomes

SUMMARY

Schools are faced with the difficult task of educating students who present diverse learning and behavioral challenges. To meet student needs, schools tap the expertise of a variety of specialists who provide services through “pull-out” programs or consultation to general educators. Unfortunately, neither strategy has proved effective to date with respect to children and youth who present significant social behavior problems. A brief review of the literature points to the shortcomings as linked to multiple treatments with varying outcomes and inconsistent implementation of best practices. One option to address the challenges schools face in light of ineffective practices to date is the development of school-wide systems of positive behavior support (PBS). Effective implementation of school-wide systems of PBS will require moving the consultation process from individual cases to working with teams of educators. Key skills and possible measurement strategies to develop and sustain school-wide consultation efforts are discussed.     

Consultation in adolescence: Hurried,terminable, interminable1

The physiological ambivalence displayed by adolescents towards dependence on adults is an obstacle, at least initially, to a course of psychotherapy. A consultation is often the only thing that can be offered to start with, and constitutes an essential stage even for those teenagers who go on to prove themselves able to benefi t from therapy or analysis. The way this fi rst meeting is handled is therefore crucial. The author investigates the diffi culties and paradoxes of this consultation, the particular nature of the ‘time’ of the consultation compared to that of therapy, and the different internal attitude this requires of the analyst. She thus identifi es two separate confi gurations‐a hurried consultation and an interminable consultation‐which can result when the consultation fails in its primary objectives. Both are connected with possible collusive elements in the professional linked to specifi c adolescent angst. The very detailed account of a consultation shows the ‘journey’ made by a 14 year‐old starting from her urgent request to be freed from a symptom experienced as an incomprehensible obstacle, to her gradual development of curiosity towards herself and towards her own mental and emotional make‐up, resulting in her acceptance of therapeutic treatment, which would initially have been impossible.     

Expanding Technical Assistance Consultation to Public Schools: District-Wide Evaluation of Instructional and Behavior Support Practices for Students with Developmental Disabilities

SUMMARY

We describe consultation to a public school district in the form of a systems-wide evaluation of instructional and behavior support practices for students with developmental disabilities. The evaluation targeted (1) the financial costs of educating students out-of-district, (2) an analysis of a representative sample of Individualized Educational Plans, and (3) interviews with and a questionnaire completed by teachers. The primary objective of the evaluation was to determine practice standards in the public school district and to make recommendations to improve service delivery. The format of the evaluative model, respective findings, suggested remedies, and implications for large-scale public school consultation are presented.     

Focus,Scope, and Practice of Behavioral Consultation to Public Schools

SUMMARY

Public school districts frequently seek professional consultation to improve educational services. This article is an overview of the focus, scope, and practice of behavioral consultation to public schools. A four-stage process of consultation is described, followed by a discussion on the expanding role of behavior support intervention. Next, the involvement of consultants in the design of individual-student, classroom-wide, and whole-school programs is considered. Contemporary approaches to consultation are presented such as positive behavior support, functional behavioral assessment, and efficacy evaluation that targets natural data sources. A concluding section sets forth recommendations for the successful practice of public school behavioral consultation.     

Supporting teacher use of interventions: effects of response dependent performance feedback on teacher implementation of a math intervention

This study examined general education teachers’ implementation of a peer tutoring intervention for five elementary students referred for consultation and intervention due to academic concerns. Treatment integrity was assessed via permanent products produced by the intervention. Following verbal instructions, intervention implementation by four teachers was consistently low or exhibited a downward trend whereas one teacher’s moderate implementation co-occurred with student gains. When classroom training was conducted with four teachers exhibiting low integrity, all teachers improved implementation. After accurately implementing the intervention for three training sessions with classroom training, response dependent performance feedback was provided when teachers independently used the intervention below 100% integrity, as determined by review of permanent products. With response dependent performance feedback, three of the four teachers implemented the intervention at levels above the verbal training performance, whereas implementation for one teacher increased following discussion of an upcoming school team meeting. Student math scores improved during intervention.     

Enhancing CBT Consultation With Multicultural Counseling Principles

The use of multicultural principles to enhance cognitive behavioral therapy (CBT) for individuals of marginalized backgrounds has received increased attention in light of the heightened national awareness of systemic oppression and racialized violence directed towards Black, Indigenous, and People of Color. However, there has been less of a focus on applying such principles to consultation for skill development. If ethical guidelines are expected to influence the behavior of clinicians in session, guidance is needed to indicate how and where and when clinicians should receive training in implementing culturally responsive CBT. Individual reading and reflection are necessary but are not sufficient in acquiring new clinical skills. Consultation is recommended and strongly suggested when clinicians are working with new populations or delivering a new treatment, or even using a new modality. Consultation can also be useful when adopting a new approach or stance to therapy. For practicing clinicians who have not developed these skills, additional consultation can and should be used to address this gap. Moving forward, integration of cultural responsiveness into standard consultation will ensure that these skills are seen as a core competency, rather than an optional additional skill that may be (or not be) elected. This paper presents core experiences that may be integral to a CBT consultation model that aims to enhance providers’ ability to provide CBT in a way that is culturally responsive to their clients. These recommendations attend to both content and process within CBT consultation and reflect guiding assumptions for helping clinicians to develop the ability to practice CBT in a culturally responsive manner, including (a) normalizing discussions of cultural identity and oppression, (b) an emphasis on cultural self-awareness, (c) emphasizing culturally informed CBT case conceptualization, and (d) skill development in applying cultural elements to CBT interventions.     

Reducing the Risk for Preschool Expulsion: Mental Health Consultation for Young Children with Challenging Behaviors

Increasing numbers of young children are being expelled from child care settings because of their problem behavior. Access to mental health consultation is related to lower rates of expulsion, but additional data are needed to document the pathways through which mental health consultation reduces the risk of expulsion. We report on outcomes from a 4-year project designed to reduce the number of children expelled for problem behavior in a large suburban county in Maryland. Two master’s-level professionals provided behavioral consultation to child care providers who identified nearly 200 children at imminent risk for expulsion. Child care providers rated children’s social skills and problem behaviors at referral and discharge using the Preschool Kindergarten Behavior Scales and the Brief Infant Toddler Social Emotional Assessment. Statistically significant increases in social skills and reductions in problem behaviors were seen for children who received individualized consultation. More than three-quarters of the children who were at risk for expulsion were able to be maintained in their current child care placement; of those that changed placements, only half (n = 13) were removed involuntarily. These findings provide additional support for mental health consultation as a promising strategy to reduce the risk for expulsion for young children with problem behaviors.     

A Teacher-Consultation Approach to Social Skills Training for Pre-Kindergarten Children: Treatment Model and Short-Term Outcome Effects

This study evaluated the post-treatment outcome effects of a classroom-based social skills program for pre-kindergarten children, using a teacher-consultation model. The pre-K RECAP (Reaching Educators, Children, and Parents) program is a semi-structured, cognitive-behavioral skills training program that provides teachers with in-classroom consultation on program implementation and classroom-wide behavior management. Data on children's social skills and behavior problems were collected from parents and teachers at pre- and post-treatment, for 149 children aged 4–5 years (of whom 56% were girls). Significant treatment effects were found for teacher but not parent reports, with treatment group children improving significantly more than comparison group children in their teacher-rated social skills and internalizing and externalizing problems. These results provide some preliminary support for the efficacy of the program on children's social skills and behavior problems, and for a teacher-consultation model for training teachers to implement school-based mental health programs.     

Is timely information always better? The effect of feedback frequency on decision making

Recent advances in information technology make it possible for decision makers to track information in real-time and obtain frequent feedback on their decisions. From a normative sense, an increase in the frequency of feedback and the ability to make changes should lead to enhanced performance as decision makers are able to respond more quickly to changes in the environment and see the consequences of their actions. At the same time, there is reason to believe that more frequent feedback can sometimes lead to declines in performance. Across four inventory management experiments, we find that in environments characterized by random noise more frequent feedback on previous decisions leads to declines in performance. Receiving more frequent feedback leads to excessive focus on and more systematic processing of more recent data as well as a failure to adequately compare information across multiple time periods. These results suggest that caution be used in the design and implementation of real-time information systems.     

Using the Performance Diagnostic Checklist to evaluate and promote paraeducators' treatment fidelity

Paraeducators, who increasingly serve as implementers for students in special education, can struggle with intervention fidelity. Special educators, often responsible for providing paraeducators with support, receive limited supervision training. From organizational behavior management, the Performance Diagnostic Checklist-Human Services (PDC-HS) is a checklist to detect reasons for poor workplace performance and identify aligned support. Utilizing a multiple baseline design, this study evaluated the impact of PDC-HS indicated implementation support on the intervention fidelity of five paraeducators implementing six behavior intervention plans for students receiving special education services. The PDC-HS indicated that all paraeducators would benefit from behavior skills training and visuals of the plans, in addition to other specific supports for individual paraeducators (e.g., prompts, feedback). These implementation supports were effective in improving intervention fidelity for four paraeducators, whereas one paraeducator (across two students) did not respond to these supports. Changes in student outcomes (i.e., academic engagement, disruptive behavior) were inconsistent, suggesting that the behavior intervention plans may have needed to be revised. Usability data suggested that the PDC-HS and indicated supports were generally acceptable, but revisions may be warranted. The implications of this study for school-based implementation support research are discussed.     

Assessing long‐term maintenance of staff performance following behavior skills training in a home‐based setting

We used a multiple baseline across participant design to evaluate the effects of behavior skills training on teaching three behavior therapists to implement discrete trial teaching (DTT) and evaluate the long‐term maintenance of skills acquired through behavioral skills training. For participants whose skills did not maintain, the authors evaluated an independent self‐evaluation procedure on their performance. Following DTT implementation training, maintenance probes were assessed at 2‐, 4‐, 6‐, and 8‐week follow‐ups. The results demonstrated that one participant maintained 100% procedural integrity (PI) through all follow‐ups, one participant decreased below mastery criterion at the 2‐week follow‐up, and one participant dropped below mastery criterion at the 4‐week follow‐up. Those participants that demonstrated decreased accuracy of implementation of DTT programs and were taught to implement a self‐evaluation procedure. Following self‐evaluation, PI maintained for up to 7 weeks for one participant. Our results suggest that if PI does not maintain, self‐evaluation may be a supplementary intervention to increase and maintain PI of new employees.     

Implementing scientific practices through case studies: Examples using home-school interventions and consultation

Integrating scientific methods into school psychological practice is an essential tenet of effective service. In this article, we present two related examples that demonstrate methods by which to conduct high-quality intervention and consultation case studies with integrity. The present case studies utilized home note and consultation-based interventions to improve task completion and accuracy in mathematics for primary grade students who had demonstrated performance difficulties despite adequate intellectual ability and academic skills. Two sets of case studies were conducted. One involved the use of a home-school note implemented by parents and teachers in conjunction with a self-instruction manual. The second set of case studies used the home note and manual instituted in the context of behavioral consultation with parents and teachers. Both interventions (home-note-only and home note with consultation) increased math completion and accuracy in underachievers. Levels of performance were stable and differences between baseline and treatment conditions were statistically significant for consultation students only. Treatment integrity, maintenance of treatment gains, and consumer acceptability were also stronger in the consultation case studies. The implications for practitioners and future research are explored.