Increasing intervention implementation in general education following consultation: a comparison of two follow-up strategies

This study examined two strategies for increasing the accuracy with which general education teachers implemented a peer tutoring intervention for reading comprehension. The intervention was implemented for 5 elementary school students who had been referred for consultation services. Initial implementation of the intervention by the teachers was variable, and the data exhibited a downward trend. When consultants held brief daily meetings with the teachers to discuss the intervention, implementation improved for 2 of 5 participants. Four of the teachers implemented the intervention at levels substantially above baseline during the performance feedback condition, whereas implementation for 1 teacher increased following discussion of an upcoming follow-up meeting with the principal. Student reading comprehension scores improved markedly during the peer tutoring intervention. Three students maintained these gains 4 weeks after the intervention ended. The implications of these findings for the maintenance of accurate treatment implementation in applied settings are discussed.     

Supporting teacher use of interventions: effects of response dependent performance feedback on teacher implementation of a math intervention

This study examined general education teachers’ implementation of a peer tutoring intervention for five elementary students referred for consultation and intervention due to academic concerns. Treatment integrity was assessed via permanent products produced by the intervention. Following verbal instructions, intervention implementation by four teachers was consistently low or exhibited a downward trend whereas one teacher’s moderate implementation co-occurred with student gains. When classroom training was conducted with four teachers exhibiting low integrity, all teachers improved implementation. After accurately implementing the intervention for three training sessions with classroom training, response dependent performance feedback was provided when teachers independently used the intervention below 100% integrity, as determined by review of permanent products. With response dependent performance feedback, three of the four teachers implemented the intervention at levels above the verbal training performance, whereas implementation for one teacher increased following discussion of an upcoming school team meeting. Student math scores improved during intervention.     

Using performance feedback to enhance implementation fidelity of the problem-solving team process

Implementation integrity is a potentially critical issue for problem-solving teams (PST) and most response-to-intervention models. The current study hypothesized that providing performance feedback, which has consistently been shown to increase implementation integrity, to PSTs would enhance the procedural integrity of the process. The PSTs for three elementary schools were provided performance feedback with a 20-item checklist created from the literature. A multiple-baseline design across schools revealed an immediate change in level after providing performance feedback. The resulting percentages of non-overlapping data were 90.9%, 90.0%, and 100%. However, PSTs still did not monitor student progress, assess the effectiveness of the intervention, or measure the integrity with which the intervention was implemented even after receiving feedback. Thus, providing performance feedback could be a method to increase the fidelity with which critical components of data-based problem-solving are implemented, but these data suggest the need for additional research.     

Using the Performance Diagnostic Checklist to evaluate and promote paraeducators' treatment fidelity

Paraeducators, who increasingly serve as implementers for students in special education, can struggle with intervention fidelity. Special educators, often responsible for providing paraeducators with support, receive limited supervision training. From organizational behavior management, the Performance Diagnostic Checklist-Human Services (PDC-HS) is a checklist to detect reasons for poor workplace performance and identify aligned support. Utilizing a multiple baseline design, this study evaluated the impact of PDC-HS indicated implementation support on the intervention fidelity of five paraeducators implementing six behavior intervention plans for students receiving special education services. The PDC-HS indicated that all paraeducators would benefit from behavior skills training and visuals of the plans, in addition to other specific supports for individual paraeducators (e.g., prompts, feedback). These implementation supports were effective in improving intervention fidelity for four paraeducators, whereas one paraeducator (across two students) did not respond to these supports. Changes in student outcomes (i.e., academic engagement, disruptive behavior) were inconsistent, suggesting that the behavior intervention plans may have needed to be revised. Usability data suggested that the PDC-HS and indicated supports were generally acceptable, but revisions may be warranted. The implications of this study for school-based implementation support research are discussed.     

Using Coaching to Support Teacher Implementation of Classroom-based Interventions

Despite the growing evidence base for the efficacy of preventive interventions, the level of implementation of these interventions in schools is often less than optimal. One promising approach to supporting teachers in implementation of interventions is the use of coaching. In this study, teachers were trained in a universal classroom management intervention and provided ongoing coaching. The association between the type and amount of coaching activities and teacher implementation of proactive classroom management over time were investigated. Results indicated that teachers who received more performance feedback had higher levels of implementation over time in comparison with teachers who received less feedback. In addition, a significant interaction between the amount of coaching a teacher received and his or her implementation of proactive classroom management was found. Increased implementation over time was observed for teachers with lower initial levels of implementation who received more coaching, whereas implementation decreased over time for teachers who received less coaching. The importance of coaching as a support system for enhancing implementation quality of classroom-based preventive interventions is discussed.     

A multi-year evaluation of the effects of a Response to Intervention (RTI) model on identification of children for special education

The purpose of this study was to examine the effects of implementation of a systematic response to intervention (RTI) model on the identification and evaluation of children for special education. Using a multiple baseline design, a systematic model of assessment and intervention was introduced in consecutive years for all elementary schools (N = 5) in the district. Effect of the RTI model on number of evaluations conducted, percentage of evaluated children who qualified for services, and proportion of identified children by sex and ethnicity before and after implementation of the model was examined. Additionally, outcomes for children who did not have an adequate response to intervention versus those who were at-risk but responded successfully to short-term intervention were examined. A cost analysis of use of the model was provided. The degree to which data obtained were used by the decision-making team was also examined. The assessment and intervention procedures, decision rules, and schoolwide training methods are described in detail and practical implications are discussed.     

Supporting Integrity of Discrete Trial Teaching via Tiered Consultation: A Pilot Study

Empirically supported consultation procedures for supporting implementation of evidence-based interventions for Autism Spectrum Disorder (ASD), such as performance feedback, may be challenging to implement in public schools. Educators may find performance feedback aversive, and support personnel may find its directiveness incompatible with their professional roles. This pilot study extends recent advances in implementation science, by investigating the influence of tiered consultation on intervention integrity of Discrete Trial Teaching (DTT). A less intensive form of implementation support was examined for its feasibility and effectiveness. Collaborative Student Data Review (CSDR) includes discussion, general feedback, and modeling. Preliminary findings indicate that CSDR is a feasible and acceptable form of implementation support, and that it shows potential as a form of implementation support for DTT.     

IMPROVING INTERVENTION INTEGRITY OF DIRECT‐SERVICE PRACTITIONERS THROUGH PERFORMANCE FEEDBACK AND PROBLEM SOLVING CONSULTATION

The purpose of this study was to improve intervention integrity of behavior support plan implementation by teachers at a specialized school. During the first phase of a multiple baseline design, performance feedback was given to the teachers during (correction) and immediately following (praise) classroom observations. In a second phase, the teachers received problem solving consultation that addressed behavior support plan misapplication and respective procedural modifications. Only one of the three teachers demonstrated improved intervention integrity with performance feedback, whereas problem solving consultation produced uniform improvement. We discuss these findings and implications for intervention integrity assessment and research. Copyright © 2014 John Wiley & Sons, Ltd.     

Accelerometer feedback to promote physical activity in adults with type 2 diabetes: a pilot study

This study examined whether giving activity feedback to obese, sedentary adults with Type 2 diabetes would improve their adherence to a home-based walking program. 49 subjects were recruited. Ten failed a treadmill cardiovascular test. Another 9 dropped out before the intervention, and 4 dropped out during the intervention. The remaining 26 adults (14 women, 12 men; age M = 48.1 yr., SD = 7.1) received behavioral counseling monthly for 3 mo. regarding exercise. All subjects wore a triaxial accelerometer during these 3 months. Randomly, half of the subjects were blind to the data from the accelerometers, i.e., no feedback, and had counseling based on their self-report diaries. The other half had access to the accelerometer data, got a computerized graph of their physical activity for the period between counseling sessions, and had counseling based on these objective data, i.e., feedback. The feedback group showed an increase in exercise over the 3 mo. The nonfeedback group showed an increase in activity at 1.5 mo. but reverted to their baseline exercise levels at 3 mo. However, analysis of variance showed there was at least an 8% probability that this effect was due to chance, so the hypothesis that feedback would improve exercise adherence could not be supported. Further studies with larger sample sizes and greater control of experimental conditions are needed to determine the utility of objective activity feedback.     

Moving European research on work and ageing forward: Overview and agenda

This longitudinal retrospective study shows that when the results of an organizational development intervention are fed back properly and action plans are made, factors associated with leadership and commitments to organization are enhanced. The study compared the effects of survey feedback, including the making of action plans and survey feedback without action plans, to the effects of no feedback at all on several psychosocial factors and sick leave over a 3-year period. The data were derived from 1236 individuals for whom baseline data were available and who completed a 3-year follow-up. Applied statistics revealed that the feedback action plan group reported greater improvements regarding leadership factors and commitment to the organization. The results reveal no differences between the feedback groups in regard to sick leave. There is a need for further studies, preferably with a randomized control design, on how to enhance both the psychosocial work environment and employee health and also how to reduce sick leave in the workplace.     

Sustainability of Teacher Implementation of School-Based Mental Health Programs

Evidence-based prevention and intervention programs are increasingly being implemented in schools and it therefore is becoming increasingly important to understand the complexities of program implementation under real-world conditions. Much research has focused on the contextual factors that influence program implementation but less work has attempted to provide an integrated understanding of mechanisms (e.g., teacher-training processes) that affect teachers' program implementation. In this paper, we review literature on factors related to teachers' implementation of school-based prevention and intervention programs, then from this review abstract what we believe are four basic ingredients that characterize potentially sustainable teacher-implemented classroom programs. Finally, we present a sequential model, based on these ingredients, of the naturalistic processes underlying sustainability of teachers' program implementation and describe how this sustainability can be enhanced through provision of teacher training and performance feedback from a classroom consultant.     

Teacher Training: Effects of Directed Rehearsal on the Teaching Skills of Physical Education Majors

Directed rehearsal was used to improve the teaching skills of six preservice elementary physical education teachers during a teaching practicum. The teaching skills lesson introduction, lesson closure, task presentation, and general and specific subject-matter feedback were assessed using a multiple baseline design across behaviors for each teacher. Social validity, in the form of the acceptability of directed rehearsal as a teacher training intervention, was also assessed using a questionnaire. Directed rehearsal was effective in improving the teaching skills of the teachers to either 100% correct implementation or, in the case of subject-matter feedback, to an established rate. In addition, the intervention was acceptable to the teachers and the practicum instructor. These results extend both the error correction and the research on teacher education literatures.     

A systematic review of training programs for parents of children with autism spectrum disorders: Single subject contributions

Aim: The purpose of this systematic review was to examine research utilizing single subject research designs (SSRD) to explore the effectiveness of interventions designed to increase parents' ability to support communication and social development in children with autism spectrum disorders (ASDs). Method: Included studies were systematically assessed for methodological quality (Logan et al., 2008; Smith et al., 2007) and intervention effects. Data examining participant characteristics, study methodology, outcomes, and analysis were systematically extracted. Results: Eleven SSRD parent-training intervention studies examining 44 participants with ASD were included. Overall, the studies were of moderate quality and reported increases in parent skills and child language and communication outcomes. Interpretation: The results supported by improvement rate difference (IRD) analysis indicated several interventions demonstrated positive effects for both parent and child outcomes. However, limited generalization and follow-up data suggested only one intervention demonstrated parents' accurate and ongoing intervention implementation beyond training.     

A replication of the use of public posting in traffic speed control

In an attempt to replicate several previous studies, a reversal design was used to assess the effects of publicly posted feedback on vehicle speed reduction. The intervention phases consisted of daily posted feedback that supplied drivers with either accurate or inaccurate information regarding the percentage of vehicles exceeding the speed limit on the road. Results from the two feedback conditions indicated no speed reduction in comparison to baseline levels. These results are discussed in light of previous studies that found strong effects for such feedback. Baseline levels of driver compliance and methodological differences may limit the effectiveness of posted feedback in reducing speeding.     

Recorded Readings: A Taped Parent-Tutoring Intervention

Parent tutoring has been successfully used to increase children’s oral reading fluency. However, commonly used procedures pose a challenge for parents who are not proficient in reading or who speak English as a second language. A taped reading program that included listening passage preview, repeated reading, and performance feedback was developed for parental implementation during home tutoring sessions. A multiple baseline design across participants was used to evaluate the effects of parent tutoring on the oral reading fluency of two elementary-aged English language learners. Data were collected on oral reading fluency using high- and low-word overlap passages. Results revealed an immediate increase in generalized oral reading fluency for one child upon intervention implementation and continued growth for both children 2 months following termination. Findings suggest recorded readings was an effective and acceptable parent tutoring intervention for increasing reading performance for language minority families.     

Training of interventionists and cultural adaptation procedures: A systematic review of culturally adapted evidence-based parenting programs in Africa

There has been an increase in the implementation of evidence-based parenting programs from high-income countries to several African countries. In this review, we systematically evaluated intervention studies of culturally adapted parenting programs in nine African countries with the objective of examining the quality of training for interventionists and cultural adaptation procedures. A total of 18 studies, obtained from an electronic search of 6 databases, met the inclusion criteria and were evaluated following PRISMA guidelines. The Ecological Validity Model was adopted to organize data on cultural adaptation procedures. Sixteen of the 18 studies reported information regarding the clinical training of interventionists and the cultural adaptations undertaken. Live and interactive workshops were the most common format used to train interventionists in the focal intervention. Overall, cultural adaptations in most studies included translation of intervention protocols into the local language. However, studies varied in the way cultural adaptation procedures were reported with some studies failing to report on cultural adaptation procedures. Concurring with previous literature, attending to issues of culture, power, privilege, access, sustainability, and other relevant concepts to increase the cultural relevance is highly encouraged in parent intervention studies in Africa. This review provides a baseline upon which future training and adaptation procedures can be built.     

CAREER SELF-MANAGEMENT: A QUASI-EXPERIMENTAL ASSESSMENT OF THE EFFECTS OF A TRAINING INTERVENTION

A growing trend is to encourage employees to become actively involved in the management of their own careers. Career self-management, the degree to which one regularly gathers information and plans for career problem solving and decision making, includes two main behaviors: developmental feedback seeking and job mobility preparedness. Although career self-management training is a commonly used employer intervention to re-socialize individuals to increase their own career management activity, it is rarely rigorously evaluated. Relying on an expectancy theory framework, the goal of this study was to evaluate the general effects of career self-management training using a quasi-experimental design. Based on data from several hundred professionals at a major U.S. employer, the results showed formal training efforts were generally not successful in resocializing people to engage in career self-management activities, and when done as an isolated human resource strategy, decreased trainees' likelihood of engaging in career self-management behaviors. To the extent that Time 2 expectancy perceptions got worse, the results showed that an individual's attitudes toward feedback seeking mediated the relationship between the training intervention and the level of preparation for job mobility conducted 6-8 months following the training.     

Reduction of residential consumption of electricity through simple monthly feedback

Feedback has been widely used in efforts to control the consumption of electricity. Previous efforts, however, have used forms of feedback that seem economically impractical. The present study examined the effects of a feasible program of monthly feedback. Forty matched nonvolunteer participants were randomly divided into two groups: a no-contact control group and a monthly feedback group. In an A-B-A design, the data showed a clear decrease in electricity consumption for the feedback group during the feedback phase. The effect was maintained during a 4-mo intervention period. Withdrawal of the feedback was associated with a return to higher levels of electricity consumption.     

Patient-Informed Treatment Development of Behavioral Smoking Cessation for People With Schizophrenia

The objective of this study was to use qualitative methodology to tailor and refine an existing smoking cessation intervention for the population of people who use cigarettes and are diagnosed with schizophrenia, schizoaffective, or psychotic disorder. Successive cohort design methodology was used to iteratively modify the treatment in response to qualitative participant, therapist, and consultant feedback on the intervention. Qualitative methodology for participant feedback included analysis of semistructured interviews with participants, visualization of app utilization data, and stakeholder feedback from study therapists and consultants. Using the successive cohort design, a tailored multicomponent mobile health smoking cessation intervention was developed. The intervention included mobile contingency management (i.e., financial compensation for confirmed abstinence from smoking), pharmacotherapy for smoking cessation, cognitive-behavioral counseling sessions, and the Stay Quit app for relapse prevention. Two cohorts (N = 13) were completed in the study; after each cohort, the treatment protocol was revised. The intervention is described, as well as the qualitative findings from each cohort and subsequent changes made to the intervention based upon patient and provider feedback. Metrics of patient engagement included treatment adherence (40% in Cohort 1 and 63% in Cohort 2). Both participants and therapists reported that the intervention was helpful. Over one third of participants self-reported abstinence at posttreatment. Since qualitative methodology is often underutilized in mental health treatment development, this study demonstrates the utility of the successive cohort design for treatment development of behavior change interventions for at-risk, vulnerable populations.