Using Planned Adaptation to Implement Evidence-Based Programs with New Populations

The Interactive Systems Framework (ISF) for Dissemination and Implementation (Wandersman et al. 2008) elaborates the functions and structures that move evidence-based programs (EBPs) from research to practice. Inherent in that process is the tension between implementing programs with fidelity and the need to tailor programs to fit the target population. We propose Planned Adaptation as one approach to resolve this tension, with the goal of guiding practitioners in adapting EBPs so that they maintain core components of program theory while taking into account the needs of particular populations. Planned Adaptation is a form of capacity building within the Prevention Support System that provides a framework to guide practitioners in adapting programs while encouraging researchers to provide information relevant to adaptation as a critical aspect of dissemination research, with the goal of promoting wider dissemination and better implementation of EBPs. We illustrate Planned Adaptation using the JOBS Program (Caplan et al. 1989), which was developed for recently laid-off, working- and middle-class workers and subsequently implemented with welfare recipients.     

Balancing fidelity and adaptation in the dissemination of empirically-supported treatments: The promise of transdiagnostic interventions

Assessing treatment fidelity is a core methodological consideration in the study of treatment outcome; it influences both the degree to which changes can be attributed to the intervention and the ability to replicate and disseminate the intervention. Efforts to increase access to evidence-based psychological treatments are receiving unprecedented support; but pressures exist to adapt treatments to service settings, running the risk of compromising fidelity. However, little evidence is available to inform the necessary conditions for the transportation of interventions to service provision settings, and the degree to which fidelity is even evaluated or emphasized in dissemination and implementation programs varies dramatically. Moreover, adaptation is associated with several benefits for dissemination efforts and may address relevant barriers to adoption. A particularly promising strategy for maximizing the benefits of both fidelity and adaptation is the use of transdiagnostic interventions. Such treatments allow for greater flexibility of the pacing and content of treatment, while still providing structure to facilitate testing and replication. Preliminary evidence supports the efficacy of this strategy, which may be particularly conducive to dissemination into service provision settings. At this time, further research is needed to evaluate the relationships among fidelity, adaptation, and outcome, and to determine the potential for transdiagnostic treatments to facilitate dissemination.     

Preschool children's development in classic Montessori, supplemented Montessori, and conventional programs

Research on the outcomes of Montessori education is scarce and results are inconsistent. One possible reason for the inconsistency is variations in Montessori implementation fidelity. To test whether outcomes vary according to implementation fidelity, we examined preschool children enrolled in high fidelity classic Montessori programs, lower fidelity Montessori programs that supplemented the program with conventional school activities, and, for comparison, conventional programs. Children were tested at the start and end of the school year on a range of social and academic skills. Although they performed no better in the fall, children in Classic Montessori programs, as compared with children in Supplemented Montessori and Conventional programs, showed significantly greater school-year gains on outcome measures of executive function, reading, math, vocabulary, and social problem-solving, suggesting that high fidelity Montessori implementation is associated with better outcomes than lower fidelity Montessori programs or conventional programs.     

Evaluation of Interactive Computerized Training to Teach Paraprofessionals to Implement Errorless Discrete Trial Instruction

Efficient and effective training strategies for paraprofessionals in special education settings face many challenges. Interactive computerized training (ICT)—a self-paced program that incorporates audio narration, video models, interactive activities, and competency checks—is one potential solution. ICT has been successful in training college students and special education teachers to implement discrete trial instruction (DTI), but its effectiveness to train paraprofessionals is unknown. Using a multiple-baseline design, we evaluated the feasibility of ICT, to train six paraprofessionals to implement DTI with an errorless learning procedure. Following ICT, the fidelity of implementation of DTI increased for all participants when implemented with a student in their classroom; however, competency varied. We added additional training components that progressed from low to more intensive feedback delivered remotely in attempt to increase fidelity to 90% or higher implementation. We also evaluated generalization to novel instructional programs and maintenance of instruction in the absence of feedback.     

Ready, Willing, and Able: Developing a Support System to Promote Implementation of School-Based Prevention Programs

While the number and scope of evidence‐based health, education, and mental health services continues to grow, the movement of these practices into schools and other practice settings remains a complex and haphazard process. The purpose of this paper is to describe and present initial support for a prevention support system designed to promote high‐quality implementation of whole school prevention initiatives in elementary and middle schools. The function and strategies of a school‐based prevention support system are discussed, including key structures and activities undertaken to identify, select, and provide technical assistance to school personnel. Data collected over a 5 year period are presented, including evidence of successful implementation support for 5 different evidence‐based programs implemented with fidelity at 12 schools and preliminary evidence of goal attainment. Findings suggest the ongoing collection of information related to organizational readiness assists in the adoption and implementation of effective practices and initiatives and provide valuable insight into the development of results‐oriented approaches to prevention service delivery. Problems, progress, and lessons learned through this process are discussed to frame future research and action steps for this school‐based prevention support system. Special Issue: Advances in Bridging Research and Practice Using the Interactive System Framework for Dissemination and Implementation; Guest Editors: Abraham Wandersman, Paul Flaspohler, Catherine A. Lesesne, Richard Puddy; Action Editor: Emilie Phillips Smith     

The center/satellite model: grand strategy for change

Whatever happened to the federal monies that used to support NDEA and EPDA counseling and guidance institute programs? Starting in 1971, an abrupt change in USOE policy led to the development of a new program, one that was no longer as much of a counseling and guidance program as it was a national and regional grand strategy for institutional and organizational change. In this article the author traces the new program from its origins during the days of the NDEA Institutes, and he also describes its potential as an instrument for change.     

Ecological interventions and the process of change for prevention: Wedding theory and research to implementation in real world settings

Considers a preventive intervention effort designed to modify the characteristics and regularities of the school environment so as to enhance the expectations conveyed to high risk students. Employing this case effort (Weinstein et al., 1991) as a base, several issues are highlighted that deserve greater attention in the design, implementation, assessment, and reporting of other preventively-focused social interventions. These issues include: (1) the degree of program implementation fidelity, (2) the impact a lack of attainment of program fidelity may have on arriving at erroneous evaluation conclusions about social programs, (3) the ways in which the theory and basic research employed in the program rationale is appropriate to and reflected in the program as designed, (4) issues of program potency and dosage, (5) the ways in which quantitative and qualitative evaluation data can enhance each other, and (6) the import of evaluability assessments of programs prior to moving to outcome evaluations. In addressing these issues, the importance of developmentally- and ecologically-informed program formulations is underscored. Finally, the critical role that program reports such as the Weinstein paper, with its careful discussion of both the process of implementation and outcome assessments, can play in the development of more effective prevention program implementation efforts is highlighted.     

CREATING NEW REALITIES: PROGRAM DEVELOPMENT AND DISSEMINATION

Program development and dissemination in human services present challenges and opportunities for social scientists. Over the past 27 years the Teaching-Family Model of group home treatment has moved from prototype development to widespread dissemination across North America. Reviewing concepts in industry related to product development and dissemination, the application of these concepts to a human services delivery system, and program replication and dissemination data offer information about how innovative human services can be widely adapted and adopted.     

BULLY PREVENTION IN POSITIVE BEHAVIOR SUPPORT

Bullying behaviors are a growing concern in U.S. schools. We present here a behavioral approach to bully prevention utilizing a schoolwide intervention. Bully prevention in positive behavior support (BP‐PBS) teaches students to withhold the social rewards hypothesized to maintain bullying. A single‐subject multiple baseline design across 6 students and three elementary schools was implemented in an empirical evaluation of the intervention's effectiveness. Results indicated that implementation was functionally related to decreased incidents of bullying for all 6 students observed. In addition, we observed a decrease in the social responses from victims and bystanders. Finally, school staff implemented the program with a high degree of fidelity and rated the program as effective and efficient. Limitations and implications of these results are discussed.     

What Happens After the Demonstration Phase? The Sustainability of Canada's At Home/Chez Soi Housing First Programs for Homeless Persons with Mental Illness

This research examined the sustainability of Canada's At Home/Chez Soi Housing First (HF) programs for homeless persons with mental illness 2 years after the end of the demonstration phase of a large (more than 2000 participants enrolled), five‐site, randomized controlled trial. Qualitative interviews were conducted with 142 participants (key informants, HF staff, and persons with lived experience) to understand sustainability outcomes and factors that influenced those outcomes. Also, a self‐report HF fidelity measure was completed for nine HF programs that continued after the demonstration project. A cross‐site analysis was performed, using the five sites as case studies. The findings revealed that nine of the 12 HF programs (75%) were sustained, and that seven of the nine programs reported a high level of fidelity (achieving an overall score of 3.5 or higher on a 4‐point scale). The sites varied in terms of the level of systems integration and expansion of HF that were achieved. Factors that promoted or impeded sustainability were observed at multiple ecological levels: broad contextual (i.e., dissemination of research evidence, the policy context), community (i.e., partnerships, the presence of HF champions), organizational (i.e., leadership, ongoing training, and technical assistance), and individual (i.e., staff turnover, changes, and capacity). The findings are discussed in terms of the implementation science literature and their implications for how evidence‐based programs like HF can be sustained.     

The KiVa antibullying curriculum and outcome: Does fidelity matter?

Research on school-based prevention suggests that the success of prevention programs depends on whether they are implemented as intended. In antibullying program evaluations, however, limited attention has been paid to implementation fidelity. The present study fills in this gap by examining the link between the implementation of the KiVa antibullying program and outcome. With a large sample of 7413 students (7–12 years) from 417 classrooms within 76 elementary schools, we tested whether the degree of implementation of the student lessons in the KiVa curriculum was related to the effectiveness of the program in reducing bullying problems in classrooms. Results of multilevel structural equation modeling revealed that after nine months of implementation, lesson adherence as well as lesson preparation time (but not duration of lessons) were associated with reductions in victimization at the classroom level. No statistically significant effects, however, were found for classroom-level bullying. The different outcomes for victimization and bullying as well as the importance of documenting program fidelity are discussed.     

AN ANALYSIS OF THE PROCEDURAL COMPONENTS OF SUPPORTED EMPLOYMENT PROGRAMS ASSOCIATED WITH EMPLOYMENT OUTCOMES

This study examined the relation between the procedural components of supported employment programs and employment outcomes for 120 individuals with disabilities. These individuals were involved in supported employment programs established through the Utah Supported Employment Project. The results suggest that successful implementation of supported employment services led to ongoing employment of study participants in community work sites, increased wages, and ongoing opportunities for workers to interact with nondisabled peers. In addition, several procedural components were found to be strongly associated with successful employment outcomes for workers. Results of the study are discussed in terms of the training needs of supported employment program staff and future research for the dissemination of a cohesive technology of supported employment.     

Career Development Programs in Business and Industry

Research and evaluation results of career development programs in business and industry indicate that a match between individual career needs and organizational goals plays a vital role in the implementation of a successful program.     

ESID,Dissemination, and Community Psychology: A Case of Partial Implementation?

Dissemination, the second stage of Experimental Social Innovation and Dissemination (ESID) is a critical, if not defining, element of this social change model. This paper attempts to assess the extent to which community psychology has adopted and implemented ESID's dissemination focus in its training and publications. We identify four levels of commitment to dissemination: dissemination advocate, dissemination activist, dissemination researcher, and experimental dissemination researcher. Content analyses of textbooks, journal publications, and conference papers and a brief survey of doctoral training in the field were conducted. Findings suggest that the dissemination aspects of ESID have been modestly and partially implemented within the field. That is, although there is some evidence of a commitment to dissemination practice (advocate, activist), there is much less evidence of a commitment to dissemination research. The implications of these findings for the effectiveness of the ESID model and for training and practice in community psychology are discussed.     

Systems change as an outcome and a process in the work of community collaboratives for health

The widespread development of comprehensive community initiatives that aim to improve community health is driven by the need to change the systems charged with delivering the services and creating the policies related to a variety of health outcomes. Georgia's Family Connection initiative is the nation's largest statewide network of community collaboratives for health, with collaboratives operating in 159 counties. Data on community context, collaborative processes, engagement in systems change, and changes in programs and activities implemented, gathered consistently at the collaborative level over 3 years, will be used to answer the following questions. How do community contexts and the structure and processes of collaboratives affect implementation of systems change? How do systems changes affect intermediate outcomes such as the type of programs offered in a community? Longitudinal change in systems change and program implementation is described and significant predictors of between‐collaborative variation in longitudinal change for each outcome are identified.     

Learning from Communities: Overcoming Difficulties in Dissemination of Prevention and Promotion Efforts

The model of prevention science advocated by the Institute of Medicine (P. J. Mrazek & R. J. Haggerty, 1994) has not lead to widespread adoption of prevention and promotion programs for four reasons. The model of dissemination of programs to communities fails to consider community and organizational capacity to implement programs, ignores the need for congruence in values between programs and host sites, displays a pro-innovation bias that undervalues indigenous practices, and assumes a simplistic model of how community organizations adopt innovations. To address these faults, researchers should locate, study, and help disseminate successful indigenous programs that fit community capacity and values. In addition, they should build on theoretical models of how locally developed programs work to make existing programs and polices more effective.     

Establishing Treatment Fidelity in Evidence-Based Parent Training Programs for Externalizing Disorders in Children and Adolescents

The current review evaluates the use of treatment fidelity strategies in evidence-based parent training programs for treating externalizing disorders. We used a broad framework for evaluating treatment fidelity developed by the National Institutes of Health Treatment Fidelity Workgroup that includes the aspects of treatment design, treatment delivery, training providers, and assessment of participant receipt of treatment and enactment of treatment skills. Sixty-five articles reporting outcome trials of evidence-based parent training programs met inclusion criteria and were coded for treatment fidelity strategies. The mean adherence to fidelity strategies was .73, which was higher than two previous review studies employing this framework in the health literature. Strategies related to treatment design showed the highest mean adherence (.83), whereas training of providers and enactment of treatment skills had the lowest (.58). In light of an increasing emphasis on effectiveness and dissemination trials, the broader treatment fidelity framework as applied in this review focuses needed attention on areas often overlooked in fidelity practices, such as training providers and generalization of treatment skills. We discuss the strengths and limitations of fidelity practices in parent training studies, implications of these findings, and areas for future research.     

The Prevention Delivery System: Organizational Context and Use of Comprehensive Programming Frameworks

The purpose of this exploratory study is to investigate organizational‐level mechanisms in the Prevention Delivery System (PDS) and their influence on implementing comprehensive programming frameworks (e.g., Communities that Care‐CtC) as the innovation. The PDS is part of the Interactive Systems Framework for Dissemination and Implementation (ISF) and describes key characteristics of innovation implementation and dissemination. The study addresses two research questions: (1) What types of organizational characteristics are related to successful use of each of the programming processes (i.e., planning, implementation, evaluation, and sustainability) that are part of comprehensive programming frameworks?; and (2) What are the similarities and differences in the organizational patterns correlated with use of each of the programming processes? Surveys, interview data, and other documents designed to assess organizational characteristics and extent of use of a comprehensive programming framework over time, were collected from 8 Community boards and 23 provider agencies. These organizations were responsible for planning and delivering substance abuse prevention services as part of a statewide initiative in Ohio. Data were analyzed using Spearman rho (and rank‐biserial) correlations, with an emphasis on effect sizes. Results indicated that leadership, shared vision, process advocates, and technical assistance were common correlates of use across programming processes. However, the role played by these organizational variables differed for each programming process, pointing to complex interactions of the organizational infrastructure with other variables (i.e., characteristics of the innovation itself and external macro‐level factors). This exploratory study provides preliminary data on the organizational‐level mechanisms of the PDS and the complexity of their relationships with the other Systems in the Interactive Systems Framework.     

Provider Perspectives on Outcome Goals for Children's Day Treatment

I describe service provider perspectives regarding outcome goals of children's day treatment and their ideas for how to make day treatment programs successful in achieving stated outcomes. Respondents were direct service staff from ten day treatment programs in one county in Washington state. Data were obtained through nominal group process methods. The primary outcome goals of day treatment reported were: community re-integration, improved adaptive functioning skills, psychological growth, and improved family functioning. Aspects of day treatment that may help achieve outcome targets included: team quality; comprehensive programming; a consistent, safe, accepting program environment; an individualized, developmental approach; linkages with other services; family involvement, and a low staff-to-client ratio. Outcome goals are discussed relative to those described within the research literature on children's mental health services. Implications for the use of outcome goals for quality improvement are described.     

Development and dissemination of collaborative family-oriented services: the case of community/day residential care in Israel

The initiation, development, and dissemination of family-oriented programs are a unifying thread that highlights family therapy's contribution to the fields of mental/physical health and social services. These demanding tasks require an ecosystemic vision, a supportive larger context, and a range of skills. This article delineates the evolution of community and day residential care in Israel by examining processes at different ecological levels: the formulation and implementation of national social policy, the follow-up of two family-oriented facilities, one of which thrived while the other eventually closed, and the residential care provided to 1 family with 3 children. The analysis of this multilevel data highlights 4 facilitating/obstructing factors that have had major impact on family-oriented programs: support by both national and local sociopolitical-professional administration, program's management autonomy, staff training, support and development, and effective facility leadership that establishes and nurtures family-oriented organizational structure and culture.