Implementation Quality and Positive Experiences in After-School Programs

Data collected during an evaluation of a multi-site trial of an enhanced after-school program were used to relate quality of program implementation to student experiences after school. The enhanced after-school program incorporated a drug use and violence prevention component that was shown to be effective in previous research. Building on Durlak and Dupre’s (Am J Community Psychol 41:327–350, 2008) dimensions of implementation, we assessed the level of dosage, quality of management and climate, participant responsiveness, and staffing quality achieved at the five program sites. We evaluated how these characteristics co-varied with self-reported positive experiences after-school. The study illustrates how multiple dimensions of program implementation can be measured, and shows that some but not all dimensions of implementation are related to the quality of student after-school experiences. Measures of quality of management and climate, participant responsiveness, and staffing stability were most clearly associated with youth experiences. The importance of measuring multiple dimensions of program implementation in intervention research is discussed.     

The Role of a State‐Level Prevention Support System in Promoting High‐Quality Implementation and Sustainability of Evidence‐Based Programs

Although numerous evidence‐based programs (EBPs) have been proven effective in research trials and are being widely promoted through federal, state, and philanthropic dollars, few have been “scaled up” in a manner likely to have a measurable impact on today's critical social problems. The Interactive Systems Framework for Dissemination and Implementation (ISF) explicates three systems that are critical in addressing the barriers that prevent these programs from having their intended public health impact. In this article we describe the relevance of these systems in a real‐world context with a specific focus on the Prevention Support System (PSS). We expand on the ISF model by presenting funders and policy‐makers as active and engaged stakeholders, and demonstrate how a state‐level PSS has used empirical evidence to inform general and program‐specific capacity‐building and support interactions among researchers, funders, and practitioners in Pennsylvania. By embracing this expanded ISF framework as a conceptual model for the wide‐scale dissemination and support of EBPs, and recognizing the need for a distinct state‐level PSS, Pennsylvania has created an infrastructure to effectively address the primary barriers to moving from lists of EBPs to achieving population‐level public health improvement. Special Issue: Advances in Bridging Research and Practice Using the Interactive System Framework for Dissemination and Implementation; Guest Editors: Abraham Wandersman, Paul Flaspohler, Catherine A. Lesesne, Richard Puddy; Action Editor: Emilie Phillips Smith     

Challenges in Supporting Community Implementation of Science-based Programs: A Critical Review of Local Partnerships for Success Plans

This paper provides the opportunity to consider local challenges to implementing science-based programs. Wandersman et al. (American Journal of Community Psychology 2008) define three systems necessary to effectively implement science-based programming. These systems include the Prevention Support System, which is responsible for assisting communities in implementing effective practices; the Prevention Research System, which is responsible for developing and packaging prevention programming; and the Prevention Delivery System, which is responsible for delivering services. The authors conduct a content analysis related to six county level efforts to implement science-based programs in Ohio. These counties are part of a larger Ohio initiative referred to as Partnerships for Success. The authors suggest that local officials are quite skilled at accessing the Prevention Research System and providers at the local level represent an effective service delivery system. The authors contend that the Prevention Support System in these counties is capable of enhancing several functions related to local infrastructure but is deficient in other important elements.     

Issues in Professional Training to Implement Evidence‐based Parenting Programs: The Preferences of Indigenous Practitioners

Indigenous children have elevated risk for poor health, behavioural, emotional, and social outcomes. Significant evidence exists that parenting programs can reduce family risk factors and improve outcomes for children and families; however, mainstream programs have had slower uptake in Indigenous communities than other communities. Culturally sensitive delivery of evidence‐based programs can enhance engagement of parents, yet the development of a workforce to deliver programs to Indigenous parents faces many obstacles. This project seeks to identify professional training processes that enhance Indigenous practitioners’ skills and confidence in delivering an evidence‐based parenting program. A survey of trained parenting practitioners via an online practitioner network assessed their views of the training and post‐training support processes they had experienced. Respondents were 57 Indigenous and 720 non‐Indigenous practitioners from 15 countries. Most training processes were rated equally helpful by Indigenous and non‐Indigenous practitioners. However, several training processes were identified as important for the delivery of culturally competent training, such as tailoring the pace of training and simplifying the language in teaching resources. Practitioners with higher ratings of the helpfulness of peer support following training reported higher program uptake and implementation. Qualitative themes also focused on the helpfulness of program resources, and having a peer support network and mentoring. Increasing access to appropriate, flexibly delivered training and post‐training support for Indigenous professionals will support the development of a skilled workforce with local knowledge and connections, and further increase the reach of evidence‐based services in Indigenous communities.     

New Perspectives on the Child‐ and Youth‐Serving Workforce in Low‐Resource Communities: Fostering Best Practices and Professional Development

The professionals and paraprofessionals who work daily with youth in low‐resource, marginalized communities are integral to youth wellbeing; yet, their professional development, and the factors that promote it, are not well understood. In this introduction to the special issue, Understanding and Strengthening the Child‐ and Youth‐Serving Workforce in Low‐Resource Communities, we focus on understudied practitioners operating in an array of sectors and settings, such as home visitors, mental health paraprofessionals, early childhood assistant teachers, teachers in low‐income countries, school resource officers, juvenile justice staff, and after‐school and community‐based program workers. We put forward a conceptual model detailing the interactive, layered set of proximal‐to‐distal ecological factors that influence the practice and professional development of these workers, and show how papers in the current issue address these layers in their examination of workforce development. We conclude with a summary of the contributions and lessons from this work – including the value of a whole‐person approach, the importance of sharing process across research stages, and the need to build on the foundation provided by community psychology and implementation science – toward the twin goals of understanding and building the skills and strengths of the workforce, and ultimately, enhancing youth development.     

Evidence‐Based Human Trafficking Policy: Opportunities to Invest in Trauma‐Informed Strategies

Human trafficking is a critical social issue characterized by chronic trauma among victims, and frequently preceded by traumatic experiences that contribute to risk of victimization. Therefore, the research‐based practice of trauma‐informed care is a highly appropriate lens for both prevention and intervention. This work examines federal legislation in the United States related to human trafficking for references to trauma, as well as how the use of research could implicitly direct public policy responses toward trauma‐informed approaches. Legislation on human trafficking has risen substantially since 1989, and the use of research and trauma language within these policies has also observed substantial increases. While the use of trauma language was associated with limited progression in the policy process, legislation using research language was more likely to pass out of Committee and become enacted. Moreover, legislation may leverage research in ways that have the potential to bolster trauma‐informed practice among human trafficking victims. Specifically, research can be used to describe the problem and causal mechanisms (e.g., impact of trauma), guide “best practice” for service delivery, and generate knowledge through studies and evaluations that guide future policy. Therefore, human trafficking legislation that implicitly guides trauma‐informed practice via the use of research may be particularly promising for the field.     

Sustaining the Utilization and High Quality Implementation of Tested and Effective Prevention Programs Using the Communities That Care Prevention System

This paper describes the extent to which communities implementing the Communities That Care (CTC) prevention system adopt, replicate with fidelity, and sustain programs shown to be effective in reducing adolescent drug use, delinquency, and other problem behaviors. Data were collected from directors of community‐based agencies and coalitions, school principals, service providers, and teachers, all of whom participated in a randomized, controlled evaluation of CTC in 24 communities. The results indicated significantly increased use and sustainability of tested, effective prevention programs in the 12 CTC intervention communities compared to the 12 control communities, during the active phase of the research project when training, technical assistance, and funding were provided to intervention sites, and 2 years following provision of such resources. At both time points, intervention communities also delivered prevention services to a significantly greater number of children and parents. The quality of implementation was high in both conditions, with only one significant difference: CTC sites were significantly more likely than control sites to monitor the quality of implementation during the sustainability phase of the project.     

Supporting Out‐of‐School Time Staff in Low Resource Communities: A Professional Development Approach

Federally funded out‐of‐school time (OST) programs provide academic support, enrichment, and safety for students and families in low‐resource communities. However, programs struggle to meet these aims, in part because of the lack of program structure and limited training and support for staff. This observational case study documents the training and technical assistance (TA) delivered to OST frontline staff and program leadership to implement Positive Behavior in Out‐of‐School Time (Positive BOOST), an adaptation of positive behavior interventions and supports conducted in multiple settings. Findings across three programs indicate that varied levels of TA (i.e., business as usual, performance feedback, coaching) are associated with different levels of staff‐ and program‐level implementation. Taken together with previous research, these findings suggest that targeted investment in developing the skills of OST staff and improving program‐wide outcomes is critical for supporting youth in low‐resource communities.     

Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation

The first purpose of this review was to assess the impact of implementation on program outcomes, and the second purpose was to identify factors affecting the implementation process. Results from over quantitative 500 studies offered strong empirical support to the conclusion that the level of implementation affects the outcomes obtained in promotion and prevention programs. Findings from 81 additional reports indicate there are at least 23 contextual factors that influence implementation. The implementation process is affected by variables related to communities, providers and innovations, and aspects of the prevention delivery system (i.e., organizational functioning) and the prevention support system (i.e., training and technical assistance). The collection of implementation data is an essential feature of program evaluations, and more information is needed on which and how various factors influence implementation in different community settings.     

The Quality Implementation Framework: A Synthesis of Critical Steps in the Implementation Process

Implementation science is growing in importance among funders, researchers, and practitioners as an approach to bridging the gap between science and practice. We addressed three goals to contribute to the understanding of the complex and dynamic nature of implementation. Our first goal was to provide a conceptual overview of the process of implementation by synthesizing information from 25 implementation frameworks. The synthesis extends prior work by focusing on specific actions (i.e., the “how to”) that can be employed to foster high quality implementation. The synthesis identified 14 critical steps that were used to construct the Quality Implementation Framework (QIF). These steps comprise four QIF phases: Initial Considerations Regarding the Host Setting, Creating a Structure for Implementation, Ongoing Structure Once Implementation Begins, and Improving Future Applications. Our second goal was to summarize research support for each of the 14 QIF steps and to offer suggestions to direct future research efforts. Our third goal was to outline practical implications of our findings for improving future implementation efforts in the world of practice. The QIF's critical steps can serve as a useful blueprint for future research and practice. Applying the collective guidance synthesized by the QIF to the Interactive Systems Framework for Dissemination and Implementation (ISF) emphasizes that accountability for quality implementation does not rest with the practitioner Delivery System alone. Instead, all three ISF systems are mutually accountable for quality implementation. Special Issue: Advances in Bridging Research and Practice Using the Interactive System Framework for Dissemination and Implementation; Guest Editors: Abraham Wandersman, Paul Flaspohler, Catherine A. Lesesne, Richard Puddy; Action Editor: Emilie Phillips Smith     

Post G20: The Challenge of Change,Implementing Evidence‐based Public Order Policing

In the wake of the 2011 ‘riots’, public order policing tactics in England and Wales have once again been brought into question. Yet, the riots came two years since police regulatory authorities in the UK called for fundamental reforms to the policing of public order. Questions are raised about why the change called for appears to have been so slow and what can be done to assist reform. This paper suggests that developing an evidence‐based policing approach within the field of public order policing to inform police decision‐making would provide the answers. By doing so, the paper addresses some of the possible barriers to implementing evidence‐based policing in public order and calls for police academic partnership to overcome these to make ‘change’ an ongoing reality. Copyright © 2012 John Wiley & Sons, Ltd.     

Challenges in Developing a Cultural Evidence‐Based Psychotherapy in the USA: Suggestions for International Studies

In this article, we underscore some of the cultural challenges and limitations of the evidence‐based psychotherapy (EBP) utilised within the USA with the hope that these problems will be avoided or ameliorated in Australia and other countries. Although the EBP research is very diverse and complex, we categorise this fast‐growing literature using a threefold classification system that underscores culture. This classification system is proposed to examine the influence of culture on EBPs. The three categories used are (a) universal evidence‐based psychotherapies or universal psychotherapies, (b) race and ethnic minority evidence‐based psychotherapies or racial/ethnic psychotherapies, and (c) cultural evidence‐based psychotherapies or cultural psychotherapies. After each category is described, its benefits and limitations are discussed, emphasising implications for culturally diverse and international studies. We conclude with suggestions to promote the advancement of a more effective and inclusive cultural EBP in which researchers from different countries can collaborate and learn from each other.     

Treating Adults With Complex Trauma: An Evidence‐Based Case Study

This article presents an overview of complex trauma and a case study describing the evidence‐informed treatment of a 25‐year‐old woman with chronic complex trauma symptoms resulting from childhood sexual, physical, and psychological abuse from a caregiver. Treatment followed a relationship‐based, cognitive behavior therapy model across 3 treatment phases. Significant reductions in trauma and general distress symptoms occurred from pre‐ to posttreatment, although firm conclusions about cause cannot be drawn from case studies. Verbatim vignettes illustrate specific interactions.     

Workforce Support for Urban After‐School Programs: Turning Obstacles into Opportunities

Organized after‐school programs can mitigate risk and build resilience for youth in urban communities. Benefits rely on high‐quality developmental experiences characterized by a supportive environment, structured youth–adult interactions, and opportunities for reflective engagement. Programs in historically disenfranchised communities are underfunded; staff are transient, underpaid, and undertrained; and youth exhibit significant mental health problems which staff are variably equipped to address. Historically, after‐school research has focused on behavior management and social‐emotional learning, relying on traditional evidence‐based interventions designed for and tested in schools. However, after‐school workforce and resource limitations interfere with adoption of empirically supported strategies and youth health promotion. We have engaged in practice‐based research with urban after‐school programs in economically vulnerable communities for nearly two decades, toward building a resource‐efficient, empirically informed multitiered model of workforce support. In this paper, we offer first‐person accounts of four academic–community partnerships to illustrate common challenges, variability across programs, and recommendations that prioritize core skills underlying risk and resilience, align with individual program goals, and leverage without overextending natural routines and resources. Reframing obstacles as opportunities has revealed the application of mental health kernels to the after‐school program workforce support and inspired lessons regarding sustainability of partnerships and practice.     

Enhancing Capacity for Trauma‐informed Care in Child Welfare: Impact of a Statewide Systems Change Initiative

Youth involved in the child welfare system (CWS) are disproportionally impacted by the negative effects of exposure to trauma. While efforts to develop trauma‐informed CWSs are accelerating, little research is available about the effects of these efforts on system capacity to respond to the needs of youth exposed to trauma. No studies evaluate longer‐term effects of these efforts. In 2011, Connecticut implemented CONCEPT, a multi‐year initiative to enhance capacity of the state's CWS to provide trauma‐informed care. CONCEPT used a multi‐component approach including workforce development, deployment of trauma screening procedures, policy change, improved access to evidence‐based trauma‐focused treatments, and focused evaluation of program effects. Changes in system capacity to deliver trauma‐informed care were assessed using statewide stratified random samples of child welfare staff at three time points (Year 1: N = 223, Year 3: N = 231, Year 5: N = 188). Significant improvements across nearly all child welfare domains were observed during the first 3 years of implementation, demonstrating system‐wide improvements in capacity to provide trauma‐informed care. These gains were maintained through the final year of implementation, with continued improvements in ratings of collaboration between child welfare and behavioral health settings on trauma‐related issues observed. Responses documented familiarity with and involvement in many of the CONCEPT activities and initiatives. Staff reported greater familiarity with efforts to increase access to specific evidence‐based services (e.g., TF‐CBT) or to enhance trauma‐related policy and practice guidelines, but less familiarity with efforts to implement new practices (e.g., trauma screening) in various sectors. Staff also reflected on the contribution of these components to enhance system capacity for trauma‐informed care.     

Top-Down,Bottom-Up,and Around the Jungle Gym: A Social Exchange and Networks Approach to Engaging Afterschool Programs in Implementing Evidence-Based Practices

This paper uses concepts from social networks and social exchange theories to describe the implementation of evidence-based practices in afterschool programs. The members of the LEGACY Together Afterschool Project team have been involved in conducting collaborative research to migrate a behavioral strategy that has been documented to reduce disruptive behaviors in classroom settings to a new setting—that of afterschool programs. We adapted the Paxis Institute’s version of the Good Behavior Game to afterschool settings which differ from in-school settings, including more fluid attendance, multiple age groupings, diverse activities that may take place simultaneously, and differences in staff training and experience (Barrish et al. in J Appl Behav Anal 2(2):119–124, 1969; Embry et al. in The Pax Good Behavior Game. Hazelden, Center City, 2003; Hynes et al. in J Child Serv 4(3):4–20, 2009; Kellam et al. in Drug Alcohol Depend 95:S5–S28, 2008; Tingstrom et al. in Behav Modif 30(2):225–253, 2006). This paper presents the experiences of the three adult groups involved in the implementation process who give first-person accounts of implementation: (1) university-based scientist-practitioners, (2) community partners who trained and provided technical assistance/coaching, and (3) an afterschool program administrator. We introduce here the AIMS model used to frame the implementation process conceptualized by this town–gown collaborative team. AIMS builds upon previous work in implementation science using four phases in which the three collaborators have overlapping roles: approach/engagement, implementation, monitoring, and sustainability. Within all four phases principles of Social Exchange Theory and Social Network Theory are highlighted.     

Valuing Our Communities: Ethical Considerations for Economic Evaluation of Community‐Based Prevention

Restricted public budgets and increasing efforts to link the impact of community interventions to public savings have increased the use of economic evaluation. While this type of evaluation can be important for program planning, it also raises important ethical issues about how we value the time of local stakeholders who support community interventions. In particular, researchers navigate issues of scientific accuracy, institutional inequality, and research utility in their pursuit of even basic cost estimates. We provide an example of how we confronted these issues when estimating the costs of a large‐scale community‐based intervention. Principles for valuing community members’ time and conducting economic evaluations of community programs are discussed.     

The impact of traditionality/modernity on identification‐ and calculus‐based trust

We conducted three studies to explore how trust is perceived, displayed or changed by participants in China, which as a country has undergone substantial cultural and historical transition. Traditionality showed a significant positive correlation with identification‐based trust (IBT) and modernity was significantly associated with calculus‐based trust (CBT; Study 1). Compared to those primed with modern Chinese culture stimuli, participants primed with traditional icons exhibited a higher level of IBT and a lower level of CBT (Study 2). The study further demonstrated more trust behaviour based on identification with traditional culture priming and more trust behaviour based on calculus when primed with modern culture (Study 3). In sum, the present findings suggest that, along with substantial social changes and economic development, people exhibit their trust differently depending on cultural context, shedding light on the trust crisis in contemporary China.     

An Evidence‐Based Model for Understanding the Mental Health Experiences of Transgender Australians

A growing body of empirical research has documented the mental health experiences of Australian transgender people. This research indicates three key factors that appear to play a role in determining outcomes for adult transgender Australians: (a) discrimination, (b) access to hormones and/or surgery, (c) community connectedness. Two theoretical frameworks clarify why these factors exist and how they can lead to either negative or positive mental health outcomes. The first is cisgenderism, which describes the ideology that delegitimises people's own understanding of their genders and bodies. Although anyone can experience cisgenderism, it is a particularly common experience for transgender people. The second is decompensation, which describes the processes through which the cumulative effects of stressors may lead to poor mental health. Drawing on both previous empirical findings and these two theoretical frameworks, this article proposes, and provides initial testing of, a model for understanding the mental health of transgender adults in Australia. The article concludes by suggesting the need for changes regarding how decisions are made about mental health service provision for transgender people, how mental health professionals understand the lives of transgender people, and how legislation can better ensure the full inclusion of transgender people in Australia.