Developing Measures of Community-Relevant Outcomes for Violence Prevention Programs: A Community-Based Participatory Research Approach to Measurement

Community-Based Participatory Research is a research paradigm that encourages community participation in designing and implementing evaluation research, though the actual outcome measures usually reflect the “external” academic researchers’ view of program effect and the policy-makers’ needs for decision-making. This paper describes a replicable process by which existing standardized psychometric scales commonly used in youth-related intervention programs were modified to measure indicators of program success defined by community partners. This study utilizes a secondary analysis of data gathered in the context of a community-based youth violence prevention program. Data were retooled into new measures developed using items from the Alabama Parenting Questionnaire, the Hare Area Specific Self-Esteem Scale, and the Youth Asset Survey. These measures evaluated two community-defined outcome indicators, “More Parental Involvement” and “Showing Kids Love.” Results showed that existing scale items can be re-organized to create measures of community-defined outcomes that are psychometrically reliable and valid. Results also show that the community definitions of parent or parenting caregivers exemplified by the two indicators are similar to how these constructs have been defined in previous research, but they are not synonymous. There are nuanced differences that are important and worthy of better understanding, in part through better measurement.     

Evaluation of the effects of a childhood depression prevention program

The study aimed to design and evaluate a program for the prevention of childhood depression (“Pozik-Bizi” [in English, “Live-Happily”]), comparing its effects with a socio-emotional intervention program based on cooperative play. The sample comprised 420 students aged 7 to 10?years from the Basque Country, 51.9% were randomly assigned to the experimental condition (“Pozik-Bizi”) program and 48.1% to the control group (“Play program”). Using a pretest-posttest repeated measures experimental design, 7 evaluation instruments were administered. When comparing the two interventions, it was confirmed that those who participated in the “Pozik-Bizi” program significantly decreased their level of clinical maladjustment, school maladjustment, emotional, and behavioral problems, and they increased positive behaviors that inhibit depression. However, the cooperative play program improved self-concept and social skills significantly more than the “Pozik-Bizi” program. The effect size in all the variables was small. The discussion analyzes the effectiveness of specific programs of prevention of childhood depression versus global programs of social-emotional development. This work provides a program to prevent childhood depression that has been shown to be effective in the reduction of clinical variables. In addition, this study confirms the positive potential of programs of cooperative play, to increase self-concept and social skills.     

Benchmarking the “Question,Persuade, Refer” Program Against Evaluations of Established Suicide Prevention Gatekeeper Trainings

Gatekeeper training is a common approach for aiding suicidal youth. This study utilizes comparative effectiveness “benchmarks” from established programs to evaluate the Question, Persuade, Refer (QPR) program. The QPR program was implemented with adults (N = 2,389) working in youth‐serving community agencies. Questionnaires assessed suicide prevention knowledge, attitudes, subjective norms, perceived behavioral control, and behaviors. Consistent with benchmarked studies, participants in QPR demonstrated knowledge and suicide prevention behavior gains compared with control groups. Future research should utilize benchmarking methods as a measure of effectiveness, and more thoroughly assess mechanisms that promote behavior change.     

Drug prevention in a community setting: A longitudinal study of the relative effectiveness of a three-year primary prevention program in boys & girls clubs across the nation

Tested the effectiveness of a youth drug prevention program in a community setting. Boys & Girls Clubs of America's Stay SMART program, adapted from a school-based personal and social competence drug prevention program, was offered, with and without a 2-year booster program, to 13-year-old members of Boys & Girls Clubs. Over 27 months, (a) 5 Boys & Girls Clubs offered the Stay SMART program, (b) 5 Boys & Girls Clubs offered the Stay SMART program with the booster programs, and (c) 4 Boys & Girls Clubs served as a control group. The Stay SMART program alone and the Stay SMART program with the booster programs showed effects for marijuana-related behavior, cigarette-related behavior, alcohol-related behavior, overall drug-related behavior, and knowledge concerning drug use. The Stay SMART program with the booster programs produced additional effects for alcohol attitudes and marijuana attitudes after each year of booster programs. Results suggest that a school-based personal and social competence program can be adapted effectively to a community setting and that booster programs might enhance program effects. Implications for alternative community models of prevention are discussed. At the time of the study, the parent organization was Boys Clubs of America.This research was supported by a grant from the Office for Substance Abuse Prevention, ADAMHA, Public Health Service, Department of Health and Human Services, Grant No. 1 H84 AD 00903-01.     

EVALUATION OF AN INPATIENT PROGRAM AIMED AT PREPARING “HARD-TO-PLACE” CHRONICALLY MENTALLY ILL FOR THE COMMUNITY

A program designed to prepare long-stay “hard-to-place” psychiatric inpatients for successful community placement was evaluated. Fifteen patients in the Intensive Learning Center (ILC) program were assessed at 5- and 10-month intervals following baseline. In addition, their functioning was compared to a criterion related comparison group of 26 patients who had been transferred to community placements 18 months earlier. Two informant driven measures were used: The Rehabilitation Evaluation of Hall and Baker (REHAB), (Baker & Hall, 1988) and, Adaptive Behavior Scale (ABS), (Nihira, Foster, Shellhaas, & Leland, 1975). The results indicated significant improvement in the ILC patients adaptive functioning after 5 months, but this improvement was not consistently sustained at the 10 month follow-up. Comparisons between the ILC and community groups were made at 10 months and suggested considerable overlap in functioning between the two groups. The implications of the findings for program development and implementation as well as for discharge planning are discussed.     

Essential Elements for Community Engagement in Evidence-Based Youth Violence Prevention

In the field of youth violence prevention, there has been increasing emphasis on “evidence based” programs and principles shown through scientific research as reaching their intended outcomes. Community mobilization and engagement play a critical role in many evidence‐based programs and strategies, as it takes a concerted effort among a wide range of people within a community to alter behavior and maintain behavioral change. How do concerned individuals and groups within a community engage others within and outside of that community to effectively plan, develop and implement appropriate EB programs as well as evaluate the outcomes and impacts of locally developed programs yet to be proven? The authors discuss five elements essential for community engagement in evidence‐based youth violence prevention based on their work in a university‐community partnership through the Asian/Pacific Islander Youth Violence Prevention Center (API Center), a National Academic Center for Excellence on Youth Violence Prevention Center supported by the Centers for Disease Control and Prevention. They include: (a) aligning EBPs with a community's shared vision and values; (b) establishing an inclusive environment for the planning, implementation and evaluation of EBPs; (c) nurturing collaboration for increased effectiveness and efficacy of EBPs; (d) building adequate leadership and community capacity to develop and sustain EBPs; and (e) building a learning community for evaluation and self‐reflection. The authors propose placing greater emphasis on “evaluative thinking” and organizational capacity for evaluation as we pursue evidence‐based practices for youth violence prevention. This is especially important for ethnic groups for which an evidence base is not well established.     

The Decline of Youth Suicidal Behavior in an Urban,Multicultural Public School System Following the Introduction of a Suicide Prevention and Intervention Program

A 5-year longitudinal study of suicidal behavior of students attending Dade County Public Schools (DCPS) in Miami, Florida, between the 1989–1990 and 1993–1994 school years, is described. As the fourth largest public school system in the United States, DCPS is representative of an urban, multicultural community. The prevalence of suicidal behavior among DCPS students necessitated the development of a districtwide Suicide Prevention and School Crisis Management Program (SPSCMP), including the creation of the “Youth in Crisis Hotline.” A review of eight national school districts' suicide prevention and intervention programs indicates similarities with the DCPS model. Statistical data compiled by the DCPS program are organized into annual, monthly, grade level, and school level classifications, and are analyzed to determine the degree and direction of self-destructive behavior among youth in a culturally and linguistically diverse school population following the introduction of a suicide prevention and intervention program. Evaluative data regarding the effectiveness of the program as well as implications for suicide prevention and intervention are discussed.     

Preventing Depression Among Adolescent Girls: Pathways Toward Effective and Sustainable Programs

Up to 25% of adolescent girls experience an episode of major depression, at least twice the rate found with adolescent boys. In addition to reducing the suffering associated with depression, prevention efforts with this high-risk population have the potential to avert short- and long-term functional impairment, reduce the risk of associated mental and physical health problems, and provide mental health services to teens who may not otherwise receive help. The effectiveness of such programs depends upon the ability to reach at-risk girls and provide effective intervention, and to accomplish these goals in ways that are sustainable in community settings such as schools. This article discusses the advantages and drawbacks of several methods of delivering prevention programs for adolescent depression, and highlights strategies to enhance the effectiveness and sustainability of these programs. Lessons learned from the implementation of a universal prevention program for Grade 9 girls (Resourceful Adolescent Program-RAP; Shochet et al., 2001) are discussed, and practical recommendations are provided with regard to systems, program, implementation, and evaluation factors. Session-by-session details of RAP are highlighted, focusing on cognitive-behavioral and interpersonal therapy concepts and methods, adapted for use in a classroom setting.     

The “Flip”—Sustaining Complexity and Multiplicity Post-Quarantine

This article describes poststructural ways of responding to the pandemic by looking for openings or gaps within more traditional ways of interacting with dilemmas. The author situates herself within a “positive deviance” epistemology, which looks for what already works rather than getting captured by the current problem. We “flip” what are the usual ways of responding and explore three different programs that illustrate inventive and assets-based approaches. Linking-Lives Storython, created specifically to utilize our COVID shelter-in-place experience; Re-Authoring Teaching: Creating a Collaboratory, begun 12 years ago to meet the needs of a widespread community; and Witness to Witness, a current response to contemporary disastrous situations—all have utilized technology to open possibilities for those who are helped as well as those who are helpers. The article shows how each program has specifically flipped more traditional ways of responding, but also how this “flip” employs a practice of “disciplined improvisation.” Each program has a built-in structure that depends on technology to make it work; each has a disciplined approach that allows the helpers to improvise to meet the needs of the receivers. It is this “flip,” this way of thinking, that can sustain us and our work in times of great complexity and multiplicity.     

Teaching Research Methods to MFT Master’s Students: A Comparison between Scientist-Practitioner and Research-Informed Approaches

The current study compares a traditional “scientist-practitioner” and a “research-informed” approach utilizing a sample of students from a COAMFTE accredited master’s program. Students were enrolled in a research methods course with either a “scientist-practitioner” or a “research-informed” focus. Results from both self assessment of core competencies, as well as basic research knowledge, found that both course types led to significant improvements; however neither course emerged as more significant than the other. Implications for best practices in teaching research methods in MFT training programs are discussed.     

How school climate relates to chronic absence: A multi–level latent profile analysis

Chronic absence is a significant problem in schools. School climate may play an important role in influencing chronic absence rates among schools, yet little research has evaluated how school climate constructs relate to chronic absence. Using multilevel latent profile analysis, we evaluated how profiles of student perceptions of school climate at both the student and school level differentiated school–level rates of chronic absence. Participants included 25,776 middle and high school students from 106 schools who completed a district administered school climate survey. Students attended schools in a large urban school district where 89% of 6th through 12th grade students were African-American and 61% were eligible for the federally subsidized school meals program. Three student–level profiles of perceptions of school climate emerged that corresponded to “positive,” “moderate,” and “negative” climate. Two predominant patterns regarding the distribution of these profiles within schools emerged that corresponded to the two school–level profiles of “marginal climate” and “climate challenged” schools. Students reporting “moderate” and “negative” climate in their schools were more likely to attend schools with higher chronic absence rates than students reporting that their school had “positive” climate. Likewise, “climate challenged” schools had significantly higher chronic absence rates than “marginal climate” schools. These results suggest that school climate shares an important relation with chronic absence among adolescent students attending urban schools. Implications for prevention and intervention programs are discussed.     

Evaluating student achievement with a relational database

Commercial database programs such as dBase and Paradox, although developed originally for business applications, are versatile and powerful tools that can be used for an academic purpose such as evaluating student performance. They can be used to write and store test questions, assemble and print classroom or on-line laboratory tests, and calculate grades, test statistics, and so forth. Databases are flexible, unlike textbook “ancillary” test bank programs that are inextricably bound to the strictly linear format and brief shelf life of specific textbook editions. A prototypical relational database program is described, with which an instructor can produce tests based on generic terms adapted from Boneau’s (1990) study of psychological literacy, as well as on behavioral learning objectives adapted from Bloom’s (1956) taxonomy of educational objectives. As a relational database, the program integrates terms, objectives, questions, tests, and test scores, and avoids unnecessary data duplication and waste of computer storage space.     

THE CLASS SPECIFIC EFFECTS OF COMPLIANCE TRAINING WITH “DO” AND “DON'T” REQUESTS: ANALOGUE ANALYSIS AND CLASSROOM APPLICATION

Two experiments are reported in which the relationship between compliance with “do” and “don't” requests was examined with developmentally disabled children. In Experiment 1, a multiple baseline design across subjects with counterbalanced treatment conditions was used to evaluate a compliance training program composed of four phases: (a) baseline, during which no consequences were delivered for compliance, (b) reinforcement for compliance with one targeted “do” request, (c) reinforcement for compliance with one targeted “don't” request, and (d) follow-up with reinforcement on a variable ratio schedule for compliance with any “do” or “don't” request. Results of probes conducted before and after training within each condition indicated that generalized compliance occurred only with requests of the same type as the target exemplar (“do” or “don't”). In Experiment 2, these results were replicated in a classroom setting. Following collection of baseline probe data on student compliance, a teacher training program was successfully implemented to increase reinforcement of compliance first with one “do” and subsequently with one “don't” request of a target student. Results of multiple baseline probes across “do” and “don't” requests indicated that the teacher generalized and maintained reinforcement of compliance with other requests of the same type and to other students, with a resulting increase in student compliance with the type of requests reinforced. The impact of treatment on both teacher and student behavior was socially validated via consumer ratings. Implications of these findings with respect to response class formation and compliance training programs are discussed.     

Education and Mental Health: New Directions for Interaction

Traditionally, mental health workers (psychiatrists, psychologists, and social workers) provide diagnostic evaluations and individual or group psychotherapeutic or casework treatment. With the advent of the community mental health movement, a new role model, that of “mental health consultant,” has emerged. This paper outlines the functions of the “mental health consultant” in the Job Corps program as a way of highlighting the functions of the mental health professional working in this new way. Educational, training, and work programs may be enriched by mental health consultation, and the new directions for interaction between education and mental health are described in this paper.     

Understanding and Improving the Effectiveness of After-School Practice

The question driving research on after‐school programs is shifting from “do programs make a difference,” to “why are some programs effective while others are not?” This article draws on the work in this volume and related studies to suggest that there is an emerging consensus on the importance of staff‐youth interactions as a determinant of program effectiveness. The commentary recommends that future research should continue to focus on understanding and improving program practices at the point‐of‐service. In doing so, two lines of inquiry seem promising. The first involves linking measures of changes in staff‐youth interactions to changes in developmental outcomes. The second is to examine how policies or other interventions intended to improve program effectiveness affect the practices of line staff.     

Collaborative Consultant: A New Role for Counselors Serving At-Risk Youths

School and community counselors have important roles in developing and implementing mental health programs for youths and families. Counselors who are working with the more complex issues of at-risk youths can enhance implementation of prevention programs by altering their roles to function within a collaborative consultation model. This model represents a new paradigm in that counselors interdependently and collaboratively partner with people previously considered their “consultees.” Such collaboration can result in more comprehensive and integrated programs, and it can prevent fragmentation and duplication of efforts. Collaborative consultation is described and contrasted with more traditional consultation models, and a 5-stage implementation process is presented. Personal and systemic factors that support the collaborative process are examined.     

Sociodemographic Factors Associated with Types of Projects Implemented by Volunteer Lay Health Educators in Their Congregations

This study focused on the association between type of community health interventions and lay health educator variables. Lay health educators are volunteers from local faith communities who complete a healthcare training program, taught by physicians in-training. Lay health educators are instructed to implement health-related initiatives in their respective communities after graduation. Of the 72 graduates since 2011, we surveyed 55 lay health educators to gain insight into their involvement with their congregation and the type of health projects they have implemented. We dichotomized the health projects into “raising awareness” and “teaching new health skills.” Using adjusted logistic regression models, variables associated with implementing health projects aimed at teaching health skills included length of time as a member of their congregation, current employment, and age. These results may help future programs prepare lay health community educators for the type of health interventions they intend to implement in their respective communities.     

Parcours : développement d’une stratégie de prévention de la récidive adaptée aux milieux ouverts

Experiments show that applying RNR principles “in the real world” within the context of probation can reduce recidivism, provided that evidence-based intervention models are used. In Quebec, probation officers agreed to take part in a pilot project whereby the Parcours program (Pathways) was transposed into a series of interview outlines to be used in a community setting. The experiment took place over a one-year period, the management of convicted offenders consisting in around 10 30 to 90-minute interviews. These interviews addressed: motivation for change, beliefs, values and cognitive distortions, as well as the adoption of relapse prevention strategies. Results were positive enough to authorize the deployment of Parcours en milieu ouvert (Pathways in community settings) in all sectors. This approach could be implemented in France, where at the instigation of the Council of Europe, new European probation rules recommend the establishment of interventions based on evidence-based models.     

Patching the Pathway and Widening the Pipeline: Models for Developing a Diverse Early Childhood Workforce in Chicago

With the growing appreciation of the importance of early learning experiences for children's healthy development, attention to the cultivation and maintenance of a qualified workforce has steadily increased. Such a workforce must have not just the knowledge and skills related to child development and early learning, but also be linguistically and culturally prepared to meet the needs of an increasingly diverse child and family population. To ensure a highly qualified workforce, programs and policymakers must attend to both the “pipeline” through which new early childhood educators (ECEs) enter the workforce and the “pathways” by which ECEs work toward and obtain the necessary education and credentials for different roles within the field. In line with the aims of this special issue, this paper leverages the first‐person account style to describe barriers to and creative solutions for the development of practitioners in low‐resourced communities in Chicago, with the goal of informing practice and policy. We describe three prior and ongoing partnership programs between community‐based organizations and institutions of higher education, each tailored to support a unique population in the ECE pipeline on the pathway for increased educational attainment and credentialing. Each program is grounded in a specific community of Chicago, a diverse city with a sizable population of children raised in non‐English speaking homes. Each program addresses specific needs of the communities they serve, especially around the recruitment, retention, and promotion of bilingual ECEs. Program administrators and community members describe each programs’ goals, development, and key components unique to their target population as well as key takeaways. We conclude with an overview of critical components that we identified across these programs in order to create pathways for change within the workforce and the communities they serve.