An Empowerment Evaluation Model for Sexual Assault Programs: Empirical Evidence of Effectiveness

Using an empowerment evaluation approach, the Sexual Assault and Rape Prevention (SARP) Evaluation Project brought together university researchers, public health evaluators, state funders, and program staff to provide evaluation training and consultation to all state-funded rape prevention and victim services programs in Michigan. In this paper, the specific activities of the SARP project are described, and process and outcome data are presented that address the effectiveness of this empowerment evaluation approach. Based on subjective reports from program staff and objective data obtained from state funders, results suggest that this evaluation model was successful in helping 90% of the prevention programs and 75% of the victim services programs successfully develop and launch program evaluations. One-year follow-up data indicate that 90% of the programs had sustained their evaluation efforts after the formal work of the SARP project had ended.     

Evaluating Ethics Education Programs: A Multilevel Approach

Although education in the responsible conduct of research is considered necessary, evidence bearing on the effectiveness of these programs in improving research ethics has indicated that, although some programs are successful, many fail. Accordingly, there is a need for systematic evaluation of ethics education programs. In the present effort, we examine procedures for evaluation of ethics education programs from a multilevel perspective: examining both within-program evaluation and cross-program evaluation. With regard to within-program evaluation, we note requisite designs and measures for conducting systematic program evaluation have been developed and multiple measures should be applied in program evaluation. With regard to cross-program evaluation, we argue that a meta-analytic framework should be employed where analyses are used to identify best practices in ethics education. The implications of this multilevel approach for improving responsible conduct of research educational programs are discussed.     

Comprehensive School Counseling Programs: A Review for Policymakers and Practitioners

This article describes components of effective school counseling programs that have emerged from 30 years of empirical research and professional standards. Results are summarized in seven sections: core principles of school counseling programs, program resources, program interventions, program evaluation, program renewal, written policies, and program climate.     

Practical Considerations and Alternative Research Methods for Evaluating HR Programs

Program evaluation methods capitalizing on quasi-experimental designs are introduced as useful alternatives for evaluating human resources (HR) programs such as selection systems, training courses, performance measurement systems, and 360 degree feedback. A case study is presented to illustrate the benefits of program evaluation methods for evaluating HR programs. In addition, several post-hoc considerations that often moderate results from HR evaluations are described. It is suggested that when these considerations are addressed, quasi-experimental program evaluation methods provide a promising tool for demonstrating bottom line impact of HR initiatives.     

Housing First and Photovoice: Transforming Lives,Communities, and Systems

This article presents findings from a community‐based participatory evaluation of a Housing First program on the Island of O'ahu. In this study, clients in a Housing First program used Photovoice to evaluate the program and to advocate for progressive housing policies. Written together by members of the Housing First Photovoice group, this collaborative article describes the outcomes from both the Housing First program and the Photovoice project and demonstrates the ways in which participatory program evaluations can interact with client‐driven programs like Housing First to produce a cumulative, transformative impact. Findings suggest that community psychologists hoping to re‐engage with community mental health systems through enacting transformative change should consider taking a community‐based participatory approach to program evaluation because increased client voice in community mental health programs and their evaluations can have far‐reaching, transformative impacts for research, practice, and policy.     

Exploring programmatic moderators of the effectiveness of marriage and relationship education programs: a meta-analytic study

This study uses meta-analytic methods to explore programmatic moderators or common factors of the effectiveness of marriage and relationship education (MRE) programs. We coded 148 evaluation reports for potential programmatic factors that were associated with stronger intervention effects, although the range of factors we could code was limited by the lack of details in the reports. Overall, we found a positive effect for program dosage: moderate-dosage programs (9-20 contact hours) were associated with stronger effects compared to low-dosage programs (1-8 contact hours). A programmatic emphasis on communication skills was associated with stronger effects on couple communication outcomes, but this difference did not reach statistical significance for the relationship quality/satisfaction outcome. There was no evidence that institutionalized MRE programs (formal manuals, ongoing presence, formal instructor training, multiple evaluations) were associated with stronger effects. Similarly, there was little evidence of differences in program setting (university/laboratory vs. religious). We discuss possible explanations for these findings and implications for program design and evaluation.     

A Meta-Analysis of Ethics Instruction Effectiveness in the Sciences

Scholars have proposed a number of courses and programs intended to improve the ethical behavior of scientists in an attempt to maintain the integrity of the scientific enterprise. In the present study, we conducted a quantitative meta-analysis based on 26 previous ethics program evaluation efforts, and the results showed that the overall effectiveness of ethics instruction was modest. The effects of ethics instruction, however, were related to a number of instructional program factors, such as course content and delivery methods, in addition to factors of the evaluation study itself, such as the field of investigator and criterion measure utilized. An examination of the characteristics contributing to the relative effectiveness of instructional programs revealed that more successful programs were conducted as seminars separate from the standard curricula rather than being embedded in existing courses. Furthermore, more successful programs were case based and interactive, and they allowed participants to learn and practice the application of real-world ethical decision-making skills. The implications of these findings for future course development and evaluation are discussed.     

Aboriginal perceptions of social and emotional wellbeing programs: A systematic review of literature assessing social and emotional wellbeing programs for Aboriginal and Torres Strait Islander Australians perspectives

Australian Aboriginal and Torres Strait Islander communities have been targets of social and emotional wellbeing programs for many years. However, the few health-care programs and services that are evaluated rarely provide insight into the participants' perspectives of program success or failure. This systematic review assessed 33 social and emotional wellbeing programs across Australia to better understand what Aboriginal community members think about the programs and how they could be improved. Results highlighted the interesting and valuable insights provided by Aboriginal participants, including what kinds of program activities and approaches are most suitable, what program characteristics are successful or desired, and their experiences of wellbeing change before and after program participation. They likewise voiced opinions about poorly received programs, culturally inappropriate services and negative experiences. This review highlighted how health and wellbeing programs must better engage Aboriginal and Torres Strait Islander clients to ensure that services are culturally safe, holistic, integrate appropriate staffing, include culturally relevant activities and value patient/participant experiences. These findings have significant implications for the health and wellbeing sector; specifically, research, policy, program design and implementation, evaluation methods, and self-determination.     

Planning and Evaluating Integrated School-Based Services

Summary

This paper reviews some of the challenges to evaluation presented by integrated school-based services, and describes field-based evaluation approaches designed to yield valid and useful information about such programs. In place of the positivistic research paradigm, a multistage evaluation approach is called for that includes qualitative and quantitative and process and outcome evaluations. This approach is characterized by close collaboration between evaluators and program personnel that makes evaluation an integral part of program planning and operation.     

Correspondence programs for smoking cessation and weight control: a comparison of two strategies in the Minnesota Heart Health Program

Mailed invitations to participate in weight loss and/or smoking cessation correspondence programs to 31,400 households in a suburban community. Two programs were offered to randomized subsets of households, a 6-month correspondence program costing +5 and the same program for free but requiring a +60 deposit to be refunded based on success in weight loss or smoking cessation. Overall, sign-up included 1,304 people for weight loss and 142 for smoking cessation. The +5 program was about 5 times as popular as the incentive program. Validated weight change after 6 months averaged about 4 lb for the +5 program and 8 lb for the incentive program. Corresponding rates of smoking cessation were about 9% and 20%, respectively. We conclude that correspondence programs for the promotion of weight control and smoking cessation are potentially cost-effective methods for reaching individuals in the community at large, many of whom would not be interested in clinic-based programs. Issues meriting further research include recruitment, especially of smokers, and evaluation of the relative trade-offs in recruitment success versus efficacy of differing treatment approaches.     

Empowering youth to change their world: Identifying key components of a community service program to promote positive development

Prior to graduating from high school, the vast majority of youth in the United States will take part in at least one community service activity. Although it is frequently assumed that community service is inherently beneficial to those that take part, the activities and processes of youth service programs tend to be unsystematic and vary widely. In addition, empirical assessment of youth service programs is inconsistent and often lacks methodological rigor. The present paper addresses these concerns in a preliminary evaluation of both the process experience and the outcomes of a theoretically grounded school-based community service program for urban adolescents. The evaluation focuses on identifying key components of a youth community service program, assessing whether the program was experienced as it was intended, and providing preliminary data on participant outcomes. Findings indicate that the program was experienced as both empowering (i.e., guided by student initiative, preferences, and strengths) and promoting a sense of community (i.e., encouraged cooperation and collective decision making). Results also suggest that taking part in the community service program increased youths' self-reported empathy and intent to be involved in future community action, as compared to a group of matched controls. Implications and recommendations for developing and evaluating service-learning or community service programs in the schools are discussed.     

AN EVALUATION OF TEACHING-FAMILY (ACHIEVEMENT PLACE) GROUP HOMES FOR JUVENILE OFFENDERS

Juvenile crime is a serious problem for which no treatment approach has been found to be reliably effective. This outcome evaluation assessed during and posttreatment effectiveness of Teaching-Family group home treatment programs for juvenile offenders. The evaluation included the original Achievement Place program, which was the prototype for the development of the Teaching-Family treatment approach, 12 replications of Achievement Place, and 9 comparison group home programs. Primary dependent measures were retrieved from court and police files and included number of alleged offenses, percentage of youths involved in those alleged offenses, and percentage of youths institutionalized. Other dependent measures were subjective ratings of effectiveness obtained from the program consumers, including the group home residents. The results showed difference during treatment favoring the Teaching-Family programs on rate of alleged criminal offenses, percentage of youths involved in those offenses, and consumer ratings of the programs. The consumer ratings provided by the youths and their school teachers were found to be inversely and significantly correlated with the reduction of criminal offenses during treatment. There were no significant differences during treatment on measures of noncriminal offenses (e.g., truancy, runaway, and curfew violations). In the posttreatment year, none of the differences between the groups was significant on any of the outcome measures. The results are discussed in terms of measurement and design issues in the evaluation of delinquency treatment programs and in relation to the evaluation of Teaching-Family group homes by Richard Jones and his colleagues.     

Future Directions for Health Promotion Program Development in Schools

Many schools will develop health promotion and disease prevention programs during this decade. Because of the rapidly changing nature of these programs, it is helpful to consider the major issues that will have an impact on health promotion program development. These issues include: (1) increased wellness focus; (2) improved database for planning; (3) computerized assessment procedures; (4) prioritized market segments; (5) standardization of service protocols; (6) confluent appraoches; (7) role delination for the school health team; (8) staff development programs; and (9) program evaluation.     

Decision analysis: An approach for multidisciplinary teams in planning special service programs

A decision analytic approach for use by multidisciplinary teams (MDTs) in planning special service programs is described, and formative evaluation information on its application by MDTs in planning individualized education programs (IEPs) is presented. Decision analysis is applied following initial determination of program goals and objectives. In using the approach, an MDT engages in a six-step process: (a) delineation of program alternatives, (b) determination of program outcomes, (c) assessment of probabilities, (d) assessment of utilities, (e) determination of overall program values, and (f) selection of the program to be developed. Utilization of the approach allows a complex program planning situation to be disaggregated into its essential elements, with discussion of program alternatives occurring relative to specific decision criteria. Strengths and limitations of the approach also are discussed, and future directions for research are briefly noted.     

After-School Youth Development Programs: A Developmental-Ecological Model of Current Research

Although there has been a rapid increase in funding and attention to after-school programs, there is little understanding of how after-school programs impact children's developmental trajectories. The heterogeneity of American children makes it very unlikely that all children need after-school programming or that there is but one brand of after-school programming suitable for all youth. We discuss the numerous developmental and contextual factors that may influence which children benefit most from after-school programs as well as the nature of the after-school programs most beneficial to children's needs. The value of utilizing an ecological and developmental perspective to after-school program evaluation are presented, including the need for improved research designs and more detailed analyses of program type and services as well as a more complete determination of which children benefit the most from after-school participation.     

The Impact of Legislative Standards on Batterer Intervention Program Practices and Characteristics

Changes in social policy are often pursued with the goal of reducing a social problem by improving prevention efforts, intervention program practices, or participant outcomes. State legislative standards for intimate partner violence intervention programs have been adopted nearly universally across the US, however, we do not know whether such standards actually achieve the intended goal of affecting programs’ policies and practices. To assess the effect that batterer intervention program (BIP) standards have on policies and practices of programs, this study used longitudinal surveys collected as part of an ongoing evaluation conducted from 2001 to the present to compare intervention program (N = 74) characteristics and practices at three time points before and after the adoption of standards in Oregon. Analyses were conducted to examine all BIPs in Oregon at each time point, as well as change among a subset of programs in existence at all survey assessments. Results indicate that across all programs, the use of mixed gender group co-facilitation increased by 14 % between 2004 and 2008, while program length increased by approximately 12 weeks. However, other practices such as programs’ coordination with community partners were unchanged. Analyses of within-program change revealed fewer differences, with only program length increasing significantly over the three assessments. These and other findings indicate that while standards affected program length as intended, other practices commonly addressed by legislative standards remained unchanged. The findings provide needed information regarding programs’ compliance with components of the standards, the potential need for compliance monitoring, and the potential impact of state standards on program effectiveness and on the prevalence of intimate partner violence.     

An Utility Based Framework for Evaluating the Financial Impact of Sales Force Training Programs

Limited attention has been devoted to the financial evaluation of sales training programs. In response to this shortcoming, this research proposes a sales training evaluation framework that integrates economic utility theory with Kirkpatrick's (1959a, 1959b, 1960a, 1960b) four-level training evaluation model. The proposed utility theory framework is tested using data derived from a sales training program conducted in Egypt. After performing the economic evaluation, sensitivity analysis is employed to demonstrate the financial trends of varying key training program variables. The paper concludes with discussions of theoretical and managerial implications, research limitations, and recommendations for future research.     

The Youth Public Arts Program: Interpersonal and Intrapersonal Outcomes for At-Risk Youth

This article provides the findings of a program evaluation of the Youth Public Arts Program, a long-standing community-based art program that serves at-risk youth in eastern North Carolina. The program evaluation was undertaken to examine the art skills, social and interpersonal skills, and community involvement of the participants. The study included four focus groups and telephone surveys with a total of 38 stakeholders (program participants, parents of participants, program staff, and referral sources). Results indicated that the at-risk youth had gains in both interpersonal and intrapersonal skills, including family relationships, peer relationships, community interaction, increased art skills, confidence, and self-regulation. This article adds to the very limited literature on community-based art programs for youth.