Reporting a program evaluation: Needs,program plan,intervention, and decisions

The approach to intervention programs varies depending on the methodological perspective adopted. This means that health professionals lack clear guidelines regarding how best to proceed, and it hinders the accumulation of knowledge. The aim of this paper is to set out the essential and common aspects that should be included in any program evaluation report, thereby providing a useful guide for the professional regardless of the procedural approach used. Furthermore, the paper seeks to integrate the different methodologies and illustrate their complementarity, this being a key aspect in terms of real intervention contexts, which are constantly changing. The aspects to be included are presented in relation to the main stages of the evaluation process: needs, objectives and design (prior to the intervention), implementation (during the intervention), and outcomes (after the intervention). For each of these stages the paper describes the elements on which decisions should be based, highlighting the role of empirical evidence gathered through the application of instruments to defined samples and according to a given procedure.     

“Here we're all in the same boat” – a qualitative study of group based rehabilitation for sick‐listed citizens with chronic pain

Musculoskeletal pain impacts upon everyday life. A degree of chronicity may pose an increased risk of sickness absence. One of two rehabilitative interventions, “Tailored Physical Activity” or “Chronic Pain Self‐Management Program”, was offered to sick‐listed citizens who experienced pain. The objectives of this paper were to: (1) Assess what factors are experienced as problematic for sick‐listed citizens in everyday life with chronic pain, and (2) Evaluate the significance of two distinct rehabilitative interventions on the future everyday lives of sick‐listed citizens. Seven semi‐structured interviews with sick‐listed citizens were analyzed using a phenomenological‐hermeneutical approach. Results were discussed by applying the theoretical framework of Antonovsky's salutogenetic model and Yaloms principles for group psychology. The potential for development of citizen's coping is evaluated based on Roessler's notion of progression. The analysis revealed four main themes: (1) Living with pain and unemployment; (2) “Putting my foot down” and “asking for help”; (3) Significance of the group, including instructors, and; (4) Aspects significant to progression. Unemployment is a major life event that promotes stress and can be accompanied by problems related to depressed mood, acceptance of the life situation, feelings of not being useful, feelings of losing control and identity conflicts. Group characteristics that gave a significant basis for progression in the self‐management program are both emotional and instrumental, while the physical training program offers a “here‐and‐now”‐experience and motivation to participate. This study indicates that the self‐management program could potentially improve coping while the physical activity program revealed one example of a means of progression.     

Specific Features of After-School Program Quality: Associations with Children’s Functioning in Middle Childhood

This longitudinal study examined associations between three after-school program quality features (positive staff–child relations, available activities, programming flexibility) and child developmental outcomes (reading and math grades, work habits, and social skills with peers) in Grade 2 and then Grade 3. Participants (n = 120 in Grade 2, n = 91 in Grade 3) attended after-school programs more than 4 days per week, on average. Controlling for child and family background factors and children’s prior functioning on the developmental outcomes, positive staff–child relations in the programs were positively associated with children’s reading grades in both Grades 2 and 3, and math grades in Grade 2. Positive staff–child relations also were positively associated with social skills in Grade 2, for boys only. The availability of a diverse array of age-appropriate activities at the programs was positively associated with children’s math grades and classroom work habits in Grade 3. Programming flexibility (child choice of activities) was not associated with child outcomes.     

Empowering youth to change their world: Identifying key components of a community service program to promote positive development

Prior to graduating from high school, the vast majority of youth in the United States will take part in at least one community service activity. Although it is frequently assumed that community service is inherently beneficial to those that take part, the activities and processes of youth service programs tend to be unsystematic and vary widely. In addition, empirical assessment of youth service programs is inconsistent and often lacks methodological rigor. The present paper addresses these concerns in a preliminary evaluation of both the process experience and the outcomes of a theoretically grounded school-based community service program for urban adolescents. The evaluation focuses on identifying key components of a youth community service program, assessing whether the program was experienced as it was intended, and providing preliminary data on participant outcomes. Findings indicate that the program was experienced as both empowering (i.e., guided by student initiative, preferences, and strengths) and promoting a sense of community (i.e., encouraged cooperation and collective decision making). Results also suggest that taking part in the community service program increased youths' self-reported empathy and intent to be involved in future community action, as compared to a group of matched controls. Implications and recommendations for developing and evaluating service-learning or community service programs in the schools are discussed.     

Methodological Overview of the Parents Matter! Program

We present an overview of the methodology employed in the Parents Matter! Program. Information on the following aspects of the program is presented: participant eligibility and recruitment; consenting procedures and administration of assessments; development and utilization of measures in the assessments; study design; intervention procedures; facilitator characteristics and training; procedures to assess treatment fidelity assessment; and procedures utilized to enhance retention of study participants.     

The Future Problem Solving Program International: An Intervention to Promote Creative Skills in Portuguese Adolescents

The Future Problem Solving Program International (FPSPI) is an internationally applied educational program that involves young people. Its theoretical foundation is both the Creative Problem Solving Model and the Futurist Thinking. It aims to promote creative and critical thinking through a futurist approach to problems. This study intended to analyze the effects of the program on creative skills evaluated by the Torrance Tests of Creative Thinking (Figural Version). The participants’ perceptions of the efficacy of the program were also assessed. This intervention was carried out with 131 adolescents over a period of 7 months in an extra‐curricular context. The evaluation of the program takes into account periods both before and after interventions, using similar experimental and control groups. The results showed significant statistical differences for the all skills studied and very positive perceptions of the efficacy of FPSPI. Two significant gender differences in creative performance were also found. The results are described and discussed in order to promote awareness for future research concerning this program.     

Enhancing Program Quality in a National Sample of After‐school Settings: The Role of Youth–Staff Interactions and Staff/Organizational Functioning

Using multilevel data from the national evaluation of Boys and Girls Clubs of America (BGCA), this study examined associations among programmatic structures, workplace and workforce characteristics, and relational practices of program staff as they relate to young people's ratings of their experience attending local clubs. The sample included 57,710 members and 5,231 staff members at 740 BGCA sites throughout the United States. Staff relational practices—including establishing caring relationships, setting high expectations, positive behavior management, encouraging youth input and agency, and cultural sensitivity—explained associations between staffing and organizational functioning and youths’ perceptions of the quality of their clubs. Findings suggest a central role of staff relational practices in establishing conditions that youth experience positively, and that staffing and organizational processes, including community engagement and teamwork and efficiency can be viewed as foundations for establishing a culture of positive adult‐youth interaction, which in turn can contribute to the promotion of positive youth development. Further, identification with the experiences of youth had a direct association with youths’ perceptions of club quality. These results underscore the importance of staff workforce development initiatives as key to improving youth experiences in after‐school programs.     

Youth Risk and Mentoring Relationship Quality: The Moderating Effect of Program Experiences

Because program experiences are more amendable than mentor or mentee characteristics, they may be important factors to consider in buffering the negative impact of youth risk on the quality of the mentor–mentee bond. Data from 455 mentees (ages 11–18; 57% male) and their undergraduate student mentors (82.3% female) from the Campus Connections mentoring program were used to assess whether youth risk and mentors’ program experiences (i.e., program structure, supportive relationships with staff, opportunities for skill building, support for efficacy and mattering, and opportunities to belong) were associated with mentoring relationship quality and whether mentors’ experience within the program moderated the association between youth risk and mentoring relationship quality. Results indicated that environmental, but not individual, risk was negatively associated with relationship quality. Mentors’ experiences with the program were positively associated with mentoring relationship quality, and in many cases, above and beyond youth level of risk. Finally, mentors’ perception of program structure, supportive relationships, and opportunities for skill building attenuated the negative relationship between environmental, but not individual, risk and relationship quality. Mentors’ experiences of program support for efficacy and mattering and opportunities to belong were not significant moderators in any model. Implications for programs and future research directions are presented.     

Using ethnographic methods for development of culture-specific interventions

Cultural specificity is considered important in developing effective school- and community-based interventions. This article illustrates the use of ethnographic research to enhance understanding of students' cultural experiences and facilitate the development of culturally specific interventions. Ethnographic data were collected during formative and intervention phases of a school-based mental health promotion project. Data were coded using an interactive deductive-inductive process. Culture specificity was defined as the integration of etic (universal) and emic (culture-bound) perspectives. Four culture-specific themes were identified: adult-sanctioned behaviors and practices, adolescents' perspectives about the present and aspirations for the future, and societal factors. These themes encompassed code categories reflecting cultural attitudes, beliefs, customs, expectations, norms, and values; and reflected the ecological framework that guided the research. Future research is needed to address the use of combined etic-emic conceptualizations of culture in the development of culture-specific interventions.