Participatory Evaluation: A Case Study of Involving Stakeholders in the Evaluation Process

ABSTRACT

One way to ensure that an evaluation has “utility” is to use a participatory evaluation approach where the evaluator partners with primary users to carry out various phases of an evaluation. This article provides a case study of how participatory evaluation was used in an out-of-school youth development and employment program at the Science Museum of Minnesota's Kitty Andersen Youth Science Center. Youth staff participated in a series of evaluation workshops where they learned about evaluation, made their own meaning of evaluation data, generated recommendations to improve their museum work, and ultimately took more ownership and control of their program. As a result of the workshops, youth also carried out their own formative evaluation of four museum programs. The program manager and the youth staff reaped a number of benefits from this process. The factors leading to its success are discussed.     

A Multi-Dimensional Framework for Evaluation of Instructional Consultation Teams

ABSTRACT

This article describes a multidimensional framework for evaluating team-based services in schools. The theory and literature on developmental, causal and confirmatory program evaluation are discussed as the foundation for this framework. The model of Instructional Consultation Teams (IC-Teams) is presented as an illustration of how program evaluation research supports program development and provides evidence of program impact. The inferred impact of IC-Teams upon over-referral and identification of students for special education, especially minority students, is investigated using the confirmatory criteria of consistency. Three studies investigating the impact of IC-Teams upon special education referrals of the IC-Team model are presented. The studies, conducted across three implementations of the program with three different methodologies, demonstrated decreases in the referral, evaluation and placement of students in special education. The need for additional research and limits of the approach are discussed.     

Can mindfulness modify your thoughts? A study in the context of social phobia

IntroductionSocial phobia is said to be the commonest anxiety disorder (Furmark, 2002). Cognitive Behavior Therapy appears to be the most recommended form of psychological treatment for this condition (Federoff & Taylor, 2001; Willutzki et al., 2012). Dismantling studies have raised doubts regarding the utility of cognitive restructuring in bringing about decrease in symptoms (Barrera et al., 2016; Pompoli et al., 2018). Mindfulness, a ‘third-wave’ concept, seems to contradict the philosophy of ‘restructuring’ with its apparent stress on non-judgmental observation (Hayes, 2004). This approach postulates that acceptance of cognitions and emotions as therapeutic in contrast to traditional cognitive behavior therapy approach, which advocates restructuring and modifications of cognitions.ObjectiveThe current study attempted to study the ‘restructuring’ effects of mindfulness in persons with social phobia.MethodThirty-nine clients with a primary diagnosis of social phobia (ICD – 10) participated in the Mindfulness Integrated Cognitive Behavior Therapy (MICBT) program, which consisted of eight sessions. MINI International Neuropsychiatry Interview Plus, Liebowitz Social Anxiety Scale, Fear of Negative Evaluation, Meta Cognitions Questionnaire and Five-Facet Mindfulness Questionnaire were administered pre and post of the therapeutic program.ResultsThe results demonstrated that MICBT had a significant effect on reducing the scores on fear of negative evaluation and bringing about changes in metacognitions.ConclusionMindfulness could be sufficient as a therapeutic strategy to bring about changes in the cognitions of clients with social phobia without using cognitive restructuring.     

Development and Implementation of a Sport Psychology Program in an Urban High School

SUMMARY

This article is based on a dissertation of the author that was centered on the process of developing a sport psychology program at the high school level. The program was developed to address academic, mental, social, and emotional needs of student-athletes and coaches. This sport psychology program, which was developed in the Linden Public School District, a large urban district in New Jersey, may be the first such program documented on a national level. In this dissertation, the author, who was a school psychologist in the district, utilized a particular program planning and evaluation framework, with attention to the first two phases-clarification and designto develop and implement a sport psychology program.     

School-Linked Services in Context

Summary

This paper describes a formative evaluation of a school-linked service delivery system for immigrant and low income families in a large suburban county in Maryland. Initially we discuss local needs that prompted the development of the program and we place problems with services for children and families with multiple needs in a broad context. Next we discuss the methods and results of the evaluation. Findings indicate the program provides a range of appropriate services to intended beneficiaries but that lack of infrastructure and articulated understandings among participating organizations present potential problems.     

Effects of an early reading intervention based on grapho-syllabic decoding and fluency training in French elementary schools

IntroductionStudies conducted in recent decades have led to the development of instructional interventions to help prevent or eliminate a large proportion of reading difficulties. However, few experiments have been conducted in French schools.ObjectiveAims for this research were to create, implement and evaluate a remediation program for French students with reading difficulties up to the end of elementary school.MethodThis paper examines the effects of a decoding-skills and fluency-based intervention on 133 struggling readers in second- to fifth-grade. Depending on the nature of their reading difficulties, students were given a year-long training based either on grapho-syllabic conversion — focusing on syllable units, rather than phonemic units, is considered a promising way of helping French-speaking beginning readers — or on repeated reading techniques, used to improve fluency.ResultsIncreases in written-word identification scores were greater for students who followed the program than they were for a control group of 184 students with reading difficulties who did not follow the program.ConclusionThe results show that programs combining early evaluation of written-word identification abilities followed by a year-long series of remediation sessions focusing on specific difficulties can help students progress in written-word identification. However, further research is needed to examine why our program was less effective with children from disadvantaged (“éducation prioritaire”) areas.     

Using Curriculum-Based Measurement to Evaluate Intervention Efficacy

SUMMARY

The use of Curriculum-Based Measurement for evaluating treatment efficacy is described and discussed. The basic methods and applications of Curriculum-Based Measurement are described, followed by examples of its application in two intervention program evaluation projects. The first project involved one individual student experiencing academic difficulties. The second project focused on evaluating a reading instruction program for a group of students. In both examples, the use of Curriculum-Based Measurement contributed significantly to monitoring student progress in response to educational program variables and making treatment evaluation decisions.     

Exploring the experience of adhering to a prescribed pre-surgical exercise program for patients with advanced rectal cancer: A phenomenological study

ObjectivesThe aim of this study was to explore the lived experience of patients with advanced rectal cancer as they attempted to adhere to a prescribed, hospital-based pre-surgical exercise program.Design/MethodsTen patients took part in three semi-structured in-depth interviews prior to (week 0), midway (week 3), and at completion (week 6) of the exercise program. Data were analysed using a phenomenological approach.ResultsMain themes that were deemed important for remaining enrolled in the program included: (1) building camaraderie and peer support, (2) experiencing a sense of structure and control, and (3) feeling safe and encouraged. Patients also reported engaging in further exercise outside the program and feeling motivated to continue exercising in the future.ConclusionsThis study shows the potential to integrate exercise prior to surgery as it was viewed as acceptable and feasible by patients with advanced rectal cancer. Moreover, it provides much needed information into the possible mechanisms that underpin patient's continued participation in exercise programs during treatment for advanced rectal cancer. Prescribed, hospital-based program should be implemented to promote opportunities for exercise and provide on-going supportive care options for patients awaiting surgery.     

Culture,psychology and ecology: An interview with Theodore Roszak

Objectives: This study investigates the efficacy of a standard cognitive behavioral group-program for chronic pain, in comparison with the same group program that integrates an existential perspective at treatment conclusion, 3-month, and 6-month follow-up. Design: This study is a quasi-experimental clinical trial with two treatment groups. Methods: We included 113 patients suffering from chronic pain who had been accepted for treatment at an interdisciplinary pain center. In addition to the specific therapy program, the patients received intense interdisciplinary treatment. Outcome measures include both pain-related disability and pain severity. Repeated-measures analyses of variance (ANOVAs) were used to test statistical significance, followed by tests of simple contrasts with pre-treatment as a reference-point. Results: Results indicated that the integration of an existential perspective leads to significantly lowered pain-related disability than the classic cognitive behavioral group-program. Additionally, existential aspects seem to be particularly important for patients with a spiritual orientation. The findings from this study support the importance of considering specific existential aspects in the cognitive-behavioral approach in treating chronic illness.     

Group training in communication skills for couples: A preliminary report

Abstract

This article reports on the initial development and evaluation of a group skill-training program for couples. The program, entitled the Communication Skills Workshop (CSW), aims at improving communication and problem-solving skills within the context of relationships. The orientation of the CSW is preventive and educational in that it (a) serves couples prior to the onset of serious relationship discord, and (b) focuses on the learning of general communication skills and problem-solving strategies rather than the amelioration of specific relationship conflicts.     

Parent perceptions of an integrated stuttering treatment and behavioral self-regulation program for early developmental stuttering

PurposeRecent research has identified approximately half of children who stutter present with self-regulation challenges. These manifest in elevated inattentive and/or impulsive behaviours, aligned with attention deficit hyperactivity disorder (ADHD) symptoms. These symptoms have been found to influence the child’s responsiveness to their stuttering treatment, and may exacerbate the psychosocial consequences of stuttering for them and their families. Early stuttering intervention identifies parents as key agents of change in the management of their children’s stuttering. This study sought feedback from parents regarding their experiences with an integrated stuttering treatment and behavioral self-regulation program for early developmental stuttering, addressing the child’s self-regulation challenges.MethodEight parents of children who stutter who had co-occurring self-regulation challenges completed the integrated program. This incorporated the Triple P--Positive Parenting Program adapted for the developmental stuttering population, and the Curtin University Stuttering Program (CUSP). Semi-structured qualitative interviews were conducted to capture parents’ reflections on, and experiences with, the integrated program.ResultsThematic analysis identified several major themes regarding the parents’ experiences with the integrated program: emotional impact on parents, child self-regulation, link between stuttering and behaviour, parent self-regulation, impact on family dynamics, and overall positive perceptions of the integrated program. All of the parents indicated they would recommend the program to future parents of children who stutter.ConclusionThis study provides insights into parents’ perceptions regarding an integrated intervention approach for early stuttering and behavior management. It also indicates how adopting a holistic approach to stuttering intervention is positive and has social validity.     

The Teacher of Reading: The First Twenty-Five Years Are the Most Challenging

This study investigated approaches to reading acquisition used in four first grades of a suburban elementary school. Individual teacher interviews established the differences in program emphases. Group A used the basal reader program; Group B, the basal reader and an additional phonics program; Group C, the basal reader and the language experience approach; Group D, the language experience approach and a wide range of supplementary readers and trade books. All groups participated in supportive language arts activities. Nine randomly selected subjects from each class participated‐‐three from each high, middle, and low reading group. Informal reading inventories were individually administered and scored for levels of word recognition in context and reading comprehension. Results support the use of a language experience approach as a viable alternative to the basal reader approach for teaching reading and writing.     

What Does Self-Care Look like for Online Graduate Mental Health Counseling Students?

ABSTRACT

The purpose of this article is to highlight what self-care looks like for online graduate mental health counseling students and implications for counselor educators. Results come from a program evaluation examining self-care practices for online mental health graduate students. Furthermore, these results will guide the examination of best practices to meet student reported needs while integrating self-care into the program’s mission, program objectives, and curriculum.     

Ethical and Legal Considerations in the Training of Mental Health Professionals

Summary

A discussion is provided regarding ethical and legal considerations that relate to the training of mental health professionals. A training program which emphasizes a proactive approach to identifying and resolving potential ethical and legal problems, and their close connection with clinical issues is described, as are the responsibilities of supervisors and supervisees in such a program. For illustrative purposes, a training vignette and a clinical vignette conclude the article.     

L’entraînement de la mémoire de travail par le programme Cogmed et le TDAH

IntroductionAttention-deficit/hyperactivity disorder (ADHD) is a prevalent neurodevelopmental disorder. Inattention, hyperactivity, and impulsivity are key symptoms of ADHD. It is typically associated with working memory deficits at the cognitive level. For this reason, interventions have been designed to train working memory in ADHD. Currently, Cogmed Working Memory Training program is the most commonly used and studied program in clinical practice and research. This program is proposed as an intervention for ADHD that targets working memory deficits with specific exercises through intensive training sessions.ObjectivesThe goal of this literature review is to examine the effects of the Cogmed program in children and adolescents with ADHD on working memory, inhibition, non-verbal reasoning, attention functioning, ADHD symptoms and academic achievement. All existing studies on the subject that included a control group (n = 8) are reviewed.ResultsIt is clear from most studies that Cogmed training program increases and verbal and visuospatial working memory (or the phonological loop and visuospatial sketchpad in Baddeley's model (1986, 2007), among ADHD participants. However, transfer of learning is not demonstrated on other components of working memory that are not directly targeted by the program such as the central executive described in Baddeley's model or the secondary memory defined by Unsworth & Engle (2007). With regards to far transfer measures, results are controversial for inhibition, non-verbal reasoning, ADHD symptoms reported by parents, and reading abilities. No improvement is demonstrated for attentional capacities, ADHD symptoms reported by teachers and mathematic reasoning.ConclusionCogmed training improves verbal and visuospatial working memory, two cognitive functions that play an important role in ADHD. However, Cogmed's exercises need to be modified in order to train more complex working memory components such as the central executive (Baddeley, 1986, 2007) and the secondary memory (Unsworth & Engle, 2007), which are more impaired in ADHD than the phonological loop and visuospatial sketchpad. Another approach would be to design programs that can tackle a larger range of cognitive functions that are impaired in ADHD (e.g., inhibition). In future, studies evaluating such modified programs, direct observation instruments that are more sensitive to short-term changes need to be included. Follow-up measures should also be systematically included.     

Le processus d’intervention en santé au travail : éclairer les enjeux de l’action par un exercice de formalisation de l’activité

IntroductionThe quality of action in occupational health requires attention to its dynamic, for example analysing the activity of the intervenant consultant: what he manages to do in his activity and the obstacles he encounters.ObjectiveThis article proposes to feed the knowledge on the intervention from a qualitative approach and centered on the intervention process.MethodUsing the case study, the article shows the real of the intervention, its obstacles and the adjustments it imposes. The case studied is in a journal.ResultsThis intervention process shows the effects of variation caused by lack of accession and support from management and the trade unions, the difficulty of passing from knowledge to action.ConclusionThe difficulties have a function of revelation which the evaluation of the effects of an intervention can take into account.     

A randomized controlled trial evaluating the efficacy of Triple P Online with parents of children with early-onset conduct problems

ObjectiveThis study examined the efficacy of Triple P Online (TPOL), an eight-module intensive online positive parenting program for parents of children with early-onset disruptive behavior problems.MethodOne hundred and sixteen parents with 2–9-year-old children displaying early-onset disruptive behavior difficulties were randomly assigned to either the intervention condition (N = 60) or an internet-use-as-usual control group (N = 56).ResultsAt post-intervention assessment, parents receiving the internet intervention TPOL had significantly better outcomes on measures of problem child behavior, dysfunctional parenting styles, parents’ confidence in their parenting role, and parental anger. At 6-month follow-up assessment intervention gains were generally maintained, and in some cases enhanced. Consumer satisfaction ratings for the program were high.ConclusionsInternet-delivered self-help parenting programs appear to make a valuable contribution to a comprehensive public health approach to parenting support.     

Effects of Physical Exercise on Resources Evaluation,Body Self-Concept and Well-Being Among Older Adults

This study addressed the effects of a 14-week moderate physical activity program on resources evaluation, body self-concept and other health-related psychological variables among older adults. In a quasi-experimental design participants were divided into three groups and completed questionnaires twice, before ( t 1) and after ( t 2) the 14-week program. The experimental group participated in a supervised mixed exercise program of moderate intensity. The placebo attention control group took part in a foreign language course. The no intervention control group did not get any intervention/attention at all. The dependent variables were resources evaluation, body self-concept, trait anxiety, and psychosomatic complaints. Using the pretest data as covariates multivariate analyses of covariance revealed significant improvements in body self-concept in the experimental group compared to the control groups. No other significant differences could be detected. Results point to the positive effects of exercise on body self-concept even among older adults.     

Changing behaviors: Using norms to promote physical activity for type 2 diabetes patients

IntroductionGiven that diabetes is a major public heath issue around the world, it is vital that we find effective means to change behaviors, especially levels of physical activity among type 2 diabetes patients.ObjectiveThis study aims to provide proof of the effectiveness of programs promoting physical activity, based on cognitive dissonance and normative focus theories which are designed to produce behavioral changes in persons with type 2 diabetes.MethodNinety-six type 2 diabetes patients were assigned to one of three programs: a traditional information program, a norm-based program, and a dissonance-based program. The participants filled out a short French version of the International Physical Activity Questionnaire (IPAQ) one week before and one week after the program.ResultsThe results showed that the participants in the norm-based program made progress compared to those in the traditional information program. No significant difference was observed between the traditional information program and dissonance-based program.ConclusionsThese findings are consistent with both the focus normative theory and previous studies showing that recalling the norm increases compliance. The procedure to be used in diabetes prevention programs should focus on the normative dimensions of the desired behaviors in order to improve patients’ quality of life.     

Grasping neither war nor peace: the folly of cosmopolitan preventive war

ABSTRACT

Some liberal-cosmopolitan theorists have sought to justify preventive war by proposing new institutions meant to ensure the accurate evaluation of non-imminent threats, and also make any war against them proportionate. In the debate over these proposals there has been little consideration of the post-war conditions any preventive war will likely produce. This is a serious omission; many theorists emphasize the degree to which the ability to secure a just peace is crucial to whether a war is proportionate. This article begins to remedy this missing piece of the debate over what it calls ‘cosmopolitan preventive war’ (CPW). After reviewing the debate, it discusses preventive war in the context of theorizations of post-war justice, or jus post bellum. It then investigates CPW’s ability to account for jus post bellum concerns through a counterfactual 2003 Iraq CPW. Showing that the proposed institutions do not do enough to account for the likely, and possibly immense, post-war harm wrought by preventive war, the article concludes with a negative evaluation of the CPW program and a brief statement on the ethics of preventive war in general.