A synoptic framework for school program evaluation

A broad-based conceptual schema, termed the Synoptic Evaluation Framework (SEF), is discussed in relation to providing school program evaluation services. Derived from evaluation theory and practice, the SEF allows for the design and conduct of a range of program evaluation studies within the school context. Moreover, the SEF has been used as a basis for training doctoral school psychology students in program evaluation. The SEF consists of the three constituent dimensions: (a) Evaluation Unit, (b) Evaluation Purpose, and (c) Evaluation Service. An understanding of the interactions among these dimensions enables an evaluator to identify the specific programmatic concerns of a decision maker and to design and conduct a responsive evaluation.     

Evaluation of a youth suicide prevention school program

Adolescent suicide is a subject of increasing national concern. In California, this problem was addressed through the implementation of a three-year youth suicide prevention school program established by legislative mandate in 1983. Findings for the first two years of the program indicate significant gains in understanding of youth suicide prevention techniques for those high school students who completed a four-hour training course. School staff and parents completing 90-minute youth suicide awareness seminars indicated that they most appreciated receiving practical advice and resources they could use in responding to a potentially suicidal young person.     

Development of a school-based treatment program for middle school youth with ADHD

The authors conduct an evaluation of a middle school-based treatment program for youth with ADHD during early stages of treatment development. The studies focus on interpreting outcome trends in preliminary data and identifying assessment issues that will be important to consider when conducting a clinical trial. Parent reports indicate that the majority of students benefit from improvements in academic, social, and overall functioning. Although teachers report beneficial effects for the majority of the participants in the program, there is little agreement about individual students. Measurement problems are associated with understanding normal change during a school year, accounting for normal behavior changes in December and May, and considerable disagreement between teachers. Suggestions to guide future work in this area are provided.     

Self-evaluation by adolescents in a psychiatric hospital school token program

Nine adolescent boys with a history of high rates of disruptive classroom behavior were selected from a psychiatric hospital school and placed in a remedial reading class after school in which various factors in a token reinforcement program involving self-evaluation were investigated. The effects of self-evaluation, in the form of a rating the students gave themselves about the appropriateness of their classroom behavior, were first assessed. While the students' ratings of their own behavior correlated highly with the teacher's ratings and evaluations made by independent observers, the self-evaluations did not lead to a reduction in disruptive behavior. A token reinforcement program, in which the teacher rated the students' level of appropriate behavior and in which the students traded earned rating points for prizes, clearly led to a reduction of disruptive behavior. When the students were given the opportunity to evaluate their own behavior and to receive rewards in exchange for the evaluation, they returned to their former rates of disruptive behavior.     

Mediation of a middle school skin cancer prevention program.

This study tested potential mediators of a school-based skin cancer prevention intervention for middle school children (6th-8th grades; N = 1,788). Ten variables were tested on 4 criteria to establish mediation, including (a) intervention related to outcome, (b) intervention related to mediators, (c) mediators related to outcome, and (d) mediated effect statistically significant. Sun-safe behaviors (e.g., sunscreen use) and potential mediators were measured with a self-report questionnaire. All criteria were met for Barriers--Sunscreen, Perceived Self-Efficacy, and Knowledge when the mediators were tested separately. In multiple mediator analyses, barriers to sunscreen use and self-efficacy satisfied Criteria 1-3 but were not statistically significant (Criterion 4). Barriers to sunscreen use, perceived self-efficacy for sun-safe behavior, and knowledge may serve as mediators.     

The school psychologist as program evaluator

School psychologists are being required to evaluate educational programs. This paper discusses the role and responsibilities of a program evaluator, the programs of measurement, and the staff fear of evaluation. A model for program evaluation is then described with an illustrative example.     

A gift from the ghetto

Counselors and their clients must communicate effectively. Because the trained counselor is not always able to relate to the problems of his clients who are both black and poor, this article examines the significant role that paraprofessionals, or indigenous counselors, can play in the ghetto.     

Guidance counselling as a whole school responsibility in the Irish post primary sector

ABSTRACT

This paper will deliberate upon the relevance of guidance counselling as a ‘whole school’ responsibility in the context of recent policy and practice changes in the Irish post primary sector since the Government of Ireland Budget 2012. Pertinent issues including the complexities of delivering a whole school approach to guidance counselling during a period of resource rationalisation will be addressed through the findings from recent case study research [Hearne, L., Geary, T., & Martin, N. (2016). A single case study of a whole school approach to guidance counselling in an Irish post-primary school; Case report. Retrieved from University of Limerick website: http://www.ul.ie/soedu/node/1481]. A single explanatory case study methodology investigated the complex phenomenon and interventions of whole school guidance counselling in a mixed gender school in the context of increasing demands for evidence-based data and methodological sophistication in lifelong guidance [European Lifelong Guidance Policy Network. (2015a). The evidence base on lifelong guidance; A guide to key findings for effective policy and practice, extended summary. Jyväskylä: Kariteam; Hearne, L. (2009). Towards an understanding of the measurement of individual progression in adult guidance (Unpublished doctoral dissertation). Waterford Institute of Technology, Waterford; Stake, R. E. (2004). Standards-based & responsive evaluation. Thousand Oaks, CA: Sage; Yin, R. K. (2009). Case study research: Design and methods (4th ed.). Thousand Oaks, CA: Sage; Yin, R. K. (2014). Case study research (5th ed.). Thousand Oaks, CA: Sage].     

Longitudinal study of a school based HIV/AIDS early prevention program for Mexican adolescents

A quasi-experimental study is reported with four measurement occasions to evaluate longer-term effects of a life-skills and HIV/AIDS school-based prevention program. Trained teachers administered the program promoting precursors of safer sex behavior to 2064 Mexican high-school students at an age before most were sexually active. The variables included in the study (knowledge about HIV/AIDS prevention, attitudes towards use of condoms, subjective norms, intentions to use condoms and life skills as decision-making skills, partner communication and individual responsibility) have been reported as precursors of protective sexual behavior. The results demonstrate the stability of training effects and a positive impact on these precursors over 1 year of follow-up.     

Designing internally-based program evaluations in school settings

The worth of evaluation activities depends heavily on the use made of evaluation results. This paper discusses some ways a psychologist carrying out evaluation within a school system can design evaluation studies to maximize the usefulness of the results. A format and procedure for designs based on providing information for decision making are presented. Examples of the experiences of one internal office are discussed, and some general suggestions for optimizing evaluation utilization are provided.     

Learning with microcomputers: Training primary school children on a problem-solving program

This study investigated the effects of structured training at two informational levels on children's performance on a problem-solving computer program. An informed-training group (strategy training with detailed explanations) performed at a superior level to a blind-training group (strategy training with no explanations) and a control group. This superiority was maintained 3 weeks later on retest. The results are discussed in relation to the processes that mediate and facilitate training.     

Free writing: Evaluation of a preventive program with elementary school children

Eight-session free-writing workshops focusing on developing elementary schoolchildren's use of different describing words, reference to self, expression and clarification of feelings, and expression of stress responses were introduced to 324 fifth-grade children living on the northern border of Israel where the probability of war-related crisis is particularly high. The purpose of the study was to evaluate (a) the effectiveness of the workshop and (b) the applicability of free writing in an actual stressful event. Results indicated that the workshop group exhibited higher mean scores than the nonintervention control group on all behavior measures, both after the workshop and following a stressful event (shelling of the town). The theoretical background for free-writing conceptualization is given and the implications of the results are discussed in light of large-scale preventive intervention.     

Dose response effects of cognitive-behavioral therapy in a school mental health program

ABSTRACT

School mental health (SMH) programs have been shown to be effective in providing evidence-based interventions to underserved youth. However, limitations of SMH programs are that they can entail holiday breaks, typically do not operate through summer, and often require pulling students from class to receive therapy. These limitations suggest that treatment must be expeditious and potent. Although researchers have investigated dose response to treatment, no studies were located that addressed dose response to treatment in SMH programs. The present study addressed this gap by evaluating the dose response to SMH treatment in a sample of 133 adolescents. Adolescents were assessed at baseline, post-treatment, and at multiple time points throughout treatment. An average treatment response of a 26.81-point decrease in Youth Outcome Questionnaire (YOQ-30) score was found across 14 sessions of cognitive-behavioral therapy (CBT). Further, adolescents exhibited reliable change in YOQ-30 score within an average of 2.91 sessions. Finally, it was found that baseline scores on the Depression and Hyperactivity subscales of the Behavior Assessment System for Children, 2nd Edition, along with YOQ-30 score, predicted treatment response. These findings advance our understanding of dose response to CBT in SMH settings, and create opportunities to better inform effective treatment strategies in similar contexts.     

A token reinforcement program in a public school: a replication and systematic analysis

A base rate of disruptive behavior was obtained for seven children in a second-grade class of 21 children. Rules, Educational Structure, and Praising Appropriate Behavior while Ignoring Disruptive Behavior were introduced successively; none of these procedures consistently reduced disruptive behavior. However, a combination of Rules, Educational Structure, and Praise and Ignoring nearly eliminated disruptive behavior of one child. When the Token Reinforcement Program was introduced, the frequency of disruptive behavior declined in five of the six remaining children. Withdrawal of the Token Reinforcement Program increased disruptive behavior in these five children, and reinstatement of the Token Reinforcement Program reduced disruptive behavior in four of these five. Follow-up data indicated that the teacher was able to transfer control from the token and back-up reinforcers to the reinforcers existing within the educational setting, such as stars and occasional pieces of candy. Improvements in academic achievement during the year may have been related to the Token Program, and attendance records appeared to be enhanced during the Token phases. The Token Program was utilized only in the afternoon, and the data did not indicate any generalization of appropriate behavior from the afternoon to the morning.