The Importance of Program Evaluation: How Can it be Applied to Diverse Genetics Education Settings?

The role of a genetic counselor often entails providing education to patient, community and/or health professional groups. While counseling supervision assists genetic counselors to be reflective about their clinical work and to enhance clinical skills, evaluation is a rather analogous process in the provision of education. Program evaluation of education activities can be applied to provide information about the needs of the target group (needs assessment), the delivery of the program (process evaluation) as well as determining the extent to which the education activity has met its intended aims (summative evaluation). Evaluation assists the educator to assess the impact of their program and provides an evidence base about genetics education. Although program evaluation can be a complex activity, the tools are ones that can be used by individuals to evaluate single or simple education activities. The components of evaluation are discussed with reference to genetic counseling practice and three very different examples of actual evaluations are provided to illustrate the diversity of evaluation strategy and programs to which it can be applied.     

Enhancing Therapeutic Gains: Examination of Fidelity to the Model for the Intensive Mental Health Program

Given that the development of treatment fidelity assessment protocol is an integral but too frequently ignored aspect of clinical trials for psychological treatments, the Intensive Mental Health Program (IMHP) sought to build fidelity activities into training, program evaluation, and clinical recordkeeping from the outset of a 3 year study period. We describe (a) operational definitions and measurement of the IMHP treatment model, (b) planned fidelity-promoting activities, (c) evaluation methods, and (d) fair to appreciable accomplishment of IMHP fidelity targets. Discussion of the circumstances and processes possibly hindering better fidelity for several IMHP treatment features accompanies presentation of fidelity data. We summarize pitfalls and compensatory strategies identified during IMHP program evaluation and offer practical suggestions to enhance accountability and evaluative activity in future clinical research.     

More Than Attendance: The Importance of After-School Program Quality

A central theme of the articles featured in this issue is the need to improve the quality of after-school programs. In this commentary, we discuss why student engagement, program characteristics and implementation, staff training, and citywide policy are key considerations in the effort to define and achieve high quality programs for youth.     

Dilemmas of Practice: Challenges to Program Quality Encountered by Youth Program Leaders

To create and sustain high quality youth development programs it is important to understand the challenging situations and dilemmas that emerge in program leaders’ daily work with youth. In this research the experiences of leaders in 12 programs were followed over a 2–9 month period, which led to the identification of 250 dilemma situations. Qualitative analyses identified 5 categories and 12 subcategories of dilemmas that reflected distinct types of considerations (e.g., youth’s personalities, relationships with the community). The analyses also found that the experienced leaders in the study typically responded to these dilemmas in ways that were youth-centered and that balanced multiple considerations. It is argued that researchers need to go beyond identifying features of high quality programs, and more fully examine how effective leaders create and sustain high quality in response to the challenging situations of practice.     

Soldiers’ perspectives of the Married Army Couples Program: A review of perceived problems and proposed solutions

The current study investigated soldiers’ perceptions of the Married Army Couples Program (MACP) using data from focus groups conducted with Army personnel in dual-military marriages. Despite the high success rate of the program (approximately 80–83%)—success being defined as establishment of a joint domicile within a 50-mile radius of each spouse’s duty station—a variety of difficulties were reported as well as consequent negative perceptions. Problems included dissatisfaction with the parameters defining joint domicile, obstacles to career advancement, lack of quality information regarding the program, and unequal allocation depending on one’s level of leadership support. Difficulties with MACP led to negative impressions of the program and of the Army as a whole. Specifically, when asked about challenges related to the program, many participants reported perceptions of a poorly functioning program, an uncaring environment, and of the Army as a hypocritical institution. For individuals in dual-military marriages, these difficulties affect quality of life. For the institution, these negative perceptions can potentially lead to problems with retention of one or both members of a dual-military couple. We propose several strategies to improve the effectiveness of MACP and to increase positive perceptions of the program based on suggestions from the focus groups in addition to the military and work literatures.     

Predictors of Farmers’ Market Shopping among People Receiving Supplemental Nutrition Assistance Program Benefits

Promoting use of farmers’ markets (FMs) is a promising community‐level strategy to increase access to nutritious foods such as fruits and vegetables. Yet, FM shopping among people with Supplemental Nutrition Assistance Program (SNAP) benefits remains low. This research examined predictors of FM shopping among SNAP recipients living within 1 mile of a FM. A cross‐sectional survey of SNAP participants (N = 270) was conducted in 2015 in Cleveland and East Cleveland, OH, USA. Multinomial regression and zero‐truncated Poisson regression analyses were conducted to examine factors associated with FM shopping. Results indicate 48% reported shopping at a FM at least once in the past year, 26% had shopped at a FM before, but not in the last year, and 26% had never shopped at a FM. The multivariable analyses found awareness of FMs and a healthy food incentive program, and four dimensions of healthy food access are significantly associated with FM shopping among SNAP recipients. The food access dimensions included service delivery, spatial‐temporal, personal, and social access. Findings highlight modifiable leverage points for improving the reach of FMs among low‐income populations.     

Real-world inpatient programs: Shedding some light-A critique

This critique is a response to an article by Morisse, Batra, Hess, Silverman, and Corrigan (1996), in which “a token economy for the real world” is promulgated as an alternative to the comprehensive social-learning program and assessment technology presented by Paul and Lentz (1977). The article misrepresents the treatment-and-assessment procedures that have been empirically validated as the most effective and cost-efficient for inpatient programs. The article also inappropriately cites the results of prior reports as support for an oversimplified approach to the development and implementation of inpatient programs. The promoted approach is in direct opposition to the widely recognized need for empirically validated interventions and evaluations in mental health services. Not only are data lacking to support assertions of effectiveness for the resulting program but illustrative examples demonstrate technically unsound procedures as well as an ethically questionable emphasis on staff desires to the exclusion of patient needs. We attempt to correct Morisse et al.'s inaccuracies and misconceptions regarding the work of Paul and colleagues, note the major problems with their perfunctory approach, and provide recommendations for implementation and maintenance of empirically validated procedures for inpatients.     

The general causality orientations scale: Self-determination in personality

This paper describes the development and validation of a general causality orientations scale. Causality orientations are conceptualized as relatively enduring aspects of people that characterize the source of initiation and regulation, and thus the degree of self-determination, of their behavior. Three orientations—autonomy, control, and impersonal—are measured by the three subscales of the instrument. Individuals are given a score on each orientation, thus allowing the use of the theoretically appropriate subscale (or, in some cases, a combination of subscales) to predict affects, cognitions, and behaviors. The scale was shown to have internal consistency and temporal stability. The orientations were shown to fit appropriately into a nomological network of constructs and to relate to various behaviors that were hypothesized to be theoretically relevant.     

EEG asymmetry and sleep mentation during REM and NREM

The hypothesis that dreaming is mediated by the right hemisphere was evaluated by monitoring EEG power asymmetry during REM and NREM sleep, and obtaining mentation reports when short-term temporal shifts in the EEG indicated relative left- or right-hemispheric dominance. Content analyses provided no support for the right-hemisphere hypothesis; indeed, some scales showed higher content during relative left-hemispheric dominance. In contrast to earlier reports, no difference between REM and NREM in EEG asymmetry was observed.