Quality benchmarks in undergraduate psychology programs

Performance benchmarks are proposed to assist undergraduate psychology programs in defining their missions and goals as well as documenting their effectiveness. Experienced academic program reviewers compared their experiences to formulate a developmental framework of attributes of undergraduate programs focusing on activity in 8 domains: curriculum, assessment issues, student learning outcomes, program resources, student development, faculty characteristics, program climate, and administrative support. A continuum of performance was conceptualized for each attribute in each of the domains to characterize underdeveloped, developing, effective, and distinguished achievement for undergraduate programs. The authors hope to inspire a national conversation about program benchmarks in psychology in order to improve program quality, encourage more effective program reviews, and help optimally functioning programs compete more successfully for resources on the basis of their distinguished achievements.     

Understanding and Improving the Effectiveness of After-School Practice

The question driving research on after‐school programs is shifting from “do programs make a difference,” to “why are some programs effective while others are not?” This article draws on the work in this volume and related studies to suggest that there is an emerging consensus on the importance of staff‐youth interactions as a determinant of program effectiveness. The commentary recommends that future research should continue to focus on understanding and improving program practices at the point‐of‐service. In doing so, two lines of inquiry seem promising. The first involves linking measures of changes in staff‐youth interactions to changes in developmental outcomes. The second is to examine how policies or other interventions intended to improve program effectiveness affect the practices of line staff.     

Moving from efficacy to effectiveness trials in prevention research

Efficacy trials test whether interventions work under optimal, highly controlled conditions whereas effectiveness trials test whether interventions work with typical clients and providers in real-world settings. Researchers, providers, and funding bodies have called for more effectiveness trials to understand whether interventions produce effects under ecologically valid conditions, which factors predict program effectiveness, and what strategies are needed to successfully implement programs in practice settings. The transition from efficacy to effectiveness with preventive interventions involves unique considerations, some of which are not shared by treatment research. The purpose of this article is to discuss conceptual and methodological issues that arise when making the transition from efficacy to effectiveness research in primary, secondary, and tertiary prevention, drawing on the experiences of two complimentary research groups as well as the existing literature. We address (a) program of research, (b) intervention design and conceptualization, (c) participant selection and characteristics, (d) providers, (e) context, (f) measurement and methodology, (g) outcomes, (h) cost, and (i) sustainability. We present examples of research in eating disorder prevention that demonstrate the progression from efficacy to effectiveness trials.     

The KiVa antibullying curriculum and outcome: Does fidelity matter?

Research on school-based prevention suggests that the success of prevention programs depends on whether they are implemented as intended. In antibullying program evaluations, however, limited attention has been paid to implementation fidelity. The present study fills in this gap by examining the link between the implementation of the KiVa antibullying program and outcome. With a large sample of 7413 students (7–12 years) from 417 classrooms within 76 elementary schools, we tested whether the degree of implementation of the student lessons in the KiVa curriculum was related to the effectiveness of the program in reducing bullying problems in classrooms. Results of multilevel structural equation modeling revealed that after nine months of implementation, lesson adherence as well as lesson preparation time (but not duration of lessons) were associated with reductions in victimization at the classroom level. No statistically significant effects, however, were found for classroom-level bullying. The different outcomes for victimization and bullying as well as the importance of documenting program fidelity are discussed.     

Engagement in After-School Programs as a Predictor of Social Competence and Academic Performance

Using the experience sampling method, this study examined two questions related to outcomes associated with after‐school programming. First, does the quality of experience in after‐school programs mediate the effect of program participation on social competence and academic performance? Second, among program participants, is the difference in quality of experience when in programs versus other settings after school related to higher social competence and academic performance? Middle school students (N = 196) attending eight programs in three Midwestern states reported a total of 4,970 randomly sampled experiences in and out of after‐school programs during 1 week in the fall and spring of the 2001–2002 academic year. Engagement during after‐school hours partially mediated the relationship between participation in after‐school programs and social competence. In addition, relative perceptions of engagement, challenge, and importance when in after‐school programs compared to elsewhere after school predicted higher English and math grades. Results suggest that the quality of experiences in after‐school programs may be a more important factor than quantity of experiences (i.e., dosage) in predicting positive academic outcomes.     

Modified TC for MICA offenders: crime outcomes

The study randomly assigned male inmates with co-occurring serious mental illness and chemical abuse (MICA) disorders to either modified therapeutic community (MTC) or mental health (MH) treatment programs. On their release from prison, MICA inmates who completed the prison MTC program could enter the MTC aftercare program. The results, obtained from an intent-to-treat analysis of all study entries, showed that inmates randomized into the MTC group had significantly lower rates of reincarceration compared with those in the MH group. The results also show that differences between the MTC + aftercare and comparison group across a variety of crime outcomes (i.e. any criminal activity, and alcohol or drug related criminal activity) are consistent and significant, and persist after an examination of various threats to validity (e.g. initial motivation, duration of treatment, exposure to risk). This study provides some support for the effectiveness of the prison TC only condition. The findings are encouraging and consonant with other studies of integrated prison and aftercare TC programs for substance abusing non-MICA offenders, although qualified by the possibility that selection bias (i.e. differences in motivation on entry into aftercare) may be operating. Nevertheless, given the available evidence and the need for effective programming for MICA offenders, program and policy makers should strongly consider developing integrated prison and aftercare modified TC programs for MICA offenders.     

Children's self-selection into prevention programs: Evaluation of an innovative recruitment strategy for children of alcoholics

Systematically evaluated the efficacy of a self-selection strategy to recruit elementary-aged children into a school-based prevention program for children of alcoholics. Recruitment involved: a film about parental alcoholism, a follow-up meeting, and an invitation to participate in a prevention program. Of the 844 4th-6th graders exposed to recruitment, 67% showed no interest in the program, 26% attended the follow-up meeting, and 11% obtained parental permission. Analyses focused on group differences according to selection classification on measures of symptomatology and resources available to the child. Significant differences were noted on level of concern about parents' drinking, depression, conduct disorder, and self-worth. Children interested in the program tended to score higher on measures of symptomatology. The potential of a self-selection model for identifying children at risk and areas for future research are discussed.     

Effects of a family literacy program on low-literate children and their parents: findings from an evaluation of the Even Start family literacy program

A randomized experiment was conducted to test the effectiveness of Even Start, a federally supported family literacy program providing early childhood education, adult education, parenting education, and joint parent-child literacy activities to children and parents from low-literate families. The evaluation of 18 Even Start projects followed 463 families for 2 years and found no statistically significant or educationally important impacts on Even Start families when they were compared with control families on child literacy outcomes, parent literacy outcomes, or parent-child interactions. The study concludes that Even Start projects were able to properly implement family literacy programs, and the observed lack of effectiveness is attributed to a combination of 2 factors: (a) a lack of full participation on the part of families and (b) instructional services that may be ineffective because of the curriculum content or the instructional approach.     

Predicting Response to an Internet-Delivered Parenting Program for Anxiety in Early Childhood

Previous research has identified factors related to outcome in child anxiety treatment and parent training programs for child behavior problems. However, it is unclear what factors predict outcomes in interventions delivered online to parents of young children at risk of anxiety. This study investigated predictors of child anxiety outcomes among 433 families with young children (3–6 years) who participated in a randomized controlled trial of Cool Little Kids Online, an eight-module early intervention program for child anxiety based on cognitive-behavioral therapy (CBT). Potential predictors included baseline demographic factors, child and parent mental health factors, and indicators of program use, including number of online modules completed and frequency of homework practice. Results showed that only access to a printer moderated intervention effectiveness. Printer access predicted lower child anxiety in the Cool Little Kids Online group, but had no effect on outcomes in the wait-list group. In both groups, higher levels of child anxiety symptoms, child-inhibited temperament, and poorer parent mental health at baseline predicted higher levels of child anxiety symptoms at 6-month follow-up. The amount of online program use was not related to improvements in child anxiety symptoms. However, parents who reported practicing the program skills more frequently showed greater reductions in child anxiety, and access to a printer was related to frequency of program skills practice. These findings provide empirical support for the important role of skills practice in online CBT interventions, and suggest that practicing program skills may be more important than completing the online modules.     

Individual Differences in the Effectiveness of Error Management Training for Developing Negotiation Skills

This study investigated whether the effectiveness of an error management approach to training negotiation knowledge and skill depended on individual differences in conscientiousness, extraversion, and openness to experience. Participants were randomly assigned to two training programs that incorporated key elements of an error management and behavioral modeling approach to training, and were trained in the complex interpersonal skill of negotiation. At the end of training, declarative knowledge acquisition, procedural knowledge acquisition, declarative knowledge retention, and transfer performance were assessed at different points in time. Results suggested that the effectiveness of the error management training program was dependent on individual levels of conscientiousness and extraversion. For several learning outcomes, the performance of highly conscientious and extraverted individuals was superior in the error management condition, while the performance of less conscientious and introverted individuals was superior in the behavioral modeling condition. The implications of these findings, and suggestions for future research, are discussed.     

What Difference Does Youth Group Make? A Longitudinal Analysis of Religious Youth Group Participation Outcomes

Though many congregations structure youth ministry programs to foster desired religious and social outcomes, questions remain regarding whether participation in religious-based youth programs is beneficial for the youth involved. Studies show that religion typically results in positive life outcomes for youth, but less is known about the causal mechanisms through which religion makes a difference in their lives. Data analyzed are from two waves of the National Study of Youth and Religion (NSYR). This article examines the effect of youth participation on adult support, church connections, and moral values. A multivariate analysis with lagged dependent variables indicates some effects of youth group participation as well as self-selection processes.     

The Children Helping Children Program: A Case Illustration

This article highlights the need for community and school-based programs for children of divorce and describes The Children Helping Children Program which was designed to respond to this need. A review of literature indicates that children of divorce typically experience school related problems including poor academic performance, depression, and disturbances in peer relationships. The review also points out that few support group programs are presently available, either in schools or through agencies for these children. Evaluative evidence consisting of parent comments at the conclusion of the program, followup questionnaire data collected at least four months after the program, and pre-post data pertaining to parenting effectiveness suggest that Children Helping Children may be a promising program. The need for further evaluative data in future implementations is discussed.     

Restoring Equity or Introducing Bias? A Contingency Model of Attitudes Toward Affirmative Action Programs1

We developed a model to explain how an individual's attitude toward the group targeted by affirmative action impacts support for the program. In this model, attitude toward the targeted group influences the extent to which an individual perceives discrimination to be responsible for workforce disparities. Perceived discrimination affects fairness judgments of affirmative action programs with the effect contingent on the extent to which the remedy involves preferential treatment. To test this, participants were told about the selection system in a company in which minorities were underrepresented. Participants evaluated the extent to which they believed that discrimination occurs in the hiring process and 3 possible remedies. Results supported attitudes toward the targeted minority group as an antecedent of perceived discrimination and found that the amount of perceived discrimination was negatively related to fairness judgments of opportunity enhancement programs, but positively related to evaluations of programs that involved preferential treatment. Fairness judgments were positively related to support for all 3 affirmative action programs.     

Enhancing Social Responsibility and Prosocial Leadership to Prevent Aggression,Peer Victimization,and Emotional Problems in Elementary School Children

Testing the theories that form the basis of prevention programs can enhance our understanding of behavioral change and inform the development, coordination, and adaptation of prevention programs. However, theories of change showing the linkages from intervention program components to risk or protective factors to desired outcomes across time are rarely specified or tested. In this 2‐year longitudinal study, we test the theory that increases in two protective factors (i.e., children's prosocial leadership and their teachers’ expectations of social responsibility) targeted by the WITS Programs (Walk Away, Ignore, Talk it Out, and Seek Help) would be associated with declines in peer victimization, aggression, and emotional problems. Participants included Canadian students, in grades 1–4 at baseline (n = 1329) and their parents and teachers. Consistent with our theory of change, variability in program implementation (adherence and integration) and in children's use of program skills (child responsiveness) are related to increases in both protective factors. Increases in these protective factors are associated with subsequent declines in children's aggression, victimization, and emotional problems. We discuss how enhancement of these protective factors may operate to improve child outcomes and the need for theory‐based research to refine and improve the effectiveness of intervention strategies and to improve program scale‐up.     

THE INFLUENCE OF TRAINEE ATTITUDES ON TRAINING EFFECTIVENESS: TEST OF A MODEL

Studies of the influence of trainee characteristics on training effectiveness have focused on the ability level necessary to learn program content. Motivational and environmental influences on training effectiveness have received little attention. The purpose of this study was to test an exploratory model describing the influence on trainee career and job attitudes on training outcomes (learning, behavior change, performance improvement). Results of the study suggest that job involvement and career planning are antecedents of learning and behavior change. Future research directions and practical implications of the results are discussed.     

What Works Best for Reducing Symptoms and Improving Quality of Life? A 6-Months Follow-up Study on the Effectiveness of Group Cognitive Behaviour Therapy and Group Information and Support for Adults Suffering from Depression

A non-randomized effectiveness trial was conducted in an urban secondary care public mental health service in Ireland. The effectiveness of an 8 session group cognitive behavior therapy program (gCBT-8), where the focus was on behavioral activation and cognitive restructuring, was compared with that of an enhanced 12 session group cognitive behavior therapy program (gCBT-12), which contained additional sessions on schema work, and a 12 session information and support control group program (gIS-12) in which there was no CBT skills training. One hundred and eighty-one adults with unipolar mood disorders were allocated to gCBT-8 (N?=?67), gCBT-12 (N?=?62), or gIS-12 (N?=?52). Before and after treatment, and at 6 months follow-up participants completed the Beck Depression Inventory II (BDI-II), the Beck Anxiety Inventory, the Clinical Outcome and Routine Evaluation, the Dysfunctional Attitudes Scale, and the brief World Health Organization Quality of Life Scale. An intention-to-treat analysis of mean scores showed that the three programs had similar outcomes. There were significant improvements on all dependent variables with post-treatment gains maintained at follow-up. The three programs also led to similar remission and combined reliable improvement and remission rates on the BDI-II. It was concluded that all three programs had similar levels of effectiveness.     

Formal mentoring programs: The relationship of program design and support to mentors’ perceptions of benefits and costs

This study investigates the relationship of formal mentoring program design elements (i.e., voluntary participation, input to matching, and effectiveness of training) and management support to the benefits and costs perceived by formal mentors. Data were collected from 97 formal mentors from a Midwestern financial institution. Multiple regressions were performed controlling for time as a mentor in the program, hours spent mentoring, and number of protégés. Voluntary mentor participation was positively related to perceiving rewarding experiences and negatively related to being more trouble than it was worth. Input to the matching process was negatively related to nepotism, and perceptions of training effectiveness were positively related to generativity. Finally, perceived management support for the program was positively related to rewarding experience and recognition, and negatively related to generativity and bad reflection. Three supplemental group interviews were conducted to further explore some of the survey findings. Directions for future research and implications for formal workplace mentoring programs as well as mentoring programs in cross-disciplinary contexts are discussed.     

Exploring programmatic moderators of the effectiveness of marriage and relationship education programs: a meta-analytic study

This study uses meta-analytic methods to explore programmatic moderators or common factors of the effectiveness of marriage and relationship education (MRE) programs. We coded 148 evaluation reports for potential programmatic factors that were associated with stronger intervention effects, although the range of factors we could code was limited by the lack of details in the reports. Overall, we found a positive effect for program dosage: moderate-dosage programs (9-20 contact hours) were associated with stronger effects compared to low-dosage programs (1-8 contact hours). A programmatic emphasis on communication skills was associated with stronger effects on couple communication outcomes, but this difference did not reach statistical significance for the relationship quality/satisfaction outcome. There was no evidence that institutionalized MRE programs (formal manuals, ongoing presence, formal instructor training, multiple evaluations) were associated with stronger effects. Similarly, there was little evidence of differences in program setting (university/laboratory vs. religious). We discuss possible explanations for these findings and implications for program design and evaluation.     

STEM enrichment programs and graduate school matriculation: the role of science identity salience

Improving the state of science education in the United States has become a national priority. One response to this problem has been the implementation of STEM enrichment programs designed to increase the number of students that enter graduate programs in science. Current research indicates enrichment programs have positive effects for student performance, degree completion, interest in science and graduate enrollment. Moreover, research suggests that beyond improving performance in STEM, and providing access to research experience and faculty mentoring, enrichment programs may also increase the degree to which students identify as scientists. However, researchers investigating the role of science identity on student outcomes have focused primarily on subjective outcomes, leaving a critical question of whether science identity also influences objective outcomes such as whether students attend graduate school. Using identity theory, this study addresses this issue by investigating science identity as a mechanism linking enrichment program participation to matriculation into graduate science programs. Quantitative results from a panel study of 694 students indicate that science identity salience, along with research experience and college GPA, mediate the effect of enrichment program participation on graduate school matriculation. Further, results indicate that although the social psychological process by which science identity salience develops operates independently from student GPA, science identity amplifies the effect of achievement on graduate school matriculation. These results indicate that policies seeking to increase the efficacy of enrichment programs and increase representation in STEM graduate programs should be sensitive to the social and academic aspects of STEM education.     

Decision analysis: An approach for multidisciplinary teams in planning special service programs

A decision analytic approach for use by multidisciplinary teams (MDTs) in planning special service programs is described, and formative evaluation information on its application by MDTs in planning individualized education programs (IEPs) is presented. Decision analysis is applied following initial determination of program goals and objectives. In using the approach, an MDT engages in a six-step process: (a) delineation of program alternatives, (b) determination of program outcomes, (c) assessment of probabilities, (d) assessment of utilities, (e) determination of overall program values, and (f) selection of the program to be developed. Utilization of the approach allows a complex program planning situation to be disaggregated into its essential elements, with discussion of program alternatives occurring relative to specific decision criteria. Strengths and limitations of the approach also are discussed, and future directions for research are briefly noted.