A Systematic Review of Randomized Controlled Trials to Assess Outcomes of Genetic Counseling

With the advancements in precision medicine and health care reform, it is critical that genetic counseling practice respond to emerging evidence to maximize client benefit. The objective of this review was to synthesize evidence on outcomes from randomized controlled trials (RCTs) of genetic counseling to inform clinical practice. Seven databases were searched in conducting this review. Studies were selected for inclusion if they were: (a) RCTs published from 1990 to 2015, and (b) assessed a direct outcome of genetic counseling. Extracted data included study population, aims, and outcomes. Risk of bias was evaluated using the Cochrane Handbook for Systematic Reviews of Interventions guidelines. A review of 1654 abstracts identified 58 publications of 54 unique RCTs that met inclusion criteria, the vast majority of which were conducted in cancer genetic counseling setting. Twenty-seven publications assessed ‘enhancements’ to genetic counseling, and 31 publications compared delivery modes. The methodological rigor varied considerably, highlighting the need for attention to quality criteria in RCT design. While most studies assessed several client outcomes hypothesized to be affected by genetic counseling (e.g., psychological wellbeing, knowledge, perceived risk, patient satisfaction), disparate validated and reliable scales and other assessments were often used to evaluate the same outcome(s). This limits opportunity to compare findings across studies. While RCTs of genetic counseling demonstrate enhanced client outcomes in a number of studies and pave the way to evidence-based practice, the heterogeneity of the research questions suggest an important need for more complementary studies with consistent outcome assessments.     

The Impact of Evidence-Based Practices on Requests for Out-of-Home Placements in the Context of System Reform

Beginning in January 2001 and continuing through April 2008 New Jersey completed statewide implementation of the basic infrastructure of a system of care (SOC) for child and adolescent behavioral health services. In 2008, the state began to enhance its service infrastructure through a large scale implementation of two community-based treatment programs that have been recognized as evidence-based: multisystemic therapy (MST) and functional family therapy (FFT). Multiple evaluations for both therapies have assessed the efficacy of the interventions at the client level. Using interrupted time series analysis, we evaluated the impact of the addition of these new treatment options to New Jersey’s SOC on a system level outcome: statewide requests for out-of-home placements. Results indicate a significant reduction in requests for out-of-home placements associated with the implementation of MST and FFT and a net overall savings to the state.     

Do haphazard reviews provide sound directions for dissemination efforts?

Comments on The dissemination and implementation of evidence-based psychological treatments: A review of current efforts (see record 2010-02208-010) by Kathryn R. McHugh and David H. Barlow. The lead article in the February–March issue by McHugh and Barlow (2010) emphasized the need for “dissemination and implementation of evidence-based psychological treatments.” The authors identified a number of intervention programs as evidence based and in need of dissemination. One is multisystemic therapy (MST). They claimed that this program is among “the most successful dissemination efforts...pursued by treatment developers” (p. 79). McHugh and Barlow’s (2010) discussion of the implementation of MST in Hawaii is troubling, because it neglected to mention concerns about the perceived lack of cultural sensitivity of the MST program in that state.     

Economic Impact of the Statewide Implementation of an Evidence-Based Treatment: Multisystemic Therapy in New Mexico

Several states have made considerable investments into large-scale implementation of evidence-based treatments (EBTs), yet little is known about key success indicators for these implementation efforts such as cost and sustainability. To that end, the present study examined the economic impact of statewide implementation of multisystemic therapy (MST; Henggeler, Schoenwald, Borduin, Rowland, & Cuningham, 2009), a family- and community-based behavioral EBT for serious juvenile offenders in New Mexico. Participants were 1,869 youth who received MST across 23 teams during the study period. We conducted a cost-benefit analysis using metrics from state data sources that compared the cost of MST to its benefits (i.e., avoided expenses from pre- to posttreatment) in two domains: (a) behavioral health services (i.e., Medicaid claims) and (b) juvenile crime (i.e., taxpayer expenses, tangible and intangible expenses to crime victims). MST costs were based on Medicaid claims, which were reimbursed at an enhanced billing rate that was intended to cover expenses for both clinical and implementation (e.g., training, quality assurance) activities. Results suggest that implementation of MST in New Mexico over the 7-year study period may have produced net benefits, through 2 years posttreatment, of more than $4,643 per youth in avoided behavioral health claims and $15,019 per youth through reductions in juvenile crime. Stated differently, every dollar that New Mexico spent on MST appeared to have returned $3.34 for a total benefit of $64.2 million over the course of the study. We discuss implications of these findings for policymakers, administrators, and researchers who are interested in increasing the sustainability of complex EBTs in community settings.     

The Influence of Parental Factors on Therapist Adherence in Multi-systemic Therapy

Therapist treatment adherence has received a great deal of attention in recent years, in part because of its relation to treatment outcomes. Although certain therapist behaviors have been found to be associated with treatment outcomes, little is known about client factors impacting on therapists’ ability to adhere to treatment protocols. In this study, we evaluated effects of parental beliefs, psychopathology, and interaction styles on therapists’ adherence to Multi-systemic Therapy (MST) principles. Eighty-two parents participating in a clinical trial of MST completed baseline measures assessing psychopathology, family functioning, and treatment expectations. Analyses indicated that parental perceptions of therapist adherence were established within the first 4 weeks of treatment, and that parental psychopathology, motivation, expectations, and child rearing practices were related to parental ratings of therapist adherence. Results were essentially unchanged when controlling for parental positive response style. Clinical and research implications of the findings are discussed.     

Performance of Mental Health Counselors in Integrated Primary and Behavioral Health Care

Integrated primary and behavioral health care (IPBH) is becoming a preferred mode of service delivery in the United States. Integrated care includes the participation of medical and mental health professionals, such as mental health counselors. The clinical outcomes of these professionals need to be studied to determine their effectiveness in such settings. We examined the performance of 10 mental health counselors on the clinical outcome of 1,747 clients treated in an IPBH center. Analyses using growth curve modeling and pre‐post test design revealed that mental health counselors were effective overall, but they differed in client dropout rates and efficiency in reducing clients' initial symptoms. We used the analyses to rank order counselors based on their effectiveness. Counselors who were the most effective varied in their efficiency but demonstrated the lowest client dropout rates. Implications for future research and counseling practice were discussed.     

Multisystemic therapy: monitoring treatment fidelity

The challenges of specifying a complex and individualized treatment model and measuring fidelity thereto are described, using multisystemic therapy (MST) as an example. Relations between therapist adherence to MST principles and instrumental and ultimate outcome variables are examined, as are relations between clinical supervision and therapist adherence. The findings provide modest support for the associations between MST adherence measures and instrumental and ultimate outcomes. Results also show that adherence can be altered when clinical supervision and adherence monitoring procedures are fortified. The modest associations between adherence measures and youth outcomes argue for further refinement and validation of the MST adherence measure, especially in light of the well-established effectiveness of MST with challenging clinical populations and the increasing dissemination of MST programs.     

Multiple-family group treatment of outpatients with schizophrenia: impact on service utilization

The impact of multiple-family group treatment (MFGT) on outpatient and inpatient mental health service utilization of 97 persons with schizophrenia was investigated. Participants were randomly assigned to standard care (n = 44) or standard care plus MFGT (n = 53). Service use for a year prior to randomization, the 2-year study period, and a 1-year follow-up were examined. Relative to standard care participants, the MFGT group had reduced community hospitalization during year 1 of the intervention and reduced state hospitalization at follow-up. During the intervention period, MFGT participants demonstrated a significant increase in outpatient utilization as a direct consequence of the intervention. However, when service use was summed across 3 years post-randomization, no group differences were observed. Results suggest that implementation of MFGT in a community mental health setting reduces inpatient service at specific time periods, without significantly increasing outpatient service utilization. These findings add to other outcomes from this study that demonstrate decreased psychiatric symptoms and caregiver distress.     

A Longitudinal Analysis of Family Empowerment and Client Outcomes

We examined the relationship between family empowerment, parent satisfaction, and mental health outcome across time. Based on the Vanderbilt Family Empowerment Project Model, increased empowerment should lead to positive changes in client outcomes. Data consisted of the Family Empowerment Scale (FES), which was used to assess the caregiver's perception of empowerment, Child and Adolescent Functional Assessment Scale (CAFAS), which measures the degree of disruption in the youth's current functioning, Client Satisfaction Questionnaire (CSQ), and demographic information gathered from families receiving services from a county mental health service system at intake and discharge. The results showed that the CAFAS and CSQ were related to empowerment at intake and discharge. Results also indicated significant increases in the knowledge subscale of the FES and the CAFAS and moderate increases in the advocacy subscale of the FES and the CSQ. We discuss the implications of these findings for systems of care, such as stronger parent-professional relationships.     

Client Versus Clinicians’ Standards of Clinically Meaningful Change and the Effects of Treatment Expectations on Therapeutic Outcomes in Individuals With Posttraumatic Stress Disorder

There is limited research on the concordance between client perceptions and clinician standards of the degree of symptom change required to achieve meaningful therapeutic improvement. This was investigated in an adult sample (N = 147) who received trauma-focused cognitive-behavioral therapies for posttraumatic stress disorder (PTSD). We examined whether clients’ benchmarks of change were related to actual outcomes and the relationship between client expectations and their treatment outcomes. Clients completed measures indexing the level of symptom reduction required (in their view) to reflect a benefit or recovery from treatment and treatment expectations. Actual PTSD severity was indexed pre- and posttreatment via self-report and clinician-administered interview. Results demonstrated that the amount of change clients said they required to experience a benefit or recovery was significantly larger than typical clinical research standards. Nonetheless, the majority of client benchmarks of change (79.7–81.8%) were consistent with clinical research standards of what constitutes benefit or recovery. Client benchmarks were generally positively correlated with their actual outcomes. Clients’ belief that treatment would be successful was associated with greater reductions in PTSD symptoms. These findings provide preliminary evidence that the standards used to determine clinically significant change are somewhat consistent with clients’ own perceptions of required symptom change.     

Therapist Self-Efficacy in Delivering Cognitive Processing Therapy in a Randomized Controlled Implementation Trial

Efforts to improve the implementation of evidence-based treatments (EBT) have recently made important strides. One such example is understanding the vital role that weekly consultation plays as therapists learn to deliver an EBT. Because mechanism-based research can further support EBT implementation, the present study sought to examine the potential relationship between therapist self-efficacy in relation to treatment fidelity and outcomes.We examined therapist self-efficacy ratings from 80 therapists working with 188 patients. These data were collected as part of a randomized controlled implementation trial testing cognitive processing therapy (CPT). Across post-workshop training conditions, we ran multilevel models to assess (1) changes in therapist self-efficacy, (2) therapist self-efficacy in relation to treatment fidelity, and (3) therapist-self-efficacy in relation to patient PTSD symptom outcomes.We found that therapist self-efficacy significantly improved over the course of 6 months of CPT training. Baseline therapist self-efficacy was differentially associated with client outcomes based on post-workshop training condition. Specifically, therapists with low self-efficacy that did not receive post-workshop consultation tended to have poorer outcomes than therapists with low self-efficacy that received consultation. In the present sample, therapist self-efficacy was not related to treatment fidelity.As this was the first study to examine therapist self-efficacy in the implementation of an evidence-based treatment, our findings suggest that self-efficacy may be an important implementation factor in treatment outcomes and worthy of ongoing research.     

The practice research network: Research into practice,practice into research

The purpose of the broader project on which this report is based was to utilize a Practice Research Network, an alternative to traditional clinical studies to 1) identify demographically one practice group, namely professional counselors in the United States, 2) describe their practice patterns, and 3) identify their client population and the efficacy of their clinical work through client outcomes based on actual client reports. This article describes the development of one such nation‐wide Practice Research Network (PRN) and provides some preliminary findings for each phase of the project. Use of the PRN model can help overcome the traditional “research to practice” gap and provide research results into community practice thus integrating outcomes measures into the daily practice of mental health professionals. During the early phases of this project, a software program was developed upon which the participating practitioners could identify the data elements within the parameters of the focus on their agreed‐upon research interests, enter data following client visits over time, and by comparing their own composite profiles and effectiveness with those of other participating practitioners who had clients presenting with similar diagnoses, could alter their own service delivery patterns, therapeutic interventions, or approaches while working with the clients and thereby improve the efficacy of their service. By participating, the practitioners benefited by access to downloadable and printable reports on profiles of 1) their specific clients, 2) their service delivery patterns, and 3) the outcomes or efficacy by their clients’ direct feedback. They were also able to compare these results with the composite data from other practitioners.     

Multisystemic therapy for young offenders: families' experiences of therapeutic processes and outcomes

Multisystemic Therapy (MST) has been found to be effective in reducing youth antisocial behavior, but little is known about the process and impact of MST from the perspective of families themselves. This qualitative study explored parents' and young people's experiences of MST, focusing on aspects of the intervention that promoted or limited change. Thirty-seven semistructured interviews were conducted with a consecutive sample of 21 families (21 parent interviews, 16 young people) who had participated in a randomized controlled trial of MST for young offenders in the United Kingdom. Thematic analysis yielded 10 themes, organized into two domains: (a) engagement in MST and initial processes of change captures the central importance of the therapeutic relationship and the MST engagement model in families' positive experiences of MST; and (b) outcomes are complex reflects the range of positive outcomes reported (notably increased parental confidence and skills, improved family relationships, a return to education, and greater reflection and aspiration on the part of the young person) and mixed behavior outcomes. Even when the young person had reoffended, respondents indicated a range of other benefits for the family. The findings support the MST theory of change as well as point to some outcomes not usually measured in MST outcome studies. They also suggest some adaptations that may increase the impact of the intervention, including more attention to the influence of deviant peers, and ongoing support for families struggling to maintain strategies beyond the prescribed treatment period.     

An evaluation of a hospital-based trainee counselling psychology practitioner's service

Counselling psychology is in its infancy in the National Health Service. There is a professional need to evaluate what its practitioners do and with what outcomes. This study is of the work of a trainee counselling psychologist, in a clinical psychology department, and covers his work over 30 months from first appointment. Numerical data were accessed from historical records relating to the author's three working areas: a psychiatric hospital's outpatient department; a university's student health service; and a short-contact primary care counselling clinic. A sample of closed-case clients from the hospital outpatients and the student health service was contacted, and sent questionnaires to elicit their views on the service received. Primary care clients were excluded from this part of the study because of service dissimilarity to the other working domains and access problems. Case loads, case types, frequencies and distributions of client attendances and non-attendances were examined, compared and contrasted using tables, graphs and statistical methods. Clients’ questionnaire responses were tabulated and commented on. It was apparent that although the same general pattern of anxiety-clustered cases predominated in each working area, in other respects the service demands were different. The data have relevance for service design, staffing and management.     

Short versus longer term effectiveness of an outreach program for the homeless mentally Ill

Presents 12-month follow-up results from an outreach/linkage intervention with persons who are homeless and mentally ill, contrasting these with results obtained at 4 months. Both sets reflect the success of the program in placing individuals in independent housing. However, longer term data provide useful information regarding client movement patterns and increased tenure in nonhomeless living arrangements beyond the termination of specialized services. Analyses of 12-month residential outcomes identified four variables as significant predictors: recruitment source, project service duration, CMH service duration, and client age. In contrast to 4-month predictors, variables reflecting baseline client functioning were no longer significantly related to outcome, suggesting that the positive effects of the intervention may take longer to achieve with some clients. Discussion focuses on the implications of these effectiveness results for future research designs and measures as well as the utility and limitations of preexperimental approaches for evaluating innovative service models when implementation and efficacy experiences are lacking. This research was supported in part by a grant from the National Institute of Mental Health, #H87 MH44373, to the Michigan Department of Mental Health.     

What results when firms implement practices: the differential relationship between specific practices, firm financial performance, customer service, and quality

Previous research on organizational practices is replete with contradictory evidence regarding their effects. Here, the authors argue that these contradictory findings may have occurred because researchers have often examined complex practice combinations and have failed to investigate a broad variety of firm-level outcomes. Thus, past research may obscure important differential effects of specific practices on specific firm-level outcomes. Extending this research, the authors develop hypotheses about the effects of practices that (a) enable information sharing, (b) set boundaries, and (c) enable teams on 3 different firm-level outcomes: financial performance, customer service, and quality. Relationships are tested in a sample of observations from over 200 Fortune 1000 firms. Results indicate that information-sharing practices were positively related to financial performance 1 year following implementation of the practices, boundary-setting practices were positively related to firm-level customer service, and team-enabling practices were related to firm-level quality. No single set of practices predicted all 3 firm-level outcomes, indicating practice-specific effects. These findings help resolve the theoretical tension in the literature regarding the effects of organizational practices and offer guidance as to how to best target practices to increase specific work-related outcomes. Implications for theory, research, and practice are discussed.     

Predictors of successful discharge from out-of-home care among children with complex needs

We examined the predictors for successful discharge from out-of-home care of children with complex needs placed in a novel comprehensive service intervention (Manatee Model) and compared their discharge experiences to their out-of-home counterparts from the same county. The study design consisted of a longitudinal two-year comparison of these two groups. A total of 147 cases were included in the analysis, and the propensity scoring technique was used to equate the groups. Among children admitted to the Manatee Model, boys, White children, younger children, those who came from a single-parent family, those who were abused, and those who had a disability (physical or emotional disability or medical condition) were found less likely to successfully exit within the two years when these predictors were examined separately. Multivariate analyses indicated that only age at the time of removal and health status were significantly associated with successful discharge. Implications of the findings are discussed.