The impact of methodological factors on child psychotherapy outcome research: A meta-analysis for researchers

Two recent meta-analyses have generated evidence for child and adolescent psychotherapy effects. However, critics note that such meta-analyses often include studies with methodological shortcomings which might invalidate their results. In the present study, we explored whether the results of the most extensive child/adolescent meta-analysis might have been influenced by such methodological variables, focusing on internal validity and external validity factors. Together, these factors accounted for two-thirds as much variance as the substantive factors (e.g., type of therapy, age) in the original metaanalysis. This suggests that relative to these therapy and child-characteristic variables, methodological factors have a substantial, though smaller, impact on metaanalysis results. In general, increased experimental rigor was related to larger effect sizes; this argues against the hypothesis that methodologically weak studies have led to an overestimate of therapy effects. No significant interactive relations were found between validity factors and predictors of outcome; this suggests that the relations noted in previous metaanalyses between outcome and various variables were notdistorted by the validity factors tested here.The research described here was supported in part by grants from the North Carolina Department of Human Resources, Division of Mental Health, Mental Retardation, and Substance Abuse Services (41626), and from the National Institute of Mental Health (1 RO3 MH38450). The authors gratefully acknowledge the assistance of Todd Morton, who provided reliability coding.