Single-Case Experimental Designs in Family Therapy Research: Limitations and Considerations

Recent authors have encouraged the use of single-case experimental design in family therapy research. However, several issues related to the application of these designs have not been addressed. The present article suggests that the applicability of single-case methods to family therapy research may be limited. The first issue raised is the general lack of adequate dependent measures of family interaction that fulfill the requirements of the single-case experimental designs. Suggestions for development of appropriate measures are given. Second, the necessary reliance on relatively weak single-case designs often allows only relatively weak conclusions. Finally, the strategy of beginning the single-case study of family functioning and family therapy in laboratory settings is suggested.     

A Gradual Effects Model for Single-Case Designs

Single-case designs are a class of repeated measures experiments used to evaluate the effects of interventions for small or specialized populations, such as individuals with low-incidence disabilities. There has been growing interest in systematic reviews and syntheses of evidence from single-case designs, but there remains a need to further develop appropriate statistical models and effect sizes for data from the designs. We propose a novel model for single-case data that exhibit nonlinear time trends created by an intervention that produces gradual effects, which build up and dissipate over time. The model expresses a structural relationship between a pattern of treatment assignment and an outcome variable, making it appropriate for both treatment reversal and multiple baseline designs. It is formulated as a generalized linear model so that it can be applied to outcomes measured as frequency counts or proportions, both of which are commonly used in single-case research, while providing readily interpretable effect size estimates such as log response ratios or log odds ratios. We demonstrate the gradual effects model by applying it to data from a single-case study and examine the performance of proposed estimation methods in a Monte Carlo simulation of frequency count data.     

Single-case experimental designs: Characteristics,changes, and challenges

Tactics of Scientific Research (Sidman, 1960) provides a visionary treatise on single-case designs, their scientific underpinnings, and their critical role in understanding behavior. Since the foundational base was provided, single-case designs have proliferated especially in areas of application where they have been used to evaluate interventions with an extraordinary range of clients, settings, and target foci. This article highlights core features of single-case experimental designs, how key and ancillary features of the designs have evolved, the special strengths of the designs, and challenges that have impeded their integration in many areas where their contributions are sorely needed. The article ends by placing the methodological approach in the context of other research traditions. In this way, the discussion moves from the specific designs toward foundations and philosophy of science issues in keeping with the strengths of the person and book we are honoring.     

Meta-analysis for single-case consultation outcomes: Applications to research and practice

The application of meta-analysis holds much appeal for single-case consultation outcome research. We review a meta-analytic method for using within-study treatment effect sizes in reporting consultation outcomes. The strengths and limitations of traditional group design meta-analysis are examined. Various methods for analyzing single-case outcomes are discussed briefly, followed by an examination of the use of meta-analysis in single-case reviews across independent studies. Within-study meta-analytic results are presented that were derived from treatments implemented in consultations in natural settings. To conclude the article, an illustration is offered of a single-case data analysis display that incorporates meta-analytic results along with other indices of treatment outcome. Recommendations are provided for using meta-analytic methods to evaluate outcomes of single-case consultation treatment.     

An Idiographic Single-Case Study Examining the Use of Rational Emotive Behavior Therapy (REBT) with Three Amateur Golfers to Alleviate Social Anxiety

Performance anxiety has been studied in relation to golf performance, but one phenomenon that has received scant attention is social anxiety. One potential intervention that could reduce social anxiety in golfers is rational emotive behavior therapy (REBT), a cognitive-behavioral approach for which research interest is growing. The current study used an idiographic single-case study design to assess the effects of REBT on the social anxiety of 3 male amateur golfers. REBT was employed both on and off the golf course to ensure integration of REBT into the golfers’ performance, offering a methodological advancement of past research. Data were collected prior to, during, and after the REBT intervention. Visual analysis following single-case guidelines revealed substantial reductions in irrational beliefs and social anxiety in all three golfers. Social validation data indicated the positive receipt of REBT by the golfers and supported the visual analysis findings. This current study supports the effectiveness of REBT and extends the research by applying REBT in a “real-world” performance setting, offering methodological advances and providing clear implications for future research and practice.     

Meta- and statistical analysis of single-case intervention research data: Quantitative gifts and a wish list

In this commentary, we add to the spirit of the articles appearing in the special series devoted to meta- and statistical analysis of single-case intervention-design data. Following a brief discussion of historical factors leading to our initial involvement in statistical analysis of such data, we discuss: (a) the value added by including statistical-analysis recommendations in the What Works Clearinghouse Standards for single-case intervention designs; (b) the importance of visual analysis in single-case intervention research, along with the distinctive role that could be played by single-case effect-size measures; and (c) the elevated internal validity and statistical-conclusion validity afforded by the incorporation of various forms of randomization into basic single-case design structures. For the future, we envision more widespread application of quantitative analyses, as critical adjuncts to visual analysis, in both primary single-case intervention research studies and literature reviews in the behavioral, educational, and health sciences.     

A Meta-analysis of the Taped Problems Intervention

Taped problems is an intervention strategy for addressing mathematics fluency that has been evaluated in multiple single-case design studies. Although its efficacy has been supported in individual studies, no comprehensive quantitative synthesis has been conducted on taped problems. The purpose of this study was to synthesize the literature that examined this intervention using parametric and nonparametric effect size computations. We examined (a) the effectiveness of taped problems across effect size comparisons, (b) maintenance over time, (c) the degree to which studies met single-case design research standards, and (d) potential moderating variables. Parametric and nonparametric analyses yielded moderate and large effect sizes and indicated that taped problems appears to be an efficacious intervention.     

Say something nice: A meta-analytic review of peer reporting interventions

Peer reporting interventions (i.e., Positive Peer Reporting and tootling) are commonly used peer-mediated interventions in schools. These interventions involve training students to make reports about peers' prosocial behaviors, whether in oral or written form. Although peer reporting interventions have been included in meta-analyses of group contingencies, this study is the first meta-analytic review of single-case research focusing exclusively on peer reporting interventions. The literature search and application of inclusion criteria yielded 21 studies examining the impact of a peer reporting intervention on student behavior compared to baseline conditions. All studies used single-case experimental designs including at least three demonstrations of an effect and at least three data points per phase. Several aspects of studies, participants, and interventions were coded. Log response ratios and Tau were calculated as effect size estimates. Effect size estimates were synthesized in a multi-level meta-analysis with random effects for (a) studies and (b) cases within studies. Overall results indicated peer reporting interventions had a non-zero and positive impact on student outcomes. This was also true when data were subset by outcome (i.e., disruptive behavior, academically engaged behavior, and social behavior). Results were suggestive of more between- than within-study variability. Moderator analyses were conducted to identify aspects of studies, participants, or peer reporting interventions associated with differential effectiveness. Moderator analyses suggested published studies were associated with higher effect sizes than unpublished studies (i.e., theses/dissertations). This meta-analysis suggests peer reporting interventions are effective in improving student behavior compared to baseline conditions. Implications and directions for future investigation are discussed.     

Issues in the Assessment of Outcome in Family Therapy

This paper examines several issues in family therapy outcome research. These include the need to consider numerous population and treatment variables, the influence of treatment goals and values upon the research, the difficulties in defining and operationalizing family treatment, the choice between emphasizing integrity of treatment or randomness of sampling, the selection of measurement methods, the controls needed in research design, the extent of generalizability of results, the importance of efficiency of treatment, and the special role of deterioration effects. Throughout, the need for a multivariate schema for conceptualizing this research and special attention to the role of values and assumptions in it are highlighted.     

Characteristics of single-case designs used to assess intervention effects in 2008

This article reports the results of a study that located, digitized, and coded all 809 single-case designs appearing in 113 studies in the year 2008 in 21 journals in a variety of fields in psychology and education. Coded variables included the specific kind of design, number of cases per study, number of outcomes, data points and phases per case, and autocorrelations for each case. Although studies of the effects of interventions are a minority in these journals, within that category, single-case designs are used more frequently than randomized or nonrandomized experiments. The modal study uses a multiple-baseline design with 20 data points for each of three or four cases, where the aim of the intervention is to increase the frequency of a desired behavior; but these characteristics vary widely over studies. The average autocorrelation is near to but significantly different from zero; but autocorrelations are significantly heterogeneous. The results have implications for the contributions of single-case designs to evidence-based practice and suggest a number of future research directions.     

Using response ratios for meta-analyzing single-case designs with behavioral outcomes

Methods for meta-analyzing single-case designs (SCDs) are needed to inform evidence-based practice in clinical and school settings and to draw broader and more defensible generalizations in areas where SCDs comprise a large part of the research base. The most widely used outcomes in single-case research are measures of behavior collected using systematic direct observation, which typically take the form of rates or proportions. For studies that use such measures, one simple and intuitive way to quantify effect sizes is in terms of proportionate change from baseline, using an effect size known as the log response ratio. This paper describes methods for estimating log response ratios and combining the estimates using meta-analysis. The methods are based on a simple model for comparing two phases, where the level of the outcome is stable within each phase and the repeated outcome measurements are independent. Although auto-correlation will lead to biased estimates of the sampling variance of the effect size, meta-analysis of response ratios can be conducted with robust variance estimation procedures that remain valid even when sampling variance estimates are biased. The methods are demonstrated using data from a recent meta-analysis on group contingency interventions for student problem behavior.     

Single-case design standards: An update and proposed upgrades

In this paper, we provide a critique focused on the What Works Clearinghouse (WWC) Standards for Single-Case Research Design (Standards 4.1). Specifically, we (a) recommend the use of visual-analysis to verify a single-case intervention study's design standards and to examine the study's operational issues, (b) identify limitations of the design-comparable effect-size measure and discuss related statistical matters, (c) review the applicability and practicality of Standards 4.1 to single-case designs (SCDs), and (d) recommend inclusion of content pertaining to diversity, equity, and inclusion in future standards. Within the historical context of the WWC Pilot Standards for Single-Case Design (1.0), we suggest that Standards 4.1 may best serve as standards for meta-analyses of SCDs but will need to make clear distinctions among the various types of SCD studies that are included in any research synthesis. In this regard, we argue for transparency in SCD studies that meet design standards and those that do not meet design standards in any meta-analysis emanating from the WWC. The intent of these recommendations is to advance the science of SCD research both in research synthesis and in promoting evidence-based practices.     

The Application of Intensive Longitudinal Methods to Investigate Change: Stimulating the Field of Applied Family Research

The use of intensive longitudinal methods (ILM)—rapid in situ assessment at micro timescales—can be overlaid on RCTs and other study designs in applied family research. Particularly, when done as part of a multiple timescale design—in bursts over macro timescales—ILM can advance the study of the mechanisms and effects of family interventions and processes of family change. ILM confers measurement benefits in accurately assessing momentary and variable experiences and captures fine-grained dynamic pictures of time-ordered processes. Thus, ILM allows opportunities to investigate new research questions about intervention effects on within-subject (i.e., within-person, within-family) variability (i.e., dynamic constructs) and about the time-ordered change process that interventions induce in families and family members beginning with the first intervention session. This paper discusses the need and rationale for applying ILM to family intervention evaluation, new research questions that can be addressed with ILM, example research using ILM in the related fields of basic family research and the evaluation of individual-based interventions. Finally, the paper touches on practical challenges and considerations associated with ILM and points readers to resources for the application of ILM.     

Single-case effect size calculation: Comparing regression and non-parametric approaches across previously published reading intervention data sets

Growing from demands for accountability and research-based practice in the field of education, there is recent focus on developing standards for the implementation and analysis of single-case designs. Effect size methods for single-case designs provide a useful way to discuss treatment magnitude in the context of individual intervention. Although a standard effect size methodology does not yet exist within single-case research, panel experts recently recommended pairing regression and non-parametric approaches when analyzing effect size data. This study compared two single-case effect size methods: the regression-based, Allison-MT method and the newer, non-parametric, Tau-U method. Using previously published research that measured the Words read Correct per Minute (WCPM) variable, these two methods were examined by comparing differences in overall effect size scores and rankings of intervention effect. Results indicated that the regression method produced significantly larger effect sizes than the non-parametric method, but the rankings of the effect size scores had a strong, positive relation. Implications of these findings for research and practice are discussed.     

A generalized least squares regression approach for computing effect sizes in single-case research: application examples

A new method for deriving effect sizes from single-case designs is proposed. The strategy is applicable to small-sample time-series data with autoregressive errors. The method uses Generalized Least Squares (GLS) to model the autocorrelation of the data and estimate regression parameters to produce an effect size that represents the magnitude of treatment effect from baseline to treatment phases in standard deviation units. In this paper, the method is applied to two published examples using common single case designs (i.e., withdrawal and multiple-baseline). The results from these studies are described, and the method is compared to ten desirable criteria for single-case effect sizes. Based on the results of this application, we conclude with observations about the use of GLS as a support to visual analysis, provide recommendations for future research, and describe implications for practice.     

Family therapy as a treatment for children: a critical review of outcome research.

The value of family therapy as a treatment for child psychopathology is considered by reviewing pertinent outcome research. Fourteen studies that met three criteria are included in the review: (a) a child or adolescent was the identified patient or referral; (b) therapy included at least one parent and the child; and (c) outcome was evaluated in terms of the child's symptoms. There are major shortcomings in most of the available data, with only two well-controlled studies. Some empirical evidence does exist that family therapy is an effective treatment for children; the data from studies of adolescents are especially encouraging. However, insufficient data are available for comparing the relative merits of conjoint family treatment and individual child therapy. If the value of family therapy as a treatment alternative or, ideally, as the "treatment of choice" for a referred individual child is to be established, more and better controlled comparative outcome studies will be necessary. Suggestions for future research are presented emphasizing the need for a developmental perspective by recommending, for example, the use of factorial designs in which the intervenaction of treatment and age can be analyzed.     

An empirical investigation of the statistical properties of generalized single-case randomization tests

Adapting Edgington's [J. Psychol. 90 (1975) 57] randomly determined intervention start-point model, Levin and Wampold [Sch. Psychol. Quart. 14 (1999) 59] proposed a set of nonparametric randomization tests for analyzing the data from single-case designs. In the present study, the performance of Levin and Wampold's four basic tests (independent start-point general and comparative effectiveness, simultaneous start-point general and comparative effectiveness) was examined with respect to their Type I error rates and statistical power. Of Levin and Wampold's four tests, all except the independent start-point comparative effectiveness test maintained their empirical Type I error rates and had acceptable power at larger sample-size and effect-size combinations. The one-tailed comparative intervention effectiveness test for the independent start-point model was found to be too liberal, in that it did not maintain its Type I error rate. Although a two-tailed application of that test was found to be conservative at longer series lengths, it had acceptable power at larger sample-size and effect-size combinations. The results support the utility of a versatile new class of single-case designs that permit both within- and between-unit statistical assessments of intervention effectiveness.