Abstract: A Meta-Analysis of the Autocorrelation in Single Case Designs

Single case design (SCD) experiments in the behavioral sciences utilize just one participant from whom data is collected over time. This design permits causal inferences to be made regarding various intervention effects, often in clinical or educational settings, and is especially valuable when between-participant designs are not feasible or when interest lies in the effects of an individualized treatment. Regression techniques are the most common quantitative practice for analyzing time series data and provide parameter estimates for both treatment and trend effects. However, the presence of serially correlated residuals, known as autocorrelation, can severely bias inferences made regarding these parameter estimates. Despite the severity of the issue, few researchers test or correct for the autocorrelation in their analyses.

Shadish and Sullivan (in press) recently conducted a meta-analysis of over 100 studies in order to assess the prevalence of the autocorrelation in the SCD literature. Although they found that the meta-analytic weighted average of the autocorrelation was close to zero, the distribution of autocorrelations was found to be highly heterogeneous. Using the same set of SCDs, the current study investigates various factors that may be related to the variation in autocorrelation estimates (e.g., study and outcome characteristics). Multiple moderator variables were coded for each study and then used in a metaregression in order to estimate the impact these predictor variables have on the autocorrelation.

This current study investigates the autocorrelation using a multilevel meta-analytic framework. Although meta-analyses involve nested data structures (e.g., effect sizes nested within studies nested within journals), there are few instances of meta-analysts utilizing multilevel frameworks with more than two levels. This is likely attributable to the fact that very few software packages allow for meta-analyses to be conducted with more than two levels and those that do allow this provide sparse documentation on how to implement these models. The proposed presentation discusses methods for carrying out a multilevel meta-analysis. The presentation also discusses the findings from the metaregression on the autocorrelation and the implications these findings have on SCDs.