Institution: | 1. Department of Statistics, Northwestern University, Evanston, Illinois, USA
Contribution: Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Supervision, Writing - original draft, Writing - review & editing;2. Department of Statistics, Northwestern University, Evanston, Illinois, USA
Contribution: Conceptualization, Data curation, Formal analysis, Methodology, Supervision, Writing - original draft, Writing - review & editing;3. Department of Statistics, University of Washington, Seattle, Washington, USA;4. Consortium for Policy Research in Education, University of Pennsylvania, Philadelphia, Pennsylvania, USA
Contribution: Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Validation, Visualization, Writing - original draft, Writing - review & editing |
Abstract: | It is common practice in both randomized and quasi-experiments to adjust for baseline characteristics when estimating the average effect of an intervention. The inclusion of a pre-test, for example, can reduce both the standard error of this estimate and—in non-randomized designs—its bias. At the same time, it is also standard to report the effect of an intervention in standardized effect size units, thereby making it comparable to other interventions and studies. Curiously, the estimation of this effect size, including covariate adjustment, has received little attention. In this article, we provide a framework for defining effect sizes in designs with a pre-test (e.g., difference-in-differences and analysis of covariance) and propose estimators of those effect sizes. The estimators and approximations to their sampling distributions are evaluated using a simulation study and then demonstrated using an example from published data. |