Correcting Overestimated Effect Size Estimates in Multiple Trials

In a simulation study, Brand, Bradley, Best, and Stoica (2011 Brand, A., Bradley, M. T., Best, L. A. and Stoica, G. 2011. Multiple trials may yield exaggerated effect size estimates. The Journal of General Psychology, 138: 1–11. Taylor &; Francis Online], Web of Science ®] , Google Scholar]) have shown that Cohen's d is notably overestimated if computed for data aggregated over multiple trials. Although the phenomenon is highly important for studies and meta-analyses of studies structurally similar to the simulated scenario, the authors do not comprehensively address how the problem could be handled. In this comment, we first suggest a corrective term d ^′ _c that includes the number and correlation of trials. Next, the results of a simulation study provide evidence that the proposed d_c′ results in a more precise estimation of trial-level effects. We conclude that, in practice, d ^′ _c together with plausible estimates of inter-trial correlation will produce a more precise effect size range compared to that suggested by Brand and colleagues (2011 Brand, A., Bradley, M. T., Best, L. A. and Stoica, G. 2011. Multiple trials may yield exaggerated effect size estimates. The Journal of General Psychology, 138: 1–11. Taylor &; Francis Online], Web of Science ®] , Google Scholar]).