Sample size for multiple regression: obtaining regression coefficients that are accurate, not simply significant

An approach to sample size planning for multiple regression is presented that emphasizes accuracy in parameter estimation (AIPE). The AIPE approach yields precise estimates of population parameters by providing necessary sample sizes in order for the likely widths of confidence intervals to be sufficiently narrow. One AIPE method yields a sample size such that the expected width of the confidence interval around the standardized population regression coefficient is equal to the width specified. An enhanced formulation ensures, with some stipulated probability, that the width of the confidence interval will be no larger than the width specified. Issues involving standardized regression coefficients and random predictors are discussed, as are the philosophical differences between AIPE and the power analytic approaches to sample size planning.     

Hierarchical linear models for the quantitative integration of effect sizes in single-case research

In this article, the calculation of effect size measures in single-case research and the use of hierarchical linear models for combining these measures are discussed. Special attention is given to meta-analyses that take into account a possible linear trend in the data. We show that effect size measures that have been proposed for this situation appear to be systematically affected by the duration of the experiment and fail to distinguish between effects on level and slope. To avoid these flaws, we propose to perform a multivariate meta-analysis on the standardized ordinary least squares regression coefficients from the study-specific regression equations describing the response variable.     

An effect size measure and Bayesian analysis of single-case designs

This article describes a linear modeling approach for the analysis of single-case designs (SCDs). Effect size measures in SCDs have been defined and studied for the situation where there is a level change without a time trend. However, when there are level and trend changes, effect size measures are either defined in terms of changes in R² or defined separately for changes in slopes and intercept coefficients. We propose an alternate effect size measure that takes into account changes in slopes and intercepts in the presence of serial dependence and provides an integrated procedure for the analysis of SCDs through estimation and inference based directly on the effect size measure. A Bayesian procedure is described to analyze the data and draw inferences in SCDs. A multilevel model that is appropriate when several subjects are available is integrated into the Bayesian procedure to provide a standardized effect size measure comparable to effect size measures in a between-subjects design. The applicability of the Bayesian approach for the analysis of SCDs is demonstrated through an example.     

Multilevel meta-analysis of single-subject experimental designs: A simulation study

One way to combine data from single-subject experimental design studies is by performing a multilevel meta-analysis, with unstandardized or standardized regression coefficients as the effect size metrics. This study evaluates the performance of this approach. The results indicate that a multilevel meta-analysis of unstandardized effect sizes results in good estimates of the effect. The multilevel meta-analysis of standardized effect sizes, on the other hand, is suitable only when the number of measurement occasions for each subject is 20 or more. The effect of the treatment on the intercept is estimated with enough power when the studies are homogeneous or when the number of studies is large; the power of the effect on the slope is estimated with enough power only when the number of studies and the number of measurement occasions are large.     

Point-biserial correlation: Interval estimation,hypothesis testing,meta-analysis,and sample size determination

The point-biserial correlation is a commonly used measure of effect size in two-group designs. New estimators of point-biserial correlation are derived from different forms of a standardized mean difference. Point-biserial correlations are defined for designs with either fixed or random group sample sizes and can accommodate unequal variances. Confidence intervals and standard errors for the point-biserial correlation estimators are derived from the sampling distributions for pooled-variance and separate-variance versions of a standardized mean difference. The proposed point-biserial confidence intervals can be used to conduct directional two-sided tests, equivalence tests, directional non-equivalence tests, and non-inferiority tests. A confidence interval for an average point-biserial correlation in meta-analysis applications performs substantially better than the currently used methods. Sample size formulas for estimating a point-biserial correlation with desired precision and testing a point-biserial correlation with desired power are proposed. R functions are provided that can be used to compute the proposed confidence intervals and sample size formulas.     

Confidence intervals and sample size calculations for the standardized mean difference effect size between two normal populations under heteroscedasticity

The use of effect sizes and associated confidence intervals in all empirical research has been strongly emphasized by journal publication guidelines. To help advance theory and practice in the social sciences, this article describes an improved procedure for constructing confidence intervals of the standardized mean difference effect size between two independent normal populations with unknown and possibly unequal variances. The presented approach has advantages over the existing formula in both theoretical justification and computational simplicity. In addition, simulation results show that the suggested one- and two-sided confidence intervals are more accurate in achieving the nominal coverage probability. The proposed estimation method provides a feasible alternative to the most commonly used measure of Cohen’s d and the corresponding interval procedure when the assumption of homogeneous variances is not tenable. To further improve the potential applicability of the suggested methodology, the sample size procedures for precise interval estimation of the standardized mean difference are also delineated. The desired precision of a confidence interval is assessed with respect to the control of expected width and to the assurance probability of interval width within a designated value. Supplementary computer programs are developed to aid in the usefulness and implementation of the introduced techniques.     

Estimating the mean effect size in meta-analysis: Bias,precision, and mean squared error of different weighting methods

Although use of the standardized mean difference in meta-analysis is appealing for several reasons, there are some drawbacks. In this article, we focus on the following problem: that a precision-weighted mean of the observed effect sizes results in a biased estimate of the mean standardized mean difference. This bias is due to the fact that the weight given to an observed effect size depends on this observed effect size. In order to eliminate the bias, Hedges and Olkin (1985) proposed using the mean effect size estimate to calculate the weights. In the article, we propose a third alternative for calculating the weights: using empirical Bayes estimates of the effect sizes. In a simulation study, these three approaches are compared. The mean squared error (MSE) is used as the criterion by which to evaluate the resulting estimates of the mean effect size. For a meta-analytic dataset with a small number of studies, theMSE is usually smallest when the ordinary procedure is used, whereas for a moderate or large number of studies, the procedures yielding the best results are the empirical Bayes procedure and the procedure of Hedges and Olkin, respectively.     

Effect size, power, and sample size determination for structured means modeling and mimic approaches to between-groups hypothesis testing of means on a single latent construct

While effect size estimates, post hoc power estimates, and a priori sample size determination are becoming a routine part of univariate analyses involving measured variables (e.g., ANOVA), such measures and methods have not been articulated for analyses involving latent means. The current article presents standardized effect size measures for latent mean differences inferred from both structured means modeling and MIMIC approaches to hypothesis testing about differences among means on a single latent construct. These measures are then related to post hoc power analysis, a priori sample size determination, and a relevant measure of construct reliability.I wish to convey my appreciation to the reviewers and Associate Editor, whose suggestions extended and strengthened the article's content immensely, and to Ralph Mueller of The George Washington University for enhancing the clarity of its presentation.     

Common language effect size for correlations

The Pearson correlation coefficient can be translated to a common language effect size, which shows the probability of obtaining a certain value on one variable, given the value on the other variable. This common language effect size makes the size of a correlation coefficient understandable to laypeople. Three examples are provided to demonstrate the application of the common language effect size in interpreting Pearson correlation coefficients and multiple correlation coefficients.     

Visual marking: dissociating effects of new and old set size

Visual marking makes it possible to ignore old items during search. In a typical study, old items are previewed 1 s before adding an equal number of new items, one of which is the target. Previewing half of the items reduces the search slope relating response time (RT) to overall set size by half. However, this manipulation sometimes only reduces overall RT but not search slope (Experiment 1). By orthogonally varying the numbers of old and new items, Experiment 2 shows that old and new set sizes interactively affect visual marking. Given a constant new set size, the size of the old set has negligible effect on RT. However, increasing the new set size reduces the preview benefit in overall RT. Experiment 3 shows that this reduction may be restricted to paradigms that use temporal segregation cues. Studies should vary old and new set size orthogonally to avoid missing a visual marking effect where one may be present.     

Measures of effect size

Two different approaches have been used to derive measures of effect size. One approach is based on the comparison of treatment means. The standardized mean difference is an appropriate measure of effect size when one is merely comparing two treatments, but there is no satisfactory analogue for comparing more than two treatments. The second approach is based on the proportion of variance in the dependent variable that is explained by the independent variable. Estimates have been proposed for both fixed-factor and random-factor designs, but their sampling properties are not well understood. Nevertheless, measures of effect size can allow quantitative comparisons to be made across different studies, and they can be a useful adjunct to more traditional outcome measures such as test statistics and significance levels.     

基于两水平回归模型的调节效应分析及其效应量

使用多元回归法进行调节效应分析在社科领域已常有应用。简述了目前多元回归法的调节效应分析存在的不足,包括人为变换检验模型、自变量和调节变量区分不足、误差方差齐性的假设难以满足、调节效应量指标ΔR²没有直接测量调节变量对自变量与因变量关系的调节程度。比较好的方法是用两水平回归模型进行调节效应分析并使用相应的效应量指标。在介绍新方法和新效应量后,总结出一套调节效应的分析流程,通过一个例子来演示如何用Mplus软件进行两水平回归模型的调节效应及其效应量分析。最后讨论了两水平回归模型的调节效应分析的发展,包括稳健的调节效应分析、潜变量的调节效应分析、有调节的中介效应分析和有中介的调节效应分析等。     

Accuracy in parameter estimation for ANCOVA and ANOVA contrasts: sample size planning via narrow confidence intervals

Contrasts of means are often of interest because they describe the effect size among multiple treatments. High-quality inference of population effect sizes can be achieved through narrow confidence intervals (CIs). Given the close relation between CI width and sample size, we propose two methods to plan the sample size for an ANCOVA or ANOVA study, so that a sufficiently narrow CI for the population (standardized or unstandardized) contrast of interest will be obtained. The standard method plans the sample size so that the expected CI width is sufficiently small. Since CI width is a random variable, the expected width being sufficiently small does not guarantee that the width obtained in a particular study will be sufficiently small. An extended procedure ensures with some specified, high degree of assurance (e.g., 90% of the time) that the CI observed in a particular study will be sufficiently narrow. We also discuss the rationale and usefulness of two different ways to standardize an ANCOVA contrast, and compare three types of standardized contrast in the ANCOVA/ANOVA context. All of the methods we propose have been implemented in the freely available MBESS package in R so that they can be easily applied by researchers.     

Heterogeneity Coefficients for Mahalanobis' D as a Multivariate Effect Size

The Mahalanobis distance D is the multivariate generalization of Cohen's d and can be used as a standardized effect size for multivariate differences between groups. An important issue in the interpretation of D is heterogeneity, that is, the extent to which contributions to the overall effect size are concentrated in a small subset of variables rather than evenly distributed across the whole set. Here I present two heterogeneity coefficients for D based on the Gini coefficient, a well-known index of inequality among values of a distribution. I discuss the properties and limitations of the two coefficients and illustrate their use by reanalyzing some published findings from studies of gender differences.     

Biases and Standard Errors of Standardized Regression Coefficients

The paper obtains consistent standard errors (SE) and biases of order O(1/n) for the sample standardized regression coefficients with both random and given predictors. Analytical results indicate that the formulas for SEs given in popular text books are consistent only when the population value of the regression coefficient is zero. The sample standardized regression coefficients are also biased in general, although it should not be a concern in practice when the sample size is not too small. Monte Carlo results imply that, for both standardized and unstandardized sample regression coefficients, SE estimates based on asymptotics tend to under-predict the empirical ones at smaller sample sizes.     

Figural after-effects, retinal size, and apparent size

The results of earlier experiments on the question of whether figural after-effects are affected by apparent as opposed to retinal size are shown to be inconclusive. A new hypothesis is proposed namely that both factors may be responsible for producing figural aftereffects, and four experiments have been made to test it. Situations were used in which the apparent sizes of the figures were determined by the size-constancy effect. It was found that where retinal sizes of test and inspection figures are the same and apparent sizes are different, figural after-effects in the direction which would be predicted on the basis of apparent sizes are obtained. It was further shown that where retinal and apparent sizes are in conflict, whether a figural after-effect is seen or not, and the direction of the figural after-effect, depends upon the balance between these two factors.     

Intelligence, birth order, and family size

The analysis of the National Child Development Study in the United Kingdom (n = 17,419) replicates some earlier findings and shows that genuine within-family data are not necessary to make the apparent birth-order effect on intelligence disappear. Birth order is not associated with intelligence in between-family data once the number of siblings is statistically controlled. The analyses support the admixture hypothesis, which avers that the apparent birth-order effect on intelligence is an artifact of family size, and cast doubt on the confluence and resource dilution models, both of which claim that birth order has a causal influence on children's cognitive development. The analyses suggest that birth order has no genuine causal effect on general intelligence.     

Correlations between speakers' body size and acoustic parameters of voice

Generally, there is a significant relationship between some acoustic measures (F0 and formant parameters) and the body size of speakers; however, data become less clear when age and sex variables are controlled. To date, no other vocal parameter apart from F0 has been studied in relation to body size. In the present study, correlations between a set of 27 parameters of the Multi-dimensional Voice Program (Kay Elemetrics Corp.) and 4 body measures were obtained from 134 speakers of both sexes belonging to one age group (20-29 years). Correlations within sex groups were null or very weak, and all significant coefficients were below .35.     

A probability-based measure of effect size: robustness to base rates and other factors

Calculating and reporting appropriate measures of effect size are becoming standard practice in psychological research. One of the most common scenarios encountered involves the comparison of 2 groups, which includes research designs that are experimental (e.g., random assignment to treatment vs. placebo conditions) and nonexperimental (e.g., testing for gender differences). Familiar measures such as the standardized mean difference (d) or the point-biserial correlation (rpb) characterize the magnitude of the difference between groups, but these effect size measures are sensitive to a number of additional influences. For example, R. E. McGrath and G. J. Meyer (2006) showed that rpb is sensitive to sample base rates, and extending their analysis to situations of unequal variances reveals that d is, too. The probability-based measure A, the nonparametric generalization of what K. O. McGraw and S. P. Wong (1992) called the common language effect size statistic, is insensitive to base rates and more robust to several other factors (e.g., extreme scores, nonlinear transformations). In addition to its excellent generalizability across contexts, A is easy to understand and can be obtained from standard computer output or through simple hand calculations.     

Metric properties of spatial images generated from locomotion: The effect of absolute size on mental scanning

The mental scanning paradigm has demonstrated that the time taken to scan the mental visual image of a map is linearly related to the scanned distances. The experiments reported here aimed at exploring the influence of two factors on mental scanning: (1) the modality through which the environment is explored, that is visual locomotion; and (2) the size of the environment. Three groups of participants had to study three paths of different sizes: large, intermediate and small. After walking on the paths, participants had to mentally scan them. In Experiment 1 an effect of the absolute size was found, which was due to the participants who took longer and relied on a kinaesthetic strategy to study the path. Experiment 2 was a repetition of the first but the duration of the learning time was controlled. A significant difference between size conditions due to longer scanning times for the larger configuration was found. Experiment 3 compared two learning strategies: visual and kinaesthetic. Also in this case a significant size effect due to longer latencies for scanning over the larger configuration was found. In all experiments a significant effect of distance and the typical time–distance coefficients of mental scanning were also found. Results suggest that the learning time duration and the spatial kinaesthetic strategy are both important for encoding the actual size of the environment.