On robust estimation of effect size under semiparametric models

Estimation of effect size is of interest in many applied fields such as Psychology, Sociology and Education. However there are few nonparametric estimators of effect size proposed in the existing literature, and little is known about the distributional characteristics of these estimators. In this article, two estimators based on the sample quantiles are proposed and studied. The first one is the estimator suggested by Hedges and Olkin (see page 93 of Hedges & Olkin, 1985) for the situation where a treatment effect is evaluated against a control group (Case A). A modified version of the robust estimator by Hedges and Olkin is also proposed for the situation where two parallel treatments are compared (Case B). Large sample distributions of both estimators are derived. Their asymptotic relative efficiencies with respect to the normal maximum likelihood estimators under several common distributions are evaluated. The robust properties of the proposed estimators are discussed with respect to the sample-wise breakdown points proposed by Akritas (1991). Simulation studies are provided in which the performing characteristics of the proposed estimator are compared to that of the nonparametric estimators by Kraemer and Andrews (1982). Interval estimation of the effect sizes is also discussed. In an example, interval estimates for the data set in Kraemer and Andrews (1982) are calculated for both cases A and B.     

Measures of effect size

Two different approaches have been used to derive measures of effect size. One approach is based on the comparison of treatment means. The standardized mean difference is an appropriate measure of effect size when one is merely comparing two treatments, but there is no satisfactory analogue for comparing more than two treatments. The second approach is based on the proportion of variance in the dependent variable that is explained by the independent variable. Estimates have been proposed for both fixed-factor and random-factor designs, but their sampling properties are not well understood. Nevertheless, measures of effect size can allow quantitative comparisons to be made across different studies, and they can be a useful adjunct to more traditional outcome measures such as test statistics and significance levels.     

从效应量应有的性质看中介效应量的合理性

效应量的作用有两个方面, 一是弥补了统计检验的不足, 二是使得效应有可比性。结合统计显著性和效应量, 才能得出适当的统计结论。效应量应当具有一些基本性质, 包括与测量单位无关、单调性、不受样本容量的影响。国际上流行的中介效应量κ平方就是因为缺乏单调性而引发质疑和研究, 从而被彻底终结了其作为中介效应量的合法性。R平方型中介效应量同样有缺乏单调性的问题。文末讨论了如何报告中介效应量以及有待研究的问题。     

Common language effect size for correlations

The Pearson correlation coefficient can be translated to a common language effect size, which shows the probability of obtaining a certain value on one variable, given the value on the other variable. This common language effect size makes the size of a correlation coefficient understandable to laypeople. Three examples are provided to demonstrate the application of the common language effect size in interpreting Pearson correlation coefficients and multiple correlation coefficients.     

Group size and the framing effect:

Past research provides conflicting evidence for the role of value in the appearance of framing effects. In this study, the effects of frame and group size were examined using scenarios about less valuable and more valuable groups (animal vs. human). In addition, two picture manipulations, intended to increase the value of the group, were presented. Choice patterns differed for the human and animal groups, with participants exhibiting greater risk seeking overall for the human scenario and showing a framing effect for humans but not animals when no pictures were presented. A small group size increased the proportion of risky choices for both the animal and human scenarios. Presenting pictures with names did lead to framing effects for animals, but providing pictures or pictures and names eliminated framing effects for the human scenario. These findings suggest that the relationship between value and framing effects is a matter of degree.     

Dissociation of the distance effect and size effect in one-digit numbers

Magnitude comparison of single digits is robustly characterized by a distance effect (close numbers are more difficult to compare than numbers further apart) and a size effect (for a given distance, comparison difficulty increases with increasing size). The distance effect indicates access to the mental number line (Dehaene, 1997), and the size effect is usually interpreted as indicating that the mental number line represents larger numbers more vaguely than smaller ones. In contrast, we have argued earlier (Verguts, Fias, & Stevens, 2005) that for symbolic numbers (Arabic or verbal notation), the size effect does not originate from the mental number line but, instead, originates from mappings to relevant output components that are specific for magnitude comparison. If the latter is true, it should be possible to dissociate the distance effect from the size effect in tasks other than magnitude comparison. In two experiments, we observed a robust distance effect insame/different judgments, which implies access to the mental number line. Yet the size effect was absent. Consistent with our prediction, this finding establishes a dissociation between the size effect and the distance effect.     

r equivalent: A simple effect size indicator

The purpose of this article is to propose a simple effect size estimate (obtained from the sample size, N, and a p value) that can be used (a) in meta-analytic research where only sample sizes and p values have been reported by the original investigator, (b) where no generally accepted effect size estimate exists, or (c) where directly computed effect size estimates are likely to be misleading. This effect size estimate is called r(equivalent) because it equals the sample point-biserial correlation between the treatment indicator and an exactly normally distributed outcome in a two-treatment experiment with N/2 units in each group and the obtained p value. As part of placing r(equivalent) into a broader context, the authors also address limitations of r(equivalent).     

Multiple trials may yield exaggerated effect size estimates

Published psychological research attempting to support the existence of small and medium effect sizes may not have enough participants to do so accurately, and thus, repeated trials or the use of multiple items may be used in an attempt to obtain significance. Through a series of Monte-Carlo simulations, this article describes the results of multiple trials or items on effect size estimates when the averages and aggregates of a dependent measure are analyzed. The simulations revealed a large increase in observed effect size estimates when the numbers of trials or items in an experiment were increased. Overestimation effects are mitigated by correlations between trials or items, but remain substantial in some cases. Some concepts, such as a P300 wave or a test score, are best defined as a composite of measures. Troubles may arise in more exploratory research where the interrelations among trials or items may not be well described.     

Useful effect size interpretations for single case research

An obstacle to broader acceptability of effect sizes in single case research is their lack of intuitive and useful interpretations. Interpreting Cohen's d as "standard deviation units difference" and R2 as "percent of variance accounted for" do not resound with most visual analysts. In fact, the only comparative analysis widely supported in single case research (SCR) is "percent of nonoverlapping data." This article explores five alternative interpretations of Cohen's d and R2 effect sizes that may be more acceptable to the SCR field. They are: (a) Cohen's (Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum) "Percent of Nonoverlapping Data" (CPND), (b) Parker et al.'s (Parker, R.I., Cryer, J., Byrns, G., 2006. Controlling trend in single case research. School Psychology Quarterly, 21, 418-440, Parker, R.I., Hagan-Burke, S., Vannest, K., in press. Percent of all non-overlapping data (PAND): An alternative to PND. Journal of Special Education) "Percent of All Nonoverlapping Data" (PAND), (c) Rosenthal et al. (Rosenthal, R., Rosnow, R., & Rubin, D. (2000). Contrasts and effect sizes in behavioral research: A correlational approach. Cambridge: Cambridge University Press.) "Binomial Effect Size Display" (BESD), (d) "Percentile Rank in Control Group" (PR), and (e) McGraw and Wong's (McGraw, K. O., & Wong, S. P. (1992). A common language effect-size statistic. Psychological Bulletin, 111, 361-365) "Common Language Effect Size" (CLES). Each of the five interpretation schemes are applied to a published data set and are evaluated according to (a) intuitive appeal, (b) relevance to visual analysis, (c) ease of calculation, and (d) technical adequacy. Three of the five appear to be improvements over prevailing practice.     

Recommended effect size statistics for repeated measures designs

Investigators, who are increasingly implored to present and discuss effect size statistics, might comply more often if they understood more clearly what is required. When investigators wish to report effect sizes derived from analyses of variance that include repeated measures, past advice has been problematic. Only recently has a generally useful effect size statistic been proposed for such designs: generalized eta squared (η_G²; Olejnik & Algina, 2003). Here, we present this method, explain that η_G² is preferred to eta squared and partial eta squared because it provides comparability across between-subjects and within-subjects designs, show that it can easily be computed from information provided by standard statistical packages, and recommend that investigators provide it routinely in their research reports when appropriate.     

Required sample size to detect the mediated effect

Mediation models are widely used, and there are many tests of the mediated effect. One of the most common questions that researchers have when planning mediation studies is, "How many subjects do I need to achieve adequate power when testing for mediation?" This article presents the necessary sample sizes for six of the most common and the most recommended tests of mediation for various combinations of parameters, to provide a guide for researchers when designing studies or applying for grants.     

Attenuation of size illusion effect in dual-task conditions

We over-estimate or under-estimate the size of an object depending its background structure (e.g., the Ebbinghaus illusion). Since deciding and preparing to execute a movement is based on perception, motor performance deteriorates due to the faulty perception of information. Therefore, such cognitive process can be a source of a failure in motor performance, although we feel in control of our performance through conscious cognitive activities. If a movement execution process can avoid distraction by the illusion-deceived conscious process, the effect of the visual illusion on visuomotor performance can be eliminated or attenuated. This study investigated this hypothesis by examining two task performances developed for a target figure inducing the Ebbinghaus size illusion: showing visually perceived size of an object by index finger-thumb aperture (size-matching), and reaching out for the object and pretending to grasp it (pantomimed grasping). In these task performances, the size of the index finger-thumb aperture becomes larger or smaller than the actual size, in accordance with the illusion effect. This study examined whether the size illusion effect can be weakened or eliminated by the dual-task condition where actors’ attention to judge the object’s size and to produce the aperture size is interrupted. 16 participants performed the size-matching and pantomimed grasping tasks while simultaneously executing a choice reaction task (dual task) or without doing so (single task). Using an optical motion capture system, the size-illusion effect was analyzed in terms of the aperture size, which indicates the visually perceived object size. The illusion effect was attenuated in the dual task condition, compared to it in the single task condition. This suggests that the dual task condition modulated attention focus on the aperture movement and therefore the aperture movement was achieved with less distraction caused by illusory information.     

An explanation of the indentation size effect in ceramics

Abstract

A quantitative model is proposed to explain the indentation size effect (ISE) often observed in the hardness response of hard brittle materials, namely that hardness is observed to increase with decreasing indentation size. The model is based on a mixed elastic/plastic materials deformation response whereby plastic deformation occurs in a discrete manner progressively to relieve stresses created by elastic flexure of the surface at the edges of the indentation. During unloading of the indenter, recovery of the elastic increment of deformation, which precedes each new band of plastic deformation, results in the indentation appearing smaller than expected, particularly as the indentation sizes decrease to approach the scale of the plastic deformation band spacing. The model fits observed experimental data well and analysis of hardness/size data in this way is shown to allow both a bulk hardness value and a characteristic deformation band scale to be calculated for a given sample.     

Accuracy of effect size estimates from published psychological research

A Monte-Carlo simulation was used to model the biasing of effect sizes in published studies. The findings from the simulation indicate that, when a predominant bias to publish studies with statistically significant results is coupled with inadequate statistical power, there will be an overestimation of effect sizes. The consequences such an effect size overestimation will then have on meta-analyses and power analyses are highlighted and discussed along with measures which can be taken to reduce the problem.     

编码强度对字体大小效应的影响

本研究采用3个实验考察编码强度对字体大小效应的影响,探讨由于知觉特征而引发的元认知错觉的内在产生机制(实验1)与有效的矫正措施(实验2和实验3)。结果发现:(1)大字体词语的知觉流畅性显著优于小字体,并且贝叶斯多层中介分析结果表明,知觉流畅性对字体大小效应起部分中介作用(实验1);(2)随着编码强度的增加,由字体大小引起的学习判断错觉逐渐消失(实验2和实验3)。以上结果表明,刺激的知觉特征(字体大小)对个体学习判断的影响,随编码强度激活线索的增加而逐渐减弱。这一结果为真实教学情境中提高学习者的编码强度,进而削弱学习判断对知觉特征线索的依赖,并准确地监测自身的学习进程提供了科学依据。