A measure of effect size for r x c contingency tables

Goodman and Kruskal's tau measure of categorical association is advanced as a replacement for conventional measures of effect size for r x c contingency tables. Goodman and Kruskal's tau is an asymmetric measure of categorical association which is based entirely on the observed data and possesses a clear interpretation in terms of proportional reduction in error. Comparisons with conventional measures of effect size based on chi-squared such as Pearson's phi2, Tschuprow's T2, and Cramer's V2 demonstrate the advantages of employing tau as a measure of effect size.     

An effect size measure and Bayesian analysis of single-case designs

This article describes a linear modeling approach for the analysis of single-case designs (SCDs). Effect size measures in SCDs have been defined and studied for the situation where there is a level change without a time trend. However, when there are level and trend changes, effect size measures are either defined in terms of changes in R² or defined separately for changes in slopes and intercept coefficients. We propose an alternate effect size measure that takes into account changes in slopes and intercepts in the presence of serial dependence and provides an integrated procedure for the analysis of SCDs through estimation and inference based directly on the effect size measure. A Bayesian procedure is described to analyze the data and draw inferences in SCDs. A multilevel model that is appropriate when several subjects are available is integrated into the Bayesian procedure to provide a standardized effect size measure comparable to effect size measures in a between-subjects design. The applicability of the Bayesian approach for the analysis of SCDs is demonstrated through an example.     

Measures of effect size

Two different approaches have been used to derive measures of effect size. One approach is based on the comparison of treatment means. The standardized mean difference is an appropriate measure of effect size when one is merely comparing two treatments, but there is no satisfactory analogue for comparing more than two treatments. The second approach is based on the proportion of variance in the dependent variable that is explained by the independent variable. Estimates have been proposed for both fixed-factor and random-factor designs, but their sampling properties are not well understood. Nevertheless, measures of effect size can allow quantitative comparisons to be made across different studies, and they can be a useful adjunct to more traditional outcome measures such as test statistics and significance levels.     

Quantifying construct validity: two simple measures

Construct validity is one of the most central concepts in psychology. Researchers generally establish the construct validity of a measure by correlating it with a number of other measures and arguing from the pattern of correlations that the measure is associated with these variables in theoretically predictable ways. This article presents 2 simple metrics for quantifying construct validity that provide effect size estimates indicating the extent to which the observed pattem of correlations in a convergent-discriminant validity matrix matches the theoretically predicted pattern of correlations. Both measures, based on contrast analysis, provide simple estimates of validity that can be compared across studies, constructs, and measures meta-analytically, and can be implemented without the use of complex statistical procedures that may limit their accessibility.     

A multivariate projection‐type analogue of the Wilcoxon — Mann — Whitney test

The problem of comparing two independent groups based on mulitivariate data is considered. Many such methods have been proposed, but it is difficult to gain a perspective on the extent to which the groups differ. The basic strategy here is to determine a robust measure of location for each group, project the data onto the line connecting these measures of location, and then compare the groups based on the ordering of the projected points. In the univariate case the method uses the same measure of effect size employed by the Wilcoxon — Mann — Whitney test. Under general conditions, the projected points are dependent, causing difficulties when testing hypotheses. Two methods are found to be effective when trying to avoid Type I error probabilities above the nominal level. The relative merits of the two methods are discussed. The projected data provide not only a useful (numerical) measure of effect size, but also a graphical indication of the extent to which groups differ.     

A measure of effect size for experimental designs with heterogeneous variances

A recent trend in the psychological literature has been to include measures of effect size when reporting probability values. The several measures of effect size associated with the Student t test for two independent samples are appropriate only when the variances are homogeneous. In this paper, commonly used measures of effect size are considered and compared, using four data sets. A chance-corrected measure of effect size is provided for two or more treatment groups characterized by either homogeneous or heterogeneous variances.     

Setting things straight: A comparison of measures of saccade trajectory deviation

In eye movements, saccade trajectory deviation has often been used as a physiological operationalization of visual attention, distraction, or the visual system’s prioritization of different sources of information. However, there are many ways to measure saccade trajectories and to quantify their deviation. This may lead to noncomparable results and poses the problem of choosing a method that will maximize statistical power. Using data from existing studies and from our own experiments, we used principal components analysis to carry out a systematic quantification of the relationships among eight different measures of saccade trajectory deviation and their power to detect the effects of experimental manipulations, as measured by standardized effect size. We concluded that (1) the saccade deviation measure is a good default measure of saccade trajectory deviation, because it is somewhat correlated with all other measures and shows relatively high effect sizes for two well-known experimental effects; (2) more generally, measures made relative to the position of the saccade target are more powerful; and (3) measures of deviation based on the early part of the saccade are made more stable when they are based on data from an eyetracker with a high sampling rate. Our recommendations may be of use to future eye movement researchers seeking to optimize the designs of their studies.     

A semantic memory sentence verification model based on relative judgment theory

A subjective referent model of sentence verification in semantic memory tasks based on the relative judgment theory of Link and Heath (1975), together with the derivation of a discriminability index, are presented in this paper. An attractive feature of the model is its consideration of both error rates and response times (RTs) in the calculation of the discriminability index. The model is also able to account for the frequent finding in semantic memory tasks that error RTs are longer than correct RTs. A partial replication of Experiment 2 of McCloskey and Glucksberg's (1979) sentence verification context effect studies, in which we employed 44 subjects and 28 categories, and controlled for item familiarity, revealed that error RTs were consistently longer than correct RTs--a finding inconsistent with the McCloskey and Glucksberg property comparison model, but in accord with the subjective referent model. An important fortuitous result was the detection of a context effect by the discriminability measure, an effect not detected by the RT data alone. The discriminability measures yielded a near perfect correlation with estimates of the mean step size of the random walk obtained by application of the parameter estimation program FITTRW (Heath, 1983).     

Holistic processing in the composite task depends on face size

Holistic processing is a hallmark of face processing. There is evidence that holistic processing is strongest for faces at identification distance 2–10 metres from the observer. However, this evidence is based on tasks that have been little used in the literature and that are indirect measures of holistic processing. We use the composite task—a well validated and frequently used paradigm—to measure the effect of viewing distance on holistic processing. In line with previous work, we find a congruency x alignment effect that is strongest for faces that are close (2 m equivalent distance) than for faces that are further away (24 m equivalent distance). In contrast, the alignment effect for same trials, used by several authors to measure holistic processing, produced results that are difficult to interpret. We conclude that our results converge with previous findings providing more direct evidence for an effect of size on holistic processing.     

Development of a Distance-Based Effect Size Metric for Single-Case Research: Ratio of Distances

This article describes the development of an effect size measure called Ratio of Distances (RD). The goal was to develop a measure of level change for single case experimental research that met several practical requirements: (a) the measure is adaptable to designs with varying numbers of observations per, and across, phases; (b) the measure is adaptable to situations in which slope does and does not exist; (c) the measure has no ceiling, as is the limitation with commonly used overlap-based measures of effect size; and (d) the measure is computationally transparent and easily computed using widely available analysis tools (e.g., Microsoft Excel). The measure is applicable to single cases and meta-analyses.     

从效应量应有的性质看中介效应量的合理性

效应量的作用有两个方面, 一是弥补了统计检验的不足, 二是使得效应有可比性。结合统计显著性和效应量, 才能得出适当的统计结论。效应量应当具有一些基本性质, 包括与测量单位无关、单调性、不受样本容量的影响。国际上流行的中介效应量κ平方就是因为缺乏单调性而引发质疑和研究, 从而被彻底终结了其作为中介效应量的合法性。R平方型中介效应量同样有缺乏单调性的问题。文末讨论了如何报告中介效应量以及有待研究的问题。     

有中介的调节模型的拓展及其效应量

传统的有中介的调节(mediated moderation, meMO)模型关于误差方差齐性的假设经常被违背, 应用研究中也缺乏测量meMO效应大小的指标。对于单层数据, 本文借助于两层建模的思想, 提出了一种可用于处理方差非齐性的两层有中介的调节(2meMO)模型; 给出了用于测量meMO分析中总调节效应、直接调节效应和有中介调节效应大小的效应量。通过Monte Carlo模拟研究, 比较了meMO和2meMO模型在参数和效应量估计上的表现。并通过实际案例解释了2meMO模型的应用以及效应量的计算和解释。     

A mere exposure effect for transformed three-dimensional objects: Effects of reflection, size, or color changes on affect and recognition

If the mere exposure effect is based on implicit memory, recognition and affect judgments should be dissociated by experimental variables in the same manner as other explicit and implicit measures. Consistent with results from recognition and picture naming or object decision priming tasks (e.g., Biederman & E. E. Cooper, 1991, 1992; L. A. Cooper, Schacter, Ballesteros, & Moore, 1992), the present research showed that recognition memory but not affective preference was impaired by reflection or size transformations of three-dimensional objects between study and test. Stimulus color transformations had no effect on either measure. These results indicate that representations that support recognition memory code spatial information about an object’s left-right orientation and size, whereas representations that underlie affective preference do not. Insensitivity to surface feature changes that do not alter object form appears to be a general characteristic of implicit memory measures, including the affective preference task.     

Sample size and bentler and Bonett's nonnormed fit index

Bentler and Bonett's nonnormed fit index is a widely used measure of goodness of fit for the analysis of covariance structures. This note shows that contrary to what has been claimed the nonnormed fit index is dependent on sample size. Specifically for a constant value of a fitting function, the nonnormed index is inversely related to sample size. A simple alternative fit measure is proposed that removes this dependency. In addition, it is shown that this new measure as well as the old nonnormed fit index can be applied to any fitting function that measures the deviation of the observed covariance matrix from the covariance matrix implied by the parameter estimates for a model.     

Correlation and explaining variance: To square or not to square?

Despite previous articles dating back 80 years, the questions of whether and when to square correlations continue to puzzle and confuse researchers. In this editorial, I point out that correlations can serve two independent purposes: they can be measures of effect size in themselves and their function as regression coefficients can be used to estimate proportion of variance in one measure for which another measure accounts. Using examples relevant to intelligence researchers, I show that the answer to the question of whether or not to square a correlation is ‘it depends.’ It depends on the purpose and it depends on the underlying theoretical model of the causal association between the variables.     

Effect size, power, and sample size determination for structured means modeling and mimic approaches to between-groups hypothesis testing of means on a single latent construct

While effect size estimates, post hoc power estimates, and a priori sample size determination are becoming a routine part of univariate analyses involving measured variables (e.g., ANOVA), such measures and methods have not been articulated for analyses involving latent means. The current article presents standardized effect size measures for latent mean differences inferred from both structured means modeling and MIMIC approaches to hypothesis testing about differences among means on a single latent construct. These measures are then related to post hoc power analysis, a priori sample size determination, and a relevant measure of construct reliability.I wish to convey my appreciation to the reviewers and Associate Editor, whose suggestions extended and strengthened the article's content immensely, and to Ralph Mueller of The George Washington University for enhancing the clarity of its presentation.     

The psychometric properties of several measures of body image

Despite the recent upsurge of interest in the construct of body image, there is relatively little information on the psychometric properties of the instruments used to measure it. This study investigated the reliability and validity of several measures of body image and compared bulimics and normals on these measures. One hundred ten normal weight females, half of whom were diagnosed as bulimic, were administered two measures of affect toward one's body, two measures of perceptions of one's entire body, and three measures of perceptions of the size of specific body sites (face, shoulders, waist, and hips). In themain, the measures provided reliable indices of body image. Examination of the correlation matrix for the measures disclosed convergence for the affective measures of body image and for all but one of the perceptual measures of body image. There was also significant covariation between the affective and the perceptual measures. The multitrait-multimethod technique was used to investigate the construct validity of the measures concerned with perceptions of the size of body sites. The multitrait-multimethod matrices disclosed substantial convergence between perceptions of face, shoulder, waist, and hip size across the three measures. However, the measure which used kinesthetic estimates of body-site size produced low reliabilities and all three of the measures showed substantial method variance. Bulimics and normals differed significantly on both the affective and the perceptual components of body image.This study is based on the first author's masters thesis. Portions of this study were represented at the 1986 meeting of the Southeastern Psychological Association. A grant to the third author from the Anclote Psychiatric Center provided support for this research project.     

Measuring recognition memory.

Recent years have seen an expanded interest in recognition memory tasks. This resurgence of interest has also renewed concerns with measurement problems. Comparing 4 models of recognition memory, Snodgrass and Corwin (1988) found that measures of bias from the distribution-free (nonparametric) model were inadequate. However, their analysis was based on bias measures that can be shown a priori to be nonindependent of discrimination. This article traces the history of the nonparametric model and develops a better measure of bias. The consequence of developing this better measure is that the nonparametric model deserves serious consideration.     

Mental chronometry and individual differences: Modeling reliabilities and correlations of reaction time means and effect sizes

We used a general stage-based model of reaction time (RT) to investigate the psychometric properties of mean RTs and experimental effect sizes (i.e., differences in mean RTs). Using the model, formulas were derived for the reliabilities of mean RTs and RT difference scores, and these formulas provide guidance about the number of trials per participant needed to obtain reliable estimates of these measures. In addition, formulas were derived for various different types of correlations computed in RT research (e.g., correlations between a mean RT and an external non-RT measure, between two mean RTs, between a mean RT and an RT effect size). The analysis revealed that observed RT-based correlations depend on many parameters of the underlying processes contributing to RT. We conclude that these correlations often fail to support the inferences drawn from them and that their proper interpretation is far more complex than is generally acknowledged.     

Internal Consistency and Power When Comparing Total Scores from Two Groups

Researchers now know that when theoretical reliability increases, power can increase, decrease, or stay the same. However, no analytic research has examined the relationship of power to the most commonly used type of reliability—internal consistency—and the most commonly used measures of internal consistency, coefficient alpha and ICC(A,k). We examine the relationship between the power of independent samples t tests and internal consistency. We explicate the mathematical model upon which researchers usually calculate internal consistency, one in which total scores are calculated as the sum of observed scores on K measures. Using this model, we derive a new formula for effect size to show that power and internal consistency are influenced by many of the same parameters but not always in the same direction. Changing an experiment in one way (e.g., lengthening the measure) is likely to influence multiple parameters simultaneously; thus, there are no simple relationships between such changes and internal consistency or power. If researchers revise measures to increase internal consistency, this might not increase power. To increase power, researchers should increase sample size, select measures that assess areas where group differences are largest, and use more powerful statistical procedures (e.g., ANCOVA).