M > 1

Increasingly, communication experiments are incorporating replication/actors for the purpose of controlling confounds and increasing generalizability. If replications are considered to be samples of possible treatment implementations, treating the replication factor as random is more appropriate than treating it as fixed. Study 1 shows that treating sampled replications as a fixed effect leads to potentially serious alpha inflation in the test of the treatment effect while treating sampled replications as random controls alpha at its nominal level. Study 2 addresses a common objection to treating replications as random: the argument that to do so will lead to unacceptably low power in statistical testing. Although experiments with very few replications are likely to be deficient in power, the results of Study 2 establish that power can be improved to an unexpected degree by a relatively modest increase in the number of replications.     

Statistical Power Analysis in Psychological Research

Reviews of the psychological literature suggest that many studies lack sufficient statistical power to detect effects of interest. Increased attention to statistical power by journal editors, reviewers, and funding agencies has led to a need for researchers to consider power carefully when designing studies. Our goal is to present an overview of issues that influence statistical power in the context of traditional research designs and analytic methods. We then extend the discussion of statistical power to complex designs and analyses providing readers with sources useful for evaluating power in the design stage of conducting research. Finally, we advocate the use of simulation and Monte Carlo methods as a flexible general strategy for designing research studies with adequate statistical power.     

The power of statistical tests for moderators in meta-analysis

Calculation of the statistical power of statistical tests is important in planning and interpreting the results of research studies, including meta-analyses. It is particularly important in moderator analyses in meta-analysis, which are often used as sensitivity analyses to rule out moderator effects but also may have low statistical power. This article describes how to compute statistical power of both fixed- and mixed-effects moderator tests in meta-analysis that are analogous to the analysis of variance and multiple regression analysis for effect sizes. It also shows how to compute power of tests for goodness of fit associated with these models. Examples from a published meta-analysis demonstrate that power of moderator tests and goodness-of-fit tests is not always high.     

The High Cost of Complexity in Experimental Design and Data Analysis: Type I and Type II Error Rates in Multiway ANOVA

The availability of statistical software packages has led to a sharp increase in use of complex research designs and complex statistical analyses in communication research. An informal examination of studies from 2 leading communication journals suggests that the analysis of variance (ANOVA) is often the statistic of choice, and a substantial proportion of published research reports using ANOVA employ complex (k ≥ 3) factorial designs, often involving multiple dependent variables. This article reports a series of Monte Carlo simulations which demonstrate that this complexity may come at a heavier cost than many communication researchers realize. As frequently used, complex factorial ANOVA yield Type I and Type II error rates that many communication scholars would likely consider unacceptable. Consequently, quality of statistical inference in many studies is highly suspect. Communication researchers are warned about problems associated with design and statistical complexity and solutions are suggested.     

A REASSESSMENT OF STATISTICAL POWER ANALYSIS IN HUMAN COMMUNICATION RESEARCH

This study conducted a statistical power analysis of 64 articles appearing in the first four volumes of Human Communication Research, 1974–1978. Each article was examined, using Cohen's revised handbook, assuming nondirectional null hypotheses and an alpha level of .05. Statistical power, the probability of rejecting a false null hypothesis, was calculated for small, medium, and large experimental effect sizes and averaged by article and volume. Results indicated that the average probability of beta errors appears to have decreased over time, providing a greater chance of rejecting false null hypotheses, but this also raised several power-related issues relevant to communication research in general.     

Hierarchically distributed processing for psychology

The advantages of loosely coupled hierarchical computer networks for psychological research are discussed. Hardware methods for interconnecting computers are considered, and software communication protocols are detailed. A hierarchy of interconnected computers is formed: (1) large campus computer for statistical analyses, (2) medium scale computer for data concentration and program development, and (3) laboratory computers for data collection. This hierarchy provides efficient computing at all levels and insures maximum computing power at minimum cost.     

Relationship Education for Military Couples: A Pilot Randomized Controlled Trial of the Effects of Couple CARE in Uniform

Military life can place excess strain on couple relationships. The Couple CARE relationship education program was tailored to address the challenges of military couples. Thirty-two Australian military couples participated in a pilot feasibility study assessing the Couple CARE in Uniform adaptation against an active control. Relationship satisfaction and communication improved in both conditions, with no significant difference between the conditions. Couples' high relationship satisfaction on presentation, paired with modest statistical power, might have contributed to the null results. However, Couple CARE in Uniform had significantly higher consumer satisfaction than the comparison condition, suggesting it is a program worthy of further evaluation.     

Précis of statistical significance: rationale, validity, and utility

The null-hypothesis significance-test procedure (NHSTP) is defended in the context of the theory-corroboration experiment, as well as the following contrasts: (a) substantive hypotheses versus statistical hypotheses, (b) theory corroboration versus statistical hypothesis testing, (c) theoretical inference versus statistical decision, (d) experiments versus nonexperimental studies, and (e) theory corroboration versus treatment assessment. The null hypothesis can be true because it is the hypothesis that errors are randomly distributed in data. Moreover, the null hypothesis is never used as a categorical proposition. Statistical significance means only that chance influences can be excluded as an explanation of data; it does not identify the nonchance factor responsible. The experimental conclusion is drawn with the inductive principle underlying the experimental design. A chain of deductive arguments gives rise to the theoretical conclusion via the experimental conclusion. The anomalous relationship between statistical significance and the effect size often used to criticize NHSTP is more apparent than real. The absolute size of the effect is not an index of evidential support for the substantive hypothesis. Nor is the effect size, by itself, informative as to the practical importance of the research result. Being a conditional probability, statistical power cannot be the a priori probability of statistical significance. The validity of statistical power is debatable because statistical significance is determined with a single sampling distribution of the test statistic based on H0, whereas it takes two distributions to represent statistical power or effect size. Sample size should not be determined in the mechanical manner envisaged in power analysis. It is inappropriate to criticize NHSTP for nonstatistical reasons. At the same time, neither effect size, nor confidence interval estimate, nor posterior probability can be used to exclude chance as an explanation of data. Neither can any of them fulfill the nonstatistical functions expected of them by critics.     

THE PERCEPTIONS AND USAGE OF STATISTICAL POWER IN APPLIED PSYCHOLOGY AND MANAGEMENT RESEARCH

We first assess the current level of statistical power across articles in seven leading journals that represent a broad sample of applied psychology and management research. We next survey the authors of these articles to examine their perceptions and usage of statistical power analysis. Finally, we examine the perceptions and usage of power analysis in a survey of authors of regression-based research appearing in leading journals. Findings from the assessment of power and surveys of researchers indicate that power analyses are not typically conducted, researchers perceive little need for statistical power, and power in published research is low. We conclude by discussing implications of low power for the field and recommending avenues for improving researchers' awareness and usage of statistical power.     

Implications of statistical power for confidence intervals

The statistical power of a hypothesis test is closely related to the precision of the accompanying confidence interval. In the case of a z-test, the width of the confidence interval is a function of statistical power for the planned study. If minimum effect size is used in power analysis, the width of the confidence interval is the minimum effect size times a multiplicative factor φ. The index φ, or the precision-to-effect ratio, is a function of the computed statistical power. In the case of a t-test, statistical power affects the probability of achieving a certain width of confidence interval, which is equivalent to the probability of obtaining a certain value of φ. To consider estimate precision in conjunction with statistical power, we can choose a sample size to obtain a desired probability of achieving a short width conditional on the rejection of the null hypothesis.     

Statistical power in network neuroscience

Network neuroscience has emerged as a leading method to study brain connectivity. The success of these investigations is dependent not only on approaches to accurately map connectivity but also on the ability to detect real effects in the data – that is, statistical power. We review the state of statistical power in the field and discuss sample size, effect size, measurement error, and network topology as key factors that influence the power of brain connectivity investigations. We use the term 'differential power' to describe how power can vary between nodes, edges, and graph metrics, leaving traces in both positive and negative connectome findings. We conclude with strategies for working with, rather than around, power in connectivity studies.     

How meta-analysis increases statistical power

One of the most frequently cited reasons for conducting a meta-analysis is the increase in statistical power that it affords a reviewer. This article demonstrates that fixed-effects meta-analysis increases statistical power by reducing the standard error of the weighted average effect size (T.) and, in so doing, shrinks the confidence interval around T.. Small confidence intervals make it more likely for reviewers to detect nonzero population effects, thereby increasing statistical power. Smaller confidence intervals also represent increased precision of the estimated population effect size. Computational examples are provided for 3 effect-size indices: d (standardized mean difference), Pearson's r, and odds ratios. Random-effects meta-analyses also may show increased statistical power and a smaller standard error of the weighted average effect size. However, the authors demonstrate that increasing the number of studies in a random-effects meta-analysis does not always increase statistical power.     

Evaluating and Reporting Statistical Power in Counseling Research

Despite recommendations from the Publication Manual of the American Psychological Association (6th ed.) to include information on statistical power when publishing quantitative results, authors seldom include analysis or discussion of statistical power. The rationale for discussing statistical power is addressed, approaches to using G*Power to report statistical power are presented, and examples for reporting statistical power are provided.     

Attributions and depression: why is the literature so inconsistent?

A large body of literature examining the relations between depression and causal attributions has produced inconsistent findings. Many studies have clearly had inadequate statistical power, however, so that negative findings cannot be readily interpreted. In this review, statistical power was computed for all published analyses relating depression to attributions to any of the following: internal, stable, or global causes, or their composite, ability/character, effort/behavior, luck, or task difficulty. On average, the power of these analyses was very poor. For example, only 8 of the 87 analyses had a probability of .80 or better of detecting a small-medium true population effect (e.g., r = .20). Separating studies by levels of power helped to clarify the inconsistencies in the literature. Whereas across all published studies depression was fairly consistently related only to the composite of internal, stable, and global attributions, those few studies with fairly high power all reported significant relations of depression to stable and global attributions as well as to the composite. It is suggested that increased attention be paid to the power of statistical analyses in planning studies and in drawing conclusions from completed studies.     

The power of statistical tests in meta-analysis

Calculations of the power of statistical tests are important in planning research studies (including meta-analyses) and in interpreting situations in which a result has not proven to be statistically significant. The authors describe procedures to compute statistical power of fixed- and random-effects tests of the mean effect size, tests for heterogeneity (or variation) of effect size parameters across studies, and tests for contrasts among effect sizes of different studies. Examples are given using 2 published meta-analyses. The examples illustrate that statistical power is not always high in meta-analysis.     

Power equivalence in structural equation modelling

Implementing large‐scale empirical studies can be very expensive. Therefore, it is useful to optimize study designs without losing statistical power. In this paper, we show how study designs can be improved without changing statistical power by defining power equivalence, a relation between structural equation models (SEMs) that holds true if two SEMs have the same power on a likelihood ratio test to detect a given effect. We show systematic operations of SEMs that maintain power, and give an algorithm that efficiently reduces SEMs to power‐equivalent models with a minimal number of observed parameters. In this way, optimal study designs can be found without reducing statistical power. Furthermore, the algorithm can be used to drastically increase the speed of power computations when using Monte Carlo simulations or approximation methods.     

A comparison of methods to test mediation and other intervening variable effects

A Monte Carlo study compared 14 methods to test the statistical significance of the intervening variable effect. An intervening variable (mediator) transmits the effect of an independent variable to a dependent variable. The commonly used R. M. Baron and D. A. Kenny (1986) approach has low statistical power. Two methods based on the distribution of the product and 2 difference-in-coefficients methods have the most accurate Type I error rates and greatest statistical power except in 1 important case in which Type I error rates are too high. The best balance of Type I error and statistical power across all cases is the test of the joint significance of the two effects comprising the intervening variable effect.     

结构方程模型统计检验力分析：原理与方法

结构方程模型是心理学、管理学、社会学等学科中重要的统计工具之一。然而, 大量使用结构方程模型的研究忽视了对该方法的统计检验力进行必要的分析和报告, 在一定程度上降低了这些研究的结果的证明效力。结构方程模型的统计检验力分析方法主要有Satorra-Saris法、MacCallum法与Monte Carlo法三类。其中Satorra-Saris法适用于备择模型清晰、检验对象相对简单、检验方法基于χ²分布的情形; MacCallum法适用于基于χ²分布的模型拟合检验且备择模型不明的情形; Monte Carlo法适用于检验对象相对复杂、采用模拟或重抽样方法进行检验的情形。在实际应用中, 研究者应当首先判断检验的目的、方法以及是否有明确的备择模型, 并根据这些信息选择具体的分析方法。     

A learning algorithm for boltzmann machines

The computational power of massively parallel networks of simple processing elements resides in the communication bandwidth provided by the hardware connections between elements. These connections can allow a significant fraction of the knowledge of the system to be applied to an instance of a problem in a very short time. One kind of computation for which massively parallel networks appear to be well suited is large constraint satisfaction searches, but to use the connections efficiently two conditions must be met: First, a search technique that is suitable for parallel networks must be found. Second, there must be some way of choosing internal representations which allow the preexisting hardware connections to be used efficiently for encoding the constraints in the domain being searched. We describe a general parallel search method, based on statistical mechanics, and we show how it leads to a general learning rule for modifying the connection strengths so as to incorporate knowledge about a task domain in an efficient way. We describe some simple examples in which the learning algorithm creates internal representations that are demonstrably the most efficient way of using the preexisting connectivity structure.