Generalized sample size determination formulas for experimental research with hierarchical data

Hierarchical data sets arise when the data for lower units (e.g., individuals such as students, clients, and citizens) are nested within higher units (e.g., groups such as classes, hospitals, and regions). In data collection for experimental research, estimating the required sample size beforehand is a fundamental question for obtaining sufficient statistical power and precision of the focused parameters. The present research extends previous research from Heo and Leon (2008) and Usami (2011b), by deriving closed-form formulas for determining the required sample size to test effects in experimental research with hierarchical data, and by focusing on both multisite-randomized trials (MRTs) and cluster-randomized trials (CRTs). These formulas consider both statistical power and the width of the confidence interval of a standardized effect size, on the basis of estimates from a random-intercept model for three-level data that considers both balanced and unbalanced designs. These formulas also address some important results, such as the lower bounds of the needed units at the highest levels.     

It's in the sample: the effects of sample size and sample diversity on the breadth of inductive generalization

Developmental studies have provided mixed evidence with regard to the question of whether children consider sample size and sample diversity in their inductive generalizations. Results from four experiments with 105 undergraduates, 105 school-age children (M = 7.2 years), and 105 preschoolers (M = 4.9 years) showed that preschoolers made a higher rate of projections from large samples than from small samples when samples were diverse (Experiments 1 and 3) but not when samples were homogeneous (Experiment 4) and not when the task required a choice between two samples (Experiment 2). Furthermore, when a property occurred in large and diverse samples, preschoolers exhibited a broad pattern of projection, generalizing the property to items from categories not represented in the evidence. In contrast, adults followed a normative pattern of induction and never attributed properties to items from categories not represented in the evidence. School-age children showed a mixed pattern of results.     

Belief bias in the perception of sample size adequacy

In two experiments participants were instructed to set aside their own, complete knowledge of a statistical population parameter and to take the perspective of an agent whose knowledge was limited to a random sample. Participants rated the appropriateness of the agent's conclusion about the adequacy of the sample size (which, objectively, was more than adequate). They also rated the agent's intelligence. Whereas previous work suggests that unbelievable statistical conclusions impact reasoning by provoking critical thought which enhances the detection of research flaws, the present studies presented participants an unflawed scenario designed to assess effects of believability on bias. The results included the finding that participants’ complete knowledge did indeed bias their perceptions not only of the adequacy of the sample size, but also of the rationality of the agent drawing the conclusion from the sample. The findings were interpreted in the context of research on belief bias, social attribution, and Theory of Mind.     

Multiple correlation: exact power and sample size calculations

This article discusses power and sample size calculations for observational studies in which the values of the independent variables cannot be fixed in advance but are themselves outcomes of the study. It reviews the mathematical framework applicable when a multivariate normal distribution can be assumed and describes a method for calculating exact power and sample sizes using a series expansion for the distribution of the multiple correlation coefficient. A table of exact sample sizes for level .05 tests is provided. Approximations to the exact power are discussed, most notably those of Cohen (1977). A rigorous justification of Cohen's approximations is given. Comparisons with exact answers show that the approximations are quite accurate in many situations of practical interest. More extensive tables and a computer program for exact calculations can be obtained from the authors.     

取样大小对因果强度推理的影响研究

摘 要：使用纸笔测验探讨表格集中呈现信息条件下取样大小对单一因果关系强度推理的影响,并比较五种模型ΔP、效力PC、SS效力、Support和χ2的预测与实验数据的相关。结果显示：（1）取样大小对不同的问题有不同的影响：高取样在ΔP=0时导致了低估计值,在0<|ΔP|=PPC时没有效果,在|ΔP|相似文献   

Subclinical delusional ideation and appreciation of sample size and heterogeneity in statistical judgment

Previous studies demonstrate that people high in delusional ideation exhibit a data‐gathering bias on inductive reasoning tasks. The current study set out to investigate the factors that may underpin such a bias by examining healthy individuals, classified as either high or low scorers on the Peters et al. Delusions Inventory (PDI). More specifically, whether high PDI scorers have a relatively poor appreciation of sample size and heterogeneity when making statistical judgments. In Expt 1, high PDI scorers made higher probability estimates when generalizing from a sample of 1 with regard to the heterogeneous human property of obesity. In Expt 2, this effect was replicated and was also observed in relation to the heterogeneous property of aggression. The findings suggest that delusion‐prone individuals are less appreciative of the importance of sample size when making statistical judgments about heterogeneous properties; this may underpin the data gathering bias observed in previous studies. There was some support for the hypothesis that threatening material would exacerbate high PDI scorers' indifference to sample size.     

汉字识别的计算机模拟

本文总结了作者近年来在连结主义的理论框架下 ,用计算机模拟汉字识别的工作。文章分三部分。第一部分说明了计算机模拟与人工智能的关系。第二部分介绍了作者提出的两个模型 :汉字识别与命名的连结主义模型和基于语义的词汇判断的计算模型。两个模型分别成功地模拟了汉字识别中的频率效应、形声字读音中的规则效应、声旁效应、语义启动效应、语境与频率的交互作用等。第三部分讨论了模拟工作的意义、分布表征、学习算法等问题。研究表明 :认知的计算机模拟能验证人类认知实验的结果 ,对结果提出合理的解释 ,并能指导进一步的实验研究。     

Effect of sample size on ductility of metallic glass

With decreasing sample dimension, the compressive plastic strain of a Zr-based metallic glass increases from near zero to as high as 80% without failure. This indicates that macroscopically brittle or ductile deformation behaviour can occur in chemically and structurally identical metallic glass. A concept of critical shear offset is proposed to explain the strong size effect on the enhanced plasticity of metallic glass by taking the shear fracture energy density into account. This finding provides new understanding on the principle that for metallic glass ‘smaller is more ductile’, even on the macroscopic scale.     

Acquisition of dispositional attributions: effects of sample size and covariation

Two experiments examined whether dispositional attributions are sensitive to the sample size of the evidence indicating a given level of covariation between person and behavior. Participants were given high or low levels of covariation (i.e. consensus and distinctiveness), and the acquisition of dispositional attributions was monitored by requesting dispositional trait ratings at fixed intervals. The results showed that dispositional attributions were sensitive to sample size, and increased given more evidence on high person‐behavior covariation while they decreased given more evidence on low person‐behavior covariation. Additional analyses suggested that in making dispositional inferences (e.g. about the actor), there was a slight preference for agreement information (e.g. low distinctiveness) over difference information (e.g. low consensus). The effects of sample size are inconsistent with current statistical or probabilistic models of covariation, but are in line with connectionist networks using an error‐correcting learning algorithm. Copyright © 2003 John Wiley & Sons, Ltd.     

Optimization of sample size in controlled experiments: The CLAST rule

Sequential rules are explored in the context of null hypothesis significance testing. Several studies have demonstrated that the fixed-sample stopping rule, in which the sample size used by researchers is determined in advance, is less practical and less efficient than sequential stopping rules. It is proposed that a sequential stopping rule called CLAST (composite limited adaptive sequential test) is a superior variant of COAST (composite open adaptive sequential test), a sequential rule proposed by Frick (1998). Simulation studies are conducted to test the efficiency of the proposed rule in terms of sample size and power. Two statistical tests are used: the one-tailed t test of mean differences with two matched samples, and the chi-square independence test for twofold contingency tables. The results show that the CLAST rule is more efficient than the COAST rule and reflects more realistically the practice of experimental psychology researchers.     

Power of pairwise comparisons in the equal variance and unequal sample size case

A Monte Carlo simulation was conducted to compare five, pairwise multiple comparison procedures. The number of means varied from 4 to 6 and the sample size ratio varied from 1 to 60. Procedures were evaluated on the basis of Type I errors, any‐pair power and all‐pairs power. Four procedures were shown to be conservative, while the fifth provided adequate control of Type I errors only for restricted values of sample size ratios. No procedure was found to be uniformly most powerful. The Tukey‐Kramer procedure was found to provide the best any‐pair power provided it is applied without requiring a significant overall F test. In most cases, the Hayter‐Fisher modification of the Tukey‐Kramer was found to provide very good any‐pair power and to be uniformly more powerful than the Tukey‐Kramer when a significant overall F test is required. A partition‐based version of Peritz's method usually provided the greatest all‐pairs power. A modification of the Shaffer‐Welsch was found to be useful in certain conditions.     

Computer simulation: A case of conflict

Herein is described an 18-month adventure involving the translation to computer-aided instruction (CAI) format of a traditional module that had been successfully used for years in an experimental psychology course. The primary focus of the paper is on the lessons learned during this transition. The major problems of translation that were encountered are discussed, as are unanticipated, often seemingly trivial, human-factors problems that were the most troublesome, and, consequently, the most educational.     

Effect size,statistical power,and sample size for assessing interactions between categorical and continuous variables

The reporting and interpretation of effect size estimates are widely advocated in many academic journals of psychology and related disciplines. However, such concern has not been adequately addressed for analyses involving interactions between categorical and continuous variables. For the purpose of improving current practice, this article presents fundamental features and theoretical developments for the variance of standardized slopes as a desirable standardized effect size measure for the degree of disparity between several slope coefficients. To estimate the effect size, a consistent and nearly unbiased estimator is described and a simple refinement is emphasized for extreme situations whenever appropriate. The essential problems of power and sample size calculations for testing the equality of slope coefficients are also considered. According to the analytic justification and empirical assessment, the exact approach has a clear advantage over the approximate methods. Both SAS and R computer codes are provided to facilitate practical accessibility of the proposed techniques in interaction studies.     

Power function charts for specification of sample size in analysis of variance

The specification of sample size is an important aspect of the planning of every experiment. When the investigator intends to use the techniques of analysis of variance in the study of treatments effects, he should, in specifying sample size, take into consideration the power of theF tests which will be made. The charts presented in this paper make possible a simple and direct estimate of the sample size required forF tests of specified power.     

The role and impact of mental simulation in design

Although theories of mental simulations have used different formulations of the premises of ‘thought experiments’, they can be fitted under a minimalist hypothesis stating that mental simulations are run under situations of uncertainty to turn that uncertainty into approximate answers. Three basic assumptions of mental simulations were tested by using naturalistic data from engineering design. Results from the design protocols showed (1) initial representations in mental simulation had higher than base‐rate uncertainty, (2) uncertainty in mental simulations were lowered after simulation runs, (3) resulting representations had more approximations than base‐rate or initial representations. Further, the reference to external representational systems (sketches and prototypes) was examined. It was found that prototypes had fewer technical/functional simulations compared to sketches or unsupported cognition. Although prototypes were associated with more approximation than unsupported cognition, the different external representation categories did not differ in information uncertainty. The results support the minimalist hypothesis of mental simulations. Copyright © 2008 John Wiley & Sons, Ltd.