非正态分布下概化理论方差分量变异量估计

方差分量估计是概化理论的必用技术,但受限于抽样,需要对其变异量进行探讨。采用Monte Carlo数据模拟技术,探讨非正态数据分布对四种方法估计概化理论方差分量变异量的影响。结果表明：（1）不同非正态数据分布下,各种估计方法的“性能”表现出差异性;（2）数据分布对方差分量变异量估计有影响,适合于非正态分布数据的方差分量变异量估计方法不一定适合于正态分布数据。     

基于概化理论的方差分量变异量估计

概化理论广泛应用于心理与教育测量实践中, 方差分量估计是进行概化理论分析的关键。方差分量估计受限于抽样, 需要对其变异量进行探讨。采用蒙特卡洛(Monte Carlo)数据模拟技术, 在正态分布下讨论不同方法对基于概化理论的方差分量变异量估计的影响。结果表明: Jackknife方法在方差分量变异量估计上不足取; 不采取Bootstrap方法的“分而治之”策略, 从总体上看, Traditional方法和有先验信息的MCMC方法在标准误及置信区间这两个变异量估计上优势明显。     

概化理论方差分量及其变异量估计:跨分布的模拟研究

为考察概化理论中方差分量及其变异量估计的准确性,采用模拟研究的方法,探究Traditional法、Jackknife法、Bootstrap法和MCMC法在p×i×h和p×(i:h)2种双侧面设计和正态、二项、多项、偏态分布4种数据类型下的表现。结果显示:(1)4种方法均能准确估计方差分量;(2)估计方差分量的标准误时,若数据正态分布,Traditional法最优,非正态分布时Bootstrap法最优;(3)估计方差分量的90%置信区间时,Bootstrap法在不同分布的数据下表现稳定,但容易受到侧面水平数的影响。综合来说,若数据呈正态分布,建议选用Traditional法; 若数据呈非正态分布,建议选用Bootstrap法。     

How meta-analysis increases statistical power

One of the most frequently cited reasons for conducting a meta-analysis is the increase in statistical power that it affords a reviewer. This article demonstrates that fixed-effects meta-analysis increases statistical power by reducing the standard error of the weighted average effect size (T.) and, in so doing, shrinks the confidence interval around T.. Small confidence intervals make it more likely for reviewers to detect nonzero population effects, thereby increasing statistical power. Smaller confidence intervals also represent increased precision of the estimated population effect size. Computational examples are provided for 3 effect-size indices: d (standardized mean difference), Pearson's r, and odds ratios. Random-effects meta-analyses also may show increased statistical power and a smaller standard error of the weighted average effect size. However, the authors demonstrate that increasing the number of studies in a random-effects meta-analysis does not always increase statistical power.     

校正的Bootstrap方法对概化理论方差分量及其变异量估计的改善

Bootstrap方法是一种有放回的再抽样方法, 可用于概化理论的方差分量及其变异量估计。用Monte Carlo技术模拟四种分布数据, 分别是正态分布、二项分布、多项分布和偏态分布数据。基于p×i设计, 探讨校正的Bootstrap方法相对于未校正的Bootstrap方法, 是否改善了概化理论估计四种模拟分布数据的方差分量及其变异量。结果表明：跨越四种分布数据, 从整体到局部, 不论是“点估计”还是“变异量”估计, 校正的Bootstrap方法都要优于未校正的Bootstrap方法, 校正的Bootstrap方法改善了概化理论方差分量及其变异量估计。     

A Bayesian Power Analysis Procedure Considering Uncertainty in Effect Size Estimates from a Meta-analysis

In conventional frequentist power analysis, one often uses an effect size estimate, treats it as if it were the true value, and ignores uncertainty in the effect size estimate for the analysis. The resulting sample sizes can vary dramatically depending on the chosen effect size value. To resolve the problem, we propose a hybrid Bayesian power analysis procedure that models uncertainty in the effect size estimates from a meta-analysis. We use observed effect sizes and prior distributions to obtain the posterior distribution of the effect size and model parameters. Then, we simulate effect sizes from the obtained posterior distribution. For each simulated effect size, we obtain a power value. With an estimated power distribution for a given sample size, we can estimate the probability of reaching a power level or higher and the expected power. With a range of planned sample sizes, we can generate a power assurance curve. Both the conventional frequentist and our Bayesian procedures were applied to conduct prospective power analyses for two meta-analysis examples (testing standardized mean differences in example 1 and Pearson's correlations in example 2). The advantages of our proposed procedure are demonstrated and discussed.     

Cognitive Bias for the Distribution of Ball Landing Positions in Amateur Tennis Players (Cognitive Bias for the Motor Variance in Tennis)

This study aimed to investigate whether the isotropy bias (estimating one's own motor variance as an approximately circular distribution rather than a vertically elongated distribution) arises in tennis players for the estimation of the two-dimensional variance for forehand strokes in tennis (Experiment 1), as well as the process underlying the isotropy bias (Experiment 2). In Experiment 1, 31 tennis players were asked to estimate prospectively their distribution of ball landing positions. They were then instructed to hit 50 forehand strokes. We compared the eccentricity of the ellipse calculated from estimated and observed landing positions. Eccentricity was significantly smaller in the estimated ellipse than in the observed ellipse. We assumed that the isotropy bias for the estimated ellipse comes from the process of variance estimation. In Experiment 2, nine participants estimated the 95% confidence interval of 300 dots. Eccentricity was significantly smaller in their estimated ellipses than it was in the ellipses for the dots, though the magnitude of bias decreased for the estimation of dots. These results suggest that the isotropy bias in tennis ball landing position includes the bias of recognizing landing position and the bias of estimating the ellipse confidence interval from the recognized landing position.     

Confidence Intervals for the Probability of Superiority Effect Size Measure and the Area Under a Receiver Operating Characteristic Curve

It is good scientific practice to the report an appropriate estimate of effect size and a confidence interval (CI) to indicate the precision with which a population effect was estimated. For comparisons of 2 independent groups, a probability-based effect size estimator (A) that is equal to the area under a receiver operating characteristic curve and closely related to the popular Wilcoxon-Mann-Whitney nonparametric statistical tests has many appealing properties (e.g., easy to understand, robust to violations of parametric assumptions, insensitive to outliers). We performed a simulation study to compare 9 analytic and 3 empirical (bootstrap) methods for constructing a CI for A that can yield very different CIs for the same data. The experimental design crossed 6 factors to yield a total of 324 cells representing challenging but realistic data conditions. Results were examined using several criteria, with emphasis placed on the extent to which observed CI coverage probabilities approximated nominal levels. Based on the simulation study results, the bias-corrected and accelerated bootstrap method is recommended for constructing a CI for the A statistic; bootstrap methods also provided the least biased and most accurate standard error of A. An empirical illustration examining score differences on a citation-based index of scholarly impact across faculty at low-ranked versus high-ranked research universities underscores the importance of choosing an appropriate CI method.     

An alternative to Cohen's standardized mean difference effect size: a robust parameter and confidence interval in the two independent groups case

The authors argue that a robust version of Cohen's effect size constructed by replacing population means with 20% trimmed means and the population standard deviation with the square root of a 20% Winsorized variance is a better measure of population separation than is Cohen's effect size. The authors investigated coverage probability for confidence intervals for the new effect size measure. The confidence intervals were constructed by using the noncentral t distribution and the percentile bootstrap. Over the range of distributions and effect sizes investigated in the study, coverage probability was better for the percentile bootstrap confidence interval.     

Effect size estimates: current use, calculations, and interpretation

The Publication Manual of the American Psychological Association (American Psychological Association, 2001, American Psychological Association, 2010) calls for the reporting of effect sizes and their confidence intervals. Estimates of effect size are useful for determining the practical or theoretical importance of an effect, the relative contributions of factors, and the power of an analysis. We surveyed articles published in 2009 and 2010 in the Journal of Experimental Psychology: General, noting the statistical analyses reported and the associated reporting of effect size estimates. Effect sizes were reported for fewer than half of the analyses; no article reported a confidence interval for an effect size. The most often reported analysis was analysis of variance, and almost half of these reports were not accompanied by effect sizes. Partial η2 was the most commonly reported effect size estimate for analysis of variance. For t tests, 2/3 of the articles did not report an associated effect size estimate; Cohen's d was the most often reported. We provide a straightforward guide to understanding, selecting, calculating, and interpreting effect sizes for many types of data and to methods for calculating effect size confidence intervals and power analysis.     

Bootstrap standard error and confidence intervals for the correlations corrected for indirect range restriction

The standard Pearson correlation coefficient, r, is a biased estimator of the population correlation coefficient, ρ(XY) , when predictor X and criterion Y are indirectly range-restricted by a third variable Z (or S). Two correction algorithms, Thorndike's (1949) Case III, and Schmidt, Oh, and Le's (2006) Case IV, have been proposed to correct for the bias. However, to our knowledge, the two algorithms did not provide a procedure to estimate the associated standard error and confidence intervals. This paper suggests using the bootstrap procedure as an alternative. Two Monte Carlo simulations were conducted to systematically evaluate the empirical performance of the proposed bootstrap procedure. The results indicated that the bootstrap standard error and confidence intervals were generally accurate across simulation conditions (e.g., selection ratio, sample size). The proposed bootstrap procedure can provide a useful alternative for the estimation of the standard error and confidence intervals for the correlation corrected for indirect range restriction.     

Estimating the mean effect size in meta-analysis: Bias,precision, and mean squared error of different weighting methods

Although use of the standardized mean difference in meta-analysis is appealing for several reasons, there are some drawbacks. In this article, we focus on the following problem: that a precision-weighted mean of the observed effect sizes results in a biased estimate of the mean standardized mean difference. This bias is due to the fact that the weight given to an observed effect size depends on this observed effect size. In order to eliminate the bias, Hedges and Olkin (1985) proposed using the mean effect size estimate to calculate the weights. In the article, we propose a third alternative for calculating the weights: using empirical Bayes estimates of the effect sizes. In a simulation study, these three approaches are compared. The mean squared error (MSE) is used as the criterion by which to evaluate the resulting estimates of the mean effect size. For a meta-analytic dataset with a small number of studies, theMSE is usually smallest when the ordinary procedure is used, whereas for a moderate or large number of studies, the procedures yielding the best results are the empirical Bayes procedure and the procedure of Hedges and Olkin, respectively.     

Accuracy in parameter estimation for ANCOVA and ANOVA contrasts: sample size planning via narrow confidence intervals

Contrasts of means are often of interest because they describe the effect size among multiple treatments. High-quality inference of population effect sizes can be achieved through narrow confidence intervals (CIs). Given the close relation between CI width and sample size, we propose two methods to plan the sample size for an ANCOVA or ANOVA study, so that a sufficiently narrow CI for the population (standardized or unstandardized) contrast of interest will be obtained. The standard method plans the sample size so that the expected CI width is sufficiently small. Since CI width is a random variable, the expected width being sufficiently small does not guarantee that the width obtained in a particular study will be sufficiently small. An extended procedure ensures with some specified, high degree of assurance (e.g., 90% of the time) that the CI observed in a particular study will be sufficiently narrow. We also discuss the rationale and usefulness of two different ways to standardize an ANCOVA contrast, and compare three types of standardized contrast in the ANCOVA/ANOVA context. All of the methods we propose have been implemented in the freely available MBESS package in R so that they can be easily applied by researchers.     

A Nondegenerate Penalized Likelihood Estimator for Variance Parameters in Multilevel Models

Group-level variance estimates of zero often arise when fitting multilevel or hierarchical linear models, especially when the number of groups is small. For situations where zero variances are implausible a priori, we propose a maximum penalized likelihood approach to avoid such boundary estimates. This approach is equivalent to estimating variance parameters by their posterior mode, given a weakly informative prior distribution. By choosing the penalty from the log-gamma family with shape parameter greater than 1, we ensure that the estimated variance will be positive. We suggest a default log-gamma(2,λ) penalty with λ→0, which ensures that the maximum penalized likelihood estimate is approximately one standard error from zero when the maximum likelihood estimate is zero, thus remaining consistent with the data while being nondegenerate. We also show that the maximum penalized likelihood estimator with this default penalty is a good approximation to the posterior median obtained under a noninformative prior. Our default method provides better estimates of model parameters and standard errors than the maximum likelihood or the restricted maximum likelihood estimators. The log-gamma family can also be used to convey substantive prior information. In either case—pure penalization or prior information—our recommended procedure gives nondegenerate estimates and in the limit coincides with maximum likelihood as the number of groups increases.     

内部分配改革的职务评价技术探新

建立工资标准系统的关键是确定职务工资率。职务间可比价值成分变异越大,对确定职务工资率的贡献也越大。根据以上研究设想采用方差分析方法进行职务评价。评价步骤包括：职务描述;对职务要素作主成分分析;对职务样本作聚类分析和判别分析;通过方差分析为可比价值各成分建构权重系数ωｉ。ωｉ‘满足：（１）ωｉ≥０;（２）Σωｉ＝１;（３）ωｉ,间可直接比较;（４）ωｉ的大小与对应的可比价值成分变异一致。最后将职务评价值线性变换为工资率。在线性方程中配一个常数。和调节系数α以适合组织的管理约束条件。配合两个企业内部分配改革的研究结果显示了方差分析法的有效性和实用性。     

Replies to comments on "Which method delivers greater signal-to-noise ratio: Structural equation modelling or regression analysis with weighted composites?" by Yuan and Fang (2023)

Observational data typically contain measurement errors. Covariance-based structural equation modelling (CB-SEM) is capable of modelling measurement errors and yields consistent parameter estimates. In contrast, methods of regression analysis using weighted composites as well as a partial least squares approach to SEM facilitate the prediction and diagnosis of individuals/participants. But regression analysis with weighted composites has been known to yield attenuated regression coefficients when predictors contain errors. Contrary to the common belief that CB-SEM is the preferred method for the analysis of observational data, this article shows that regression analysis via weighted composites yields parameter estimates with much smaller standard errors, and thus corresponds to greater values of the signal-to-noise ratio (SNR). In particular, the SNR for the regression coefficient via the least squares (LS) method with equally weighted composites is mathematically greater than that by CB-SEM if the items for each factor are parallel, even when the SEM model is correctly specified and estimated by an efficient method. Analytical, numerical and empirical results also show that LS regression using weighted composites performs as well as or better than the normal maximum likelihood method for CB-SEM under many conditions even when the population distribution is multivariate normal. Results also show that the LS regression coefficients become more efficient when considering the sampling errors in the weights of composites than those that are conditional on weights.     

On the reliability of a weighted composite

A general formula for the reliability of a weighted composite has been derived by which that reliability can be estimated from a knowledge of the weights whatever their source, reliabilities, dispersions, and intercorrelations of the components. The Spearman-Brown formula has been shown to be a special case of the general statement. The effect of the internal consistency or intercorrelation of the components has been investigated and the conditions defining the set of weights yielding maximum reliability shown to be that the weight of a component is proportional to the sum of its intercorrelations with the remaining components and inversely proportional to its error variance.     

认知诊断属性分类一致性信度区间估计三种方法

摘要：引入了三种可以估计认知诊断属性分类一致性信度置信区间的方法：Bootstrap法、平行测验法和平行测验配对法。用模拟研究验证和比较了这三种方法的表现,结果发现,平行测验法和Bootstrap法在被试量比较少、题目数量比较少的情况下,估计的标准误和置信区间较接近,但是随着被试量的增加,Bootstrap法的估计精度提高较快,在被试量大和题目数量较多时基本接近平行测验配对法的结果。Bootstrap法的所需时间最少,平行测验配对法计算过程复杂且用时较长,推荐用Bootstrap法估计认知诊断属性分类一致性信度的置信区间。     

The variance of d' estimates obtained in yes-no and two-interval forced choice procedures

In studies of detection and discrimination, data are often obtained in the form of a 2 x 2 matrix and then converted to an estimate of d' based on the assumptions that the underlying decision distributions are Gaussian and equal in variance. The statistical properties of the estimate of d', d' are well understood for data obtained using the yes-no procedure, but less effort has been devoted to the more commonly used two-interval forced choice (2IFC) procedure. The variance associated with d' is a function of true d' in both procedures, but for small values of true d' the variance of d' obtained using the 2IFC procedure is predicted to be less than the variance of d' obtained using yes-no; for large values of true d', the variance of d' obtained using the 2IFC procedure is predicted to be greater than the variance of d' from yes-no. These results follow from standard assumptions about the relationship between the two procedures. The present paper reviews the statistical properties of d' obtained using the two standard procedures and compares estimates of the variance of d' as a function of true d' with the variance observed in values of d' obtained with a 2IFC procedure.     

Nearly unbiased estimators for the three‐parameter Weibull distribution with greater efficiency than the iterative likelihood method

The maximum likelihood estimation (MLE) method is the most commonly used method to estimate the parameters of the three‐parameter Weibull distribution. However, it returns biased estimates. In this paper, we show how to calculate weights which cancel the biases contained in the MLE equations. The exact weights can be computed when the population parameters are known and the expected weights when they are not. Two of the three weights' expected values are dependent only on the sample size, whereas the third also depends on the population shape parameters. Monte Carlo simulations demonstrate the practicability of the weighted MLE method. When compared with the iterative MLE technique, the bias is reduced by a factor of 7 (irrespective of the sample size) and the variability of the parameter estimates is also reduced by a factor of 7 for very small sample sizes, but this gain disappears for large sample sizes.