Profile-likelihood Confidence Intervals in Item Response Theory Models

Confidence intervals (CIs) are fundamental inferential devices which quantify the sampling variability of parameter estimates. In item response theory, CIs have been primarily obtained from large-sample Wald-type approaches based on standard error estimates, derived from the observed or expected information matrix, after parameters have been estimated via maximum likelihood. An alternative approach to constructing CIs is to quantify sampling variability directly from the likelihood function with a technique known as profile-likelihood confidence intervals (PL CIs). In this article, we introduce PL CIs for item response theory models, compare PL CIs to classical large-sample Wald-type CIs, and demonstrate important distinctions among these CIs. CIs are then constructed for parameters directly estimated in the specified model and for transformed parameters which are often obtained post-estimation. Monte Carlo simulation results suggest that PL CIs perform consistently better than Wald-type CIs for both non-transformed and transformed parameters.     

A Note on Confidence Intervals for Two-Group Latent Mean Effect Size Measures

This note suggests delta method implementations for deriving confidence intervals for a latent mean effect size measure for the case of 2 independent populations. A hypothetical kindergarten reading example using these implementations is provided, as is supporting LISREL syntax.     

Experimental Power for Indirect Effects in Group-randomized Studies with Group-level Mediators

Mediation analyses have provided a critical platform to assess the validity of theories of action across a wide range of disciplines. Despite widespread interest and development in these analyses, literature guiding the design of mediation studies has been largely unavailable. Like studies focused on the detection of a total or main effect, an important design consideration is the statistical power to detect indirect effects if they exist. Understanding the sensitivity to detect indirect effects is exceptionally important because it directly influences the scale of data collection and ultimately governs the types of evidence group-randomized studies can bring to bear on theories of action. However, unlike studies concerned with the detection of total effects, literature has not established power formulas for detecting multilevel indirect effects in group-randomized designs. In this study, we develop closed-form expressions to estimate the variance of and the power to detect indirect effects in group-randomized studies with a group-level mediator using two-level linear models (i.e., 2-2-1 mediation). The results suggest that when carefully planned, group-randomized designs may frequently be well positioned to detect mediation effects with typical sample sizes. The resulting power formulas are implemented in the R package PowerUpR and the PowerUp!-Mediator software (causalevaluation.org).     

从效应量应有的性质看中介效应量的合理性

效应量的作用有两个方面, 一是弥补了统计检验的不足, 二是使得效应有可比性。结合统计显著性和效应量, 才能得出适当的统计结论。效应量应当具有一些基本性质, 包括与测量单位无关、单调性、不受样本容量的影响。国际上流行的中介效应量κ平方就是因为缺乏单调性而引发质疑和研究, 从而被彻底终结了其作为中介效应量的合法性。R平方型中介效应量同样有缺乏单调性的问题。文末讨论了如何报告中介效应量以及有待研究的问题。     

决策信心校准水平对元认知监控作用的影响

决策信心是个体对自身决策正确性的主观评价,是对决策过程的元认知体验。决策信心校准指决策信心水平与实际的决策正确率之间的匹配程度,其指标有信心水平和决策正确率的相关系数及Type II信号检测论中的ROC曲线下面积（Aroc）等。已有研究发现进行决策信心评估能够增强对当前或后续决策的元认知监控作用,但目前尚不清楚这种效应是否依赖于个体的决策信心校准水平。本研究通过设置知觉决策后是否进行决策信心评估（有信心评估与无信心评估）两种条件,考察个体决策信心的校准水平（Aroc）对元认知监控作用的影响。结果显示：1）与无信心评估条件相比,有信心评估的决策反应时显著增长,决策正确率显著提高（p<0.005）;2）Aroc与有、无信心评估条件下决策正确率的增加值显著正相关（r=0.25,p=0.034）,且高Aroc组的决策正确率增加值显著高于低Aroc组（p<0.05）。结果表明,在知觉决策过程中加入决策信心评估具有增强元认知监控作用的效应,体现为决策时间的增长和决策正确率的提高。并且,这种效应的大小依赖于个体的决策信心校准水平,校准水平越高元认知监控作用越好。     

A Comparative Investigation of Confidence Intervals for IndependentVariables in Linear Regression

In linear regression, the most appropriate standardized effect size for individual independent variables having an arbitrary metric remains open to debate, despite researchers typically reporting a standardized regression coefficient. Alternative standardized measures include the semipartial correlation, the improvement in the squared multiple correlation, and the squared partial correlation. No arguments based on either theoretical or statistical grounds for preferring one of these standardized measures have been mounted in the literature. Using a Monte Carlo simulation, the performance of interval estimators for these effect-size measures was compared in a 5-way factorial design. Formal statistical design methods assessed both the accuracy and robustness of the four interval estimators. The coverage probability of a large-sample confidence interval for the semipartial correlation coefficient derived from Aloe and Becker was highly accurate and robust in 98% of instances. It was better in small samples than the Yuan-Chan large-sample confidence interval for a standardized regression coefficient. It was also consistently better than both a bootstrap confidence interval for the improvement in the squared multiple correlation and a noncentral interval for the squared partial correlation.     

The statistical recommendations of the American Psychological Association Publication Manual: Effect sizes,confidence intervals,and meta‐analysis

Estimation based on effect sizes, confidence intervals, and meta‐analysis usually provides a more informative analysis of empirical results than does statistical significance testing, which has long been the conventional choice in psychology. The sixth edition of the American Psychological Association Publication Manual now recommends that psychologists should, wherever possible, use estimation and base their interpretation of research results on point and interval estimates. We outline the Manual's recommendations and suggest how they can be put into practice: adopt an estimation framework, starting with the formulation of research aims as ‘How much?’ or ‘To what extent?’ questions. Calculate from your data effect size estimates and confidence intervals to answer those questions, then interpret. Wherever appropriate, use meta‐analysis to integrate evidence over studies. The Manual's recommendations can assist psychologists improve they way they do their statistics and help build a more quantitative and cumulative discipline.     

A Regression Framework for Causal Mediation Analysis with Applications to Behavioral Science

We introduce and extend the classical regression framework for conducting mediation analysis from the fit of only one model. Using the essential mediation components (EMCs) allows us to estimate causal mediation effects and their analytical variance. This single-equation approach reduces computation time and permits the use of a rich suite of regression tools that are not easily implemented on a system of three equations. Additionally, we extend this framework to non-nested mediation systems, provide a joint measure of mediation for complex mediation hypotheses, propose new visualizations for mediation effects, and explain why estimates of the total effect may differ depending on the approach used. Using data from social science studies, we also provide extensive illustrations of the usefulness of this framework and its advantages over traditional approaches to mediation analysis. The example data are freely available for download online and we include the R code necessary to reproduce our results.     

Confidence Intervals for the Probability of Superiority Effect Size Measure and the Area Under a Receiver Operating Characteristic Curve

It is good scientific practice to the report an appropriate estimate of effect size and a confidence interval (CI) to indicate the precision with which a population effect was estimated. For comparisons of 2 independent groups, a probability-based effect size estimator (A) that is equal to the area under a receiver operating characteristic curve and closely related to the popular Wilcoxon-Mann-Whitney nonparametric statistical tests has many appealing properties (e.g., easy to understand, robust to violations of parametric assumptions, insensitive to outliers). We performed a simulation study to compare 9 analytic and 3 empirical (bootstrap) methods for constructing a CI for A that can yield very different CIs for the same data. The experimental design crossed 6 factors to yield a total of 324 cells representing challenging but realistic data conditions. Results were examined using several criteria, with emphasis placed on the extent to which observed CI coverage probabilities approximated nominal levels. Based on the simulation study results, the bias-corrected and accelerated bootstrap method is recommended for constructing a CI for the A statistic; bootstrap methods also provided the least biased and most accurate standard error of A. An empirical illustration examining score differences on a citation-based index of scholarly impact across faculty at low-ranked versus high-ranked research universities underscores the importance of choosing an appropriate CI method.     

Sample Size Planning for Detecting Mediation Effects: A Power Analysis Procedure Considering Uncertainty in Effect Size Estimates

Abstract

When planning mediation studies, researchers are often interested in the sample size needed to achieve adequate power for testing mediation. Power depends on population effect sizes, which are unknown in practice. In conventional power analysis, effect size estimates, however, are often used as population values, which could result in underpowered studies. Uncertainty in effect size estimates has been considered in other sample size planning contexts (e.g., t-test, ANOVA), but has not been handled properly for planning mediation studies. In the current study, we proposed an easy-to-use sample size planning method for testing mediation with uncertainty in effect size estimates considered. We conducted simulation studies to demonstrate the impact of uncertainty in effect size estimates on power of testing mediation, and to provide sample size suggestions under different levels of uncertainty. Empirical examples were provided to illustrate the application of our method. R functions and a web application were developed to facilitate implementation.     

中介效应的检验方法和效果量测量:回顾与展望

通过中介效应检验方法之间的比较和效果量指标之间的比较,建议放弃将总效应c显著作为中介效应检验的前提条件,放弃基于直接效应c'显著性的完全和部分中介的提法,推荐使用偏差校正的百分位Bootstrap法直接对中介效应ab进行检验,使用κ²、R_med²等中介效果量指标并报告效果量的置信区间。作为示例,用R软件的MBESS软件包对某消防员饮食健康调查进行了中介效应检验和效果量测量。随后展望了中介效应检验方法和效果量测量的拓展方向。     

损益框架和损益概率对不同年级儿童风险决策的影响

摘 要 为探讨任务框架和自信水平对不同年级儿童风险决策的影响,采用3（年级：二/四/六年级）×2（任务框架：获益/损失）×2（自信水平：过度自信/自信不足）的组间实验设计,在情境性实验任务中探查177名儿童风险决策的发展特点。结果表明：（1）不同年级儿童在风险决策中表现出不同的发展特点,二年级儿童比六年级儿童更倾向于冒险,而四年级前后则是儿童风险决策发展的关键期、转折期;（2）儿童风险决策中存在框架效应,表现出偏好反转现象,与在获益情境下更偏好于保守决策相比,在损失情境下的儿童更偏好冒险决策;（3）自信水平并未对儿童的风险决策产生影响。     

中介效应的三类区间估计方法

由于中介效应ab的估计量通常不是正态分布, 因此需用不对称置信区间进行中介效应分析。详述了三类获得不对称置信区间的方法, 包括乘积分布法(M法和经验M法)、Bootstrap方法(偏差校正和未校正的非参数百分位Bootstrap方法、偏差校正和未校正的参数百分位残差Bootstrap方法)和马尔科夫链蒙特卡罗(MCMC)方法。比较了三类方法在单层(简单和多重)和多层中介效应分析中的表现, 发现三类方法的表现相近, 与乘积分布法相比, 偏差校正的百分位Bootstrap方法表现较好, 但有先验信息的MCMC方法能更有效降低均方误。最后对中介效应不对称置信区间研究的拓展方向做了展望。     

决策信心的认知机制与神经基础

决策信心是指在进行判断或决策时, 个体对其决策最优性或正确性的信心程度。决策信心的单过程理论认为决策与信心评估是并行的同一加工过程; 而决策后理论认为决策和信心评估是串行的两个加工过程。决策信心存在两种典型的偏差效应：过分自信和信心不足。决策信心的神经机制研究指出, 信心评估可能是决策过程中最基本并且普遍存在的一个成分; 对决策信心评估敏感的脑区包括扣带回、背外侧前额叶和顶上小叶等。未来研究应该在决策信心产生的机制、信心与决策如何交互的心理及神经机制等领域展开大量研究。     

决策信心在信息化从众中的作用

研究采用JAS范式,通过操纵群体信息的性质（支持和反对）以及一致性程度,考察了决策信心在信息化从众过程中的作用。结果表明：（1）个体仅在接收到反对信息时会发生决策的偏转,表现出从众;（2）群体参照信息能够显著影响个体的信心：支持性的群体信息使个体的信心显著增高,而反对性的群体信息使信心显著降低,且表现出一种“负性偏向”（即个体对来自群体的负性信息更加敏感）;（3）在反对条件下,个体信心降低的程度能够很好地预测其决策偏转的概率,即信息化从众行为。这说明,决策信心可能在信息化从众中起着核心的中介作用——反对性的群体信息使得决策信心下降,而决策信心的下降导致了决策的偏转,从而表现出从众行为。     

What you are confident of determines your choices in context: Confidence in yourself versus in your ability

The notion that consumers' preference is constructed by decision context is well established. Two of such salient manifestations are compromise effect and attraction effect. Although literature has explored the moderators of these effects from the perspective of a decision maker, little is known about whether a significant difference exists between the effects of individual differences as a situational state and as a stable personality. This article approaches this question by examining how specific self‐confidence and general self‐confidence shape consumer's preference for context options. Four studies find that compromise effect is greater for consumer with high specific self‐confidence, whereas attraction effect is greater for consumer with low specific self‐confidence. The two context effects are greater for consumers with low general self‐confidence only in the presence of social influence. In addition, low (vs. high) general self‐confidence strengthens (vs. weakens) the impact of specific self‐confidence on context effects under this condition. This article concludes by discussing the theoretical and practical implications of the findings.     

Confidence in accuracy moderates the benefits of retrieval practice

ABSTRACT

The effect of Retrieval Practice refers to the phenomenon that taking a practice test is more effective for learning than re-study, probably due to the benefit from processes underlying successful retrievals during practice. However, it is rarely studied whether other processes (e.g., metacognitive monitoring) during retrieval practice may also play an important role. In two experiments, we examined whether the effectiveness of retrieval was affected by subjects’ confidence in their retrieval success. Subjects studied word-pairs and rated their confidence after each practice test trial (cued-recall in Exp.1, multiple choice in Exp.2), or re-study trial. In both experiments, we observed the classic retrieval practice effect. Moreover, in Exp.2, the benefits differed as a function of subjects’ confidence in their retrieval performance: the effect of retrieval practice only emerged for high-confidence trials with higher than 56% confidence. In summary, the retrieval practice only facilitates the retention of correct answers with high confidence.     

父母支持、友谊质量对孤独感和抑郁的影响：检验一个间接效应模型

以560名5、7、10年级学生为被试, 检验父母支持、友谊质量与孤独感、抑郁的关系是否符合间接效应模型及其模型适用性问题。结构方程建模及群组分析结果发现：(1)总体上, 间接效应模型成立, 父母支持既直接影响孤独感和抑郁, 也通过影响友谊质量进而影响孤独感和抑郁状况; (2)从具体适应问题看, 间接效应模型对于孤独感和抑郁均显著, 但更适用于对孤独感的预测; (3)从发展阶段看, 间接效应模型仅适用于童年晚期和青少年早期, 且更适用于童年晚期; (4)从性别角度看, 间接效应模型对两性均显著, 但更适用于男生。结果提示, 父母支持、友谊质量对孤独感和抑郁的影响均符合间接效应模型, 但该模型的适用性可能因具体适应问题、发展阶段和性别而有差异。     

不同列队呈现方式下反馈对目击证人辨认自信心的影响

采用辨认后反馈范式研究不同队列呈现方式下反馈对目击证人辨认自信心的影响。结果发现：（1）所有辨认者中,反馈组被试自信心显著高于控制组被试的自信心;反馈方式与辨认选择的交互作用显著,辨认选择与列队呈现方式的交互作用显著;（2）积极辨认者中,反馈组被试的自信心显著高于控制组被试的自信心;正确辨认被试的自信心显著高于错误辨认被试的自信心。结果表明：目击辨认中存在辨认后反馈效应;积极辨认者中辨认自信心可以作为辨认准确性的一个参考指标。