A Comparison of ML,WLSMV, and Bayesian Methods for Multilevel Structural Equation Models in Small Samples: A Simulation Study

Multilevel structural equation models are increasingly applied in psychological research. With increasing model complexity, estimation becomes computationally demanding, and small sample sizes pose further challenges on estimation methods relying on asymptotic theory. Recent developments of Bayesian estimation techniques may help to overcome the shortcomings of classical estimation techniques. The use of potentially inaccurate prior information may, however, have detrimental effects, especially in small samples. The present Monte Carlo simulation study compares the statistical performance of classical estimation techniques with Bayesian estimation using different prior specifications for a two-level SEM with either continuous or ordinal indicators. Using two software programs (Mplus and Stan), differential effects of between- and within-level sample sizes on estimation accuracy were investigated. Moreover, it was tested to which extent inaccurate priors may have detrimental effects on parameter estimates in categorical indicator models. For continuous indicators, Bayesian estimation did not show performance advantages over ML. For categorical indicators, Bayesian estimation outperformed WLSMV solely in case of strongly informative accurate priors. Weakly informative inaccurate priors did not deteriorate performance of the Bayesian approach, while strong informative inaccurate priors led to severely biased estimates even with large sample sizes. With diffuse priors, Stan yielded better results than Mplus in terms of parameter estimates.     

因子分析元分析的三阶段整合模型

因子分析元分析(meta-analysis of factor analyses, MAFA)整合了因子分析和元分析两种方法学传统。开展MAFA方法学研究可结束过去几十年MAFA研究缺乏根据的状况, 服务心理学等领域内的知识生产。三阶段整合模型认为MAFA包括数据准备、数据合成和数据分析三个主要阶段。模型及数据合成技术的有效性将用心理测量学方法, 包括蒙特卡洛模拟研究进行检验。研究结果将阐明正确运用MAFA的一般规则、操作步骤和注意事项, 进而增进我们对MAFA的认识, 丰富应用心理等领域的方法学体系, 为MAFA方法的推广应用奠定基础。     

多水平随机中介效应估计及其比较

本文在综述各类多水平中介模型的基础上, 聚焦于自变量、中介变量、因变量都来自多水平结构中较低水平的多水平随机中介效应模型, 通过蒙特卡洛模拟研究比较该模型与简化的多水平固定中介效应模型、传统中介效应模型的差别, 并考察了目前用于多水平随机中介效应的三种参数估计方法：限制性极大似然、极大似然、最小方差二次无偏估计在不同情况下对随机中介效应估计的优劣。研究结果显示：当数据符合多水平随机中介效应模型时, 使用简化模型将错误估计中介效应及其标准误, 得到不正确的统计检验结果; 使用多水平随机中介效应模型能够实现对中介效应的正确估计和检验, 其中限制性极大似然或极大似然估计方法优于最小方差二次无偏估计方法。     

An alternative approach to relapse analysis: Using Monte Carlo methods and proportional rates of response

Relapse is the recovery of a previously suppressed response. Animal models have been useful in examining the mechanisms underlying relapse (e.g., reinstatement, renewal, reacquisition, resurgence). However, there are several challenges to analyzing relapse data using traditional approaches. For example, null hypothesis significance testing is commonly used to determine whether relapse has occurred. However, this method requires several a priori assumptions about the data, as well as a large sample size for between‐subjects comparisons or repeated testing for within‐subjects comparisons. Monte Carlo methods may represent an improved analytic technique, because these methods require no prior assumptions, permit smaller sample sizes, and can be tailored to account for all of the data from an experiment instead of some limited set. In the present study, we conducted reanalyses of three studies of relapse (Berry, Sweeney, & Odum, 2014 ; Galizio et al., 2018 ; Odum & Shahan, 2004 ) using Monte Carlo techniques to determine if relapse occurred and if there were differences in rate of response based on relevant independent variables (such as group membership or schedule of reinforcement). These reanalyses supported the previous findings. Finally, we provide general recommendations for using Monte Carlo methods in studies of relapse.     

R-RUM的参数估计及性能评价研究

认知诊断作为21世纪一种新的测量范式,在国内外越来越受到重视。该文运用MCMC算法实现了R-RUM的参数估计,并采用Monte Carlo模拟方法探讨其性能。研究结果表明:(1)R-RUM参数估计方法可行,估计精度较高;(2)Q矩阵复杂性和模型参数水平对模型参数估计精度有较大影响,随着r_(jk)*值的增大和Q矩阵复杂性的增加,项目参数和被试参数估计精度逐渐下降;(3)在特定情形下,R-RUM具有一定的稳健性。     

Model Evaluation and Multiple Strategies in Cognitive Diagnosis: An Analysis of Fraction Subtraction Data

This paper studies three models for cognitive diagnosis, each illustrated with an application to fraction subtraction data. The objective of each of these models is to classify examinees according to their mastery of skills assumed to be required for fraction subtraction. We consider the DINA model, the NIDA model, and a new model that extends the DINA model to allow for multiple strategies of problem solving. For each of these models the joint distribution of the indicators of skill mastery is modeled using a single continuous higher-order latent trait, to explain the dependence in the mastery of distinct skills. This approach stems from viewing the skills as the specific states of knowledge required for exam performance, and viewing these skills as arising from a broadly defined latent trait resembling the θ of item response models. We discuss several techniques for comparing models and assessing goodness of fit. We then implement these methods using the fraction subtraction data with the aim of selecting the best of the three models for this application. We employ Markov chain Monte Carlo algorithms to fit the models, and we present simulation results to examine the performance of these algorithms. The work reported here was performed under the auspices of the External Diagnostic Research Team funded by Educational Testing Service. Views expressed in this paper does not necessarily represent the views of Educational Testing Service.     

基于概化理论的方差分量变异量估计

概化理论广泛应用于心理与教育测量实践中, 方差分量估计是进行概化理论分析的关键。方差分量估计受限于抽样, 需要对其变异量进行探讨。采用蒙特卡洛(Monte Carlo)数据模拟技术, 在正态分布下讨论不同方法对基于概化理论的方差分量变异量估计的影响。结果表明: Jackknife方法在方差分量变异量估计上不足取; 不采取Bootstrap方法的“分而治之”策略, 从总体上看, Traditional方法和有先验信息的MCMC方法在标准误及置信区间这两个变异量估计上优势明显。     

A New Concurrent Calibration Method for Nonequivalent Group Design under Nonrandom Assignment

This study proposes a new item parameter linking method for the common-item nonequivalent groups design in item response theory (IRT). Previous studies assumed that examinees are randomly assigned to either test form. However, examinees can frequently select their own test forms and tests often differ according to examinees’ abilities. In such cases, concurrent calibration or multiple group IRT modeling without modeling test form selection behavior can yield severely biased results. We proposed a model wherein test form selection behavior depends on test scores and used a Monte Carlo expectation maximization (MCEM) algorithm. This method provided adequate estimates of testing parameters.     

中介效应的三类区间估计方法

由于中介效应ab的估计量通常不是正态分布, 因此需用不对称置信区间进行中介效应分析。详述了三类获得不对称置信区间的方法, 包括乘积分布法(M法和经验M法)、Bootstrap方法(偏差校正和未校正的非参数百分位Bootstrap方法、偏差校正和未校正的参数百分位残差Bootstrap方法)和马尔科夫链蒙特卡罗(MCMC)方法。比较了三类方法在单层(简单和多重)和多层中介效应分析中的表现, 发现三类方法的表现相近, 与乘积分布法相比, 偏差校正的百分位Bootstrap方法表现较好, 但有先验信息的MCMC方法能更有效降低均方误。最后对中介效应不对称置信区间研究的拓展方向做了展望。     

含题组的测验等值

题组越来越多地出现在各类考试中, 采用标准的IRT模型对有题组的测验等值, 可能因忽略题组的局部相依性导致等值结果的失真。为解决此问题, 我们采用基于题组的2PTM模型及IRT特征曲线法等值, 以等值系数估计值的误差大小作为衡量标准, 以Wilcoxon符号秩检验为依据, 在几种不同情况下进行了大量的Monte Carlo模拟实验。实验结果表明, 考虑了局部相依性的题组模型2PTM绝大部分情况下都比2PLM等值的误差小且有显著性差异。另外, 用6种不同等值准则对2PTM等值并评价了不同条件下等值准则之间的优劣。