认知诊断测验中的项目功能差异检测方法比较

检测项目功能差异(DIF)是认知诊断测验中很重要的问题。首先将逻辑斯蒂克回归法(LR)引入认知诊断测验DIF检测, 然后将LR法与MH法和Wald检验法的DIF检验效果进行比较。在比较中同时考察了匹配变量、DIF种类、DIF大小和受测者人数的影响。结果表明：(1) LR法在认知诊断测验DIF检测中, 检验力较高, 一类错误率较低。(2) LR法在检测认知诊断测验的DIF时, 不受认知诊断方法的影响。(3) LR法可以有效区分一致性DIF和非一致性DIF, 并有较高检验力和较低一类错误率。(4)采用知识状态作为匹配变量, 能够得到较理想的检验力和一类错误率。(5) DIF越大, 受测者人数越多, 统计检验力越高, 但一类错误率不受影响。     

项目功能差异对于认知诊断测验估计准确性的影响

为探讨项目功能差异对于认知诊断测验估计准确性的影响,采用模拟研究的方式在3种DIF题目所占比例,3种DIF量下,检测了4种认知诊断测验中存在的DIF对于被试能力估计准确性和题目参数估计准确性的影响。结果发现:(1)DIF对于目标组被试能力估计准确性影响较大;(2)含有DIF的题目所占比例和DIF量增大,都会降低目标组被试能力估计的准确性;(3)非一致性DIF对于被试能力估计准确性的影响大于一致性DIF;(4)只有含有DIF题目的题目参数估计准确性会下降;(5)随着DIF量增大,含有DIF题目的题目参数估计准确性下降增多,但不受含DIF题目所占比例的影响。     

运用均数与协方差结构模型侦查项目功能差异

阐释了运用多组均数与协方差结构(MACS)模型侦查多级反应项目的一致性与非一致性项目功能差异(DIF)的原理与程序, 以道德自我概念量表DIF的侦查进行示例, 并对该方法进行了评价。与项目反应理论比照, MACS采用系统的、迭代的方式利用修正指数来侦查DIF, 并提供多个拟合指数协同评价模型拟合;与标准验证性因素分析相较, MACS不仅能侦查非一致性DIF, 而且能侦查一致性DIF。运用MACS侦查DIF是一种值得推荐的方法。     

变通的题组项目功能差异检验方法在篇章阅读测验中的应用

篇章形式的阅读测验在语文学科考试与语言能力测试中占有越来越重要的地位。篇章阅读测验是一种典型的题组测验, 因此需要采用能够处理题组效应的统计方法进行分析。在进行项目功能差异(DIF)检验时, 也需要采用与之匹配的DIF检验方法。目前能够处理题组效应的DIF检验方法主要包括变通的题组DIF检验方法和基于题组反应模型的DIF检验方法, 基于题组反应模型的DIF检验方法由于实现过程繁琐, 目前只停留在理论探讨阶段。本研究将变通的题组DIF检验方法及其效应值指标引入篇章阅读测验的DIF检验中, 能够解决篇章阅读测验中DIF检验与测量的问题, 效应值指标能够为如何处理有DIF效应的题组项目提供重要依据。本研究首先选用非题组DIF检验方法与变通的题组DIF检验方法对一份试卷进行DIF检验, 两种方法的比较结果体现了进行题组DIF检验的必要性与优越性, 然后选用变通的题组DIF检验方法中有代表性的四种方法对某阅读成就测验进行题组DIF检验。研究结果表明, 在篇章阅读测验中, 能够处理题组效应的DIF检验方法较传统的DIF检验方法具有较大的优越性。     

题组项目功能差异（DIF）检验方法的应用探索

篇章形式的阅读测验是一种典型的题组测验,在进行项目功能差异（DIF）检验时需要采用与之匹配的DIF检验方法.基于题组反应模型的DIF检验方法是真正能够处理题组效应的DIF检验方法,能够提供题组中每个项目的DIF效应测量,是题组DIF检验方法中较有理论优势的一种,主要使用的方法是Rasch题组DIF检验方法.该研究将Rasch题组DIF检验方法引入篇章阅读测验的DIF检验中,对某阅读成就测验进行题组DIF检验,结果显示,该测验在内容维度和能力维度的部分子维度上出现了具有显著DIF效应的项目,研究从测验公平的角度对该测验的进一步修改及编制提出了一定的建议.研究中进一步将Rasch题组DIF检验方法与基于传统Rasch模型的DIF检验方法以及变通的题组DIF检验方法的结果进行比较,研究结果体现了进行题组DIF检验的必要性与优越性.研究结果表明,在篇章阅读测验中,能够真正处理题组效应的题组DIF检验方法更加具有理论优势且对于阅读测验的编制与质量的提高具有更重要的意义.     

LP方法及其与三种常用DIF检测方法的比较

本研究基于项目反应理论,提出了一种检验力高且犯Ⅰ类错误率小的检测DIF的新方法：LP法（Likelihood Procedure）,且以2PLM下对题目进行DIF检验为例介绍此法。本文通过与MH方法、Lord卡方检验法和Raju面积测量法三种常用的检验DIF的方法比较研究LP法的有效性,同时探讨样本容量、测验长度、目标组和参照组能力分布的差异、DIF值大小等相关因素对LP法有效性可能产生的影响。通过模拟研究,得到以下结论：(1)LP法比MH法及Lord卡方法更灵敏且更稳健;(2) LP法比Raju面积测量法更合理;(3)LP法的检验力随着被试样本容量或DIF值的增大而增大;（4）当参照组与目标组的能力无差异时,LP法在各种条件下的检验力比参照组与目标组的能力有差异时的检验力高;（5）LP法对一致性DIF和非一致性DIF都有良好的检验力,且LP法对一致性DIF的检验力比对非一致性DIF的检验力高。LP法可以简便的扩展并运用到多维度、多级评分项目上。     

中国少数民族考生与外国考生HSK成绩的公平性分析

该研究利用项目功能差异(Differential Item Functioning,简称DIF)理论,对HSK考生中不同两个群体——外国人和中国国内的少数民族,进行题目的反应分析。考查HSK的题目是否存在不利于某一群体。具体做法：采用MH和SIBTEST方法检测DIF,利用标准化的离散分析方法和SIBTEST的项目束分析法鉴别DIF的真伪,并寻找造成DIF的原因。由数据分析的结果可知,HSK(初、中等)A卷对对外国考生和中国国内的少数民族考生存在一些有DIF的题目。     

IRT_Δb法和修正LR法对矩阵取样DIF检验的有效性

矩阵取样测验包含多个题册,单个题册的总分不能直接作为匹配变量用于 DIF 检测。本研究首先基于模拟数据,同时采用 I RT_Δb法,以及用 I RT模型估计的考生能力作为匹配变量修订后的 L R法对矩阵取样测验进行DIF检测,分析二者进行DIF检测的有效性及其相关影响因素;并根据已有的LR法DIF判断标准划定出I RT_Δb法分类标准;最后使用实证数据加以验证。结果显示：矩阵取样测验中, I RT_Δb法和修正LR法均能较好地区分DIF量不同的题目;样本量、题册中DIF题目的比例和考生群体间真实能力的差异对两种方法的检验力、犯I类错误的概率和分类结果都有较大影响。     

题目参数漂移:概念厘定及相关研究

题目参数漂移(Item Parameter Drift,IPD)指锚题参数值在若干连续性测试场合或测验水平之间的变化。该概念和项目功能差异(Differential Item Function,DIF)有实质区别。当前的IPD研究涵盖五个方面:IPD的实际存在、产生原因、检测方法、链接结果效应,及极端锚题的处理策略。横向IPD领域需要更进一步和具有综合性的研究,尤其需研究DIF检测方法在IPD检测中的语境适用性,开发针对IPD的检测方法,以及发展修正的链接程序。纵向IPD研究需要进行系统、深入探索。     

关于DIF的一个简单LOG模型

本文提出了解决DIF问题的基本想法,并通过实例计算,介绍了利用Logistic函数构建的一种简单的DIF模型。     

An investigation of DIF mechanisms in the context of differential testlet effects

This study investigated differential item functioning (DIF) mechanisms in the context of differential testlet effects across subgroups. Specifically, we investigated DIF manifestations when the stochastic ordering assumption on the nuisance dimension in a testlet does not hold. DIF hypotheses were formulated analytically using a parametric marginal item response function approach and compared with empirical DIF results from a unidimensional item response theory approach. The comparisons were made in terms of type of DIF (uniform or non‐uniform) and direction (whether the focal or reference group was advantaged). In general, the DIF hypotheses were supported by the empirical results, showing the usefulness of the parametric approach in explaining DIF mechanisms. Both analytical predictions of DIF and the empirical results provide insights into conditions where a particular type of DIF becomes dominant in a specific DIF direction, which is useful for the study of DIF causes.     

汉语词汇测验中的项目功能差异初探

该文运用两种不同的方法对实际的汉语词汇测验中的３６个词汇进行了ＤＩＦ探测。对于１４００多劬的初三学生分别作了男女生与城郊学生间的比较。在男女组分析中检出７个属于一致性ＤＩＦ的项目;对于城郊学生组经两种方法同时确定的ＤＩＦ项目有７个,其中５个是一致性ＤＩＦ,２个是非一致性ＤＩＦ的项目。该文还讨论了产生ＤＩＦ的可能因素。     

The Effect of Propensity Scores on DIF Analysis: Inference on the Potential Cause of DIF

Differential item functioning (DIF) analysis is important in terms of test fairness. While DIF analyses have mainly been conducted with manifest grouping variables, such as gender or race/ethnicity, it has been recently claimed that not only the grouping variables but also contextual variables pertaining to examinees should be considered in DIF analyses. This study adopted propensity scores to incorporate the contextual variables into the gender DIF analysis. In this study, propensity scores were used to control for the contextual variables that potentially affect the gender DIF. Subsequent DIF analyses with the Mantel-Haenszel (MH) procedure and the Logistic Regression (LR) model were run with the propensity score applied reference (males) and focal groups (females) through propensity score matching. The propensity score embedded MH model and LR model detected fewer number of gender DIF than the conventional MH and LR models. The propensity score embedded models, as a confirmatory approach in DIF analysis, could contribute to hypothesizing an inference on the potential cause of DIF. Also, salient advantages of propensity score embedded DIF analysis models are discussed.     

A new procedure for detection of crossing DIF

The purpose of this paper is to present a hypothesis testing and estimation procedure, Crossing SIBTEST, for detecting crossing DIF. Crossing DIF exists when the difference in the probabilities of a correct answer for the two examinee groups changes signs as ability level is varied. In item response theory terms, crossing DIF is indicated by two crossing item characteristic curves. Our new procedure, denoted as Crossing SIBTEST, first estimates the matching subtest score at which crossing occurs using least squares regression analysis. A Crossing SIBTEST statistic then is used to test the hypothesis of crossing DIF. The performance of Crossing SIBTEST is evaluated in this study.This research was partially supported by a grant from the Law School Admission Council and by National Science Foundation Mathematics Grant NSF-DMS-94-04327. The research reported here is collaborative in every respect and the order of authorship is alphabetical. The authors thank Jeff Douglas and Louis Roussos for their useful comments and discussions.     

A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF

A model-based modification (SIBTEST) of the standardization index based upon a multidimensional IRT bias modeling approach is presented that detects and estimates DIF or item bias simultaneously for several items. A distinction between DIF and bias is proposed. SIBTEST detects bias/DIF without the usual Type 1 error inflation due to group target ability differences. In simulations, SIBTEST performs comparably to Mantel-Haenszel for the one item case. SIBTEST investigates bias/DIF for several items at the test score level (multiple item DIF called differential test functioning: DTF), thereby allowing the study of test bias/DIF, in particular bias/DIF amplification or cancellation and the cognitive bases for bias/DIF.This research was partially supported by Office of Naval Research Cognitive and Neural Sciences Grant N0014-90-J-1940, 4421-548 and National Science Foundation Mathematics Grant NSF-DMS-91-01436. The research reported here is collaborative in every respect and the order of authorship is alphabetical. The assistance of Hsin-hung Li and Louis Roussos in conducting the simulation studies was of great help. Discussions with Terry Ackerman, Paul Holland, and Louis Roussos were very helpful.     

认知诊断框架下多级评分题目的DIF检测方法及其应用

本文对多级计分认知诊断测验的DIF概念进行了界定,并通过模拟实验以及实证研究对四种常见的多级计分DIF检验方法的适用性进行理论以及实践性的探索。研究结果表明：四种方法均能对多级计分认知诊断中的DIF进行有效的检验,且各方法的表现受模型的影响不大;相较于以总分为匹配变量,以KS为匹配变量时更利于DIF的检测;以KS为匹配变量的LDFA方法以及以KS为匹配变量的曼特尔检验方法在检测DIF题目时有着最高的检验力。