多维测验项目参数的估计：基于SEM与MIRT方法的比较

作者简要回顾了SEM框架下分类数据因素分析(CCFA)模型和MIRT框架下测验题目和潜在能力的关系模型, 对两种框架下的主要参数估计方法进行了总结。通过模拟研究, 比较了SEM框架下WLSc和WLSMV估计方法与MIRT框架下MLR和MCMC估计方法的差异。研究结果表明：(1) WLSc得到参数估计的偏差最大, 且存在参数收敛的问题; (2)随着样本量增大, 各种项目参数估计的精度均提高, WLSMV方法与MLR方法得到的参数估计精度差异很小, 大多数情况下不比MCMC方法差; (3)除WLSc方法外, 随着每个维度测验题目的增多参数估计的精度逐渐增高; (4)测验维度对区分度参数和难度参数的影响较大, 而测验维度对项目因素载荷和阈值的影响相对较小; (5)项目参数的估计精度受项目测量维度数的影响, 只测量一个维度的项目参数估计精度较高。另外文章还对两种方法在实际应用中应该注意的问题提供了一些建议。     

从整合的角度看联结记忆中的项目再认

联结记忆由三种成分构成：项目1, 项目2以及项目1-项目2之间的联结, 其中, 对项目1和项目2的再认称之为项目再认, 而对项目1-项目2之间联结的再认称之为联结再认。双加工理论认为项目再认可以由熟悉性和回想加工来完成, 而联结再认只能由回想加工来完成。但近期有大量的研究发现：当要学习的项目对被整合为一个新的整体表征时, 熟悉性也能够支持联结再认。而关于整合对联结记忆中项目再认的研究较少, 总结已有研究提出两种观点：一种是“只有受益”观点(benefits-only)认为整合在增加联结再认的同时不影响项目再认; 另一种是“收支平衡”观点(costs and benefits)认为整合增加联结再认是以牺牲项目再认为代价的。未来研究应该关注整合对联结记忆中项目再认的影响及其神经机制, 了解整合对联结再认和项目再认的具体作用, 有助于针对具体记忆任务选择合适的编码方式来提高记忆表现。     

认知诊断CAT中具有非统计约束选题方法的比较

项目曝光控制和内容约束关系到测验安全、测验的信度和效度, 是计算机化自适应测验(Computerized Adaptive Testing, CAT)中两类重要的非统计约束条件。本文在认知诊断CAT中针对内容约束和项目曝光控制要求, 运用5种方法选择测验项目。它们分别是：(1) Monte Carlo方法与项目合格方法相结合, 记为MC-IE; (2) Monte Carlo方法与最大优先指标方法相结合, 记为MC-MPI; (3) Monte Carlo方法与限制阈值方法相结合, 记为MC-RT; (4) Monte Carlo方法与限制进度指标方法相结合, 记为MC-RPG以及(5) Monte Carlo方法与最大后验概率方法相结合, 记为MC-PP。然后通过在线性、收敛、发散、无结构和独立五种属性结构下构建题库并运用重参化融融统和模型模拟被试反应比较它们的选题表现。研究发现, (1) 相同选题方法在不同属性结构下项目曝光率的分布类似, 测量精度按线性、收敛、发散、无结构和独立结构的顺序依次降低; (2) 相同属性结构下, 不同方法的测量精度高低依次为MC-PP、MC-IE、MC-RT、MC-MPI和MC-RPG方法; 项目曝光均匀性优劣依次为MC-RPG、MC-MPI、MC-RT、MC-IE和MC-PP方法。统一量纲值表明, MC-RPG方法的综合表现最好, MC-MPI方法的表现次之。     

计算机适应性测验条件下认知设计项目预测参数的影响

自动化项目生成(Automatic Item Generation)中的项目参数是基于认知项目设计的刺激特征集预测的, 在不确定性来源上较之用经验数据标定的参数更为复杂。文章通过实证研究分析了在计算机适应性测验条件下基于认知设计系统法生成的抽象推理测验(ART)项目预测参数对能力参数估计的精确性。研究表明, 项目预测参数比相应标定参数分布更为趋中。这种回归效应既影响到能力参数估计误差大小, 也导致适应性测验过程中项目选择的差异。在控制了项目选择差异之后, 能力参数估计误差较之基于项目标定参数的能力估计误差大, 但差别并不明显。两者相应的能力估计值相关很高, 对应能力值之间的差异很小, 且几乎贯彻整个能力分布区间。     

Biodata Validity Decay and Score Inflation with Faking: Do Item Attributes Explain Variance Across Items?

Rating scales were developed to assess the biodata dimensions offered by Mael (1991). Biodata items assessing conscientiousness were administered under honest-responding and faking-good conditions. Item attributes were examined to determine their value in predicting item validity for honest respondents and item validity for faking respondents. Analyses were also conducted to determine whether the degree of item faking was related to item attributes. Item attributes associated with item validity for honest respondents are not the same as the item attributes indicative of item validity for the faking respondents. We suggest that this makes it very difficult to develop a biodata questionnaire which will be equally valid for both honest and faking respondents.     

A Unidimensional Latent Trait Model for Continuous Item Responses

A general linear latent trait model for continuous item responses is described. The special unidimensional case for continuous item responses is Joreskog's (1971) model of congeneric item responses. In the context of the unidimensional case model for continuous item responses the concepts of item and test information functions, specific objectivity, item bias, and reliability are discussed; also the application of the model to test construction is shown. Finally, the correspondence with latent trait theory for dichotomous item responses is discussed.     

题库结构对原始题在线属性标定准确性之影响研究

目前已有研究证明可达阵在认知诊断测验编制中起重要作用,但迄今为止并没有引起普遍注意。本文主要讨论当题库缺少某些可达阵对应的项目类,对原始题的属性向量在线标定的准确性的影响。本文对含6个属性的独立型结构进行了模拟试验,结果显示：如果题库不充要,原始题的属性标定准确性受到影响,题库中非可达阵中项目对标定有一定的弥补作用。间接印证了可达阵在认知诊断题库起到非常重要的作用。     

Does the rose still smell as sweet? Item variability across test forms and revisions

This article examines item stability when the same item appears in different contexts. The 1st section considers the assumptions in classical test theory and item response theory concerning the relationship between the item and the trait it is presumed to measure. The 2nd section presents contextualist challenges to the measurement theory assumptions about item properties and shows the instability of item characteristics across different testing contexts. The 3rd section describes methods for checking the relationship between items and traits. Classical test methods, item response methods, and structural equation methods for assessing item stability are reviewed. The instability of item characteristics across contexts should caution researchers to assess, and not assume, that items operate the same way on different test versions. Item instability also indicates the need for a more detailed understanding of the psychological processes that occur between item and answer.     

The effect of being in a tip-of-the-tongue state on subsequent items

The tip-of-the-tongue state (TOT) is the feeling that an inaccessible item will be recalled. In the TOT induction paradigm, participants are given a list of general information questions or word definitions, and the participants indicate whether they are in a TOT for each item. The present study explored the effect that being in a TOT for one item (N) has on the recall and the likelihood of a TOT for the subsequent item (N + 1). Three experiments were conducted. All three experiments showed that TOTs do not affect the rate of recall for the next item but decrease the likelihood of a TOT for the next item. This effect extended to items occurring two items after the initial TOT (N + 2) in two experiments. Thus, TOTs are less likely to occur after another TOT than after an item not in a TOT. These data are interpreted within a metacognitive framework.     

Some relations between guttman's principal components of scale analysis and other psychometric theory

Guttman's principal components for the weighting system are the item scoring weights that maximize the generalized Kuder-Richardson reliability coefficient. The principal component for any item is effectively the same as the factor loading of the item divided by the item standard deviation, the factor loadings being obtained from an ordinary factor analysis of the item intercorrelation matrix.     

The revelation effect for item and associative recognition: Familiarity versus recollection

The revelation effect occurs when items on a recognition test are more likely to be judged as being old if they are preceded by a cognitive task that involves the processing of similar types of stimuli. This effect was examined for item (single-word) and associative (word-pair) recognition. We found, in Experiments 1 and 2, a revelation effect for item, but not for associative recognition under normal study conditions. A revelation effect for both item and associative recognition was observed in Experiments 3 and 4 when study time was extremely brief, thus limiting the encoding of information that would support recall or recollection. In Experiment 5, we demonstrated that the revelation effect for item recognition is eliminated when item recognition decisions are made in the context of a study item. The results show that the revelation task influenced recognition decisions based on familiarity, but not decisions that involved recall or recollection.     

CTTITEM: SAS macro and SPSS syntax for classical item analysis

This article describes the functions of a SAS macro and an SPSS syntax that produce common statistics for conventional item analysis including Cronbach’s alpha, item difficulty index (p-value or item mean), and item discrimination indices (D-index, point biserial and biserial correlations for dichotomous items and item-total correlation for polytomous items). These programs represent an improvement over the existing SAS and SPSS item analysis routines in terms of completeness and user-friendliness. To promote routine evaluations of item qualities in instrument development of any scale, the programs are available at no charge for interested users. The program codes along with a brief user’s manual that contains instructions and examples are downloadable from suen.ed.psu.edu/~pwlei/plei.htm.     

Investigating the relationship between item exposure and test overlap: Item sharing and item pooling

To date, exposure control procedures that are designed to control item exposure and test overlap simultaneously are based on the assumption of item sharing between pairs of examinees. However, examinees may obtain test information from more than one examinee in practice. This larger scope of information sharing needs to be taken into account in refining exposure control procedures. To control item exposure and test overlap among a group of examinees larger than two, the relationship between the two indices needs to be identified first. The purpose of this paper is to analytically derive the relationships between item exposure rate and each of the two forms of test overlap, item sharing and item pooling, for fixed‐length computerized adaptive tests. Item sharing is defined as the number of common items shared by all examinees in a group, while item pooling is the number of overlapping items that an examinee has with a group of examinees. The accuracy of the derived relationships was verified using numerical examples. The relationships derived will lay the foundation for future development of procedures to simultaneously control item exposure and item sharing or item pooling among a group of examinees larger than two.     

Sensitivity of a linear composite of predictor items to differential item weighting

Many authors have demonstrated for idealized item configurations that equal item weights are often virtually as good for a particular predictive purpose as the item weights that are theoretically optimal. What has not been heretofore clear, however, is what happens to the similarity between weighted and unweighted composites of the same items when the item configuration's variance structure is complex.     

Asymmetric Item Characteristic Curves and Item Complexity: Insights from Simulation and Real Data Analyses

While item complexity is often considered as an item feature in test development, it is much less frequently attended to in the psychometric modeling of test items. Prior work suggests that item complexity may manifest through asymmetry in item characteristics curves (ICCs; Samejima in Psychometrika 65:319–335, 2000). In the current paper, we study the potential for asymmetric IRT models to inform empirically about underlying item complexity, and thus the potential value of asymmetric models as tools for item validation. Both simulation and real data studies are presented. Some psychometric consequences of ignoring asymmetry, as well as potential strategies for more effective estimation of asymmetry, are considered in discussion.     

兼顾测验效率和题库使用率的CD-CAT选题策略

CD–CAT中已有选题策略较注重测验效率,而对题库使用率不够重视。针对此问题,基于DINA模型,引入两种新的选题策略KLED和RHA,同时对HA进行模拟研究。结果显示：PWKL与KLED只在测验效率上具有优势;KLED若按属性向量分层,题库使用率有所提高,KLED比ED更容易推广到其他有显式表达的诊断模型场合;HA、RHA和RP–PWKL可较好兼顾测验效度和题库使用率,但RP-PWKL需设置项目的最大曝光率阈值。两种新选题方法在定长和变长CD-CAT都具有一定的应用价值。     

Conjunctive item response theory kernels

Conjunctive item response models are introduced such that (a) sufficient statistics for latent traits are not necessarily additive in item scores; (b) items are not necessarily locally independent; and (c) existing compensatory (additive) item response models including the binomial, Rasch, logistic, and general locally independent model are special cases. Simple estimates and hypothesis tests for conjunctive models are introduced and evaluated as well. Conjunctive models are also identified with cognitive models that assume the existence of several individually necessary component processes for a global ability. It is concluded that conjunctive models and methods may show promise for constructing improved tests and uncovering conjunctive cognitive structure. It is also concluded that conjunctive item response theory may help to clarify the relationships between local dependence, multidimensionality, and item response function form.I appreciate the many helpful suggestions that were given by the reviewers and Ivo Molenaar.     

粗糙集在项目认知属性标定中的应用

认知诊断是新一代测量理论的核心, 对形成性教学评估具有重要意义。项目认知属性标定是认知诊断中一项基础而重要的工作,现有的项目认知属性辅助标定方法的研究工作很少, 并且在应用上存在诸多局限。课堂评估是认知诊断应用的理想场所,但课堂评估中项目的选取具有随意性, 教师难以在短时间内准确标识项目认知属性。本研究首次提出采用粗糙集方法对项目认知属性进行标定, 该方法无需太多被试和项目, 亦无需已知项目参数, 且能当场诊断出结果, 适于采用纸笔测验的课堂评估。通过Monte Carlo模拟研究表明：采用粗糙集方法能迅速地对项目认知属性进行标定, 并具有较高的标定准确率; 而且, 项目认知属性越少、或被试估计判准率越高、或失误率越小则项目认知属性标定的准确率越高。粗糙集方法的引入, 对拓展认知诊断的应用范围, 真正实现其辅助性教学功能, 具有重要作用。     

Consequences of Ignoring Guessing Effects on Measurement Invariance Analysis

Pseudo-guessing parameters are present in item response theory applications for many educational assessments. When sample size is not sufficiently large, the guessing parameters may be ignored from the analysis. This study examines the impact of ignoring pseudo-guessing parameters on measurement invariance analysis, specifically, on item difficulty, item discrimination, and mean and variance of ability distribution. Results show that when non-zero guessing parameters are ignored from the measurement invariance analysis, item discrimination estimates tend to decrease particularly for more difficult items, and item difficulty estimates decrease unless the items are highly discriminating and difficult. As the guessing parameter increases, the size of the decrease in item discrimination and difficulty tends to increase, and the estimated mean and variance of ability distribution tend to be inaccurate. When two groups have heterogeneous ability distributions, ignoring the guessing parameter affects the reference group and the focal group differently. Implications of result findings are discussed.     

不同认知成分在图形推理测验项目难度预测中的作用

理解项目难度变异的来源是实现计算机自动化项目生成的第一步。通过文献综述, 总结出影响图形推理测验项目难度的四个方面的因素, 再通过操控构图元素熟悉性、属性的抽象性、知觉组织的和谐性以及规则类型与数目这些因素, 编制8套图形推理测验, 共包含112个与高级瑞文推理类似的项目。采用铆测验等值设计, 在每套测验中嵌入10个高级瑞文推理测验项目为铆题, 通过网络施测于6323名被试。使用BILOG MG估算项目参数, 并使用IRTEQ进行测验等值, 将后七套测验上所有项目的项目参数都转换到第一套测验的单位系统上。以项目难度为因变量, 项目题干特征变量为预测变量进行回归分析, 结果发现这四个因素均对项目难度有显著预测作用。优势分析的结果显示记忆负荷(即规则类型与数目的组合)是项目难度的最重要的预测变量, 其他依次为属性的抽象性、知觉组织的和谐性和构图元素熟悉性。