A general proof of consistency of heuristic classification for cognitive diagnosis models

The Asymptotic Classification Theory of Cognitive Diagnosis (Chiu et al., 2009, Psychometrika, 74, 633–665) determined the conditions that cognitive diagnosis models must satisfy so that the correct assignment of examinees to proficiency classes is guaranteed when non‐parametric classification methods are used. These conditions have only been proven for the Deterministic Input Noisy Output AND gate model. For other cognitive diagnosis models, no theoretical legitimization exists for using non‐parametric classification techniques for assigning examinees to proficiency classes. The specific statistical properties of different cognitive diagnosis models require tailored proofs of the conditions of the Asymptotic Classification Theory of Cognitive Diagnosis for each individual model – a tedious undertaking in light of the numerous models presented in the literature. In this paper a different way is presented to address this task. The unified mathematical framework of general cognitive diagnosis models is used as a theoretical basis for a general proof that under mild regularity conditions any cognitive diagnosis model is covered by the Asymptotic Classification Theory of Cognitive Diagnosis.     

Validity of the German Test Anxiety Inventory (TAI‐G) in an Australian sample

Test anxiety (TA) is a prevalent issue among students that can result in deleterious consequences, such as underachievement. However, a contemporary measure that has been validated for use with Australian students seems to be lacking. This study, therefore, investigated the suitability of the German Test Anxiety Inventory (TAI‐G) for use with Australian university students. While the original TAI‐G contains 30 items and was designed to measure four factors (worry, emotionality, interference, and lack of confidence), differing factorial models have been supported in the literature using either the original or a shortened 17‐item version of the measure. These differing TAI‐G models were tested and compared in the current study via confirmatory factor analysis using 224 Australian university students. As expected, results supported the superior fit of the 17‐item four‐factor model. Additionally, the convergent validity of the measure was supported since measures of self‐esteem, self‐efficacy, and general anxiety were all found to correlate significantly with the TAI‐G in the hypothesised directions. Finally, the finding that all of the TAI‐G subscales had acceptably high reliabilities led to the conclusion that the 17‐item TAI‐G is a valid and reliable measure of TA in an Australian university population.     

Empirical Priors in Polytomous Computerized Adaptive Tests: Risks and Rewards in Clinical Settings

The use of empirical prior information about participants has been shown to substantially improve the efficiency of computerized adaptive tests (CATs) in educational settings. However, it is unclear how these results translate to clinical settings, where small item banks with highly informative polytomous items often lead to very short CATs. We explored the risks and rewards of using prior information in CAT in two simulation studies, rooted in applied clinical examples. In the first simulation, prior precision and bias in the prior location were manipulated independently. Our results show that a precise personalized prior can meaningfully increase CAT efficiency. However, this reward comes with the potential risk of overconfidence in wrong empirical information (i.e., using a precise severely biased prior), which can lead to unnecessarily long tests, or severely biased estimates. The latter risk can be mitigated by setting a minimum number of items that are to be administered during the CAT, or by setting a less precise prior; be it at the expense of canceling out any efficiency gains. The second simulation, with more realistic bias and precision combinations in the empirical prior, places the prevalence of the potential risks in context. With similar estimation bias, an empirical prior reduced CAT test length, compared to a standard normal prior, in 68% of cases, by a median of 20%; while test length increased in only 3% of cases. The use of prior information in CAT seems to be a feasible and simple method to reduce test burden for patients and clinical practitioners alike.     

Evaluating Equating Transformations in IRT Observed-Score and Kernel Equating Methods

Test equating is a statistical procedure to ensure that scores from different test forms can be used interchangeably. There are several methodologies available to perform equating, some of which are based on the Classical Test Theory (CTT) framework and others are based on the Item Response Theory (IRT) framework. This article compares equating transformations originated from three different frameworks, namely IRT Observed-Score Equating (IRTOSE), Kernel Equating (KE), and IRT Kernel Equating (IRTKE). The comparisons were made under different data-generating scenarios, which include the development of a novel data-generation procedure that allows the simulation of test data without relying on IRT parameters while still providing control over some test score properties such as distribution skewness and item difficulty. Our results suggest that IRT methods tend to provide better results than KE even when the data are not generated from IRT processes. KE might be able to provide satisfactory results if a proper pre-smoothing solution can be found, while also being much faster than IRT methods. For daily applications, we recommend observing the sensibility of the results to the equating method, minding the importance of good model fit and meeting the assumptions of the framework.     

Quantitative assessment of drawing tests in children with dyslexia and dysgraphia

Drawing tests in children diagnosed with dyslexia and dysgraphia were quantitatively compared. Fourteen children with dysgraphia, 19 with dyslexia and 13 normally developing were asked to copy 3 figures: a circle, a square and a cross. An optoelectronic system allowed the acquisition of the drawing track in three-dimensions. The participants’ head position and upper limb movements were measured as well. A set of parameters including movement duration, velocity, length of the trace, Range of Motion of the upper limb, was computed and compared among the 3 groups. Children with dyslexia traced the circle faster than the other groups. In the cross test, dyslexic participants showed a reduced execution time and increased velocity while drawing the horizontal line. Children with dyslexia were also faster in drawing certain sides of square with respect to the other groups.     

Diagnostic Classification Models for a Mixture of Ordered and Non-ordered Response Options in Rating Scales

When developing ordinal rating scales, we may include potentially unordered response options such as “Neither Agree nor Disagree,” “Neutral,” “Don’t Know,” “No Opinion,” or “Hard to Say.” To handle responses to a mixture of ordered and unordered options, Huggins-Manley et al. (2018) proposed a class of semi-ordered models under the unidimensional item response theory framework. This study extends the concept of semi-ordered models into the area of diagnostic classification models. Specifically, we propose a flexible framework of semi-ordered DCMs that accommodates most earlier DCMs and allows for analyzing the relationship between those potentially unordered responses and the measured traits. Results from an operational study and two simulation studies show that the proposed framework can incorporate both ordered and non-ordered responses into the estimation of the latent traits and thus provide useful information about both the items and the respondents.     

初级军官心理选拔的预测性

目的：通过建立院校学员胜任特征及初级军官评价模型,探讨初级军官心理选拔检测系统的预测性。方法：某军校340名男性本科学员入校时完成初级军官心理选拔检测,毕业前进行院校胜任特征评价,对162名学员毕业后1~5年进行部队工作表现追踪评价。结果：中国MBTI-G人格类型测验对院校学员管理能力、影响力、约束力、口头表达、军人气质、人际关系等有较好的预测性,预测符合率72.13%;中国军人明尼苏达个性调查表6项精神障碍维度对情绪稳定性有较好的预测,预测符合率71.90%;院校学员一般能力倾向测验对学业成绩、文字表达、口头表达、人际关系和决策能力等胜任特征也有一定的预测性,预测符合率66.70%;三项测验院校胜任特征总预测符合率达81.70%,部队胜任特征总预测符合率76~87%。结论：为我军初级军官心理选拔提供了实用工具,为提高心理选拔预测准确性提供了科学依据     

基于项目区分度的双目标CD-CAT选题策略

针对双目标CD-CAT,将六种项目区分度（鉴别力D、一般区分度GDI、优势比OR、2PL的区分度a、属性区分度ADI、认知诊断区分度CDI）分别与IPA方法结合,得到新的选题策略。模拟研究比较了它们的表现,还考察了区分度分层在控制项目曝光的表现。结果发现：新方法都能明显提高知识状态的判准率和能力估计精度;分层选题均能很好地提高题库利用率。总体上,OR加权能显著提高测量精度;OR分层选题在保证测量精度条件下显著提高项目曝光均匀性。     

不同铆测验设计下多维IRT等值方法的比较

实际应用中测验往往具有多维结构, 如果仍采用单维IRT方法进行等值, 会得到不准确的结果。因此对于多维结构的测验, 需要使用多维IRT等值方法来实现参数的转换。基于共同题设计, 文章通过模拟研究的方法, 考察了不同铆测验设计下几种多维IRT等值方法的表现, 同时考虑了测验长度、两个维度题目数量的比例、铆测验长度、铆测验的选择策略、两个维度之间的相关和等值群体的能力水平差异六个因素的影响。所比较的多维IRT等值方法有：均值/均值(MM)方法, 均值/标准差(MS)方法, Stoking-Lord (SL)方法, Haebara (HB)方法, 最小平方(LS)方法。结果显示：(1) SL, HB和LS方法得到的等值误差均方根最小, 且在各条件下表现较为稳定。(2) MM和MS方法在非等组条件下呈现出很大的误差均方根。(3)铆测验设计对SL, HB和LS方法的等值结果没有显著影响。(4)在两个维度之间的相关较高, 测验长度和铆测验长度较长, 等值群体的能力水平没有差异的条件下, SL, HB和LS方法得到的等值误差均方根最小。