Dissecting the expanded cognitive reflection test: an item response theory analysis

This study presents a psychometric evaluation of the Expanded Cognitive Reflection Test (CRT7) based on item response theory. The participants (N?=?1204) completed the CRT7 and provided self-reported information about their cognitive styles through the Preference for Intuition and Deliberation Scale (PID). A two-parameter logistic model was fitted to the data to obtain the item difficulty and discrimination parameters of the CRT7. The results showed that the items had good discriminatory power (αs?=?.80 ? 2.92), but the range of difficulty was restricted (βs ranged from ?.60 to .32). Moreover, the CRT7 showed a pattern of correlations with the PID which was similar to that of the original CRT. When taken together, these results are evidence of the adequacy of the CRT7 as an expanded tool for measuring cognitive reflection; however, one of the newer items (the pig item) was consistently problematic across analyses, and so it is recommended that in future studies it should be removed from the CRT7.     

应用Rasch模型分析Rosenberg自尊量表

采用Rosenberg自尊量表(RSES)对425名在校大学生进行施测,应用项目反应理论的Rasch模型对项目指标进行分析及DIF检验。结果表明,Rosenberg自尊量表具有单维性,量表的信度为0.84; 除项目8以外,其他项目拟合指标良好,较适用来区分中等及偏低自尊水平的个体,项目功能差异检验发现在项目1和项目5上存在DIF,表现为男生自尊水平要高于女生。相对于经典测量理论,应用Rasch模型分析Rosenberg自尊量表具有优势,为进一步的完善和使用该自尊量表提供依据。     

四参数Logistic模型和传统模型对被试作答拟合能力的比较研究

针对测验中高能力被试答错容易试题的睡眠现象,可使用四参数Logistic模型分析数据。研究选取了来自心理测验和成就测验的实际数据,分别采用传统模型和四参数Logistic模型进行拟合,对不同模型的拟合指标及参数估计结果进行比较。结果表明,四参数Logistic模型能够提高拟合程度,增强估计结果的准确性,有效纠正高能力被试能力被低估的现象。建议在必要时使用四参数Logistic模型进行数据分析。     

考虑题目选项信息的非参数认知诊断计算机自适应测验

选择题中的作答选项能提供额外诊断信息, 为充分利用选项信息, 研究提出认知诊断计算机自适应测验(CD-CAT)中两种处理选择题选项信息的非参数选题策略和变长终止规则。模拟研究的结果发现：(1)定长条件下两种非参数选题策略的分类准确性整体要高于参数选题策略; (2)两种非参数选题策略较参数选题策略具有更加均衡的题库使用情况; (3)非参数选题策略在两种新的变长终止规则下具有更高的分类准确率; (4)两种非参数选题策略均适用于选择题CD-CAT情境, 使用者可任选其一进行测验分析。     

认知诊断框架下多级评分题目的DIF检测方法及其应用

本文对多级计分认知诊断测验的DIF概念进行了界定,并通过模拟实验以及实证研究对四种常见的多级计分DIF检验方法的适用性进行理论以及实践性的探索。研究结果表明：四种方法均能对多级计分认知诊断中的DIF进行有效的检验,且各方法的表现受模型的影响不大;相较于以总分为匹配变量,以KS为匹配变量时更利于DIF的检测;以KS为匹配变量的LDFA方法以及以KS为匹配变量的曼特尔检验方法在检测DIF题目时有着最高的检验力。     

The Mental Vulnerability Questionnaire: A psychometric evaluation

Eplov, L.F., Petersen, J., Jørgensen, T., Johansen, C., Birket‐Smith, M., Lyngberg, A. C. & Mortensen, E. L. (2010). The Mental Vulnerability Questionnaire: A psychometric evaluation. Scandinavian Journal of Psychology 51, 548–554. The Mental Vulnerability Questionnaire was originally a 22 item scale, later reduced to a 12 item scale. In population studies the 12 item scale has been a significant predictor of health and illness. The scale has not been psychometrically evaluated for more than 30 years, and the aim of the present study was both to evaluate the psychometric properties of the 22 and 12 item scales and of three new scales. The main study sample was a community sample comprising more than 6,000 men and women. In this sample the coefficients of homogeneity were all over 0.30 for the three new scales, but below 0.30 for the 12 and the 22 item scales. All five Mental Vulnerability scales had positively skewed score distributions which were associated significantly with both SCL‐90‐R symptom scores and NEO‐PI‐R personality scales (primarily Neuroticism and Extraversion). Coefficient alpha was highest for the 22 and 12 item scales, and the two scales also showed the highest long‐term stability. The three new scales reflect relatively independent dimensions of Psychosomatic Symptoms, Mental Symptoms, and Interpersonal Problems, but because of reliability problems it remains an open question whether they will prove useful as predictors of health and morbidity.     

Comparison of classical and modern methods for measuring and correcting for acquiescence

Likert-type self-report scales are frequently used in large-scale educational assessment of social-emotional skills. Self-report scales rely on the assumption that their items elicit information only about the trait they are supposed to measure. However, different response biases may threaten this assumption. Specifically, in children, the response style of acquiescence is an important source of systematic error. Balanced scales, including an equal number of positively and negatively keyed items, have been proposed as a solution to control for acquiescence, but the reasons why this design feature worked from the perspective of modern psychometric models have been underexplored. Three methods for controlling for acquiescence are compared: classical method by partialling out the mean; an item response theory method to measure differential person functioning (DPF); and multidimensional item response theory (MIRT) with random intercept. Comparative analyses are conducted on simulated ratings and on self-ratings provided by 40,649 students (aged 11–18) on a fully balanced 30-item scale assessing conscientious self-management. Acquiescence bias was explained as DPF and it was demonstrated that: the acquiescence index is highly related to DPF; balanced scales produce scores controlled for DPF; and MIRT factor scores are highly related to scores controlled for DPF and the random intercept is highly related to DPF.     

Cognitive diagnosis models for multiple strategies

Cognitive diagnosis models (CDMs) have been used as psychometric tools in educational assessments to estimate students’ proficiency profiles. However, most CDMs assume that all students adopt the same strategy when approaching problems in an assessment, which may not be the case in practice. This study develops a generalized multiple-strategy CDM for dichotomous response data. The proposed model provides a unified framework to accommodate various condensation rules (e.g., conjunctive, disjunctive, and additive) and different strategy selection approaches (i.e., probability-matching, over-matching, and maximizing). Model parameters are estimated using the marginal maximum likelihood estimation via expectation-maximization algorithm. Simulation studies showed that the parameters of the proposed model can be adequately recovered and that the proposed model was relatively robust to some types of model misspecifications. A set of real data was analysed as well to illustrate the use of the proposed model in practice.     

Does cross‐frequency phase coupling of oscillatory brain activity contribute to a better understanding of visual working memory?

Nesting of fast rhythmical brain activity (gamma) into slower brain waves (theta) has frequently been suggested as a core mechanism of multi‐item working memory (WM) retention. It provides a better understanding of WM capacity limitations, and, as we discuss in this review article, it can lead to applications for modulating memory capacity. However, could cross‐frequency coupling of brain oscillations also constructively contribute to a better understanding of the neuronal signatures of working memory compatible with theoretical approaches that assume flexible capacity limits? Could a theta‐gamma code also be considered as a neural mechanism of flexible sharing of cognitive resources between memory representations in multi‐item WM? Here, we propose potential variants of theta‐gamma coupling that could explain WM retention beyond a fixed memory capacity limit of a few visual items. Moreover, we suggest how to empirically test these predictions in the future.     

Assessing the validity of inferences from scores on the cognitive reflection test

Decision‐making researchers purport that a novel cognitive ability construct, cognitive reflection, explains variance in intuitive thinking processes that traditional mental ability constructs do not. However, researchers have questioned the validity of the primary measure because of poor construct conceptualization and lack of validity studies. Prior studies have not adequately aligned the analytical techniques with the theoretical basis of the construct, dual‐processing theory of reasoning. The present study assessed the validity of inferences drawn from the cognitive reflection test (CRT) scores. We analyzed response processes with an item response tree model, a method that aligns with the dual‐processing theory in order to interpret CRT scores. Findings indicate that the intuitive and reflective factors that the test purportedly measures were indistinguishable. Exploratory, post hoc analyses demonstrate that CRT scores are most likely capturing mental abilities. We suggest that future researchers recognize and distinguish between individual differences in cognitive abilities and cognitive processes.