Invariance on a reading comprehension test in European Portuguese: A differential item functioning analysis between students from rural and urban areas

The aim of this study was to determine whether the items from a reading comprehension test in European Portuguese function differently across students from rural and urban areas, which biases the test validity and the equity in assessment. The sample was composed of 653 students from second, third and fourth grades. The presence of differential item functioning (DIF) was analysed using logistic regression and the Mantel–Haenszel procedure. Although 17 items were flagged with DIF, only five items showed non-negligible DIF in all effect-size measures. The evidence of invariance across students with rural or urban backgrounds for most of the items supports the validity of the test though the five identified items should be further investigated.     

The bidirectional relation between positive mental health and psychopathology in a longitudinal representative panel study

Background: There is accumulating evidence that positive mental health and psychopathology should be seen as separate indicators of mental health. This study contributes to this evidence by investigating the bidirectional relation between positive mental health and psychopathological symptoms over time. Methods: Positive mental health (MHC-SF) and psychopathological symptoms (BSI) were longitudinally measured in a representative adult sample (N?=?1932) on four measurement occasions in nine months. A cross-lagged panel design was applied and evaluated with a latent growth model combined with an item response theory measurement model. Results: Psychopathological symptoms were longitudinally related to positive mental health and vice versa, controlling for initial levels. The changes over time were even more important than the absolute levels of psychopathological symptoms and positive mental health, respectively. Conclusions: The results underline the need for a comprehensive perspective on mental health, incorporating both the treatment of symptoms and the enhancement of well-being.     

融合反应时的多级评分IRT模型开发及其应用研究

当前大多数融合反应时的IRT模型仅适用于0-1评分数据资料,极大的限制了IRT反应时模型在实际中的应用。本文在传统的二级计分反应时IRT模型基础上,拟开发一种多级评分反应时模型。在层次建模框架下,分别采用拓广分部评分模型(GPCM)和对数正态模型构建融合反应时的多级评分IRT模型(本文记为JRT-GPCM),并采用全息贝叶斯MCMC算法实现新模型的参数估计。为验证新开发的JRT-GPCM模型的可行性及其在实践中的应用,本文开展了两项研究:研究1为模拟实验研究,研究2为新模型在大五人格-神经质分量表中的应用。研究1结果表明,JRT-GPCM模型的估计精度较高,且具有较好的稳健性。研究2表明,被试的潜在特质与作答速度具有一定的正相关,且本研究结果支持Ferrando和Lorenzo-Seva(2007)提出的“距离-困难度假设”,即当被试的潜在特质与项目的难度阈限距离越远,那么被试会花费更多的时间对项目进行作答。总之,本研究为拓展反应时信息在心理测量及教育中的应用提供新的方法支持。     

年龄、焦虑与情节记忆之内容与来源的关系

将情节记忆区分为内容记忆和来源记忆,筛选年老和年轻的高、低特质焦虑者完成记忆任务,以探讨情节记忆年老化的选择性和变异性及其与焦虑之间的联系。两项实验结果表明来源记忆是否随增龄而有选择性受损受来源性质影响,与来源呈现方式和来源识记意识性无关。焦虑干扰了老年组的记忆操作,是老年人情节记忆存在个体差异的原因之一;但焦虑对青年组没有不利影响,提示焦虑调节两类记忆的年龄差异。     

A Biodata Inventory Administered via Interactive Voice Response (IVR) Technology: Predictive Validity, Utility, and Subgroup Differences

Although organizations are increasingly using interactive voice response (IVR) systems to help screen and select job applicants, few if any published studies have investigated the effectiveness of such systems. The current study responded to this need by evaluating a biodata inventory administered via IVR technology. A predictive validity study revealed that both a 42-item biodata measure and a more parsimonious 6-item composite correlated with supervisor ratings of performance and did not result in adverse impact or differential prediction. In addition, a utility analysis indicated that both IVR biodata measures would likely result in considerable financial gains to the hiring organization. The implications of these results for selection research and practice are discussed.     

A test for conditional independence between response time and accuracy

An important distinction between different models for response time and accuracy is whether conditional independence (CI) between response time and accuracy is assumed. In the present study, a test for CI given an exponential family model for accuracy (for example, the Rasch model or the one‐parameter logistic model) is proposed and evaluated in a simulation study. The procedure is based on the non‐parametric Kolmogorov–Smirnov tests. As an illustrative example, the CI test was applied to data from an arithmetics test for secondary education.     

Detection of differential item functioning in Rasch models by boosting techniques

Methods for the identification of differential item functioning (DIF) in Rasch models are typically restricted to the case of two subgroups. A boosting algorithm is proposed that is able to handle the more general setting where DIF can be induced by several covariates at the same time. The covariates can be both continuous and (multi‐)categorical, and interactions between covariates can also be considered. The method works for a general parametric model for DIF in Rasch models. Since the boosting algorithm selects variables automatically, it is able to detect the items which induce DIF. It is demonstrated that boosting competes well with traditional methods in the case of subgroups. The method is illustrated by an extensive simulation study and an application to real data.     

The Development and Testing of a New Version of the Cognitive Reflection Test Applying Item Response Theory (IRT)

The cognitive reflection test (CRT) is a short measure of a person's ability to resist intuitive response tendencies and to produce a normatively correct response, which is based on effortful reasoning. Although the CRT is a very popular measure, its psychometric properties have not been extensively investigated. A major limitation of the CRT is the difficulty of the items, which can lead to floor effects in populations other than highly educated adults. The present study aimed at investigating the psychometric properties of the CRT applying item response theory analyses (a two‐parameter logistic model) and at developing a new version of the scale (the CRT‐long), which is appropriate for participants with both lower and higher levels of cognitive reflection. The results demonstrated the good psychometric properties of the original, as well as the new scale. The validity of the new scale was also assessed by measuring correlations with various indicators of intelligence, numeracy, reasoning and decision‐making skills, and thinking dispositions. Moreover, we present evidence for the suitability of the new scale to be used with developmental samples. Finally, by comparing the performance of adolescents and young adults on the CRT and CRT‐long, we report the first investigation into the development of cognitive reflection. Copyright © 2015 John Wiley & Sons, Ltd.     

Developing Pairwise Preference-Based Personality Test and Experimental Investigation of Its Resistance to Faking Effect by Item Response Model

Recent years have shown increased awareness of the importance of personality tests in educational, clinical, and occupational settings, and developing faking-resistant personality tests is a very pragmatic issue for achieving more precise measurement. Inspired by Stark (2002) and Stark, Chernyshenko, and Drasgow (2005), we develop a pairwise preference-based personality test that aims to measure multidimensional personality traits using a large-scale statement bank. An experiment compares the resistance of the developed personality test to faking with that of rating scale-based personality tests in the item response theory model framework. Results show that latent traits estimated from the personality test based on the rating scale method are severely biased, and that faking effect can be pragmatically ignored in the personality test developed based on the pairwise preference method.     

Optimal designs for the generalized partial credit model

Analysing ordinal data is becoming increasingly important in psychology, especially in the context of item response theory. The generalized partial credit model (GPCM) is probably the most widely used ordinal model and has found application in many large-scale educational assessment studies such as PISA. In the present paper, optimal test designs are investigated for estimating persons’ abilities with the GPCM for calibrated tests when item parameters are known from previous studies. We find that local optimality may be achieved by assigning non-zero probability only to the first and last categories independently of a person's ability. That is, when using such a design, the GPCM reduces to the dichotomous two-parameter logistic (2PL) model. Since locally optimal designs require the true ability to be known, we consider alternative Bayesian design criteria using weight distributions over the ability parameter space. For symmetric weight distributions, we derive necessary conditions for the optimal one-point design of two response categories to be Bayes optimal. Furthermore, we discuss examples of common symmetric weight distributions and investigate under what circumstances the necessary conditions are also sufficient. Since the 2PL model is a special case of the GPCM, all of these results hold for the 2PL model as well.