Reliability coefficients for multiple group item response theory models

Reliability of scores from psychological or educational assessments provides important information regarding the precision of measurement. The reliability of scores is however population dependent and may vary across groups. In item response theory, this population dependence can be attributed to differential item functioning or to differences in the latent distributions between groups and needs to be accounted for when estimating the reliability of scores for different groups. Here, we introduce group-specific and overall reliability coefficients for sum scores and maximum likelihood ability estimates defined by a multiple group item response theory model. We derive confidence intervals using asymptotic theory and evaluate the empirical properties of estimators and the confidence intervals in a simulation study. The results show that the estimators are largely unbiased and that the confidence intervals are accurate with moderately large sample sizes. We exemplify the approach with the Montreal Cognitive Assessment (MoCA) in two groups defined by education level and give recommendations for applied work.     

认知诊断模型的标准误与置信区间估计：并行自助法

认知诊断模型的标准误(Standard Error, SE; 或方差—协方差矩阵)与置信区间(Confidence Interval, CI)在模型参数估计不确定性的度量、项目功能差异检验、项目水平上的模型比较、Q矩阵检验以及探索属性层级关系等领域有重要的理论与实践价值。本研究提出了两种新的SE和CI计算方法：并行参数化自助法和并行非参数化自助法。模拟研究发现：模型完全正确设定时, 在高质量及中等质量项目条件下, 这两种方法在计算模型参数的SE和CI时均有好的表现; 模型参数存在冗余时, 在高质量及中等质量项目条件下, 对于大部分允许存在的模型参数而言, 其SE和CI有好的表现。通过实证数据展示了新方法的价值及计算效率提升效果。     

Identifying the Effects of Unjustified Confidence versus Overconfidence: Lessons Learned from Two Analytic Methods

One of the most common findings in behavioral decision research is that people have unrealistic beliefs about how much they know. However, demonstrating that misplaced confidence exists does not necessarily mean that there are costs to it. This paper contrasts two approaches toward answering whether misplaced confidence is good or bad, which we have labeled the overconfidence and unjustified confidence approach. We first consider conceptual and analytic issues distinguishing these approaches. Then, we provide findings from a set of simulations designed to determine when the approaches produce different conclusions across a range of possible confidence–knowledge–outcome relationships. Finally, we illustrate the main findings from the simulations with three empirical examples drawn from our own data. We conclude that the unjustified confidence approach is typically the preferred approach, both because it is appropriate for testing a larger set of psychological mechanisms as well as for methodological reasons. Copyright © 2013 John Wiley & Sons, Ltd.     

Intrinsic goals for leisure-time physical activity predict children's daily step counts through autonomous motivation

ObjectivesThe present study tested a motivational sequence in which children's goals for leisure-time sport and physical activity (PA) from Goal Content Theory predicted pedometer-determined PA through behavioural regulation constructs from Self-Determination Theory.DesignQuantitative cross-sectional study.MethodsA sample of 1272 Flemish (Dutch speaking part of Belgium) sixth graders (M age = 11.42 years) filled in a questionnaire on PA goal content, behavioural regulations and pubertal status. Children's height and weight were measured to assess Body Mass Index (BMI). To measure daily PA behaviour, participants wore a pedometer for seven consecutive days. Structural equation modelling was performed to test the proposed motivational sequence and to examine the mediation effect of behavioural regulations towards leisure-time sport and PA on the relation between PA goal content and daily step counts.ResultsA structural equation model supported the hypothesized sequence in which intrinsic goals for leisure-time sport and PA predicted children's daily step counts through autonomous motivation towards leisure-time sport and PA. Multi-sample invariance testing revealed that the proposed model was largely invariant across BMI groups, pubertal status and gender.ConclusionsThe findings highlight the importance of emphasizing intrinsic goals for participation in leisure-time sport and PA in children aged 10–12 years. By stimulating children to participate in PA and sports by referring to intrinsic goals, children are more likely to enjoy PA, which in turn might increase the likelihood of a regular and long-term PA engagement.     

Inhibitory mechanisms in single negative priming from ignored and briefly flashed primes: The key role of the inter-stimulus interval

The influence of interstimulus intervals (ISIs) on priming effects was investigated using a single-prime negative priming (NP) paradigm. In all experiments, a brief (16 ms), centrally displayed prime (a Chinese character, to be ignored) appeared, followed by a pattern mask and then a centrally displayed target (another semantically related or unrelated Chinese character); the task required semantic categorization (animate/inanimate) of the target. An ISI could occur either between prime and mask (Experiments 1 and 5) or between mask and target (Experiments 2–4). The results revealed NP when a 470 ms ISI occurred between prime and mask (Experiments 1 and 5) and when a 700 ms ISI occurred between mask and target (Experiments 3 and 4). In contrast, when a long prime-target SOA was maintained but the mask-target ISI was shortened, NP disappeared (Experiment 4). The results indicated that a persisting mask/distractor (without ISI) located in the same position as the following target interfered with the buildup of inhibition, but an ISI between prime and mask or mask and target eliminated this interference, and that inhibition processes induced by an ignore instruction were implemented faster with an ISI placed between prime and mask than with an ISI placed between mask and target.     

The Role of Individual Differences in the Accuracy of Confidence Judgments

Generally, self-assessment of accuracy in the cognitive domain produces overconfidence, whereas self-assessment of visual perceptual judgments results in under-confidence. Despite contrary empirical evidence, in models attempting to explain those phenomena, individual differences have often been disregarded. The authors report on 2 studies in which that shortcoming was addressed. In Experiment 1, participants (N = 520) completed a large number of cognitive-ability tests. Results indicated that individual differences provide a meaningful source of overconfidence and that a metacognitive trait might mediate that effect. In further analysis, there was only a relatively small correlation between test accuracy and confidence bias. In Experiment 2 (N = 107 participants), both perceptual and cognitive ability tests were included, along with measures of personality. Results again indicated the presence of a confidence factor that transcended the nature of the testing vehicle. Furthermore, a small relationship was found between that factor and some self-reported personality measures. Thus, personality traits and cognitive ability appeared to play only a small role in determining the accuracy of self-assessment. Collectively, the present results suggest that there are multiple causes of miscalibration, which current models of over- and underconfidence fail to encompass.     

An Event History Analysis of First-Term Soldier Attrition

The study of attrition has largely focused on identifying demographic or biographic characteristics that predict whether soldiers complete their enlistment term. As a result, much is known about who attrites. Less is known about why soldiers attrite. This research assessed the influence of two psychological factors on U.S. Army soldier attrition: self-reported confidence that one could complete one’s term of service and ambivalence regarding the decision to enlist. The study sample consisted of first-term enlisted soldiers (N = 14,808) who were respondents to Army surveys. Results of fitting three longitudinal models indicated that confidence in being able to complete one’s term of obligation was more predictive of attrition for those reporting greater ambivalence regarding the decision to enlist. This effect was significant throughout a 3-year period. This work adds to our understanding of attrition by highlighting the role of confidence and ambivalence. Implications for strategies to reduce attrition are discussed.     

α系数的区间估计方法比较

多数情况下,α系数可以用来评价测验信度。诸多研究建议,在报告测验信度的时候应当包括其置信区间。通过蒙特卡洛模拟研究,比较了7种α系数区间估计方法,包括Fisher法、Bonett-02法、Bonett-10法、精确Koning-Franses法、渐近ID法、渐近Koning-Franses法和ADF法。结果发现Bonett-10法和精确Koning-Franses法较好,它们的结果相差很小。这两种方法都比较简单,只需要样本的α值、测验题数、被试人数及F临界值,通过简单的运算便可得到α系数的置信区间。