Correlation and prediction in ordinal test theory

Based on the test theory model for ordinal measurements proposed by Schulman and Haden, the present paper considers correlations between tests, attenuation, regressions involving true and observed scores, and prediction of test reliability.The population correlation between tests is shown to be related to the expected sample correlation for samples of sizen ₁ andn ₂. Errors of estimation, measurement and prediction are found to be similar to their counterparts in interval test theory, while attenuation is identical to its counterpart. The bias in estimating population reliability from sample data is compared for Kendall's tau and Spearman's rho.The author wishes to thank the referees for their helpful comments on an earlier draft of this paper, and in particular, for the suggested alternative methods of establishing some of the presented results.     

The impacts of ignoring individual mobility across clusters in estimating a piecewise growth model

A three-level piecewise growth model (3L-PGM) can be used to break up nonlinear growth into multiple components, providing the opportunity to examine potential sources of variation in individual and contextual growth within different segments of the model. The conventional 3L-PGM assumes that the data are strictly hierarchical in nature, where measurement occasions (level 1) are nested within individuals (level 2) who are members of a single cluster (level 3). However, in longitudinal research, it is sometimes difficult for data structures to remain purely clustered during a study, such as when some students change classrooms or schools over time. One resulting data structure in this situation is known as a multiple membership structure, where some lower-level units are members of more than one higher-level unit. The new multiple membership PGM (MM-PGM) extends the 3L-PGM to handle multiple membership data structures frequently found in the social sciences. This study sought to examine the consequences of ignoring individual mobility across clusters when estimating a 3L-PGM in comparison to estimating a MM-PGM. MM-PGM estimates were less biased (especially in the cluster-level coefficient estimates), although we found substantial bias in cluster-level variance components across some conditions for both models.     

How representations of number and numeracy predict decision paradoxes: A fuzzy-trace theory approach

Higher numeracy has been associated with decision biases in some numerical judgment-and-decision problems. According to fuzzy-trace theory, understanding such paradoxes involves broadening the concept of numeracy to include processing the gist of numbers—their categorical and ordinal relations—in addition to objective (verbatim) knowledge about numbers. We assess multiple representations of gist, as well as numeracy, and use them to better understand and predict systematic paradoxes in judgment and decision-making. In two samples (Ns = 978 and 957), we assessed categorical (some vs. none) and ordinal gist representations of numbers (higher vs. lower, as in relative magnitude judgment, estimation, approximation, and simple ratio comparison), objective numeracy, and a nonverbal, nonnumeric measure of fluid intelligence in predicting: (a) decision preferences exhibiting the Allais paradox and (b) attractiveness ratings of bets with and without a small loss in which the loss bet is rated higher than the objectively superior no-loss bet. Categorical and ordinal gist tasks predicted unique variance in paradoxical decisions and judgments, beyond objective numeracy and intelligence. Whereas objective numeracy predicted choosing or rating according to literal numerical superiority, appreciating the categorical and ordinal gist of numbers was pivotal in predicting paradoxes. These results bring important paradoxes under the same explanatory umbrella, which assumes three types of representations of numbers—categorical gist, ordinal gist, and objective (verbatim)—that vary in their strength across individuals.     

Multilevel Reliability Measures of Latent Scores Within an Item Response Theory Framework

Abstract

This paper evaluated multilevel reliability measures in two-level nested designs (e.g., students nested within teachers) within an item response theory framework. A simulation study was implemented to investigate the behavior of the multilevel reliability measures and the uncertainty associated with the measures in various multilevel designs regarding the number of clusters, cluster sizes, and intraclass correlations (ICCs), and in different test lengths, for two parameterizations of multilevel item response models with separate item discriminations or the same item discrimination over levels. Marginal maximum likelihood estimation (MMLE)-multiple imputation and Bayesian analysis were employed to evaluate the accuracy of the multilevel reliability measures and the empirical coverage rates of Monte Carlo (MC) confidence or credible intervals. Considering the accuracy of the multilevel reliability measures and the empirical coverage rate of the intervals, the results lead us to generally recommend MMLE-multiple imputation. In the model with separate item discriminations over levels, marginally acceptable accuracy of the multilevel reliability measures and empirical coverage rate of the MC confidence intervals were found in a limited condition, 200 clusters, 30 cluster size, .2 ICC, and 40 items, in MMLE-multiple imputation. In the model with the same item discrimination over levels, the accuracy of the multilevel reliability measures and the empirical coverage rate of the MC confidence intervals were acceptable in all multilevel designs we considered with 40 items under MMLE-multiple imputation. We discuss these findings and provide guidelines for reporting multilevel reliability measures.     

Links between self-reported and laboratory behavioral impulsivity

Havik, M., Jakobson, A., Tamm, M., Paaver, M., Konstabel, K., Uusberg, A., Allik, J., Ööpik, V. & Kreegipuu, K. (2012). Links between self‐reported and laboratory behavioral impulsivity. Scandinavian Journal of Psychology 53, 216–223. A major problem in the research considering impulsivity is the lack of mutual understanding on how to measure and define impulsivity. Our study examined the relationship between self‐reported impulsivity, behavioral excitatory and inhibitory processes and time perception. Impulsivity – fast, premature, thoughtless or disinhibited behavior – was assessed in 58 normal, healthy participants (30 men, mean age 21.9 years). Self‐reported impulsivity as measured by Adaptive and Maladaptive Impulsivity Scale (AMIS) and behavioral excitatory and inhibitory processes as measured by Stop Signal Task were not directly related. Time perception, measured by the retrospective Time Estimation Task, was related to both. The length of the perceived time interval was positively correlated to AMIS Disinhibition subscale and negatively to several Stop Signal Task parameters. The longer subjects perceived the duration to last, the higher was their score on Disinhibition scale and the faster were their reactive responses in the Stop Signal Task. In summary our findings support the idea of cognitive tempo as a possible mechanism underlying impulsive behavior.     

The effect of food toppings on calorie estimation and consumption

In this research we examine the effect of adding a food topping to the base food on consumers' calorie estimation and consumption of the augmented food (base food plus topping). We show that consumers underestimate the calorie content of augmented food with an unhealthy base, especially when the topping is healthy. However, consumers are less likely to underestimate the calorie content of augmented food with a healthy base, regardless of whether the topping is healthy or unhealthy. Further, we show that adding a healthy topping to an unhealthy base makes consumers not only underestimate the calorie content but also eat more of the augmented food.     

潜变量交互效应标准化估计：方法比较与选用策略

标准化估计对模型的解释和效应大小的比较有重要作用。虽然潜变量交互效应的恰当标准化估计公式已经面世超过10年, 国内外都在使用和引用, 但至今未见到关于不同估计方法得到的恰当标准化估计的系统比较。通过模拟实验, 比较了乘积指标法、潜调节结构方程(LMS)、无先验信息和有先验信息的贝叶斯法的潜变量交互效应标准化估计在不同条件下的表现。结果发现, 在正态条件下, LMS和有信息贝叶斯法表现较好; 而在非正态条件下, 乘积指标法比较稳健, 但需要较大的样本(不小于500), 小样本且外生潜变量之间相关很低时可使用无信息贝叶斯法。     

认知诊断缺失数据处理方法的比较：零替换、多重插补与极大似然估计法

数据缺失在测验中经常发生, 认知诊断评估也不例外, 数据缺失会导致诊断结果的偏差。首先, 通过模拟研究在多种实验条件下比较了常用的缺失数据处理方法。结果表明：(1)缺失数据导致估计精确性下降, 随着人数与题目数量减少、缺失率增大、题目质量降低, 所有方法的PCCR均下降, Bias绝对值和RMSE均上升。(2)估计题目参数时, EM法表现最好, 其次是MI, FIML和ZR法表现不稳定。(3)估计被试知识状态时, EM和FIML表现最好, MI和ZR表现不稳定。其次, 在PISA2015实证数据中进一步探索了不同方法的表现。综合模拟和实证研究结果, 推荐选用EM或FIML法进行缺失数据处理。     

Spatial numerical associations in preschoolers

ABSTRACT

Three-to-five-year-old French children were asked to add or remove objects to or from linear displays. The hypothesis of a universal tendency to represent increasing number magnitudes from left to right led to predict a majority of manipulations at the right end of the rows, whatever children's hand laterality. Conversely, if numbers are not inherently associated with space, children were expected to favour laterality-consistent manipulations. The results showed a strong tendency to operate on the right end of the rows in right-handers, but no preference in left-handers. These findings suggest that the task elicited a left-to-right oriented representation of magnitudes that counteracted laterality-related responses in left-handed children. The young age of children and the lack of a developmental trend towards right preference weaken the hypothesis of a cultural origin of this oriented representation. The possibility that our results are due to weaker brain lateralisation in left-handers compared to right-handers is addressed in Discussion section.