Essential independence and likelihood-based ability estimation for polytomous items

A definition ofessential independence is proposed for sequences of polytomous items. For items satisfying the reasonable assumption that the expected amount of credit awarded increases with examinee ability, we develop a theory ofessential unidimensionality which closely parallels that of Stout. Essentially unidimensional item sequences can be shown to have a unique (up to change-of-scale) dominant underlying trait, which can be consistently estimated by a monotone transformation of the sum of the item scores. In more general polytomous-response latent trait models (with or without ordered responses), anM-estimator based upon maximum likelihood may be shown to be consistent for under essentially unidimensional violations of local independence and a variety of monotonicity/identifiability conditions. A rigorous proof of this fact is given, and the standard error of the estimator is explored. These results suggest that ability estimation methods that rely on the summation form of the log likelihood under local independence should generally be robust under essential independence, but standard errors may vary greatly from what is usually expected, depending on the degree of departure from local independence. An index of departure from local independence is also proposed.This work was supported in part by Office of Naval Research Grant N00014-87-K-0277 and National Science Foundation Grant NSF-DMS-88-02556. The author is grateful to William F. Stout for many helpful comments, and to an anonymous reviewer for raising the questions addressed in section 2. A preliminary version of section 6 appeared in the author's Ph.D. thesis.     

Correlation and prediction in ordinal test theory

Based on the test theory model for ordinal measurements proposed by Schulman and Haden, the present paper considers correlations between tests, attenuation, regressions involving true and observed scores, and prediction of test reliability.The population correlation between tests is shown to be related to the expected sample correlation for samples of sizen ₁ andn ₂. Errors of estimation, measurement and prediction are found to be similar to their counterparts in interval test theory, while attenuation is identical to its counterpart. The bias in estimating population reliability from sample data is compared for Kendall's tau and Spearman's rho.The author wishes to thank the referees for their helpful comments on an earlier draft of this paper, and in particular, for the suggested alternative methods of establishing some of the presented results.     

The impacts of ignoring individual mobility across clusters in estimating a piecewise growth model

A three-level piecewise growth model (3L-PGM) can be used to break up nonlinear growth into multiple components, providing the opportunity to examine potential sources of variation in individual and contextual growth within different segments of the model. The conventional 3L-PGM assumes that the data are strictly hierarchical in nature, where measurement occasions (level 1) are nested within individuals (level 2) who are members of a single cluster (level 3). However, in longitudinal research, it is sometimes difficult for data structures to remain purely clustered during a study, such as when some students change classrooms or schools over time. One resulting data structure in this situation is known as a multiple membership structure, where some lower-level units are members of more than one higher-level unit. The new multiple membership PGM (MM-PGM) extends the 3L-PGM to handle multiple membership data structures frequently found in the social sciences. This study sought to examine the consequences of ignoring individual mobility across clusters when estimating a 3L-PGM in comparison to estimating a MM-PGM. MM-PGM estimates were less biased (especially in the cluster-level coefficient estimates), although we found substantial bias in cluster-level variance components across some conditions for both models.     

Stopping rules for multi-category computerized classification testing

Computerized classification testing (CCT) aims to classify persons into one of two or more possible categories to make decisions such as mastery/non-mastery or meet most/meet all/exceed. A defining feature of CCT is its stopping criterion: the test terminates when there is enough confidence to make a decision. There is abundant research on CCT with a single cut-off, and two common stopping criteria are the sequential probability ratio test (SPRT) statistic and the generalized likelihood ratio statistic (GLR). However, there is a relative scarcity of research extending the SPRT to the multi-hypothesis case for when there is more than one cut-off. In this paper, we propose a new multi-category GLR (mGLR) statistic as well as a stochastically curtailed version of the CCT with three or more categories. A simulation study was conducted to show that the mGLR statistic outperformed the existing stopping rules by generating shorter average test length without sacrificing classification accuracy. Results also revealed that the stochastically curtailed mGLR successfully increased test efficiency in certain testing conditions.     

On the existence and uniqueness of JML estimates for the partial credit model

A necessary and sufficient condition is given in this paper for the existence and uniqueness of the maximum likelihood (the so-called joint maximum likelihood) estimate of the parameters of the Partial Credit Model. This condition is stated in terms of a structural property of the pattern of the data matrix that can be easily verified on the basis of a simple iterative procedure. The result is proved by using an argument of Haberman (1977). The author wishes to thank the Editor and the anonymous reviewers for their comments that helped to substantially improve the final version of this paper. This research was supported in part by a MURST grant (ex 60%).     

Multilevel Reliability Measures of Latent Scores Within an Item Response Theory Framework

Abstract

This paper evaluated multilevel reliability measures in two-level nested designs (e.g., students nested within teachers) within an item response theory framework. A simulation study was implemented to investigate the behavior of the multilevel reliability measures and the uncertainty associated with the measures in various multilevel designs regarding the number of clusters, cluster sizes, and intraclass correlations (ICCs), and in different test lengths, for two parameterizations of multilevel item response models with separate item discriminations or the same item discrimination over levels. Marginal maximum likelihood estimation (MMLE)-multiple imputation and Bayesian analysis were employed to evaluate the accuracy of the multilevel reliability measures and the empirical coverage rates of Monte Carlo (MC) confidence or credible intervals. Considering the accuracy of the multilevel reliability measures and the empirical coverage rate of the intervals, the results lead us to generally recommend MMLE-multiple imputation. In the model with separate item discriminations over levels, marginally acceptable accuracy of the multilevel reliability measures and empirical coverage rate of the MC confidence intervals were found in a limited condition, 200 clusters, 30 cluster size, .2 ICC, and 40 items, in MMLE-multiple imputation. In the model with the same item discrimination over levels, the accuracy of the multilevel reliability measures and the empirical coverage rate of the MC confidence intervals were acceptable in all multilevel designs we considered with 40 items under MMLE-multiple imputation. We discuss these findings and provide guidelines for reporting multilevel reliability measures.     

Links between self-reported and laboratory behavioral impulsivity

Havik, M., Jakobson, A., Tamm, M., Paaver, M., Konstabel, K., Uusberg, A., Allik, J., Ööpik, V. & Kreegipuu, K. (2012). Links between self‐reported and laboratory behavioral impulsivity. Scandinavian Journal of Psychology 53, 216–223. A major problem in the research considering impulsivity is the lack of mutual understanding on how to measure and define impulsivity. Our study examined the relationship between self‐reported impulsivity, behavioral excitatory and inhibitory processes and time perception. Impulsivity – fast, premature, thoughtless or disinhibited behavior – was assessed in 58 normal, healthy participants (30 men, mean age 21.9 years). Self‐reported impulsivity as measured by Adaptive and Maladaptive Impulsivity Scale (AMIS) and behavioral excitatory and inhibitory processes as measured by Stop Signal Task were not directly related. Time perception, measured by the retrospective Time Estimation Task, was related to both. The length of the perceived time interval was positively correlated to AMIS Disinhibition subscale and negatively to several Stop Signal Task parameters. The longer subjects perceived the duration to last, the higher was their score on Disinhibition scale and the faster were their reactive responses in the Stop Signal Task. In summary our findings support the idea of cognitive tempo as a possible mechanism underlying impulsive behavior.     

The effect of food toppings on calorie estimation and consumption

In this research we examine the effect of adding a food topping to the base food on consumers' calorie estimation and consumption of the augmented food (base food plus topping). We show that consumers underestimate the calorie content of augmented food with an unhealthy base, especially when the topping is healthy. However, consumers are less likely to underestimate the calorie content of augmented food with a healthy base, regardless of whether the topping is healthy or unhealthy. Further, we show that adding a healthy topping to an unhealthy base makes consumers not only underestimate the calorie content but also eat more of the augmented food.     

潜变量交互效应标准化估计：方法比较与选用策略

标准化估计对模型的解释和效应大小的比较有重要作用。虽然潜变量交互效应的恰当标准化估计公式已经面世超过10年, 国内外都在使用和引用, 但至今未见到关于不同估计方法得到的恰当标准化估计的系统比较。通过模拟实验, 比较了乘积指标法、潜调节结构方程(LMS)、无先验信息和有先验信息的贝叶斯法的潜变量交互效应标准化估计在不同条件下的表现。结果发现, 在正态条件下, LMS和有信息贝叶斯法表现较好; 而在非正态条件下, 乘积指标法比较稳健, 但需要较大的样本(不小于500), 小样本且外生潜变量之间相关很低时可使用无信息贝叶斯法。