Constructing the exact significance level for a person fit statistic

This paper uses an extension of the network algorithm originally introduced by Mehta and Patel to construct exact tail probabilities for testing the general hypothesis that item responses are distributed according to the Rasch model. By assuming that item difficulties are known, the algorithm is applicable to the statistical tests either given the maximum likelihood ability estimate or conditioned on the total score. A simulation study indicates that the network algorithm is an efficient tool for computing the significance level of a person fit statistic based on test lengths of 30 items or less.     

Combinatorial data analysis: Association and partial association

A combinatorial data analysis strategy is reviewed that is designed to compare two arbitrary measures of proximity defined between the objects from some set. Based on a particular cross-product definition of correspondence between these two numerically specified notions of proximity (typically represented in the form of matrices), extensions are then pursued to indices of partial association that relate the observed pattern of correspondence between the first two proximity measures to a third. The attendant issues of index normalization and significance testing are discussed; the latter is approached through a simple randomization model implemented either through a Monte Carlo procedure or distributional approximations based on the first three moments. Applications of the original comparison strategy and its extensions to partial association may be developed for a variety of methodological and substantive tasks. Besides rank correlation, we emphasize the topics of spatial autocorrelation for one variable and spatial association between two and mention the connection to the usual randomization approach for one-way analysis-of-variance.     

A nonparametric approach for assessing latent trait unidimensionality

Assuming a nonparametric family of item response theory models, a theory-based procedure for testing the hypothesis of unidimensionality of the latent space is proposed. The asymptotic distribution of the test statistic is derived assuming unidimensionality, thereby establishing an asymptotically valid statistical test of the unidimensionality of the latent trait. Based upon a new notion of dimensionality, the test is shown to have asymptotic power 1. A 6300 trial Monte Carlo study using published item parameter estimates of widely used standardized tests indicates conservative adherence to the nominal level of significance and statistical power averaging 81 out of 100 rejections for examinee sample sizes and psychological test lengths often incurred in practice.The referees' comments were remarkably detailed and greatly enhanced the writeup and sensitized the author to certain pertinent issues. Discussions with Fritz Drasgow, Lloyd Humphreys, Dennis Jennings, Brian Junker, Robert Linn, Ratna Nandakumar, and Robin Shealy were also very useful.This research was supported by the Office of Naval Research under grant N00014-84-K-0186; NR 150-533, and by the National Science Foundation under grant DMS 85-03321.     

Reliability of test scores in nonparametric item response theory

Three methods for estimating reliability are studied within the context of nonparametric item response theory. Two were proposed originally by Mokken (1971) and a third is developed in this paper. Using a Monte Carlo strategy, these three estimation methods are compared with four classical lower bounds to reliability. Finally, recommendations are given concerning the use of these estimation methods.The authors are grateful for constructive comments from the reviewers and from Charles Lewis.     

Extending exploratory diagnostic classification models: Inferring the effect of covariates

Diagnostic models provide a statistical framework for designing formative assessments by classifying student knowledge profiles according to a collection of fine-grained attributes. The context and ecosystem in which students learn may play an important role in skill mastery, and it is therefore important to develop methods for incorporating student covariates into diagnostic models. Including covariates may provide researchers and practitioners with the ability to evaluate novel interventions or understand the role of background knowledge in attribute mastery. Existing research is designed to include covariates in confirmatory diagnostic models, which are also known as restricted latent class models. We propose new methods for including covariates in exploratory RLCMs that jointly infer the latent structure and evaluate the role of covariates on performance and skill mastery. We present a novel Bayesian formulation and report a Markov chain Monte Carlo algorithm using a Metropolis-within-Gibbs algorithm for approximating the model parameter posterior distribution. We report Monte Carlo simulation evidence regarding the accuracy of our new methods and present results from an application that examines the role of student background knowledge on the mastery of a probability data set.     

Stochastic ordering using the latent trait and the sum score in polytomous IRT models

In a restricted class of item response theory (IRT) models for polytomous items the unweighted total score has monotone likelihood ratio (MLR) in the latent trait. MLR implies two stochastic ordering (SO) properties, denoted SOM and SOL, which are both weaker than MLR, but very useful for measurement with IRT models. Therefore, these SO properties are investigated for a broader class of IRT models for which the MLR property does not hold.In this study, first a taxonomy is given for nonparametric and parametric models for polytomous items based on the hierarchical relationship between the models. Next, it is investigated which models have the MLR property and which have the SO properties. It is shown that all models in the taxonomy possess the SOM property. However, counterexamples illustrate that many models do not, in general, possess the even more useful SOL property.Hemker's research was supported by the Netherlands Research Council, Grant 575-67-034. Junker's research was supported in part by the National Institutes of Health, Grant CA54852, and by the National Science Foundation, Grant DMS-94.04438.     

Time-series analysis in operant research

A time-series method is presented, nontechnically, for analysis of data generated in individual-subject operant studies, and is recommended as a supplement to visual analysis of behavior change in reversal or multiple-baseline experiments. The method can be used to identify three kinds of statistically significant behavior change: (a) changes in score levels from one experimental phase to another, (b) reliable upward or downward trends in scores, and (c) changes in trends between phases. The detection of, and reliance on, serial dependency (autocorrelation among temporally adjacent scores) in individual-subject behavioral scores is emphasized. Examples of published data from the operant literature are used to illustrate the time-series method.     

Considerations in the choice of interobserver reliability estimates

Two types of interobserver reliability values may be needed in treatment studies in which observers constitute the primary data-acquisition system: trial reliability and the reliability of the composite unit or score which is subsequently analyzed, e.g., daily or weekly session totals. Two approaches to determining interobserver reliability are described: percentage agreement and "correlational" measures of reliability. The interpretation of these estimates, factors affecting their magnitude, and the advantages and limitations of each approach are presented.     

贝叶斯统计方法的应用与现状

贝叶斯统计方法是心理学数据分析的热门方法。研究全面论述贝叶斯方法在心理学领域的应用与方向。现阶段贝叶斯方法以模拟研究为主,应用方向为心理学研究常用的项目反应理论、认知诊断、计算机自适应、结构方程模型。同时,评述发现贝叶斯方法正逐步被国内心理学研究者所接受。最后,文章讨论了当下贝叶斯统计在心理学研究中应用的局限性及可能的原因,建议统计学者开发界面友好的贝叶斯软件,并在心理学课程中加入贝叶斯知识。     

Too bored for sports? Adaptive and less-adaptive latent personality profiles for exercise behavior

Physical exercise is an effective tool for improving public health, but the general population exercises too little. Drawing on recent theorizing on the combined role of boredom and self-control in guiding goal-directed behavior, we test the hypothesis that individual differences in boredom and self-control differentiate high from low exercisers. The role of boredom as a non-adaptive disposition is of particular interest, because research on boredom in sports is scarce. Here, we investigate the role of such individual differences in self-reported weekly exercise behavior (in minutes) in a sample of N = 507 participants (n = 200 female, M_age = 36.43 (±9.54)). We used the robust variant of Mahalanobis distance to detect and remove n = 51 multivariate outliers and then performed latent profile analysis to assess if boredom (boredom proneness; exercise-related boredom) and self-control (trait self-control; if-then planning) combine into identifiable latent profiles. In line with theoretical considerations, the Bayesian Information Criterion favored a solution with two latent profiles. One profile was characterized by higher-than-average exercise-related boredom and boredom proneness and lower-than-average self-control and if-then planning values. This pattern was reversed for the second profile. A one-sided Bayesian two-sample t-test supported the hypothesis that the first profile is associated with less exercise behavior than the second profile, BF = 16.93. Our results foster the notion of self-control and if-then planning as adaptive dispositions. More importantly, they point to an important role of boredom in the exercise setting: exercise-related boredom and getting easily bored in general are associated with less exercise activity. This is in line with recent theorizing on boredoms' and self-controls’ function in guiding goal-directed behavior.