The asymptotic posterior normality of the latent trait in an IRT model

It has long been part of the item response theory (IRT) folklore that under the usual empirical Bayes unidimensional IRT modeling approach, the posterior distribution of examinee ability given test response is approximately normal for a long test. Under very general and nonrestrictive nonparametric assumptions, we make this claim rigorous for a broad class of latent models.This research was partially supported by Office of Naval Research Cognitive and Neural Sciences Grant N0014-J-90-1940, 442-1548, National Science Foundation Mathematics Grant NSF-DMS-91-01436, and the National Center for Supercomputing Applications. We wish to thank Kumar Joag-dev and Zhiliang Ying for enlightening suggestions concerning the proof of the basic result.The authors wish to thank Kumar Joag-Dev, Brian Junker, Bert Green, Paul Holland, Robert Mislevy, and especially Zhiliang Ying for their useful comments and discussions.     

Correlation and prediction in ordinal test theory

Based on the test theory model for ordinal measurements proposed by Schulman and Haden, the present paper considers correlations between tests, attenuation, regressions involving true and observed scores, and prediction of test reliability.The population correlation between tests is shown to be related to the expected sample correlation for samples of sizen ₁ andn ₂. Errors of estimation, measurement and prediction are found to be similar to their counterparts in interval test theory, while attenuation is identical to its counterpart. The bias in estimating population reliability from sample data is compared for Kendall's tau and Spearman's rho.The author wishes to thank the referees for their helpful comments on an earlier draft of this paper, and in particular, for the suggested alternative methods of establishing some of the presented results.     

Force and rate relations in responding during variable-interval reinforcement

Four rats responded on one-minute variable-interval schedules with several variations in peak-force of response required for food reinforcement. Measures of peak force and rate were taken for the responses, which were the downward exertions of force against a static force-transducing operandum. The analysis distinguished responses, a generic class of measured behavior, from criterion responses, an operationally specified subclass required for reinforcement. Absolute rate of response showed no systematic change, but the rate of responses meeting a newly required criterion of peak-force invariably increased through changes in the absolute rate of response, the relative-frequency distributions of peak force, or some combination of both. The relative frequency of responses meeting an elevated force criterion during variable-interval reinforcement exceeded that maintained with the same criterion with continuous reinforcement. The requirement of more effortful responding for reinforcement does not necessarily reduce response rate. Conformity of the behavior to the requirement for reinforcement is the salient effect.     

Estimation and visualization of confusability matrices from adaptive measurement data

We present a simple but effective method based on Luce’s choice axiom [Luce, R.D. (1959). Individual choice behavior: A theoretical analysis. New York: John Wiley & Sons] for consistent estimation of the pairwise confusabilities of items in a multiple-choice recognition task with arbitrarily chosen choice-sets. The method combines the exact (non-asymptotic) Bayesian way of assessing uncertainty with the unbiasedness emphasized in the classical frequentist approach.We apply the method to data collected using an adaptive computer game designed for prevention of reading disability. A player’s estimated confusability of phonemes (or more accurately, phoneme-grapheme connections) and larger units of language is visualized in an easily understood way with color cues and explicit indication of the accuracy of the estimates. Visualization of learning-related changes in the player’s performance is considered.The empirical validity of the choice axiom is evaluated using the game data itself. The axiom appears to hold reasonably well although a small systematic violation is observable for the smallest choice-set sizes.     

Monaural Loudness Adaptation for Middle-Intensity Middle-Frequency Signals: The Importance of Measurement Technique

Using the Simple Adaptation technique (SA) and the Ipsilateral Comparison Paradigm (ICP), the authors studied monaural loudness adaptation to a middle-intensity [60 dB(A)] tone at signal frequencies of 250, 1000, and 4000 Hz in the left and right ears. Adaptation effects were absent when the SA procedure was used. However, they were observed uniformly across all frequency values with the ICP, a result that challenges the assertion in the literature, on the basis of SA measures, that loudness adaptation for middle-intensity signals occurs only at frequencies above 4000 Hz. The ICP features periodic intensity modulations (±10 dB relative to the base signal) to accommodate listeners' needs for referents by which they can gauge subtle changes in the loudness of the adapting tone, a key component that is missing in the SA method. Adaptation effects in this investigation were similar in both ears, supporting the equal susceptibility assumption common in loudness adaptation studies.     

时距估计任务中的间断期望效应

使用单任务研究程序,采用引入提示线索的方法,以产生时距作为反应指标对存在间断的时距估计任务中的间断期望效应和提示线索效应（注意效应）进行系统考察,并对间断时距的效应、产生时距与等待时距的关系问题作出进一步探讨。结果表明,间断位置（等待时距）因素是被试时间判断的主要线索,被试的产生时距随着等待时距的增加而延长。间断实验中表现出极其显著的提示线索效应,此效应既增加了时距估计的变异,又延长了被试的时距估计。无间断实验条件下,被试表现出显著的间断期望效应,被试对间断的期望有损于时间估计。     

Essential independence and likelihood-based ability estimation for polytomous items

A definition ofessential independence is proposed for sequences of polytomous items. For items satisfying the reasonable assumption that the expected amount of credit awarded increases with examinee ability, we develop a theory ofessential unidimensionality which closely parallels that of Stout. Essentially unidimensional item sequences can be shown to have a unique (up to change-of-scale) dominant underlying trait, which can be consistently estimated by a monotone transformation of the sum of the item scores. In more general polytomous-response latent trait models (with or without ordered responses), anM-estimator based upon maximum likelihood may be shown to be consistent for under essentially unidimensional violations of local independence and a variety of monotonicity/identifiability conditions. A rigorous proof of this fact is given, and the standard error of the estimator is explored. These results suggest that ability estimation methods that rely on the summation form of the log likelihood under local independence should generally be robust under essential independence, but standard errors may vary greatly from what is usually expected, depending on the degree of departure from local independence. An index of departure from local independence is also proposed.This work was supported in part by Office of Naval Research Grant N00014-87-K-0277 and National Science Foundation Grant NSF-DMS-88-02556. The author is grateful to William F. Stout for many helpful comments, and to an anonymous reviewer for raising the questions addressed in section 2. A preliminary version of section 6 appeared in the author's Ph.D. thesis.     

Dysfunctional implications of narrow window theory: variability in the intuitive assessment of correlation

Narrow Window theory, suggested by Y. Kareev ten years ago, has so far focused on one central implication of the limited capacity of working memory on intuitive correlation estimation, namely, overestimation of the distal population correlation. This paper points to additional and perhaps more dramatic implications due to the large dispersion of intuitive estimates: (a) large estimation errors, possibly causing overestimation of negligible rhos, misses of strong rhos, and distorted hierarchies of the rhos between different pairs of variables; and (b) large interpersonal differences in the estimation of any given rho and highly incongruent hierarchies of estimated correlations between different pairs of variables. These implications impede both individuals' adaptation to the empirical world and communication among themselves.     

Multilevel Reliability Measures of Latent Scores Within an Item Response Theory Framework

Abstract

This paper evaluated multilevel reliability measures in two-level nested designs (e.g., students nested within teachers) within an item response theory framework. A simulation study was implemented to investigate the behavior of the multilevel reliability measures and the uncertainty associated with the measures in various multilevel designs regarding the number of clusters, cluster sizes, and intraclass correlations (ICCs), and in different test lengths, for two parameterizations of multilevel item response models with separate item discriminations or the same item discrimination over levels. Marginal maximum likelihood estimation (MMLE)-multiple imputation and Bayesian analysis were employed to evaluate the accuracy of the multilevel reliability measures and the empirical coverage rates of Monte Carlo (MC) confidence or credible intervals. Considering the accuracy of the multilevel reliability measures and the empirical coverage rate of the intervals, the results lead us to generally recommend MMLE-multiple imputation. In the model with separate item discriminations over levels, marginally acceptable accuracy of the multilevel reliability measures and empirical coverage rate of the MC confidence intervals were found in a limited condition, 200 clusters, 30 cluster size, .2 ICC, and 40 items, in MMLE-multiple imputation. In the model with the same item discrimination over levels, the accuracy of the multilevel reliability measures and the empirical coverage rate of the MC confidence intervals were acceptable in all multilevel designs we considered with 40 items under MMLE-multiple imputation. We discuss these findings and provide guidelines for reporting multilevel reliability measures.