A note on latent trait theory and Bowker's test

Bowker's test for marginal equality in contingency tables provides a familiar chi-square test to determine whether the marginal distributions are the same across two or more factors or occasions. In this note it is shown how latent trait theory provides a theoretical framework for the development and application of this test.The research reported here was supported by a grant to the senior author from the National Institute on Aging (AG03164).     

An automated test development of parallel tests from a seed test

Binary programming models are presented to generate parallel tests from an itembank. The parallel tests are created to match item for item an existing seed test and match user supplied taxonomic specifications. The taxonomic specifications may be either obtained from the seed test or from some other user requirement. An algorithm is presented along with computational results to indicate the overall efficiency of the process. Empirical findings based on an itembank for the Arithmetic Reasoning section of the Armed Services Vocational Aptitude Battery are given.The Office of Naval Research, Program in Cognitive Science, N00014-87-C-0696 partially supported the work of Douglas H. Jones. The Rutgers Research Resource Committee of the Graduate School of Management partially supported the work of Douglas H. Jones and Ing-Long Wu. A Thomas and Betts research fellowship partially supported the work of Ing-Long Wu. The Human Resources Laboratory, United States Air Force, partially supported the work of Ronald Armstrong. The authors benefited from conversations with Dr. Wayne Shore, Operational Technologies, San Antonio, Texas. The order of authors' names is alphabetical and denotes equal authorship.     

The asymptotic posterior normality of the latent trait for polytomous IRT models

Chang and Stout (1993) presented a derivation of the asymptotic posterior normality of the latent trait given examinee responses under nonrestrictive nonparametric assumptions for dichotomous IRT models. This paper presents an extention of their results to polytomous IRT models in a fairly straightforward manner. In addition, a global information function is defined, and the relationship between the global information function and the currently used information functions is discussed. An information index that combines both the global and local information is proposed for adaptive testing applications.This research was partially supported by Educational Testing Service Allocation Project No. 79424. The author wishes to thank Charles Davis, Xuming He, Frank Jenkins, Spence Swinton, William Stout, Ming-Mai Wang, and Zhiliang Ying for their helpful comments and discussions. The author particularly wishes to thank the Editor, Shizuhiko Nishisato, the Associate Editor, and three anonymous reviewers for their thoroughness and thoughtful suggestions.     

A flexible latent trait model for response times in tests

Latent trait models for response times in tests have become popular recently. One challenge for response time modeling is the fact that the distribution of response times can differ considerably even in similar tests. In order to reduce the need for tailor-made models, a model is proposed that unifies two popular approaches to response time modeling: Proportional hazard models and the accelerated failure time model with log–normally distributed response times. This is accomplished by resorting to discrete time. The categorization of response time allows the formulation of a response time model within the framework of generalized linear models by using a flexible link function. Item parameters of the proposed model can be estimated with marginal maximum likelihood estimation. Applicability of the proposed approach is demonstrated with a simulation study and an empirical application. Additionally, means for the evaluation of model fit are suggested.     

四参数Logistic模型潜在特质参数的Warm加权极大似然估计

本文以四参数Logistic(4-parameter Logistic,4PL)模型为研究对象,根据Warm的加权极大似然估计技巧,提出了4PL模型潜在特质参数的加权极大似然估计方法,并借助模拟研究对加权极大似然估计的性质进行验证。研究结果表明,与通常的极大似然估计和后验期望估计相比,加权极大似然估计的偏差(bias)明显减小,并且具有良好的返真性能。此外,在测试的长度较短和项目的区分度较小的情况下,加权极大似然估计依然保持了良好的统计性质,表现出更加显著的优势。     

Two-group classification in latent trait theory: Scores with monotone likelihood ratio

This paper deals with two-group classification when a unidimensional latent trait,, is appropriate for explaining the data,X. It is shown that ifX has monotone likelihood ratio then optimal allocation rules can be based on its magnitude when allocation must be made to one of two groups related to. These groups may relate to probabilistically via a non-decreasing functionp(), or may be defined by all subjects above or below a selected value on.In the case where the data arise from dichotomous items, then only the assumption that the items have nondecreasing item characteristic functions is enough to ensure that the unweighted sum of responses (the number-right score or raw score) possesses this fundamental monotone likelihood ratio property.     

The asymptotic posterior normality of the latent trait in an IRT model

It has long been part of the item response theory (IRT) folklore that under the usual empirical Bayes unidimensional IRT modeling approach, the posterior distribution of examinee ability given test response is approximately normal for a long test. Under very general and nonrestrictive nonparametric assumptions, we make this claim rigorous for a broad class of latent models.This research was partially supported by Office of Naval Research Cognitive and Neural Sciences Grant N0014-J-90-1940, 442-1548, National Science Foundation Mathematics Grant NSF-DMS-91-01436, and the National Center for Supercomputing Applications. We wish to thank Kumar Joag-dev and Zhiliang Ying for enlightening suggestions concerning the proof of the basic result.The authors wish to thank Kumar Joag-Dev, Brian Junker, Bert Green, Paul Holland, Robert Mislevy, and especially Zhiliang Ying for their useful comments and discussions.     

The role of secondary covariates when estimating latent trait population distributions

The U.S. National Assessment of Educational Progress (NAEP), the Third International Mathematics and Science Study (TIMSS), and the U.S. Adult Literacy Survey collect probability samples of students (or adults) who are administered brief examinations in subject areas such as mathematics and reading (cognitive variables), along with background demographic (primary) and educational environment (secondary) questions. The demographic questions are used in the primary reporting, while the numerous explanatory secondary variables, or covariates, are only directly utilized in subsequent secondary analyses. The covariates are also used indirectly to create the plausible values (multiple imputations) that are an integral part of analyses because of the use of sparse matrix sampling of cognitive items. The improvement in the precision of the primary reporting due to the inclusion of the covariates is assessed here and contrasted with the precision of reporting using plausible values created using only the primary demographic variables.The results demonstrate that the improvement in precision depends on the matrix sampling designs for the cognitive assessments. The improvements range from essentially none for the most common designs, to moderate for some less common designs. Consequently, two potential changes in the reporting procedures that could improve the statistical and operational efficiency of primary reporting are (a) eliminate or reduce the collection of covariates and increase the number of cognitive items, (b) to avoid delays, eliminate the covariates from the creation of plausible values used for the primary reports, but include them later when creating public-use files for secondary analyses. The potential improvements in statistical and operational efficiency must be weighed against the intrinsic interest in the covariates, and the potential for small discrepancies in the primary and secondary reporting.Thanks to Donald Rubin, Robert Mislevy, and John Barnard for their helpful comments and computing assistance. This work was supported by NCES Grant 84.902B980011.     

A comment on Birnbaum's three-parameter logistic model in the latent trait theory

Birnbaum's three-parameter logistic model for the multiple-choice item in the latent trait theory is considered with respect to the item response information function and the unique maximum condition. It is clarified that with models of knowledge or random guessing nature, which include the three-parameter logistic model, the unique maximum condition is not satisfied for the correct answer, and the item response information function is negative for the interval (− ∞,θ _g ). It is suggested that we should useθ _g as a criterion in selecting optimal items for a specified group of examinees, so that we can practically avoid the possibility of non-unique maxima of the likelihood function on the response pattern given by an examinee in the group. The work described in this paper was partially done while the author was at University of New Brunswick, Canada, in 1968–1970, supported by NRC Grant APA-345 from National Research Council of Canada.     

Reliability of test scores in nonparametric item response theory

Three methods for estimating reliability are studied within the context of nonparametric item response theory. Two were proposed originally by Mokken (1971) and a third is developed in this paper. Using a Monte Carlo strategy, these three estimation methods are compared with four classical lower bounds to reliability. Finally, recommendations are given concerning the use of these estimation methods.The authors are grateful for constructive comments from the reviewers and from Charles Lewis.     

Laughter: an empirical test of some reversal theory hypotheses

Hypotheses derived from reversal theory on state-specific responses to humorous material were tested by recruitment of 10 telic (serious minded) and 10 paratelic (playful) state-dominant individuals from a larger sample of university students completing the Telic Dominance Scale. They were individually exposed to a comedy programme displayed on television in a psychophysiological laboratory. Polygraph record of respiration provided operational measurement of laughter. Subjects ratings were derived for felt and preferred level of arousal and for telidparatelic state in the laboratory situation. Results confirmed (1) that humorous material tends to induce the paratelic state, even in telic state-dominant individuals, and (2) that frequency of laughter in the paratelic state is positively correlated with degree of felt arousal and with arousal preference, thus indicating a linear , rather than ditonic, relation of hedonic tone to felt arousal in this state.     

The estimation of true score variance and error variance in the classical test theory model

The use of one-way analysis of variance tables for obtaining unbiased estimates of true score variance and error score variance in the classical test theory model is discussed. Attention is paid to both balanced (equal numbers of observations on each person) and unbalanced designs, and estimates provided for both homoscedastic (common error variance for all persons) and heteroscedastic cases. It is noted that optimality properties (minimum variance) can be claimed for estimates derived from analysis of variance tables only in the balanced, homoscedastic case, and that there they are essentially a reflection of the symmetry inherent in the situation. Estimates which might be preferable in other cases are discussed. An example is given where a natural analysis of variance table leads to estimates which cannot be derived from the set of statistics which is sufficient under normality assumptions. Reference is made to Bayesian studies which shed light on the difficulties encountered. Work on this paper was carried out at the headquarters of the American College Testing Program, Iowa City, Iowa, while the author was on leave from the University College of Wales.     

Locally dependent latent trait model for polytomous responses with application to inventory of hostility

Psychological tests often involve item clusters that are designed to solicit responses to behavioral stimuli. The dependency between individual responses within clusters beyond that which can be explained by the underlying trait sometimes reveals structures that are of substantive interest. The paper describes two general classes of models for this type of locally dependent responses. Specifically, the models include a generalized log-linear representation and a hybrid parameterization model for polytomous data. A compact matrix notation designed to succinctly represent the system of complex multivariate polytomous responses is presented. The matrix representation creates the necessary formulation for the locally dependent kernel for polytomous item responses. Using polytomous data from an inventory of hostility, we provide illustrations as to how the locally dependent models can be used in psychological measurement.     

不同认知结构被试的测验设计模式

正如不同的病症需要使用不同的医疗技术方法来诊断一样, 不同的认知结构也需要设计对应的测验模式来进行诊断, 从而保证测验具有高质量的诊断评估效果。但传统测验形式未考虑不同认知结构的针对性诊断测验需求, 导致“千人一卷”在测验效率上有所不足; 认知诊断计算机化自适应测验虽可针对不同认知结构的被试施测不同的项目, 然而支持自适应过程的题库却没有针对不同认知结构被试设计对应的项目, 导致题库使用效率较低。要解决上述问题的关键在于, 探索如何针对不同认知结构设计相对应的测验模式。本研究采用Monte Carlo模拟, 对六种属性层级关系下, 不同认知结构的测验设计模式进行探讨。实验结果表明(1)同一属性层级关系下, 不同认知结构的最佳测验设计模式不同; (2)依据不同认知结构的最佳测验设计模式构建的题库具有更高的使用效率。测验编制者可以根据实验结果针对不同认知结构优化对应的测验设计模式, 并用于指导题库建设。     

A latent trait analysis of the eysenck personality questionnaire in an elderly community sample

A latent trait model was fitted to the responses of a large probability sample of elderly Australians to the Extraversion and Neuroticism scales of the short form of the Eysenck Personality Questionnaire—Revised (EPQ-R). Item parameters obtained were compared to those available from comparable analyses of a younger sample in order to detect item bias. While some evidence of differential functioning between younger and older groups was found, the measurement properties of these scales in older persons were confirmed. The comparable functioning of the instrument in both groups that scores on these scales may be validly compared across the adult life span.     

A generalization of the bush-mosteller model with some significance tests

A generalization of the Bush-Mosteller learning model is proposed in connection with two-alternative learning situations with continuous reinforcement. The problem is to test the hypothesis that reward and non-reward are equally effective in promoting learning. Statistically this reduces to testing a hypothesis about the value of a single parameter, while a set of other parameters remains unspecified. The test presented has the property of being asymptotically locally most powerful among all tests of the same size and asymptotically similar. The application of the test is illustrated.I would like to express my gratitude to Professors J. Neyman and E. L. Scott, Department of Statistics, University of California, for their constant assistance and encouragement throughout the research that led to this paper and during its preparation, and to Professor F. W. Irwin, Department of Psychology, University of Pennsylvania, for his many helpful suggestions and comments.     

A hierarchical latent response model for inferences about examinee engagement in terms of guessing and item-level non-response

In low-stakes assessments, test performance has few or no consequences for examinees themselves, so that examinees may not be fully engaged when answering the items. Instead of engaging in solution behaviour, disengaged examinees might randomly guess or generate no response at all. When ignored, examinee disengagement poses a severe threat to the validity of results obtained from low-stakes assessments. Statistical modelling approaches in educational measurement have been proposed that account for non-response or for guessing, but do not consider both types of disengaged behaviour simultaneously. We bring together research on modelling examinee engagement and research on missing values and present a hierarchical latent response model for identifying and modelling the processes associated with examinee disengagement jointly with the processes associated with engaged responses. To that end, we employ a mixture model that identifies disengagement at the item-by-examinee level by assuming different data-generating processes underlying item responses and omissions, respectively, as well as response times associated with engaged and disengaged behaviour. By modelling examinee engagement with a latent response framework, the model allows assessing how examinee engagement relates to ability and speed as well as to identify items that are likely to evoke disengaged test-taking behaviour. An illustration of the model by means of an application to real data is presented.