Higher-order latent trait models for cognitive diagnosis

Higher-order latent traits are proposed for specifying the joint distribution of binary attributes in models for cognitive diagnosis. This approach results in a parsimonious model for the joint distribution of a high-dimensional attribute vector that is natural in many situations when specific cognitive information is sought but a less informative item response model would be a reasonable alternative. This approach stems from viewing the attributes as the specific knowledge required for examination performance, and modeling these attributes as arising from a broadly-defined latent trait resembling theϑ of item response models. In this way a relatively simple model for the joint distribution of the attributes results, which is based on a plausible model for the relationship between general aptitude and specific knowledge. Markov chain Monte Carlo algorithms for parameter estimation are given for selected response distributions, and simulation results are presented to examine the performance of the algorithm as well as the sensitivity of classification to model misspecification. An analysis of fraction subtraction data is provided as an example. This research was funded by National Institute of Health grant R01 CA81068. We would like to thank William Stout and Sarah Hartz for many useful discussions, three anonymous reviewers for helpful comments and suggestions, and Kikumi Tatsuoka and Curtis Tatsuoka for generously sharing data.     

A note on the asymptotic variance-covariance matrix of item parameter estimates in the rasch model

Lord and Wingersky have developed a method for computing the asymptotic variance-covariance matrix of maximum likelihood estimates for item and person parameters under some restrictions on the estimates which are needed in order to fix the latent scale. The method is tedious, but can be simplified for the Rasch model when one is only interested in the item parameters. This is demonstrated here under a suitable restriction on the item parameter estimates.     

Psychometric latent response models

In this paper, some psychometric models will be presented that belong to the larger class oflatent response models (LRMs). First, LRMs are introduced by means of an application in the field ofcomponential item response theory (Embretson, 1980, 1984). Second, a general definition of LRMs (not specific for the psychometric subclass) is given. Third, some more psychometric LRMs, and examples of how they can be applied, are presented. Fourth, a method for obtaining maximum likelihood (ML) and some maximum a posteriori (MAP) estimates of the parameters of LRMs is presented. This method is then applied to theconjunctive Rasch model. Fifth and last, an application of the conjunctive Rasch model is presented. This model was applied to responses to typical verbal ability items (open synonym items).This paper presents theoretical and empirical results of a research project supported by the Research Council [Onderzoeksraad] of the University of Leuven (grant number 89-9) to Paul De Boeck and Luc Delbeke.     

Continuous and discrete latent structure models for item response data

Relations are examined between latent trait and latent class models for item response data. Conditions are given for the two-latent class and two-parameter normal ogive models to agree, and relations between their item parameters are presented. Generalizationss are then made to continuous models with more than one latent trait and discrete models with more than two latent classes, and methods are presented for relating latent class models to factor models for dichotomized variables. Results are illustrated using data from the Law School Admission Test, previously analyzed by several authors.     

Random effects and extended generalized partial credit models

In this paper it is shown that under the random effects generalized partial credit model for the measurement of a single latent variable by a set of polytomously scored items, the joint marginal probability distribution of the item scores has a closed-form expression in terms of item category location parameters, parameters that characterize the distribution of the latent variable in the subpopulation of examinees with a zero score on all items, and item-scaling parameters. Due to this closed-form expression, all parameters of the random effects generalized partial credit model can be estimated using marginal maximum likelihood estimation without assuming a particular distribution of the latent variable in the population of examinees and without using numerical integration. Also due to this closed-form expression, new special cases of the random effects generalized partial credit model can be identified. In addition to these new special cases, a slightly more general model than the random effects generalized partial credit model is presented. This slightly more general model is called the extended generalized partial credit model. Attention is paid to maximum likelihood estimation of the parameters of the extended generalized partial credit model and to assessing the goodness of fit of the model using generalized likelihood ratio tests. Attention is also paid to person parameter estimation under the random effects generalized partial credit model. It is shown that expected a posteriori estimates can be obtained for all possible score patterns. A simulation study is carried out to show the usefulness of the proposed models compared to the standard models that assume normality of the latent variable in the population of examinees. In an empirical example, some of the procedures proposed are demonstrated.     

A latent class model for rating data

A latent class model for rating data is presented which is the analogue of Andrich's binomial Rasch model for Lazarsfeld's latent class analysis (LCA). The response probabilities for the rating categories follow a binomial distribution and depend on class-specific item parameters. The EM-algorithm for parameter estimation as well as goodness of fit tests for the model are described. An example using questionnaire items on interest in physics illustrates the use of the model as an alternative to the latent trait approach of analyzing test data.I would like to thank Clifford Clogg and the anonymous reviewers for their helpful comments.     

Constant latent odds-ratios models and the mantel-haenszel null hypothesis

In the present paper, a new family of item response theory (IRT) models for dichotomous item scores is proposed. Two basic assumptions define the most general model of this family. The first assumption is local independence of the item scores given a unidimensional latent trait. The second assumption is that the odds-ratios for all item-pairs are constant functions of the latent trait. Since the latter assumption is characteristic of the whole family, the models are called constant latent odds-ratios (CLORs) models. One nonparametric special case and three parametric special cases of the general CLORs model are shown to be generalizations of the one-parameter logistic Rasch model. For all CLORs models, the total score (the unweighted sum of the item scores) is shown to be a sufficient statistic for the latent trait. In addition, conditions under the general CLORs model are studied for the investigation of differential item functioning (DIF) by means of the Mantel-Haenszel procedure. This research was supported by the Dutch Organization for Scientific Research (NWO), grant number 400-20-026.     

Generalized latent trait models

In this paper we discuss a general model framework within which manifest variables with different distributions in the exponential family can be analyzed with a latent trait model. A unified maximum likelihood method for estimating the parameters of the generalized latent trait model will be presented. We discuss in addition the scoring of individuals on the latent dimensions. The general framework presented allows, not only the analysis of manifest variables all of one type but also the simultaneous analysis of a collection of variables with different distributions. The approach used analyzes the data as they are by making assumptions about the distribution of the manifest variables directly.     

Bayesian estimation and model selection in ordered latent class models for polytomous items

In a latent class IRT model in which the latent classes are ordered on one dimension, the class specific response probabilities are subject to inequality constraints. The number of these inequality constraints increase dramatically with the number of response categories per item, if assumptions like monotonicity or double monotonicity of the cumulative category response functions are postulated. A Markov chain Monte Carlo method, the Gibbs sampler, can sample from the multivariate posterior distribution of the parameters under the constraints. Bayesian model selection can be done by posterior predictive checks and Bayes factors. A simulation study is done to evaluate results of the application of these methods to ordered latent class models in three realistic situations. Also, an example of the presented methods is given for existing data with polytomous items. It can be concluded that the Bayesian estimation procedure can handle the inequality constraints on the parameters very well. However, the application of Bayesian model selection methods requires more research.     

A multidimensional latent trait model for measuring learning and change

A latent trait model is presented for the repeated measurement of ability based on a multidimensional conceptualization of the change process. A simplex structure is postulated to link item performance under a given measurement condition or occasion to initial ability and to one or more modifiabilities that represent individual differences in change. Since item discriminations are constrained to be equal within a measurement condition, the model belongs to the family of multidimensional Rasch models. Maximum likelihood estimators of the item parameters and abilities are derived, and an example provided that shows good recovery of both item and ability parameters. Properties of the model are explored, particularly for several classical issues in measuring change.     

Local dependence latent structure models

A local independence latent structure model, which assumesm latent classes, requires a minimum of 2m-1 items for the solution of the 2m ² latent parameters. If one adds 3 items to the test and if one assumes local dependence between pairs of items, thereby adding additional latent parameters, _ij , representing the association between itemsi andj, then it is possible to obtain estimates for all of the latent parameters: latent class frequencies latent probabilities, and measures of association between pairs of items. The solution consists of (1) forming (m + 1) × (m + 1) matrices of manifest data, which are singular, (2) solving for the _ij in equations that result from the singularity of the data matrices, (3) correcting the manifest data by removing the contamination due to local dependence, and (4) estimating the remaining latent parameters from the corrected data, using methods outlined in earlier literature.     

Normal ogive model on the continuous response level in the multidimensional latent space

The homogeneous case of the continuous response model is expanded to the multi-dimensional latent space, and the normal ogive model is presented. The operating density characteristic of the continuous item response and the vector of basic functions are developed. It is found out that there is a vector of sufficient statistics for estimating the subject's vector of latent traits, given the item parameter vectors. The relationship between the model and the linear factor analysis is observed. The matrix of item response information functions is introduced. Some additional observations are also made.The work described in this paper was partly carried out while the author was at Bowling Green State University with the support of its Research Associateship in the summer, 1972.     

Multicomponent latent trait models for ability tests

A major research direction for ability measurement has been to identify the information-processes that are involved in solving test items through mathematical modeling of item difficulty. However, this research has had limited impact on ability measurement, since person parameters are not included in the process models. The current paper presents some multicomponent latent trait models for reproducing test performance from both item and person parameters on processing components. Components are identified from item subtasks, in which performance is a logistic function (i.e., Rasch model) of person and item parameters, and then are combined according to a mathematical model of processing on the composite item.The author would like to thank David Thissen for his invaluable insights concerning this model and an anonymous reviewer for his suggestion about the sample space for the model.This research was partially supported by National Institute of Education grant number NIE-6-7-0156 to Susan E. Whitely, principal investigator. However the opinions expressed herein do not necessarily reflect the position or policy of the National Institute of Education, and no official endorsement by the National Institute of Education should be referred. Part of this paper was presented at the annual meeting of thePsychometric Society, Monterey, California: June, 1979.     

Locally dependent latent trait model and the dutch identity revisited

In this paper, we propose a class of locally dependent latent trait models for responses to psychological and educational tests. Typically, item response models treat an individual's multiple response to stimuli as conditional independent given the individual's latent trait. In this paper, instead the focus is on models based on a family of conditional distributions, or kernel, that describes joint multiple item responses as a function of student latent trait, not assuming conditional independence. Specifically, we examine a hybrid kernel which comprises a component for one-way item response functions and a component for conditional associations between items given latent traits. The class of models allows the extension of item response theory to cover some new and innovative applications in psychological and educational research. An EM algorithm for marginal maximum likelihood of the hybrid kernel model is proposed. Furthermore, we delineate the relationship of the class of locally dependent models and the log-linear model by revisiting the Dutch identity (Holland, 1990). The work is supported by a research grant from the Marshall School of Business, University of Southern California. The author thanks the anonymous referees for their suggestions.     

Evaluation on types of invariance in studying extreme response bias with an IRTree approach

In recent years, item response tree (IRTree) approaches have received increasing attention in the response style literature for their ability to partial out response style latent variables as well as associated item parameters. When an IRTree approach is adopted to measure extreme response styles, directional and content invariance could be assumed at the latent variable and item parameter levels. In this study, we propose to evaluate the empirical validity of these invariance assumptions by employing a general IRTree model with relaxed invariance assumptions. This would allow us to examine extreme response biases, beyond extreme response styles. With three empirical applications of the proposed evaluation, we find that relaxing some of the invariance assumptions improves the model fit, which suggests that not all assumed invariances are empirically supported. Specifically, at the latent variable level, we find reasonable evidence for directional invariance but mixed evidence for content invariance, although we also find that estimated correlations between content-specific extreme response latent variables are high, hinting at the potential presence of a general extreme response tendency. At the item parameter level, we find no directional or content invariance for thresholds and no content invariance for slopes. We discuss how the variant item parameter estimates obtained from a general IRTree model can offer useful insight to help us understand response bias related to extreme responding measured within the IRTree framework.     

Fitting and Testing Conditional Multinormal Partial Credit Models

A multinormal partial credit model for factor analysis of polytomously scored items with ordered response categories is derived using an extension of the Dutch Identity (Holland in Psychometrika 55:5?C18, 1990). In the model, latent variables are assumed to have a multivariate normal distribution conditional on unweighted sums of item scores, which are sufficient statistics. Attention is paid to maximum likelihood estimation of item parameters, multivariate moments of latent variables, and person parameters. It is shown that the maximum likelihood estimates can be found without the use of numerical integration techniques. More general models are discussed which can be used for testing the model, and it is shown how models with different numbers of latent variables can be tested against each other. In addition, multi-group extensions are proposed, which can be used for testing both measurement invariance and latent population differences. Models and procedures discussed are demonstrated in an empirical data example.     

Decomposing person and occasion-specific effects: an extension of latent state-trait (LSI) theory to hierarchical LST models

An extension of latent state-trait (LST) theory to hierarchical LST models is presented. In hierarchical LST models, the covariances between 2 or more latent traits are explained by a general 3rd-order factor, and the covariances between latent state residuals pertaining to different traits measured on the same measurement occasion are explained by 2nd-order latent occasion-specific factors. Analogous to recent developments in multitrait-multimethod methodology, all factors are interpreted in relation to factors taken as comparison standards. An empirical example from test anxiety research illustrates how estimates of additive variance components due to general trait, specific trait, occasion, state residual, method, and measurement error can be obtained using confirmatory factor analysis. Advantages and limitations of these models are discussed.     

Information Matrices and Standard Errors for MLEs of Item Parameters in IRT

The paper clarifies the relationship among several information matrices for the maximum likelihood estimates (MLEs) of item parameters. It shows that the process of calculating the observed information matrix also generates a related matrix that is the middle piece of a sandwich-type covariance matrix. Monte Carlo results indicate that standard errors (SEs) based on the observed information matrix are robust to many, but not all, conditions of model/distribution misspecifications. SEs based on the sandwich-type covariance matrix perform most consistently across conditions. Results also suggest that SEs based on other matrices are either not consistent or perform not as robust as those based on the sandwich-type covariance matrix or the observed information matrix.     

多维项目反应理论补偿性模型参数估计:基于广义回归神经网络集合

运用广义回归神经网络(GRNN)方法对小样本多维项目反应理论(MIRT)补偿性模型的项目参数进行估计,尝试解决传统参数估计方法样本数量要求较大的问题。MIRT双参数Logistic补偿模型被设置为二级计分的二维模型。首先,模拟二维能力参数、项目参数值与考生作答矩阵。其次,把通过主成分分析得到的前两个因子在每个题目上的载荷作为区分度的初始值以及题目通过率作为难度的初始值,这两个指标的初始值作为神经网络的输入。集成100个神经网络,其输出值的均值作为MIRT的项目参数估计值。最后,设置2×2种(能力相关水平:0.3和0.7; 两种估计方法:GRNN和MCMC方法)实验处理,对GRNN和MCMC估计方法的返真性进行比较。结果表明,小样本的情况下,基于GRNN集成方法的参数估计结果优于MCMC方法。     

计算机适应性测验条件下认知设计项目预测参数的影响

自动化项目生成(Automatic Item Generation)中的项目参数是基于认知项目设计的刺激特征集预测的, 在不确定性来源上较之用经验数据标定的参数更为复杂。文章通过实证研究分析了在计算机适应性测验条件下基于认知设计系统法生成的抽象推理测验(ART)项目预测参数对能力参数估计的精确性。研究表明, 项目预测参数比相应标定参数分布更为趋中。这种回归效应既影响到能力参数估计误差大小, 也导致适应性测验过程中项目选择的差异。在控制了项目选择差异之后, 能力参数估计误差较之基于项目标定参数的能力估计误差大, 但差别并不明显。两者相应的能力估计值相关很高, 对应能力值之间的差异很小, 且几乎贯彻整个能力分布区间。