Marginal maximum likelihood estimation of item response theory (IRT) equating coefficients for the common-examinee design

A method of estimating item response theory (IRT) equating coefficients by the common-examinee design with the assumption of the two-parameter logistic model is provided. The method uses the marginal maximum likelihood estimation, in which individual ability parameters in a common-examinee group are numerically integrated out. The abilities of the common examinees are assumed to follow a normal distribution but with an unknown mean and standard deviation on one of the two tests to be equated. The distribution parameters are jointly estimated with the equating coefficients. Further, the asymptotic standard errors of the estimates of the equating coefficients and the parameters for the ability distribution are given. Numerical examples are provided to show the accuracy of the method.     

Maximum likelihood estimation of item response parameters when some responses are omitted

A theoretical model is given for dealing with omitted responses. Two special cases are investigated.This work was supported in part by contract N00014-80-C-0402, project designation NR 150-453 between the Office of Naval Research and Educational Testing Service. Reproduction in whole or in part is permitted for any purpose of the United States Government.     

Marginal maximum likelihood estimation for a psychometric model of discontinuous development

Item response theory models posit latent variables to account for regularities in students' performances on test items. Wilson's “Saltus” model extends the ideas of IRT to development that occurs in stages, where expected changes can be discontinuous, show different patterns for different types of items, or even exhibit reversals in probabilities of success on certain tasks. Examples include Piagetian stages of psychological development and Siegler's rule-based learning. This paper derives marginal maximum likelihood (MML) estimation equations for the structural parameters of the Saltus model and suggests a computing approximation based on the EM algorithm. For individual examinees, empirical Bayes probabilities of learning-stage are given, along with proficiency parameter estimates conditional on stage membership. The MML solution is illustrated with simulated data and an example from the domain of mixed number subtraction. The authors' names appear in alphabetical order. We would like to thank Karen Draney for computer programming, Kikumi Tatsuoka for allowing us to use the mixed-number subtraction data, and Eric Bradlow, Chan Dayton, Kikumi Tatsuoka, and four anonymous referees for helpful suggestions. The first author's work was supported by Contract No. N00014-88-K-0304, R&T 4421552, from the Cognitive Sciences Program, Cognitive and Neural Sciences Division, Office of Naval Research, and by the Program Research Planning Council of Educational Testing Service. The second author's work was supported by a National Academy of Education Spencer Fellowship and by a Junior Faculty Research Grant from the Committee on Research, University of California at Berkeley. A copy of the Saltus computer program can be obtained from the second author.     

The linear transformation model with frailties for the analysis of item response times

The item response times (RTs) collected from computerized testing represent an underutilized source of information about items and examinees. In addition to knowing the examinees’ responses to each item, we can investigate the amount of time examinees spend on each item. In this paper, we propose a semi‐parametric model for RTs, the linear transformation model with a latent speed covariate, which combines the flexibility of non‐parametric modelling and the brevity as well as interpretability of parametric modelling. In this new model, the RTs, after some non‐parametric monotone transformation, become a linear model with latent speed as covariate plus an error term. The distribution of the error term implicitly defines the relationship between the RT and examinees’ latent speeds; whereas the non‐parametric transformation is able to describe various shapes of RT distributions. The linear transformation model represents a rich family of models that includes the Cox proportional hazards model, the Box–Cox normal model, and many other models as special cases. This new model is embedded in a hierarchical framework so that both RTs and responses are modelled simultaneously. A two‐stage estimation method is proposed. In the first stage, the Markov chain Monte Carlo method is employed to estimate the parametric part of the model. In the second stage, an estimating equation method with a recursive algorithm is adopted to estimate the non‐parametric transformation. Applicability of the new model is demonstrated with a simulation study and a real data application. Finally, methods to evaluate the model fit are suggested.     

Marginal maximum likelihood estimation for the one-parameter logistic model

Two algorithms are described for marginal maximum likelihood estimation for the one-parameter logistic model. The more efficient of the two algorithms is extended to estimation for the linear logistic model. Numerical examples of both procedures are presented. Portions of this research were presented at the meeting of the Psychometric Society in Chapel Hill, N.C. in May, 1981. Thanks to R. Darrell Bock, Gerhard Fischer, and Paul Holland for helpful comments in the course of this research.     

Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm

Maximum likelihood estimation of item parameters in the marginal distribution, integrating over the distribution of ability, becomes practical when computing procedures based on an EM algorithm are used. By characterizing the ability distribution empirically, arbitrary assumptions about its form are avoided. The Em procedure is shown to apply to general item-response models lacking simple sufficient statistics for ability. This includes models with more than one latent dimension.Supported in part by NSF grant BNS 7912417 to the University of Chicago and by SSRC (UK) grant HR6132 to the University of Lancaster.We are indebted to Mark Reiser and Robert Gibbons for computer programming. David Thissen clarified a number of points in an earlier draft.     

Weighted likelihood estimation of ability in item response theory

Applications of item response theory, which depend upon its parameter invariance property, require that parameter estimates be unbiased. A new method, weighted likelihood estimation (WLE), is derived, and proved to be less biased than maximum likelihood estimation (MLE) with the same asymptotic variance and normal distribution. WLE removes the first order bias term from MLE. Two Monte Carlo studies compare WLE with MLE and Bayesian modal estimation (BME) of ability in conventional tests and tailored tests, assuming the item parameters are known constants. The Monte Carlo studies favor WLE over MLE and BME on several criteria over a wide range of the ability scale.     

Multicategorical spline model for item response theory

Additional information contained in incorrect responses calls for a multicategorical rather than a binary analysis of multiple choice data. A nonparametric divided-by-total model for joint maximum likelihood estimation of probability-of-choice functions (for particular responses) and of latent ability is proposed. The model approximates probability functions by rational splines. Some illustrative examples of real test data analysis and the results of a Monte Carlo study are presented.The research in this paper was supported by the National Sciences and Engineering Research Council of Canada Grants OGP0105521 and APA 320 awarded to the first and the second author, respectively. The authors are indebted to R. Melzack and A. Baker for making available the data analyzed in this paper. We would also like to thank J. McKenna and B. Cont for their assistance in editing this paper.     

An accumulator model for responses and response times in tests based on the proportional hazards model

Latent trait models for responses and response times in tests often lack a substantial interpretation in terms of a cognitive process model. This is a drawback because process models are helpful in clarifying the meaning of the latent traits. In the present paper, a new model for responses and response times in tests is presented. The model is based on the proportional hazards model for competing risks. Two processes are assumed, one reflecting the increase in knowledge and the second the tendency to discontinue. The processes can be characterized by two proportional hazards models whose baseline hazard functions correspond to the temporary increase in knowledge and discouragement. The model can be calibrated with marginal maximum likelihood estimation and an application of the ECM algorithm. Two tests of model fit are proposed. The amenability of the proposed approaches to model calibration and model evaluation is demonstrated in a simulation study. Finally, the model is used for the analysis of two empirical data sets.     

An item response model with internal restrictions on item difficulty

An IRT model based on the Rasch model is proposed for composite tasks, that is, tasks that are decomposed into subtasks of different kinds. There is one subtask for each component that is discerned in the composite tasks. A component is a generic kind of subtask of which the subtasks resulting from the decomposition are specific instantiations with respect to the particular composite tasks under study. The proposed model constrains the difficulties of the composite tasks to be linear combinations of the difficulties of the corresponding subtask items, which are estimated together with the weights used in the linear combinations, one weight for each kind of subtask. Although the model does not belong to the exponential family, its parameters can be estimated using conditional maximum likelihood estimation. The approach is demonstrated with an application to spelling tasks. We thank Eric Maris for his helpful comments.     

A generalized item response tree model for psychological assessments

A new item response theory (IRT) model with a tree structure has been introduced for modeling item response processes with a tree structure. In this paper, we present a generalized item response tree model with a flexible parametric form, dimensionality, and choice of covariates. The utilities of the model are demonstrated with two applications in psychological assessments for investigating Likert scale item responses and for modeling omitted item responses. The proposed model is estimated with the freely available R package flirt (Jeon et al., 2014b).     

A nonlinear mixed model framework for item response theory

Mixed models take the dependency between observations based on the same cluster into account by introducing 1 or more random effects. Common item response theory (IRT) models introduce latent person variables to model the dependence between responses of the same participant. Assuming a distribution for the latent variables, these IRT models are formally equivalent with nonlinear mixed models. It is shown how a variety of IRT models can be formulated as particular instances of nonlinear mixed models. The unifying framework offers the advantage that relations between different IRT models become explicit and that it is rather straightforward to see how existing IRT models can be adapted and extended. The approach is illustrated with a self-report study on anger.     

A Pearson-Type-VII item response model for assessing person fluctuation

Using Lumsden’s Thurstonian fluctuation model as a starting point, this paper attempts to develop a unidimensional item response theory model intended for binary personality items. Under some additional assumptions, a new model is obtained in which the item characteristic curves are defined by a cumulative Pearson-Type-VII distribution, and the person response curves are two-parameter normal ogives. Procedures for fitting the new model are proposed. Furthermore, the relations between individual fluctuation and scalability are discussed, and a scalability index based on the new model is proposed. All the developments in this paper are illustrated using two empirical examples.     

Maximum marginal likelihood estimation for semiparametric item analysis

The item characteristic curve (ICC), defining the relation between ability and the probability of choosing a particular option for a test item, can be estimated by using polynomial regression splines. These provide a more flexible family of functions than is given by the three-parameter logistic family. The estimation of spline ICCs is described by maximizing the marginal likelihood formed by integrating ability over a beta prior distribution. Some simulation results compare this approach with the joint estimation of ability and item parameters.IRCAMThe research reported in this paper was supported by Grants APA320 and A4035 from the Natural Sciences and Engineering Research Council of Canada. It was also supported by Contract No. F41689-82-C-10020 from the Air Force Human Resources Laboratory to Educational Testing Service. The author wishes to thank M. Abrahamowicz for his assistance and R. Darrell Bock for providing the parameters for the items used in the simulations.     

Analysis of residuals for the multionmial item response model

Using the item response model as developed on the multinomial distribution, asymptotic variances are obtained for residuals associated with response patterns and first-, and second-order marginal frequencies of manifest variables. When the model does not fit well, an examination of these residuals may reveal the source of the poor fit. Finally, a limited-information test of fit for the model is developed by using residuals defined for the first-, and second-order marginals. Model evaluation based on residuals for these marginals is particularly useful when the response pattern frequencies are sparse.The author would like to thank Yasuo Amemiya and Joseph Lucke for helpful suggestions. This research was supported by a Research Incentive Grant from Arizona State University.     

Measuring change for a multidimensional test using a generalized explanatory longitudinal item response model

Even though many educational and psychological tests are known to be multidimensional, little research has been done to address how to measure individual differences in change within an item response theory framework. In this paper, we suggest a generalized explanatory longitudinal item response model to measure individual differences in change. New longitudinal models for multidimensional tests and existing models for unidimensional tests are presented within this framework and implemented with software developed for generalized linear models. In addition to the measurement of change, the longitudinal models we present can also be used to explain individual differences in change scores for person groups (e.g., learning disabled students versus non‐learning disabled students) and to model differences in item difficulties across item groups (e.g., number operation, measurement, and representation item groups in a mathematics test). An empirical example illustrates the use of the various models for measuring individual differences in change when there are person groups and multiple skill domains which lead to multidimensionality at a time point.     

Sequential sampling designs for the two-parameter item response theory model

In optimal design research, designs are optimized with respect to some statistical criterion under a certain model for the data. The ideas from optimal design research have spread into various fields of research, and recently have been adopted in test theory and applied to item response theory (IRT) models. In this paper a generalized variance criterion is used for sequential sampling in the two-parameter IRT model. Some general principles are offered to enable a researcher to select the best sampling design for the efficient estimation of item parameters.     

Optimal item difficulty for the three-parameter normal ogive response model

In tailored testing, it is important to determine the optimal difficulty of the next item to present to the examinee. This paper shows that the difference that maximizes information for the three-parameter normal ogive response model is approximately 1.7 times the optimal difference –b for the three-parameter logistic model. Under the normal model, calculation of the optimal difficulty for minimizing the Bayes risk is equivalent to maximizing an associated information function.The views expressed herein, are those of the author and do not necessarily reflect those of the Department of the Navy.     

Estimation of a four‐parameter item response theory model

We explore the justification and formulation of a four‐parameter item response theory model (4PM) and employ a Bayesian approach to recover successfully parameter estimates for items and respondents. For data generated using a 4PM item response model, overall fit is improved when using the 4PM rather than the 3PM or the 2PM. Furthermore, although estimated trait scores under the various models correlate almost perfectly, inferences at the high and low ends of the trait continuum are compromised, with poorer coverage of the confidence intervals when the wrong model is used. We also show in an empirical example that the 4PM can yield new insights into the properties of a widely used delinquency scale. We discuss the implications for building appropriate measurement models in education and psychology to model more accurately the underlying response process.