Quantile maximum likelihood estimation of response time distributions

Queen’s University, Kingston, Ontario, Canada We introduce and evaluate via a Monte Carlo study a robust new estimation technique that fits distribution functions to grouped response time (RT) data, where the grouping is determined by sample quantiles. The new estimator, quantile maximum likelihood (QML), is more efficient and less biased than the best alternative estimation technique when fitting the commonly used ex-Gaussian distribution. Limitations of the Monte Carlo results are discussed and guidance provided for the practical application of the new technique. Because QML estimation can be computationally costly, we make fast open source code for fitting available that can be easily modified     

Heteroscedastic one‐factor models and marginal maximum likelihood estimation

In the present paper, a general class of heteroscedastic one‐factor models is considered. In these models, the residual variances of the observed scores are explicitly modelled as parametric functions of the one‐dimensional factor score. A marginal maximum likelihood procedure for parameter estimation is proposed under both the assumption of multivariate normality of the observed scores conditional on the single common factor score and the assumption of normality of the common factor score. A likelihood ratio test is derived, which can be used to test the usual homoscedastic one‐factor model against one of the proposed heteroscedastic models. Simulation studies are carried out to investigate the robustness and the power of this likelihood ratio test. Results show that the asymptotic properties of the test statistic hold under both small test length conditions and small sample size conditions. Results also show under what conditions the power to detect different heteroscedasticity parameter values is either small, medium, or large. Finally, for illustrative purposes, the marginal maximum likelihood estimation procedure and the likelihood ratio test are applied to real data.     

Pseudo maximum likelihood estimation and a test for misspecification in mean and covariance structure models

Using the theory of pseudo maximum likelihood estimation the asymptotic covariance matrix of maximum likelihood estimates for mean and covariance structure models is given for the case where the variables are not multivariate normal. This asymptotic covariance matrix is consistently estimated without the computation of the empirical fourth order moment matrix. Using quasi-maximum likelihood theory a Hausman misspecification test is developed. This test is sensitive to misspecification caused by errors that are correlated with the independent variables. This misspecification cannot be detected by the test statistics currently used in covariance structure analysis.For helpful comments on a previous draft of the paper we are indebted to Kenneth A. Bollen, Ulrich L. Küsters, Michael E. Sobel and the anonymous reviewers of Psychometrika. For partial research support, the first author wishes to thank the Department of Sociology at the University of Arizona, where he was a visiting professor during the fall semester 1987.     

Learning overhypotheses with hierarchical Bayesian models

Inductive learning is impossible without overhypotheses, or constraints on the hypotheses considered by the learner. Some of these overhypotheses must be innate, but we suggest that hierarchical Bayesian models can help to explain how the rest are acquired. To illustrate this claim, we develop models that acquire two kinds of overhypotheses--overhypotheses about feature variability (e.g. the shape bias in word learning) and overhypotheses about the grouping of categories into ontological kinds like objects and substances.     

A maximum likelihood hierarchical disaggregate model for predicting choices of individuals

The absence of operational disaggregate lexicographic decision models and Tversky's observation that choice behavior is often inconsistent, hierarchical, and context dependent motivate the development of a maximum likelihood hierarchical (MLH) choice model. This new disaggregate choice model requires few assumptions and accommodates the three aspects of choice behavior noted by A. Tversky (1972, Journal of Mathematical Psychology, 9, 341–367). The model has its foundation in a prototype model developed by the authors. Unlike the deterministic prototype, however, MLH is a probabilistic model which generates maximum likelihood estimators of the aggregate “cutoff values.” The model is formulated as a concave programming problem whose solutions are therefore globally optimal. Finally, the model is applied to data from three separate studies where it is demonstrated to have superior performance over the prototype model in its predictive performance.     

Marginal maximum likelihood estimation for the one-parameter logistic model

Two algorithms are described for marginal maximum likelihood estimation for the one-parameter logistic model. The more efficient of the two algorithms is extended to estimation for the linear logistic model. Numerical examples of both procedures are presented. Portions of this research were presented at the meeting of the Psychometric Society in Chapel Hill, N.C. in May, 1981. Thanks to R. Darrell Bock, Gerhard Fischer, and Paul Holland for helpful comments in the course of this research.     

On a signal detection approach to m-alternative forced choice with bias,with maximum likelihood and Bayesian approaches to estimation

The standard signal detection theory (SDT) approach to $m$-alternative forced choice uses the proportion correct as the outcome variable and assumes that there is no response bias. The assumption of no bias is not made for theoretical reasons, but rather because it simplifies the model and estimation of its parameters. The SDT model for $m$AFC with bias is presented, with the cases of two, three, and four alternatives considered in detail. Two approaches to fitting the model are noted: maximum likelihood estimation with Gaussian quadrature and Bayesian estimation with Markov chain Monte Carlo. Both approaches are examined in simulations. SAS and OpenBUGS programs to fit the models are provided, and an application to real-world data is presented.     

Marginal maximum likelihood estimation of item response theory (IRT) equating coefficients for the common-examinee design

A method of estimating item response theory (IRT) equating coefficients by the common-examinee design with the assumption of the two-parameter logistic model is provided. The method uses the marginal maximum likelihood estimation, in which individual ability parameters in a common-examinee group are numerically integrated out. The abilities of the common examinees are assumed to follow a normal distribution but with an unknown mean and standard deviation on one of the two tests to be equated. The distribution parameters are jointly estimated with the equating coefficients. Further, the asymptotic standard errors of the estimates of the equating coefficients and the parameters for the ability distribution are given. Numerical examples are provided to show the accuracy of the method.     

Bayesian estimation and testing of structural equation models

The Gibbs sampler can be used to obtain samples of arbitrary size from the posterior distribution over the parameters of a structural equation model (SEM) given covariance data and a prior distribution over the parameters. Point estimates, standard deviations and interval estimates for the parameters can be computed from these samples. If the prior distribution over the parameters is uninformative, the posterior is proportional to the likelihood, and asymptotically the inferences based on the Gibbs sample are the same as those based on the maximum likelihood solution, for example, output from LISREL or EQS. In small samples, however, the likelihood surface is not Gaussian and in some cases contains local maxima. Nevertheless, the Gibbs sample comes from the correct posterior distribution over the parameters regardless of the sample size and the shape of the likelihood surface. With an informative prior distribution over the parameters, the posterior can be used to make inferences about the parameters underidentified models, as we illustrate on a simple errors-in-variables model.We thank David Spiegelhalter for suggesting applying the Gibbs sampler to structural equation models to the first author at a 1994 workshop in Wiesbaden. We thank Ulf Böckenholt, Chris Meek, Marijtje van Duijn, Clark Glymour, Ivo Molenaar, Steve Klepper, Thomas Richardson, Teddy Seidenfeld, and Tom Snijders for helpful discussions, mathematical advice, and critiques of earlier drafts of this paper.     

Bayesian estimation of item response curves

Item response curves for a set of binary responses are studied from a Bayesian viewpoint of estimating the item parameters. For the two-parameter logistic model with normally distributed ability, restricted bivariate beta priors are used to illustrate the computation of the posterior mode via the EM algorithm. The procedure is illustrated by data from a mathematics test.This work was supported under Contract No. N00014-85-K-0113, NR 150-535, from Personnel and Training Research Programs, Psychological Sciences Division, Office of Naval Research. The authors wish to thank Mark D. Reckase for providing the ACT data used in the illustration and Michael J. Soltys for computational assistance. They also wish to thank the editor and four anonymous reviewers for many valuable suggestions.     

Marginal maximum likelihood estimation for a psychometric model of discontinuous development

Item response theory models posit latent variables to account for regularities in students' performances on test items. Wilson's “Saltus” model extends the ideas of IRT to development that occurs in stages, where expected changes can be discontinuous, show different patterns for different types of items, or even exhibit reversals in probabilities of success on certain tasks. Examples include Piagetian stages of psychological development and Siegler's rule-based learning. This paper derives marginal maximum likelihood (MML) estimation equations for the structural parameters of the Saltus model and suggests a computing approximation based on the EM algorithm. For individual examinees, empirical Bayes probabilities of learning-stage are given, along with proficiency parameter estimates conditional on stage membership. The MML solution is illustrated with simulated data and an example from the domain of mixed number subtraction. The authors' names appear in alphabetical order. We would like to thank Karen Draney for computer programming, Kikumi Tatsuoka for allowing us to use the mixed-number subtraction data, and Eric Bradlow, Chan Dayton, Kikumi Tatsuoka, and four anonymous referees for helpful suggestions. The first author's work was supported by Contract No. N00014-88-K-0304, R&T 4421552, from the Cognitive Sciences Program, Cognitive and Neural Sciences Division, Office of Naval Research, and by the Program Research Planning Council of Educational Testing Service. The second author's work was supported by a National Academy of Education Spencer Fellowship and by a Junior Faculty Research Grant from the Committee on Research, University of California at Berkeley. A copy of the Saltus computer program can be obtained from the second author.     

Understanding memory impairment with memory models and hierarchical Bayesian analysis

The study of human episodic memory is a topic that interests cognitive and mathematical psychologists as well as clinicians interested in the diagnosis and assessment of Alzheimer’s disease and related disorders (ADRD). In this paper, we use simple cognitive models for the recognition and recall tasks typically applied in clinical assessments of ADRD to study memory performance in ADRD patients. Our models make use of hierarchical Bayesian methods as a way to model individual differences in patient performance and to facilitate the modeling of performance changes that occur during multiple recall tasks. We show how the models are able to account for different aspects of patient performance, and also discuss some of the predictive capabilities of the model. We conclude with a discussion on the scope to improve on our results by discussing the link between memory theory in psychology and clinical practice.     

Two-Stage maximum likelihood estimation in the misspecified restricted latent class model

The maximum likelihood classification rule is a standard method to classify examinee attribute profiles in cognitive diagnosis models (CDMs). Its asymptotic behaviour is well understood when the model is assumed to be correct, but has not been explored in the case of misspecified latent class models. This paper investigates the asymptotic behaviour of a two-stage maximum likelihood classifier under a misspecified CDM. The analysis is conducted in a general restricted latent class model framework addressing all types of CDMs. Sufficient conditions are proposed under which a consistent classification can be obtained by using a misspecified model. Discussions are also provided on the inconsistency of classification under certain model misspecification scenarios. Simulation studies and a real data application are conducted to illustrate these results. Our findings can provide some guidelines as to when a misspecified simple model or a general model can be used to provide a good classification result.     

How cognitive modeling can benefit from hierarchical Bayesian models

Hierarchical Bayesian modeling provides a flexible and interpretable way of extending simple models of cognitive processes. To introduce this special issue, we discuss four of the most important potential hierarchical Bayesian contributions. The first involves the development of more complete theories, including accounting for variation coming from sources like individual differences in cognition. The second involves the capability to account for observed behavior in terms of the combination of multiple different cognitive processes. The third involves using a few key psychological variables to explain behavior on a wide range of cognitive tasks. The fourth involves the conceptual unification and integration of disparate cognitive models. For all of these potential contributions, we outline an appropriate general hierarchical Bayesian modeling structure. We also highlight current models that already use the hierarchical Bayesian approach, as well as identifying research areas that could benefit from its adoption.     

Some small sample results for maximum likelihood estimation in multidimensional scaling

Some aspects of the small sample behavior of maximum likelihood estimates in multidimensional scaling are investigated by Monte Carlo. An investigation of Model M2 in the MULTISCALE program package shows that the chi-square test of dimensionality requires a correction of tabled chi-square values to be unbiased. A formula for this correction in the case of two dimensions is estimated. The power of the test of dimensionality is acceptable with as few as two replications for 15 stimuli and as few as five replications for 10 stimuli. The biases in the exponent and standard error estimates in this model are also investigated.The research reported here was supported by grant number APA 320 to the author by the National Science and Engineering Research Council of Canada.