The EM algorithm for latent class analysis with equality constraints

The EM algorithm is a popular iterative method for estimating parameters in the latent class model where at each step the unknown parameters can be estimated simply as weighted sums of some latent proportions. The algorithm may also be used when some parameters are constrained to equal given constants or each other. It is shown that in the general case with equality constraints, the EM algorithm is not simple to apply because a nonlinear equation has to be solved. This problem arises, mainly, when equality constrints are defined over probabilities indifferent combinations of variables and latent classes. A simple condition is given in which, although probabilities in different variable-latent class combinations are constrained to be equal, the EM algorithm is still simple to apply.The authors are grateful to the Editor and the anonymous reviewers for their helpful comments on an earlier draft of this paper. C. C. Clogg and R. Luijkx are also acknowledged for verifying our results with their computer programs MLLSA and LCAG, respectively.     

Wisdom's Information: Rereading a Biblical Image in the Light of Some Contemporary Science and Speculation

The biblical image of Wisdom as the power who "orders all things well" in nature and in human life can be read in the light of contemporary information theory. Some current scientific speculation offers an interpretation of reality as a vast information-processing system, in which informational situations are continuously transformed through algorithmic operations. This interpretation finds a metaphysical counterpart in the distinction between "nature natured" and "nature naturing" in the philosophical theology of Samuel Taylor Coleridge. This confluence of religious, metaphysical, and scientific imagery suggests a picture of the world in which the processes of "nature naturing,""human humaning," and "God Godding" inform and recur in each other.     

Additive structure in qualitative data: An alternating least squares method with optimal scaling features

A method is developed to investigate the additive structure of data that (a) may be measured at the nominal, ordinal or cardinal levels, (b) may be obtained from either a discrete or continuous source, (c) may have known degrees of imprecision, or (d) may be obtained in unbalanced designs. The method also permits experimental variables to be measured at the ordinal level. It is shown that the method is convergent, and includes several previously proposed methods as special cases. Both Monte Carlo and empirical evaluations indicate that the method is robust.This research was supported in part by grant MH-10006 from the National Institute of Mental Health to the Psychometric Laboratory of the University of North Carolina. We wish to thank Thomas S. Wallsten for comments on an earlier draft of this paper. Copies of the paper and of ADDALS, a program to perform the analyses discussed herein, may be obtained from the second author.     

FACAIC: Model selection algorithm for the orthogonal factor model using AIC and CAIC

This paper describes the authors' FORTRAN algorithm FACAIC for choosing the number of factors for an orthogonal factor model using Akaike's Information Criterion. FACAIC utilizes the IMSL subroutine OFCOMM.The authors dedicate this algorithm to Professor Hirotugu Akaike in appreciation of his pioneering work on AIC which was originally intended for the factor analysis and other statistical model identification problems.     

Consistent estimation in the rasch model based on nonparametric margins

Consider the class of two parameter marginal logistic (Rasch) models, for a test ofm True-False items, where the latent ability is assumed to be bounded. Using results of Karlin and Studen, we show that this class of nonparametric marginal logistic (NML) models is equivalent to the class of marginal logistic models where the latent ability assumes at most (m + 2)/2 values. This equivalence has two implications. First, estimation for the NML model is accomplished by estimating the parameters of a discrete marginal logistic model. Second, consistency for the maximum likelihood estimates of the NML model can be shown (whenm is odd) using the results of Kiefer and Wolfowitz. An example is presented which demonstrates the estimation strategy and contrasts the NML model with a normal marginal logistic model.This research was supported by NIMH traning grant, 2 T32 MH 15758-06 and by ONR contract N00014-84-K-0588. The author would like to thank Diane Lambert, John Rolph, and Stephen Fienberg for their assistance. Also, the comments of the referees helped to substantially improve the final version of this paper.     

Maximum likelihood additivity analysis

A maximum likelihood estimation procedure was developed to fit unweighted and weighted additive models to conjoint data obtained by the categorical rating, the pair comparison or the directional ranking method. The scoring algorithm used to fit the models was found to be both reliable and efficient, and the program MAXADD is capable of handling up to 300 parameters to be estimated. Practical uses of the procedure are reported to demonstrate various advantages of the procedure as a statistical method.The research reported here was supported by Grant A6394 to the author from the Natural Sciences and Engineering Research Council of Canada. Portions of this research were presented at the Psychometric Society meeting in Iowa City, Iowa, in May, 1980.Thanks are due to Jim Ramsay, Justine Sergent and anonymous reviewers for their helpful comments.Two MAXADD programs which perform the computations discussed in this paper may be obtained from the author.     

Animal foraging and the evolution of goal-directed cognition

Foraging- and feeding-related behaviors across eumetazoans share similar molecular mechanisms, suggesting the early evolution of an optimal foraging behavior called area-restricted search (ARS), involving mechanisms of dopamine and glutamate in the modulation of behavioral focus. Similar mechanisms in the vertebrate basal ganglia control motor behavior and cognition and reveal an evolutionary progression toward increasing internal connections between prefrontal cortex and striatum in moving from amphibian to primate. The basal ganglia in higher vertebrates show the ability to transfer dopaminergic activity from unconditioned stimuli to conditioned stimuli. The evolutionary role of dopamine in the modulation of goal-directed behavior and cognition is further supported by pathologies of human goal-directed cognition, which have motor and cognitive dysfunction and organize themselves, with respect to dopaminergic activity, along the gradient described by ARS, from perseverative to unfocused. The evidence strongly supports the evolution of goal-directed cognition out of mechanisms initially in control of spatial foraging but, through increasing cortical connections, eventually used to forage for information.     

High-dimensional maximum marginal likelihood item factor analysis by adaptive quadrature

Although the Bock–Aitkin likelihood-based estimation method for factor analysis of dichotomous item response data has important advantages over classical analysis of item tetrachoric correlations, a serious limitation of the method is its reliance on fixed-point Gauss-Hermite (G-H) quadrature in the solution of the likelihood equations and likelihood-ratio tests. When the number of latent dimensions is large, computational considerations require that the number of quadrature points per dimension be few. But with large numbers of items, the dispersion of the likelihood, given the response pattern, becomes so small that the likelihood cannot be accurately evaluated with the sparse fixed points in the latent space. In this paper, we demonstrate that substantial improvement in accuracy can be obtained by adapting the quadrature points to the location and dispersion of the likelihood surfaces corresponding to each distinct pattern in the data. In particular, we show that adaptive G-H quadrature, combined with mean and covariance adjustments at each iteration of an EM algorithm, produces an accurate fast-converging solution with as few as two points per dimension. Evaluations of this method with simulated data are shown to yield accurate recovery of the generating factor loadings for models of upto eight dimensions. Unlike an earlier application of adaptive Gibbs sampling to this problem by Meng and Schilling, the simulations also confirm the validity of the present method in calculating likelihood-ratio chi-square statistics for determining the number of factors required in the model. Finally, we apply the method to a sample of real data from a test of teacher qualifications.     

On local influence analysis of full information item factor models

The full information item factor (FIIF) model is very useful for analyzing relations of dichotomous variables. In this article, we present a feasible procedure to assess local influence of minor perturbations for identifying influence aspects of the FIIF model. The development is based on a Q-displacement function which is closely related with the Monte Carlo EM algorithm in the ML estimation. In the E-step of this algorithm, the conditional expectations are approximated by sample means of observations simulated by the Gibbs sampler from the appropriate conditional distributions. It turns out that these observations can be utilized for computing the building blocks of the proposed diagnostic measures. The diagnoses are based on the conformal normal curvature that can be computed easily. A number of interesting perturbation schemes are considered. The methodology is illustrated with two real examples.The research is fully supported by a grant (CUHK 4356/00H) from the Research Grant Council of the Hong Kong Special Administration Region. The authors are thankful to the Editor, Associate Editor, anonymous reviewers, and W.Y. Poon for valuable comments for improving the paper, and to ICPSR and the relevant founding agency for allowing us to use of their data. The assistance of Michael Leung and Esther Tam is gratefully acknowledged.     

The statistical analysis of general processing tree models with the EM algorithm

Multinomial processing tree models assume that an observed behavior category can arise from one or more processing sequences represented as branches in a tree. These models form a subclass of parametric, multinomial models, and they provide a substantively motivated alternative to loglinear models. We consider the usual case where branch probabilities are products of nonnegative integer powers in the parameters, 0_s1, and their complements, 1 - _s. A version of the EM algorithm is constructed that has very strong properties. First, the E-step and the M-step are both analytic and computationally easy; therefore, a fast PC program can be constructed for obtaining MLEs for large numbers of parameters. Second, a closed form expression for the observed Fisher information matrix is obtained for the entire class. Third, it is proved that the algorithm necessarily converges to a local maximum, and this is a stronger result than for the exponential family as a whole. Fourth, we show how the algorithm can handle quite general hypothesis tests concerning restrictions on the model parameters. Fifth, we extend the algorithm to handle the Read and Cressie power divergence family of goodness-of-fit statistics. The paper includes an example to illustrate some of these results.This research was supported by National Science Foundation Grant BNS-8910552 to William H. Batchelder and David M. Riefer. We are grateful to David Riefer for his useful comments, and to the Institute for Mathematical Behavior Sciences for its support.