Limited‐information goodness‐of‐fit testing of item response theory models for sparse 2P tables

Bartholomew and Leung proposed a limited‐information goodness‐of‐fit test statistic (Y) for models fitted to sparse 2^P contingency tables. The null distribution of Y was approximated using a chi‐squared distribution by matching moments. The moments were derived under the assumption that the model parameters were known in advance and it was conjectured that the approximation would also be appropriate when the parameters were to be estimated. Using maximum likelihood estimation of the two‐parameter logistic item response theory model, we show that the effect of parameter estimation on the distribution of Y is too large to be ignored. Consequently, we derive the asymptotic moments of Y for maximum likelihood estimation. We show using a simulation study that when the null distribution of Y is approximated using moments that take into account the effect of estimation, Y becomes a very useful statistic to assess the overall goodness of fit of models fitted to sparse 2^P tables.     

Constrained multidimensional scaling inN spaces

A gradient method is used to obtain least squares estimates of parameters of them-dimensional euclidean model simultaneously inN spaces, given the observation of all pairwise distances ofn stimuli for each space. The procedure can estimate an additive constant as well as stimulus projections and the metric of the reference axes of the configuration in each space. Each parameter in the model can be fixed to equal some a priori value, constrained to be equal to any other parameter, or free to take on any value in the parameter space. Two applications of the procedure are described.     

Mean Comparison: Manifest Variable Versus Latent Variable

Mean comparisons are of great importance in the application of statistics. Procedures for mean comparison with manifest variables have been well studied. However, few rigorous studies have been conducted on mean comparisons with latent variables, although the methodology has been widely used and documented. This paper studies the commonly used statistics in latent variable mean modeling and compares them with parallel manifest variable statistics. Our results indicate that, under certain conditions, the likelihood ratio and Wald statistics used for latent mean comparisons do not always have greater power than the Hotelling T² statistics used for manifest mean comparisons. The noncentrality parameter corresponding to the T² statistic can be much greater than those corresponding to the likelihood ratio and Wald statistics, which we find to be different from those provided in the literature. Under a fixed alternative hypothesis, our results also indicate that the likelihood ratio statistic can be stochastically much greater than the corresponding Wald statistic. The robustness property of each statistic is also explored when the model is misspecified or when data are nonnormally distributed. Recommendations and advice are provided for the use of each statistic. The research was supported by NSF grant DMS-0437167 and Grant DA01070 from the National Institute on Drug Abuse. We would like to thank three referees for suggestions that helped in improving the paper.     

A Nondegenerate Penalized Likelihood Estimator for Variance Parameters in Multilevel Models

Group-level variance estimates of zero often arise when fitting multilevel or hierarchical linear models, especially when the number of groups is small. For situations where zero variances are implausible a priori, we propose a maximum penalized likelihood approach to avoid such boundary estimates. This approach is equivalent to estimating variance parameters by their posterior mode, given a weakly informative prior distribution. By choosing the penalty from the log-gamma family with shape parameter greater than 1, we ensure that the estimated variance will be positive. We suggest a default log-gamma(2,λ) penalty with λ→0, which ensures that the maximum penalized likelihood estimate is approximately one standard error from zero when the maximum likelihood estimate is zero, thus remaining consistent with the data while being nondegenerate. We also show that the maximum penalized likelihood estimator with this default penalty is a good approximation to the posterior median obtained under a noninformative prior. Our default method provides better estimates of model parameters and standard errors than the maximum likelihood or the restricted maximum likelihood estimators. The log-gamma family can also be used to convey substantive prior information. In either case—pure penalization or prior information—our recommended procedure gives nondegenerate estimates and in the limit coincides with maximum likelihood as the number of groups increases.     

Modelling inter-individual differences in latent within-person variation: The confirmatory factor level variability model

Psychological theories often produce hypotheses that pertain to individual differences in within-person variability. To empirically test the predictions entailed by such hypotheses with longitudinal data, researchers often use multilevel approaches that allow them to model between-person differences in the mean level of a certain variable and the residual within-person variance. Currently, these approaches can be applied only when the data stem from a single variable. However, it is common practice in psychology to assess not just a single measure but rather several measures of a construct. In this paper we describe a model in which we combine the single-indicator model with confirmatory factor analysis. The new model allows individual differences in latent mean-level factors and latent within-person variability factors to be estimated. Furthermore, we show how the model's parameters can be estimated with a maximum likelihood estimator, and we illustrate the approach using an example that involves intensive longitudinal data.     

On the existence and uniqueness of JML estimates for the partial credit model

A necessary and sufficient condition is given in this paper for the existence and uniqueness of the maximum likelihood (the so-called joint maximum likelihood) estimate of the parameters of the Partial Credit Model. This condition is stated in terms of a structural property of the pattern of the data matrix that can be easily verified on the basis of a simple iterative procedure. The result is proved by using an argument of Haberman (1977). The author wishes to thank the Editor and the anonymous reviewers for their comments that helped to substantially improve the final version of this paper. This research was supported in part by a MURST grant (ex 60%).     

Limited Information Goodness-of-fit Testing in Multidimensional Contingency Tables

We introduce a family of goodness-of-fit statistics for testing composite null hypotheses in multidimensional contingency tables. These statistics are quadratic forms in marginal residuals up to order r. They are asymptotically chi-square under the null hypothesis when parameters are estimated using any asymptotically normal consistent estimator. For a widely used item response model, when r is small and multidimensional tables are sparse, the proposed statistics have accurate empirical Type I errors, unlike Pearson's X ². For this model in nonsparse situations, the proposed statistics are also more powerful than X ². In addition, the proposed statistics are asymptotically chi-square when applied to subtables, and can be used for a piecewise goodness-of-fit assessment to determine the source of misfit in poorly fitting models. This research has been supported by the Department of Universities, Research, and Information Society (DURSI) of the Catalan Government, by grant BSO2003-08507 of the Spanish Ministry of Science and Technology, and an NSERC Canada grant. We are grateful to the referees for comments leading to improvements.     

Assessing Approximate Fit in Categorical Data Analysis

A family of Root Mean Square Error of Approximation (RMSEA) statistics is proposed for assessing the goodness of approximation in discrete multivariate analysis with applications to item response theory (IRT) models. The family includes RMSEAs to assess the approximation up to any level of association of the discrete variables. Two members of this family are RMSEA₂, which uses up to bivariate moments, and the full information RMSEA_n. The RMSEA₂ is estimated using the M₂ statistic of Maydeu-Olivares and Joe (2005, 2006), whereas for maximum likelihood estimation, RMSEA_n is estimated using Pearson's X² statistic. Using IRT models, we provide cutoff criteria of adequate, good, and excellent fit using the RMSEA₂. When the data are ordinal, we find a strong linear relationship between the RMSEA₂ and the Standardized Root Mean Squared Residual goodness-of-fit index. We are unable to offer cutoff criteria for the RMSEA_n as its population values decrease as the number of variables and categories increase.     

A computer program for Spearman-Kärber and probit analysis of psychometric function data

PMETRIC is a computer program for the analysis of observed psychometric functions. It can estimate the parameters of these functions, using either probit analysis (a parametric technique) or the Spearman-Kärber method (a nonparametric one). For probit analysis, either a maximum likelihood or a minimum χ² criterion may be used for parameter estimation. In addition, standard errors of parameter estimates can be estimated via bootstrapping. The program can be used to analyze data obtained from either yes-no orm-alternative forced-choice tasks. To facilitate the use of PMETRIC in simulation work, an associated program, PMETGEN, is provided for the generation of simulated psychometric function data. Use of PMETRIC is illustrated with data from a duration discrimination task.     

Joint Maximum Likelihood Estimation for High-Dimensional Exploratory Item Factor Analysis

Joint maximum likelihood (JML) estimation is one of the earliest approaches to fitting item response theory (IRT) models. This procedure treats both the item and person parameters as unknown but fixed model parameters and estimates them simultaneously by solving an optimization problem. However, the JML estimator is known to be asymptotically inconsistent for many IRT models, when the sample size goes to infinity and the number of items keeps fixed. Consequently, in the psychometrics literature, this estimator is less preferred to the marginal maximum likelihood (MML) estimator. In this paper, we re-investigate the JML estimator for high-dimensional exploratory item factor analysis, from both statistical and computational perspectives. In particular, we establish a notion of statistical consistency for a constrained JML estimator, under an asymptotic setting that both the numbers of items and people grow to infinity and that many responses may be missing. A parallel computing algorithm is proposed for this estimator that can scale to very large datasets. Via simulation studies, we show that when the dimensionality is high, the proposed estimator yields similar or even better results than those from the MML estimator, but can be obtained computationally much more efficiently. An illustrative real data example is provided based on the revised version of Eysenck’s Personality Questionnaire (EPQ-R).

     

Investigating the Performance of Alternate Regression Weights by Studying All Possible Criteria in Regression Models with a Fixed Set of Predictors

We describe methods for assessing all possible criteria (i.e., dependent variables) and subsets of criteria for regression models with a fixed set of predictors, x (where x is an n×1 vector of independent variables). Our methods build upon the geometry of regression coefficients (hereafter called regression weights) in n-dimensional space. For a full-rank predictor correlation matrix, R _xx, of order n, and for regression models with constant R ² (coefficient of determination), the OLS weight vectors for all possible criteria terminate on the surface of an n-dimensional ellipsoid. The population performance of alternate regression weights—such as equal weights, correlation weights, or rounded weights—can be modeled as a function of the Cartesian coordinates of the ellipsoid. These geometrical notions can be easily extended to assess the sampling performance of alternate regression weights in models with either fixed or random predictors and for models with any value of R ². To illustrate these ideas, we describe algorithms and R (R Development Core Team, 2009) code for: (1) generating points that are uniformly distributed on the surface of an n-dimensional ellipsoid, (2) populating the set of regression (weight) vectors that define an elliptical arc in ℝ ⁿ , and (3) populating the set of regression vectors that have constant cosine with a target vector in ℝ ⁿ . Each algorithm is illustrated with real data. The examples demonstrate the usefulness of studying all possible criteria when evaluating alternate regression weights in regression models with a fixed set of predictors.     

Simultaneous factor analysis in several populations

This paper is concerned with the study of similarities and differences in factor structures between different groups. A common situation occurs when a battery of tests has been administered to samples of examinees from several populations.A very general model is presented, in which any parameter in the factor analysis models (factor loadings, factor variances, factor covariances, and unique variances) for the different groups may be assigned an arbitrary value or constrained to be equal to some other parameter. Given such a specification, the model is estimated by the maximum likelihood method yielding a large samplex ² of goodness of fit. By computing several solutions under different specifications one can test various hypotheses.The method is capable of dealing with any degree of invariance, from the one extreme, where nothing is invariant, to the other extreme, where everything is invariant. Neither the number of tests nor the number of common factors need to be the same for all groups, but to be at all interesting, it is assumed that there is a common core of tests in each battery that is the same or at least content-wise comparable.This research was supported by grant NSF-GB-12959 from National Science Foundation. My thanks are due to Michael Browne for his comments on an earlier draft of this paper and to Marielle van Thillo who checked the mathematical derivations and wrote and debugged the computer program SIFASP.Now at Statistics Department, University of Uppsala, Sweden.     

Robust estimation of the hierarchical model for responses and response times

Van der Linden's (2007, Psychometrika, 72, 287) hierarchical model for responses and response times in tests has numerous applications in psychological assessment. The success of these applications requires the parameters of the model to have been estimated without bias. The data used for model fitting, however, are often contaminated, for example, by rapid guesses or lapses of attention. This distorts the parameter estimates. In the present paper, a novel estimation approach is proposed that is robust against contamination. The approach consists of two steps. In the first step, the response time model is fitted on the basis of a robust estimate of the covariance matrix. In the second step, the item response model is extended to a mixture model, which allows for a proportion of irregular responses in the data. The parameters of the mixture model are then estimated with a modified marginal maximum likelihood estimator. The modified marginal maximum likelihood estimator downweights responses of test-takers with unusual response time patterns. As a result, the estimator is resistant to several forms of data contamination. The robustness of the approach is investigated in a simulation study. An application of the estimator is demonstrated with real data.     

Alternative approaches to factorial invariance

Special cases of the factor analysis model are developed for four selection situations. Methods are suggested whereby parameters in each case can be estimated using a maximum likelihood procedure recently developed by Jöreskog. Also, a numerical illustration is presented for each case.This research was supported by a grant from the University Research Council, Vanderbilt University.     

The Performance of Multivariate Methods for Two-Group Comparisons with Small Samples and Incomplete Data

Abstract

In intervention studies having multiple outcomes, researchers often use a series of univariate tests (e.g., ANOVAs) to assess group mean differences. Previous research found that this approach properly controls Type I error and generally provides greater power compared to MANOVA, especially under realistic effect size and correlation combinations. However, when group differences are assessed for a specific outcome, these procedures are strictly univariate and do not consider the outcome correlations, which may be problematic with missing outcome data. Linear mixed or multivariate multilevel models (MVMMs), implemented with maximum likelihood estimation, present an alternative analysis option where outcome correlations are taken into account when specific group mean differences are estimated. In this study, we use simulation methods to compare the performance of separate independent samples t tests estimated with ordinary least squares and analogous t tests from MVMMs to assess two-group mean differences with multiple outcomes under small sample and missingness conditions. Study results indicated that a MVMM implemented with restricted maximum likelihood estimation combined with the Kenward–Roger correction had the best performance. Therefore, for intervention studies with small N and normally distributed multivariate outcomes, the Kenward–Roger procedure is recommended over traditional methods and conventional MVMM analyses, particularly with incomplete data.     

Reaction of a Ni-coated Al nanoparticle to form B2-NiAl: A molecular dynamics study

The kinetic reaction in a Ni-coated Al nanoparticle with equi-atomic fractions and diameter of approximately 4.5 nm is studied by means of molecular dynamics simulation, using a potential of the embedded atom type to model the interatomic interactions. First, the large driving force for the alloying of Ni and Al initiates solid state amorphization of the nanoparticle with the formation of Ni₅₀Al₅₀ amorphous alloy. Amorphization makes intermixing of the components much easier compared to the crystalline state. The average rate of penetration of Ni atoms can be estimated to be about two times higher than Al atoms, whilst the total rate of inter-penetration can be estimated to be of the order of 10^?2 m/s. The heat of the intermixing with the formation of Ni₅₀Al₅₀ amorphous alloy can be estimated at approximately ?0.34 eV/at. Next, the crystallization of the Ni₅₀Al₅₀ amorphous alloy into B2-NiAl ordered crystal structure is observed. The heat of the crystallization can be estimated as approximately ?0.08 eV/at. Then, the B2-NiAl ordered nanoparticle melts at a temperature of approximately 1500 K. It is shown that, for the alloying reaction in the initial Ni-coated Al nanoparticle, the ignition temperature can be as low as approximately 200 K, while the adiabatic temperature for the reaction is below the melting temperature of the nanoparticle with the B2-NiAl ordered structure.     

A goodness of fit test for the rasch model

The Rasch model is an item analysis model with logistic item characteristic curves of equal slope,i.e. with constant item discriminating powers. The proposed goodness of fit test is based on a comparison between difficulties estimated from different scoregroups and over-all estimates. Based on the within scoregroup estimates and the over-all estimates of item difficulties a conditional likelihood ratio is formed. It is shown that—2 times the logarithm of this ratio isx ²-distributed when the Rasch model is true. The power of the proposed goodness of fit test is discussed for alternative models with logistic item characteristic curves, but unequal discriminating items from a scholastic aptitude test.     

On the robustness of maximum likelihood scaling for violations of the error model

A Monte Carlo study was conducted to investigate the robustness of the assumed error distribution in maximum likelihood estimation models for multidimensional scaling. Data sets generated according to the lognormal, the normal, and the rectangular distribution were analysed with the log-normal error model in Ramsay's MULTISCALE program package. The results show that violations of the assumed error distribution have virtually no effect on the estimated distance parameters. In a comparison among several dimensionality tests, the corrected version of thex ² test, as proposed by Ramsay, yielded the best results, and turned out to be quite robust against violations of the error model.     

A multistage ranking model

In this paper, we propose a (n–1)² parameter, multistage ranking model, which represents a generalization of Luce's model. We propose then×n item-rank relative frequency matrix (p-matrix) as a device for summarizing a set of rankings. As an alternative to the traditional maximum likelihood estimation, for the proposed model we suggest a method which estimates the parameters from thep-matrix. An illustrative numerical example is given. The proposed model and its differences from Luce's model are briefly discussed. We also show some specialp-matrix patterns possessed by the Thurstonian models and distance-based models.     

Comparing Latent Structures of the Grade of Membership,Rasch, and Latent Class Models

This paper focuses on model interpretation issues and employs a geometric approach to compare the potential value of using the Grade of Membership (GoM) model in representing population heterogeneity. We consider population heterogeneity manifolds generated by letting subject specific parameters vary over their natural range, while keeping other population parameters fixed, in the marginal space (based on marginal probabilities) and in the full parameter space (based on cell probabilities). The case of a 2 × 2 contingency table is discussed in detail, and a generalization to 2^J tables with J ≥ 3 is sketched. Our approach highlights the main distinction between the GoM model and the probabilistic mixture of classes by demonstrating geometrically the difference between the concepts of partial and probabilistic memberships. By using the geometric approach we show that, in special cases, the GoM model can be thought of as being similar to an item response theory (IRT) model in representing population heterogeneity. Finally, we show that the GoM item parameters can provide quantities analogous to more general logistic IRT item parameters. As a latent structure model, the GoM model might be considered a useful alternative for a data analysis when both classes of extreme responses, and additional heterogeneity that cannot be captured by those latent classes, are expected in the population. This work was supported by Award #1R03 AG18986-01 from the National Institute on Aging and NIH grant #1R01 CA94212-01. The presentation of the ideas in this paper owes much to discussions with Stephen Fienberg and Brian Junker, Carnegie Mellon University. The author thanks Jim Ramsay and two anonymous reviewers for their valuable comments on earlier drafts of this paper.