Interval estimation for the intraclass correlation in dirichlet-multinomial data

When the underlying distribution is discrete with a limited number of categories, methods for interval estimation of the intraclass correlation which assume normality are theoretically inadequate for use. On the basis of large sample theory, this paper develops an asymptotic closed-form interval estimate of the intraclass correlation for the case where there is a natural score associated with each category. This paper employs Monte Carlo simulation to demonstrate that when the underlying intraclass correlation is large, the traditional interval estimator which assumes normality can be misleading. We find that when the number of classes is 20, the interval estimator proposed here can generally perform reasonably well in a variety of situations. This paper further notes that the proposed interval estimator is invariant with respect to a linear transformation. When the data are on a nominal scale, an extension of the proposed method to account for this case, as well as a discussion on the relationship between the intraclass correlation and a kappa-type measure defined here and on the limitation of the corresponding kappa-type estimator are given.The authors wish to thank the Editor, the Associate Editor, and the three referees for many valuable comments and suggestions to improve the clarity of this paper. The works for the first, the third, and the fourth authors were partially supported by grant #R01AR43025-01 from the National Institute of Arthritis and Musculoskeletal and Skin Diseases.     

Multilevel Model Prediction

Multilevel models are proven tools in social research for modeling complex, hierarchical systems. In multilevel modeling, statistical inference is based largely on quantification of random variables. This paper distinguishes among three types of random variables in multilevel modeling—model disturbances, random coefficients, and future response outcomes—and provides a unified procedure for predicting them. These predictors are best linear unbiased and are commonly known via the acronym BLUP; they are optimal in the sense of minimizing mean square error and are Bayesian under a diffuse prior. For parameter estimation purposes, a multilevel model can be written as a linear mixed-effects model. In this way, parameters of the many equations can be estimated simultaneously and hence efficiently. For prediction purposes, we show that it is more convenient to retain the multiple equation feature of multilevel models. In this way, the efficient BLUPs are easy to compute and retain their intuitively appealing recursive form. We also derive explicit equations for standard errors of these different types of predictors. Prediction in multilevel modeling is important in a wide range of applications. To demonstrate the applicability of our results, this paper discusses prediction in the context of a study of school effectiveness. This research was supported by a grant from the Graduate School at the University of Wisconsin at Madision and the National Science Foundation, Grant number SES-0436274. We are grateful to Norman Webb at Wisconsin Center for Education Research for making available the data used in the reported application.     

Bayes modal estimation in item response models

This article describes a Bayesian framework for estimation in item response models, with two-stage prior distributions on both item and examinee populations. Strategies for point and interval estimation are discussed, and a general procedure based on the EM algorithm is presented. Details are given for implementation under one-, two-, and three-parameter binary logistic IRT models. Novel features include minimally restrictive assumptions about examinee distributions and the exploitation of dependence among item parameters in a population of interest. Improved estimation in a moderately small sample is demonstrated with simulated data.This research was supported by a grant from the Spencer Foundation, Chicago, IL. Comments and suggestions on earlier drafts by Charles Lewis, Frederic Lord, Rosenbaum, James Ramsey, Hiroshi Watanabe, the editor, and two anonymous referees are gratefully acknowledged.     

A note on moran's measure of multiple rank correlation

A measure of multiple rank correlation,T _y.12 ², is proposed for the situation with no tied observations in the variables. The measure is a weighted average of two squared Kendall taus. It is shown thatT _y.12 ² is equivalent to a statistic previously proposed by Moran and thus a new interpretation is given to Moran's statistic.The author wishes to thank Nancy Anderson, Willard Larkin, and Kent Norman for their helpful comments.     

A note on the estimation of the level of predictive precision of a fitted linear equation

A procedure that utilizes the sample multiple correlation to form a lower bound for the level of predictive precision of a fitted regression equation is suggested. The procedure is shown to yield probability statements which are true at least 100(1–)% of the time.     

A short note on the maximal point‐biserial correlation under non‐normality

The aim of this paper is to derive the maximal point‐biserial correlation under non‐normality. Several widely used non‐normal distributions are considered, namely the uniform distribution, t‐distribution, exponential distribution, and a mixture of two normal distributions. Results show that the maximal point‐biserial correlation, depending on the non‐normal continuous variable underlying the binary manifest variable, may not be a function of p (the probability that the dichotomous variable takes the value 1), can be symmetric or non‐symmetric around p = .5, and may still lie in the range from ?1.0 to 1.0. Therefore researchers should exercise caution when they interpret their sample point‐biserial correlation coefficients based on popular beliefs that the maximal point‐biserial correlation is always smaller than 1, and that the size of the correlation is always further restricted as p deviates from .5.     

Maximum likelihood estimation of the polychoric correlation coefficient

The polychoric correlation is discussed as a generalization of the tetrachoric correlation coefficient to more than two classes. Two estimation methods are discussed: Maximum likelihood estimation, and what may be called two-step maximum likelihood estimation. For the latter method, the thresholds are estimated in the first step. For both methods, asymptotic covariance matrices for estimates are derived, and the methods are illustrated and compared with artificial and real data.This paper was read at the 1978 European Meeting on Psychometrics and Mathematical Psychology in Uppsala, Sweden, June 1978.Research reported in this paper has been supported by the Bank of Sweden Tercentenary Foundation under project Structural Equation Models in the Social Sciences, project director Karl G. Jöreskog.     

Robust estimation of the hierarchical model for responses and response times

Van der Linden's (2007, Psychometrika, 72, 287) hierarchical model for responses and response times in tests has numerous applications in psychological assessment. The success of these applications requires the parameters of the model to have been estimated without bias. The data used for model fitting, however, are often contaminated, for example, by rapid guesses or lapses of attention. This distorts the parameter estimates. In the present paper, a novel estimation approach is proposed that is robust against contamination. The approach consists of two steps. In the first step, the response time model is fitted on the basis of a robust estimate of the covariance matrix. In the second step, the item response model is extended to a mixture model, which allows for a proportion of irregular responses in the data. The parameters of the mixture model are then estimated with a modified marginal maximum likelihood estimator. The modified marginal maximum likelihood estimator downweights responses of test-takers with unusual response time patterns. As a result, the estimator is resistant to several forms of data contamination. The robustness of the approach is investigated in a simulation study. An application of the estimator is demonstrated with real data.     

A note on the estimation of nonspurious correlations

A method is provided for estimating the nonspurious correlation of a part of a test with the total test. Two cases are considered: one in which the actual subtest is parallel to the total test, the other in which the actual subtest is not parallel to the total test.     

Weighted likelihood estimation of ability in item response theory

Applications of item response theory, which depend upon its parameter invariance property, require that parameter estimates be unbiased. A new method, weighted likelihood estimation (WLE), is derived, and proved to be less biased than maximum likelihood estimation (MLE) with the same asymptotic variance and normal distribution. WLE removes the first order bias term from MLE. Two Monte Carlo studies compare WLE with MLE and Bayesian modal estimation (BME) of ability in conventional tests and tailored tests, assuming the item parameters are known constants. The Monte Carlo studies favor WLE over MLE and BME on several criteria over a wide range of the ability scale.     

The relationship between multilevel models and non‐parametric multilevel mixture models: Discrete approximation of intraclass correlation,random coefficient distributions,and residual heteroscedasticity

Multilevel data structures are common in the social sciences. Often, such nested data are analysed with multilevel models (MLMs) in which heterogeneity between clusters is modelled by continuously distributed random intercepts and/or slopes. Alternatively, the non‐parametric multilevel regression mixture model (NPMM) can accommodate the same nested data structures through discrete latent class variation. The purpose of this article is to delineate analytic relationships between NPMM and MLM parameters that are useful for understanding the indirect interpretation of the NPMM as a non‐parametric approximation of the MLM, with relaxed distributional assumptions. We define how seven standard and non‐standard MLM specifications can be indirectly approximated by particular NPMM specifications. We provide formulas showing how the NPMM can serve as an approximation of the MLM in terms of intraclass correlation, random coefficient means and (co)variances, heteroscedasticity of residuals at level 1, and heteroscedasticity of residuals at level 2. Further, we discuss how these relationships can be useful in practice. The specific relationships are illustrated with simulated graphical demonstrations, and direct and indirect interpretations of NPMM classes are contrasted. We provide an R function to aid in implementing and visualizing an indirect interpretation of NPMM classes. An empirical example is presented and future directions are discussed.     

A mixture model for distributions of correlation coefficients

An old problem in personnel psychology is to characterize distributions of test validity correlation coefficients. The proposed model views histograms of correlation coefficients as observations from a mixture distribution which, for a fixed sample sizen, is a conditional mixture distributionh(r|n) = _{j j} h(r; _j ,n), whereR is the correlation coefficient, _j are population correlation coefficients and _j are the mixing weights. The associated marginal distribution ofR is regarded as the parent distribution underlying histograms of empirical correlation coefficients. Maximum likelihood estimates of the parameters _j and _j can be obtained with an EM algorithm solution and tests for the number of componentst are achieved after the (one-component) density ofR is replaced with a tractable modeling densityh(r; _j ,n). Two illustrative examples are provided.     

Bayesian estimation of item response curves

Item response curves for a set of binary responses are studied from a Bayesian viewpoint of estimating the item parameters. For the two-parameter logistic model with normally distributed ability, restricted bivariate beta priors are used to illustrate the computation of the posterior mode via the EM algorithm. The procedure is illustrated by data from a mathematics test.This work was supported under Contract No. N00014-85-K-0113, NR 150-535, from Personnel and Training Research Programs, Psychological Sciences Division, Office of Naval Research. The authors wish to thank Mark D. Reckase for providing the ACT data used in the illustration and Michael J. Soltys for computational assistance. They also wish to thank the editor and four anonymous reviewers for many valuable suggestions.     

A generalized family of coefficients of relational agreement for numerical scales

A family of coefficients of relational agreement for numerical scales is proposed. The theory is a generalization to multiple judges of the Zegers and ten Berge theory of association coefficients for two variables and is based on the premise that the choice of a coefficient depends on the scale type of the variables, defined by the class of admissible transformations. Coefficients of relational agreement that denote agreement with respect to empirically meaningful relationships are derived for absolute, ratio, interval, and additive scales. The proposed theory is compared to intraclass correlation, and it is shown that the coefficient of additivity is identical to one measure of intraclass correlation.The author thanks the Editor and anonymous reviewers for helpful suggestions.     

A parametric procedure for ultrametric tree estimation from conditional rank order proximity data

The psychometric and classification literatures have illustrated the fact that a wide class of discrete or network models (e.g., hierarchical or ultrametric trees) for the analysis of ordinal proximity data are plagued by potential degenerate solutions if estimated using traditional nonmetric procedures (i.e., procedures which optimize a STRESS-based criteria of fit and whose solutions are invariant under a monotone transformation of the input data). This paper proposes a new parametric, maximum likelihood based procedure for estimating ultrametric trees for the analysis of conditional rank order proximity data. We present the technical aspects of the model and the estimation algorithm. Some preliminary Monte Carlo results are discussed. A consumer psychology application is provided examining the similarity of fifteen types of snack/breakfast items. Finally, some directions for future research are provided.     

A note on ALSCAL: The estimation of the additive constant

A method is given for the least squares regression of a squared variabled ² on the squared variable 0², where the relation between 0 andd is linear. The problem arises in MDS-algorithms where the loss function is defined in terms of squared distances (e.g., ALSCAL). It is pointed out that the Lagrangian multiplier is a root of fourth degree polynomial.     

A note on Roy's largest root

The name Roy's largest root and similar names are used in practice to label two different but functionally related statistics—one proportional to anF, and the other, a squared canonical correlation. This note presents the logic that leads to the two formulations, states which statistic some popular statistical packages use, and shows the possible source of this inconsistency in the original work of Roy (1953) and Heck (1960).     

A note on the investigation of local and global identifiability

In general, nonlinear models such as those commonly employed for the analysis of covariance structures, are not globally identifiable. Any investigation of local identifiability must either yield a mapping of identifiability onto the entire parameter space, which will rarely be feasible in any applications of interest, or confine itself to the neighbourhood of such points of special interest as the maximum likelihood point.The author would like to thank J. Jack McArdle and Colin Fraser for their comments on this paper.