Structural Modeling of Measurement Error in Generalized Linear Models with Rasch Measures as Covariates

This paper proposes a structural analysis for generalized linear models when some explanatory variables are measured with error and the measurement error variance is a function of the true variables. The focus is on latent variables investigated on the basis of questionnaires and estimated using item response theory models. Latent variable estimates are then treated as observed measures of the true variables. This leads to a two-stage estimation procedure which constitutes an alternative to a joint model for the outcome variable and the responses given to the questionnaire. Simulation studies explore the effect of ignoring the true error structure and the performance of the proposed method. Two illustrative examples concern achievement data of university students. Particular attention is given to the Rasch model.     

Structural Equation Modeling of Social Networks: Specification,Estimation, and Application

Psychologists are interested in whether friends and couples share similar personalities or not. However, no statistical models are readily available to test the association between personalities and social relations in the literature. In this study, we develop a statistical model for analyzing social network data with the latent personality traits as covariates. Because the model contains a measurement model for the latent traits and a structural model for the relationship between the network and latent traits, we discuss it under the general framework of structural equation modeling (SEM). In our model, the structural relation between the latent variable(s) and the outcome variable is no longer linear or generalized linear. To obtain model parameter estimates, we propose to use a two-stage maximum likelihood (ML) procedure. This modeling framework is evaluated through a simulation study under representative conditions that would be found in social network data. Its usefulness is then demonstrated through an empirical application to a college friendship network.     

Multilevel multidimensional item response model with a multilevel latent covariate

In a pre‐test–post‐test cluster randomized trial, one of the methods commonly used to detect an intervention effect involves controlling pre‐test scores and other related covariates while estimating an intervention effect at post‐test. In many applications in education, the total post‐test and pre‐test scores, ignoring measurement error, are used as response variable and covariate, respectively, to estimate the intervention effect. However, these test scores are frequently subject to measurement error, and statistical inferences based on the model ignoring measurement error can yield a biased estimate of the intervention effect. When multiple domains exist in test data, it is sometimes more informative to detect the intervention effect for each domain than for the entire test. This paper presents applications of the multilevel multidimensional item response model with measurement error adjustments in a response variable and a covariate to estimate the intervention effect for each domain.     

Using analysis of covariance (ANCOVA) with fallible covariates

Analysis of covariance (ANCOVA) is used widely in psychological research implementing nonexperimental designs. However, when covariates are fallible (i.e., measured with error), which is the norm, researchers must choose from among 3 inadequate courses of action: (a) know that the assumption that covariates are perfectly reliable is violated but use ANCOVA anyway (and, most likely, report misleading results); (b) attempt to employ 1 of several measurement error models with the understanding that no research has examined their relative performance and with the added practical difficulty that several of these models are not available in commonly used statistical software; or (c) not use ANCOVA at all. First, we discuss analytic evidence to explain why using ANCOVA with fallible covariates produces bias and a systematic inflation of Type I error rates that may lead to the incorrect conclusion that treatment effects exist. Second, to provide a solution for this problem, we conduct 2 Monte Carlo studies to compare 4 existing approaches for adjusting treatment effects in the presence of covariate measurement error: errors-in-variables (EIV; Warren, White, & Fuller, 1974), Lord's (1960) method, Raaijmakers and Pieters's (1987) method (R&P), and structural equation modeling methods proposed by S?rbom (1978) and Hayduk (1996). Results show that EIV models are superior in terms of parameter accuracy, statistical power, and keeping Type I error close to the nominal value. Finally, we offer a program written in R that performs all needed computations for implementing EIV models so that ANCOVA can be used to obtain accurate results even when covariates are measured with error.     

Mixture distribution latent state-trait analysis: basic ideas and applications

Extensions of latent state-trait models for continuous observed variables to mixture latent state-trait models with and without covariates of change are presented that can separate individuals differing in their occasion-specific variability. An empirical application to the repeated measurement of mood states (N=501) revealed that a model with 2 latent classes fits the data well. The larger class (76%) consists of individuals whose mood is highly variable, whose general well-being is comparatively lower, and whose mood variability is influenced by daily hassles and uplifts. The smaller class (24%) represents individuals who are rather stable and happier and whose mood is influenced only by daily uplifts but not by daily hassles. A simulation study on the model without covariates with 5 sets of sample sizes and 5 sets of number of occasions revealed that the appropriateness of the parameter estimates of this model depends on number of observations (the higher the better) and number of occasions (the higher the better). Another simulation study estimated Type I and II errors of the Lo-Mendell-Rubin test.     

A caveat on the Savage–Dickey density ratio: The case of computing Bayes factors for regression parameters

The Savage–Dickey density ratio is a simple method for computing the Bayes factor for an equality constraint on one or more parameters of a statistical model. In regression analysis, this includes the important scenario of testing whether one or more of the covariates have an effect on the dependent variable. However, the Savage–Dickey ratio only provides the correct Bayes factor if the prior distribution of the nuisance parameters under the nested model is identical to the conditional prior under the full model given the equality constraint. This condition is violated for multiple regression models with a Jeffreys–Zellner–Siow prior, which is often used as a default prior in psychology. Besides linear regression models, the limitation of the Savage–Dickey ratio is especially relevant when analytical solutions for the Bayes factor are not available. This is the case for generalized linear models, non-linear models, or cognitive process models with regression extensions. As a remedy, the correct Bayes factor can be computed using a generalized version of the Savage–Dickey density ratio.     

Measurement bias and error correction in a two-stage estimation for multilevel IRT models

Among current state-of-the-art estimation methods for multilevel IRT models, the two-stage divide-and-conquer strategy has practical advantages, such as clearer definition of factors, convenience for secondary data analysis, convenience for model calibration and fit evaluation, and avoidance of improper solutions. However, various studies have shown that, under the two-stage framework, ignoring measurement error in the dependent variable in stage II leads to incorrect statistical inferences. To this end, we proposed a novel method to correct both measurement bias and measurement error of latent trait estimates from stage I in the stage II estimation. In this paper, the HO-IRT model is considered as the measurement model, and a linear mixed effects model on overall (i.e., higher-order) abilities is considered as the structural model. The performance of the proposed correction method is illustrated and compared via a simulation study and a real data example using the National Educational Longitudinal Survey data (NELS 88). Results indicate that structural parameters can be recovered better after correcting measurement biases and errors.     

Stochastic EM for estimating the parameters of a multilevel IRT model

An item response theory (IRT) model is used as a measurement error model for the dependent variable of a multilevel model. The dependent variable is latent but can be measured indirectly by using tests or questionnaires. The advantage of using latent scores as dependent variables of a multilevel model is that it offers the possibility of modelling response variation and measurement error and separating the influence of item difficulty and ability level. The two‐parameter normal ogive model is used for the IRT model. It is shown that the stochastic EM algorithm can be used to estimate the parameters which are close to the maximum likelihood estimates. This algorithm is easily implemented. The estimation procedure will be compared to an implementation of the Gibbs sampler in a Bayesian framework. Examples using real data are given.     

Latent variable models for partially ordered responses and trajectory analysis of anger‐related feelings

A general framework is presented for the analysis of partially ordered set (poset) data. The work is motivated by the need to analyse poset data such as multi‐componential responses in psychological measurement and partially accomplished cognitive tasks in educational measurement. It is shown how the generalized loglinear model can be used to represent poset data that form a lattice and how latent‐variable models can be constructed by further specifying the canonical parameters of the loglinear representation. The approach generalizes a class of latent‐variable models for completely ordered data. We apply the methods to analyse data on the frequency and intensity of anger‐related feelings. Furthermore, we propose a trajectory analysis to gain insight into the response function of partially ordered emotional states.     

Non‐linear structural equation models with correlated continuous and discrete data

Structural equation models (SEMs) have been widely applied to examine interrelationships among latent and observed variables in social and psychological research. Motivated by the fact that correlated discrete variables are frequently encountered in practical applications, a non‐linear SEM that accommodates covariates, and mixed continuous, ordered, and unordered categorical variables is proposed. Maximum likelihood methods for estimation and model comparison are discussed. One real‐life data set about cardiovascular disease is used to illustrate the methodologies.     

The DINA model as a constrained general diagnostic model: Two variants of a model equivalency

The ‘deterministic‐input noisy‐AND’ (DINA) model is one of the more frequently applied diagnostic classification models for binary observed responses and binary latent variables. The purpose of this paper is to show that the model is equivalent to a special case of a more general compensatory family of diagnostic models. Two equivalencies are presented. Both project the original DINA skill space and design Q ‐matrix using mappings into a transformed skill space as well as a transformed Q ‐matrix space. Both variants of the equivalency produce a compensatory model that is mathematically equivalent to the (conjunctive) DINA model. This equivalency holds for all DINA models with any type of Q ‐matrix, not only for trivial (simple‐structure) cases. The two versions of the equivalency presented in this paper are not implied by the recently suggested log‐linear cognitive diagnosis model or the generalized DINA approach. The equivalencies presented here exist independent of these recently derived models since they solely require a linear – compensatory – general diagnostic model without any skill interaction terms. Whenever it can be shown that one model can be viewed as a special case of another more general one, conclusions derived from any particular model‐based estimates are drawn into question. It is widely known that multidimensional models can often be specified in multiple ways while the model‐based probabilities of observed variables stay the same. This paper goes beyond this type of equivalency by showing that a conjunctive diagnostic classification model can be expressed as a constrained special case of a general compensatory diagnostic modelling framework.     

Measuring change for a multidimensional test using a generalized explanatory longitudinal item response model

Even though many educational and psychological tests are known to be multidimensional, little research has been done to address how to measure individual differences in change within an item response theory framework. In this paper, we suggest a generalized explanatory longitudinal item response model to measure individual differences in change. New longitudinal models for multidimensional tests and existing models for unidimensional tests are presented within this framework and implemented with software developed for generalized linear models. In addition to the measurement of change, the longitudinal models we present can also be used to explain individual differences in change scores for person groups (e.g., learning disabled students versus non‐learning disabled students) and to model differences in item difficulties across item groups (e.g., number operation, measurement, and representation item groups in a mathematics test). An empirical example illustrates the use of the various models for measuring individual differences in change when there are person groups and multiple skill domains which lead to multidimensionality at a time point.     

Random effects and extended generalized partial credit models

In this paper it is shown that under the random effects generalized partial credit model for the measurement of a single latent variable by a set of polytomously scored items, the joint marginal probability distribution of the item scores has a closed-form expression in terms of item category location parameters, parameters that characterize the distribution of the latent variable in the subpopulation of examinees with a zero score on all items, and item-scaling parameters. Due to this closed-form expression, all parameters of the random effects generalized partial credit model can be estimated using marginal maximum likelihood estimation without assuming a particular distribution of the latent variable in the population of examinees and without using numerical integration. Also due to this closed-form expression, new special cases of the random effects generalized partial credit model can be identified. In addition to these new special cases, a slightly more general model than the random effects generalized partial credit model is presented. This slightly more general model is called the extended generalized partial credit model. Attention is paid to maximum likelihood estimation of the parameters of the extended generalized partial credit model and to assessing the goodness of fit of the model using generalized likelihood ratio tests. Attention is also paid to person parameter estimation under the random effects generalized partial credit model. It is shown that expected a posteriori estimates can be obtained for all possible score patterns. A simulation study is carried out to show the usefulness of the proposed models compared to the standard models that assume normality of the latent variable in the population of examinees. In an empirical example, some of the procedures proposed are demonstrated.     

Generalized latent variable models with non‐linear effects

Until recently, item response models such as the factor analysis model for metric responses, the two‐parameter logistic model for binary responses and the multinomial model for nominal responses considered only the main effects of latent variables without allowing for interaction or polynomial latent variable effects. However, non‐linear relationships among the latent variables might be necessary in real applications. Methods for fitting models with non‐linear latent terms have been developed mainly under the structural equation modelling approach. In this paper, we consider a latent variable model framework for mixed responses (metric and categorical) that allows inclusion of both non‐linear latent and covariate effects. The model parameters are estimated using full maximum likelihood based on a hybrid integration–maximization algorithm. Finally, a method for obtaining factor scores based on multiple imputation is proposed here for the non‐linear model.     

A comparison of methods for estimating quadratic effects in nonlinear structural equation models

Two Monte Carlo simulations were performed to compare methods for estimating and testing hypotheses of quadratic effects in latent variable regression models. The methods considered in the current study were (a) a 2-stage moderated regression approach using latent variable scores, (b) an unconstrained product indicator approach, (c) a latent moderated structural equation method, (d) a fully Bayesian approach, and (e) marginal maximum likelihood estimation. Of the 5 estimation methods, it was found that overall the methods based on maximum likelihood estimation and the Bayesian approach performed best in terms of bias, root-mean-square error, standard error ratios, power, and Type I error control, although key differences were observed. Similarities as well as disparities among methods are highlight and general recommendations articulated. As a point of comparison, all 5 approaches were fit to a reparameterized version of the latent quadratic model to educational reading data.     

Latent growth curve analysis with dichotomous items: Comparing four approaches

A Monte Carlo study was used to compare four approaches to growth curve analysis of subjects assessed repeatedly with the same set of dichotomous items: A two‐step procedure first estimating latent trait measures using MULTILOG and then using a hierarchical linear model to examine the changing trajectories with the estimated abilities as the outcome variable; a structural equation model using modified weighted least squares (WLSMV) estimation; and two approaches in the framework of multilevel item response models, including a hierarchical generalized linear model using Laplace estimation, and Bayesian analysis using Markov chain Monte Carlo (MCMC). These four methods have similar power in detecting the average linear slope across time. MCMC and Laplace estimates perform relatively better on the bias of the average linear slope and corresponding standard error, as well as the item location parameters. For the variance of the random intercept, and the covariance between the random intercept and slope, all estimates are biased in most conditions. For the random slope variance, only Laplace estimates are unbiased when there are eight time points.     

Examining time‐invariance in reliability in multi‐wave,multi‐indicator models: A covariance structure analysis approach accounting for indicator specificity

A covariance structure analysis method for testing time‐invariance in reliability in multiwave, multiple‐indicator models in outlined. The approach accounts for observed variable specificity and permits, in addition, estimation of reliability in terms of ‘pure’ measurement error variance. The proposed procedure is developed within a confirmatory factor analysis framework and illustrated with data from a cognitive intervention study.     

Paired comparison,triple comparison,and ranking experiments as generalized linear models,and their implementation on GLIM

A wide variety of paired comparison, triple comparison, and ranking experiments may be viewed as generalized linear models. These include paired comparison models based on both the Bradley-Terry and Thurstone-Mosteller approaches, as well as extensions of these models that allow for ties, order of presentation effects, and the presence of covariates. Moreover, the triple comparison model of Pendergrass and Bradley, as well as models for complete rankings of more than three items, can also be represented as generalized linear models. All such models can be easily fit by maximum likelihood, using the widely available GLIM computer package. Additionally, GLIM enables the computation of likelihood ratio statistics for testing many hypotheses of interest. Examples are presented that cover a variety of cases, along with their implementation on GLIM.     

Two Studies Examining the Error Theory Underlying the Measurement Model of the Verbal Aggressiveness Scale

Meta‐analysis indicates moderate correlations between the Verbal Aggressiveness Scale (VAS) and other self‐report measures but near‐zero correlations with behavioral measures. Accurately interpreting correlations between the VAS and other variables, however, requires an examination of the untested error theory underlying the measurement model for the VAS. In two separate studies, the results of single‐factor correlated uniqueness confirmatory factor analytic models revealed a pattern of significant error covariances indicating that VAS item scores are confounded by systematic error attributable to multiple unspecified latent effects. After pruning the item sets, we identified 4 items that were free of latent variable influences other than trait verbal aggressiveness. Implications for interpreting the verbal aggressiveness literature are discussed along with recommendations for revising the VAS.     

The Thorny Relation Between Measurement Quality and Fit Index Cutoffs in Latent Variable Models

Latent variable modeling is a popular and flexible statistical framework. Concomitant with fitting latent variable models is assessment of how well the theoretical model fits the observed data. Although firm cutoffs for these fit indexes are often cited, recent statistical proofs and simulations have shown that these fit indexes are highly susceptible to measurement quality. For instance, a root mean square error of approximation (RMSEA) value of 0.06 (conventionally thought to indicate good fit) can actually indicate poor fit with poor measurement quality (e.g., standardized factors loadings of around 0.40). Conversely, an RMSEA value of 0.20 (conventionally thought to indicate very poor fit) can indicate acceptable fit with very high measurement quality (standardized factor loadings around 0.90). Despite the wide-ranging effect on applications of latent variable models, the high level of technical detail involved with this phenomenon has curtailed the exposure of these important findings to empirical researchers who are employing these methods. This article briefly reviews these methodological studies in minimal technical detail and provides a demonstration to easily quantify the large influence measurement quality has on fit index values and how greatly the cutoffs would change if they were derived under an alternative level of measurement quality. Recommendations for best practice are also discussed.