Omitted Variables in Multilevel Models

Statistical methodology for handling omitted variables is presented in a multilevel modeling framework. In many nonexperimental studies, the analyst may not have access to all requisite variables, and this omission may lead to biased estimates of model parameters. By exploiting the hierarchical nature of multilevel data, a battery of statistical tools are developed to test various forms of model misspecification as well as to obtain estimators that are robust to the presence of omitted variables. The methodology allows for tests of omitted effects at single and multiple levels. The paper also introduces intermediate-level tests; these are tests for omitted effects at a single level, regardless of the presence of omitted effects at a higher level. A simulation study shows, not surprisingly, that the omission of variables yields bias in both regression coefficients and variance components; it also suggests that omitted effects at lower levels may cause more severe bias than at higher levels. Important factors resulting in bias were found to be the level of an omitted variable, its effect size, and sample size. A real data study illustrates that an omitted variable at one level may yield biased estimators at any level and, in this study, one cannot obtain reliable estimates for school-level variables when omitted child effects exist. However, robust estimators may provide unbiased estimates for effects of interest even when the efficient estimators fail, and the one-degree-of-freedom test helps one to understand where the problem is located. It is argued that multilevel data typically contain rich information to deal with omitted variables, offering yet another appealing reason for the use of multilevel models in the social sciences. This research was supported by the National Academy of Education/Spencer Foundation and the National Science Foundation, Grant Number SES-0436274.     

Stochastic EM for estimating the parameters of a multilevel IRT model

An item response theory (IRT) model is used as a measurement error model for the dependent variable of a multilevel model. The dependent variable is latent but can be measured indirectly by using tests or questionnaires. The advantage of using latent scores as dependent variables of a multilevel model is that it offers the possibility of modelling response variation and measurement error and separating the influence of item difficulty and ability level. The two‐parameter normal ogive model is used for the IRT model. It is shown that the stochastic EM algorithm can be used to estimate the parameters which are close to the maximum likelihood estimates. This algorithm is easily implemented. The estimation procedure will be compared to an implementation of the Gibbs sampler in a Bayesian framework. Examples using real data are given.     

Estimating Incremental Validity Under Missing Data

A common form of missing data is caused by selection on an observed variable (e.g., Z). If the selection variable was measured and is available, the data are regarded as missing at random (MAR). Selection biases correlation, reliability, and effect size estimates when these estimates are computed on listwise deleted (LD) data sets. On the other hand, maximum likelihood (ML) estimates are generally unbiased and outperform LD in most situations, at least when the data are MAR. The exception is when we estimate the partial correlation. In this situation, LD estimates are unbiased when the cause of missingness is partialled out. In other words, there is no advantage of ML estimates over LD estimates in this situation. We demonstrate that under a MAR condition, even ML estimates may become biased, depending on how partial correlations are computed. Finally, we conclude with recommendations about how future researchers might estimate partial correlations even when the cause of missingness is unknown and, perhaps, unknowable.     

有调节的多层中介效应分析

多层中介和多层调节效应分析在社科领域已常有应用,但如果将多层中介和调节整合在一起,可以产生2（多层中介类型）×2（调节变量的层次）×3（调节的中介路径）共12种有调节的多层中介模型。面对有调节的多层中介效应分析,研究者往往束手无策。详述了基于多层线性模型的12种有调节的多层中介的分析方法和基于多层结构方程模型的4类有调节的多层中介分析方法,包括正交分割法,随机系数预测法,潜调节结构方程法和贝叶斯合理值法。这四类方法的核心议题在于如何处理潜调节项。当样本量足够大时,建议选择潜调节结构方程法;当样本量不足时,建议选择贝叶斯合理值法。随后用一个实际例子演示如何进行有调节的多层中介效应分析并有相应的Mplus程序。最后展望了有调节的多层中介效应分析研究的拓展方向。     

Conceptualizing and testing random indirect effects and moderated mediation in multilevel models: new procedures and recommendations

The authors propose new procedures for evaluating direct, indirect, and total effects in multilevel models when all relevant variables are measured at Level 1 and all effects are random. Formulas are provided for the mean and variance of the indirect and total effects and for the sampling variances of the average indirect and total effects. Simulations show that the estimates are unbiased under most conditions. Confidence intervals based on a normal approximation or a simulated sampling distribution perform well when the random effects are normally distributed but less so when they are nonnormally distributed. These methods are further developed to address hypotheses of moderated mediation in the multilevel context. An example demonstrates the feasibility and usefulness of the proposed methods.     

Bayesian modeling of measurement error in predictor variables using item response theory

It is shown that measurement error in predictor variables can be modeled using item response theory (IRT). The predictor variables, that may be defined at any level of an hierarchical regression model, are treated as latent variables. The normal ogive model is used to describe the relation between the latent variables and dichotomous observed variables, which may be responses to tests or questionnaires. It will be shown that the multilevel model with measurement error in the observed predictor variables can be estimated in a Bayesian framework using Gibbs sampling. In this article, handling measurement error via the normal ogive model is compared with alternative approaches using the classical true score model. Examples using real data are given.This paper is part of the dissertation by Fox (2001) that won the 2002 Psychometric Society Dissertation Award.     

Predicting group-level outcome variables: An empirical comparison of analysis strategies

This study provides a review of two methods for analyzing multilevel data with group-level outcome variables and compares them in a simulation study. The analytical methods included an unadjusted ordinary least squares (OLS) analysis of group means and a two-step adjustment of the group means suggested by Croon and van Veldhoven (2007). The Type I error control, power, bias, standard errors, and RMSE in parameter estimates were compared across design conditions that included manipulations of number of predictor variables, level of correlation between predictors, level of intraclass correlation, predictor reliability, effect size, and sample size. The results suggested that an OLS analysis of the group means, with White’s heteroscedasticity adjustment, provided more power for tests of group-level predictors, but less power for tests of individual-level predictors. Furthermore, this simple analysis avoided the extreme bias in parameter estimates and inadmissible solutions that were encountered with other strategies. These results were interpreted in terms of recommended analytical methods for applied researchers.     

School-related stress,social support,and distress: prospective analysis of reciprocal and multilevel relationships

This three-wave prospective study investigated the reciprocal relationships among school-related stress, school-related social support, and distress in a cohort of 767 secondary school students (mean age 13.9 years). Stress, support, and distress were measured at three occasions with six-month lags between. Reciprocal relationships were analyzed with multivariate multilevel modeling (MLwiN). Each of the three factors at baseline predicted change in one or two of the other factors at subsequent measurements, indicating a complex pattern of reciprocal relationships among stress, support, and distress across time. A high level of distress at baseline predicted a lower level of support and a higher level of stress six months later. High levels of stress at baseline predicted a higher level of distress and a lower level of support 12 months later. The results are consistent with a transactional and dynamic model of stress, support, and distress, and indicate the need to view school-related stress, support, and distress as mutually dependent factors.     

A 2 × 2 taxonomy of multilevel latent contextual models: accuracy-bias trade-offs in full and partial error correction models

In multilevel modeling, group-level variables (L2) for assessing contextual effects are frequently generated by aggregating variables from a lower level (L1). A major problem of contextual analyses in the social sciences is that there is no error-free measurement of constructs. In the present article, 2 types of error occurring in multilevel data when estimating contextual effects are distinguished: unreliability that is due to measurement error and unreliability that is due to sampling error. The fact that studies may or may not correct for these 2 types of error can be translated into a 2 × 2 taxonomy of multilevel latent contextual models comprising 4 approaches: an uncorrected approach, partial correction approaches correcting for either measurement or sampling error (but not both), and a full correction approach that adjusts for both sources of error. It is shown mathematically and with simulated data that the uncorrected and partial correction approaches can result in substantially biased estimates of contextual effects, depending on the number of L1 individuals per group, the number of groups, the intraclass correlation, the number of indicators, and the size of the factor loadings. However, the simulation study also shows that partial correction approaches can outperform full correction approaches when the data provide only limited information in terms of the L2 construct (i.e., small number of groups, low intraclass correlation). A real-data application from educational psychology is used to illustrate the different approaches.     

Common Misconceptions Concerning The Analysis Of Covariance

Four misconceptions about the requirements for proper use of analysis of covariance (ANCOVA) are examined by means of Monte Carlo simulation. Conclusions are that ANCOVA does not require covariates to be measured without error, that ANCOVA can be used effectively to adjust for initial group differences that result from nonrandom assignment which is dependent on observed covariate scores, that ANCOVA does not provide unbiased estimates of true treatment effects where initial group differences are due to nonrandom assignment which is dependent on the true latent covariable if the covariate contains measurement error, and that ANCOVA requires no assumption concerning the equality of within-groups and between-groups regression. Where treatments actually influence covariate scores, the hypothesis tested by ANCOVA concerns a weighted combination of effects on covariate and dependent variables.     

Simple imputation methods versus direct likelihood analysis for missing item scores in multilevel educational data

Missing data, such as item responses in multilevel data, are ubiquitous in educational research settings. Researchers in the item response theory (IRT) context have shown that ignoring such missing data can create problems in the estimation of the IRT model parameters. Consequently, several imputation methods for dealing with missing item data have been proposed and shown to be effective when applied with traditional IRT models. Additionally, a nonimputation direct likelihood analysis has been shown to be an effective tool for handling missing observations in clustered data settings. This study investigates the performance of six simple imputation methods, which have been found to be useful in other IRT contexts, versus a direct likelihood analysis, in multilevel data from educational settings. Multilevel item response data were simulated on the basis of two empirical data sets, and some of the item scores were deleted, such that they were missing either completely at random or simply at random. An explanatory IRT model was used for modeling the complete, incomplete, and imputed data sets. We showed that direct likelihood analysis of the incomplete data sets produced unbiased parameter estimates that were comparable to those from a complete data analysis. Multiple-imputation approaches of the two-way mean and corrected item mean substitution methods displayed varying degrees of effectiveness in imputing data that in turn could produce unbiased parameter estimates. The simple random imputation, adjusted random imputation, item means substitution, and regression imputation methods seemed to be less effective in imputing missing item scores in multilevel data settings.     

Multilevel IRT using dichotomous and polytomous response data

A structural multilevel model is presented where some of the variables cannot be observed directly but are measured using tests or questionnaires. Observed dichotomous or ordinal polytomous response data serve to measure the latent variables using an item response theory model. The latent variables can be defined at any level of the multilevel model. A Bayesian procedure Markov chain Monte Carlo (MCMC), to estimate all parameters simultaneously is presented. It is shown that certain model checks and model comparisons can be done using the MCMC output. The techniques are illustrated using a simulation study and an application involving students' achievements on a mathematics test and test results regarding management characteristics of teachers and principles.     

群际接触的次级转移效应(STE)的心理机制与影响因素

次级转移效应(secondary transfer effect,STE)是指群际接触的积极效应从直接接触的外群体转移到另一个未直接接触的外群体中。通过文献分析,找到了次级转移效应产生的中介变量(包括社会身份复杂性、群际移情和群际焦虑、多元文化主义以及态度的泛化)和调节变量(群体相似性、群体地位和社会支配倾向等)。今后的研究应坚持调查法与实验法等多种方法的结合,从多个维度测量群际接触与群体态度,探索和完善STE调节和中介变量,系统的验证变量之间的关系,从而完善和发展群际接触理论。     

Influences on Member Assimilation in Workgroups in High‐Reliability Organizations: A Multilevel Analysis

This multilevel investigation examined the effect of group interaction and its influence on individual‐level membership variables and group assimilation. The study is based on a model of group socialization developed by Moreland and Levine (1982) and was modified in this study to investigate the development and maintenance of highly interdependent workgroups in a high‐reliability organization: a municipal fire department. Using hierarchical linear modeling, we examined individual‐ and crew‐level influence on four assimilation outcomes: involvement, trustworthiness, commitment, and acceptance. At the individual level, acculturation predicted all the four assimilation outcomes. Involvement also was a predictor of the latter sequences of assimilation: commitment and acceptance. The study also found that one crew‐level variable—crew performance—affected and modified the influence of tenure, proactivity, involvement, acculturation, and trust on members’ commitment. Implications are offered for the influence of group interaction on member assimilation and support for continuing group‐level research on assimilation. This study also underscores the utility of multilevel analysis in examining communication at the interpersonal and group levels.     

Mediation analysis and categorical variables: The final frontier

Many scholars are interested in understanding the process by which an independent variable affects a dependent variable, perhaps in part directly and perhaps in part indirectly, occurring through the activation of a mediator. Researchers are facile at testing for mediation when all the variables are continuous, but a definitive answer had been lacking heretofore as to how to analyze the data when the mediator or dependent variable is categorical. This paper describes the problems that arise as well as the potential solutions. In the end, a solution is recommended that is both optimal in its statistical qualities as well as practical and easily implemented: compute z_Mediation.     

Predicting post-event processing in social anxiety disorder following two prototypical social situations: State variables and dispositional determinants

This study investigated self-reported state (anxiety, physical symptoms, cognitions, internally focused attention, safety behaviors, social performance) and trait (social anxiety, depressive symptoms, dysfunctional self-consciousness) predictors of post-event processing (PEP) subsequent to two social situations (interaction, speech) in participants with a primary diagnosis of social anxiety disorder (SAD) and healthy controls (HC). The speech triggered significantly more intense PEP, especially in SAD. Regardless of the type of social situation, PEP was best predicted by situational anxiety and dysfunctional cognitions among the state variables. If only trait variables were considered, PEP following both situations was accounted for by trait social anxiety. In addition, dysfunctional self-consciousness contributed to PEP-speech. If state and trait variables were jointly considered, for both situations, situational anxiety and dysfunctional cognitions were confirmed as the most powerful PEP predictors above and beyond trait social anxiety (interaction) and dysfunctional self-consciousness (speech). Hence, PEP as assessed on the day after a social situation seems to be mainly determined by state variables. Trait social anxiety and dysfunctional self-consciousness also significantly contribute to PEP depending on the type of social situation. The present findings support dysfunctional cognitions as a core cognitive mechanism for the maintenance of SAD. Implications for treatment are discussed.     

Predictors of early person reference development: Maternal language input,attachment and neurodevelopmental markers

In a longitudinal natural language development study in Germany, the acquisition of verbal symbols for present persons, absent persons, inanimate things and the mother–toddler dyad was investigated. Following the notion that verbal referent use is more developed in ostensive contexts, symbolic play situations were coded for verbal person reference by means of noun and pronoun use. Depending on attachment classifications at twelve months of age, effects of attachment classification and maternal language input were studied up to 36 months in four time points. Hierarchical regression analyses revealed that, except for mother absence, maternal verbal referent input rates at 17 and 36 months were stronger predictors for all referent types than any of the attachment organizations, or any other social or biological predictor variable. Attachment effects accounted for up to 9.8% of unique variance proportions in the person reference variables. Perinatal and familial measures predicted person references dependent on reference type. The results of this investigation indicate that mother-reference, self-reference and thing-reference develop in similar quantities measured from the 17-month time point, but are dependent of attachment quality.     

Ignoring Clustering in Confirmatory Factor Analysis: Some Consequences for Model Fit and Standardized Parameter Estimates

In many situations, researchers collect multilevel (clustered or nested) data yet analyze the data either ignoring the clustering (disaggregation) or averaging the micro-level units within each cluster and analyzing the aggregated data at the macro level (aggregation). In this study we investigate the effects of ignoring the nested nature of data in confirmatory factor analysis (CFA). The bias incurred by ignoring clustering is examined in terms of model fit and standardized parameter estimates, which are usually of interest to researchers who use CFA. We find that the disaggregation approach increases model misfit, especially when the intraclass correlation (ICC) is high, whereas the aggregation approach results in accurate detection of model misfit in the macro level. Standardized parameter estimates from the disaggregation and aggregation approaches are deviated toward the values of the macro- and micro-level standardized parameter estimates, respectively. The degree of deviation depends on ICC and cluster size, particularly for the aggregation method. The standard errors of standardized parameter estimates from the disaggregation approach depend on the macro-level item communalities. Those from the aggregation approach underestimate the standard errors in multilevel CFA (MCFA), especially when ICC is low. Thus, we conclude that MCFA or an alternative approach should be used if possible.     

ANCOVA Versus CHANGE From Baseline in Nonrandomized Studies: The Difference

The pretest-posttest control group design can be analyzed with the posttest as dependent variable and the pretest as covariate (ANCOVA) or with the difference between posttest and pretest as dependent variable (CHANGE). These 2 methods can give contradictory results if groups differ at pretest, a phenomenon that is known as Lord's paradox. Literature claims that ANCOVA is preferable if treatment assignment is based on randomization or on the pretest and questionable for preexisting groups. Some literature suggests that Lord's paradox has to do with measurement error in the pretest. This article shows two new things: First, the claims are confirmed by proving the mathematical equivalence of ANCOVA to a repeated measures model without group effect at pretest. Second, correction for measurement error in the pretest is shown to lead back to ANCOVA or to CHANGE, depending on the assumed absence or presence of a true group difference at pretest. These two new theoretical results are illustrated with multilevel (mixed) regression and structural equation modeling of data from two studies.     

Studying Multivariate Change Using Multilevel Models and Latent Curve Models

In longitudinal research investigators often measure multiple variables at multiple points in time and are interested in investigating individual differences in patterns of change on those variables. In the vast majority of applications, researchers focus on studying change in one variable at a time. In this article we consider methods for studying relations1.1ips between patterns of change on different variables. We show how the multilevel modeling framework, which is often used to study univariate change, can be extended to the multivariate case to yield estimates of covariances of parameters representing aspects of change on different variables. We illustrate this approach using data from a study of physiological response to marital conflict in older married couples, showing a substantial correlation between rate of linear change on different stress-related hormones during conflict. We also consider how similar issues can be studied using extensions of latent curve models to the multivariate case, and we show how such models are related to multivariate multilevel models.