Robust Methods for Moderation Analysis with a Two-Level Regression Model

Moderation analysis has many applications in social sciences. Most widely used estimation methods for moderation analysis assume that errors are normally distributed and homoscedastic. When these assumptions are not met, the results from a classical moderation analysis can be misleading. For more reliable moderation analysis, this article proposes two robust methods with a two-level regression model when the predictors do not contain measurement error. One method is based on maximum likelihood with Student's t distribution and the other is based on M-estimators with Huber-type weights. An algorithm for obtaining the robust estimators is developed. Consistent estimates of standard errors of the robust estimators are provided. The robust approaches are compared against normal-distribution-based maximum likelihood (NML) with respect to power and accuracy of parameter estimates through a simulation study. Results show that the robust approaches outperform NML under various distributional conditions. Application of the robust methods is illustrated through a real data example. An R program is developed and documented to facilitate the application of the robust methods.     

Improved Regression Calibration

The likelihood for generalized linear models with covariate measurement error cannot in general be expressed in closed form, which makes maximum likelihood estimation taxing. A popular alternative is regression calibration which is computationally efficient at the cost of inconsistent estimation. We propose an improved regression calibration approach, a general pseudo maximum likelihood estimation method based on a conveniently decomposed form of the likelihood. It is both consistent and computationally efficient, and produces point estimates and estimated standard errors which are practically identical to those obtained by maximum likelihood. Simulations suggest that improved regression calibration, which is easy to implement in standard software, works well in a range of situations.     

The Impacts of Ignoring a Crossed Factor in Analyzing Cross-Classified Data

Cross-classified random-effects models (CCREMs) are used for modeling nonhierarchical multilevel data. Misspecifying CCREMs as hierarchical linear models (i.e., treating the cross-classified data as strictly hierarchical by ignoring one of the crossed factors) causes biases in the variance component estimates, which in turn, results in biased estimation in the standard errors of the regression coefficients. Analytical studies were conducted to provide closed-form expressions for the biases. With balanced design data structure, ignoring a crossed factor causes overestimation of the variance components of adjacent levels and underestimation of the variance component of the remaining crossed factor. Moreover, ignoring a crossed factor at the kth level causes underestimation of the standard error of the regression coefficient of the predictor associated with the ignored factor and overestimation of the standard error of the regression coefficient of the predictor at the (k?1)th level. Simulation studies were also conducted to examine the effect of different structures of cross-classification on the biases. In general, the direction and magnitude of the biases depend on the level of the ignored crossed factor, the level with which the predictor is associated at, the magnitude of the variance component of the ignored crossed factor, the variance components of the predictors, the sample sizes, and the structure of cross-classification. The results were further illustrated using the Early Childhood Longitudinal Study-Kindergarten Cohort data.     

Augmenting Data With Published Results in Bayesian Linear Regression

In most research, linear regression analyses are performed without taking into account published results (i.e., reported summary statistics) of similar previous studies. Although the prior density in Bayesian linear regression could accommodate such prior knowledge, formal models for doing so are absent from the literature. The goal of this article is therefore to develop a Bayesian model in which a linear regression analysis on current data is augmented with the reported regression coefficients (and standard errors) of previous studies. Two versions of this model are presented. The first version incorporates previous studies through the prior density and is applicable when the current and all previous studies are exchangeable. The second version models all studies in a hierarchical structure and is applicable when studies are not exchangeable. Both versions of the model are assessed using simulation studies. Performance for each in estimating the regression coefficients is consistently superior to using current data alone and is close to that of an equivalent model that uses the data from previous studies rather than reported regression coefficients. Overall the results show that augmenting data with results from previous studies is viable and yields significant improvements in the parameter estimation.     

Regression tests for several samples

In many situations it is desirable or necessary to administer a set of tests to several different groups, and to ask if the results obtained in the different groups may be regarded as being essentially the same in some sense. In the case of two variables (one dependent and one independent) one may, for instance, ask if the errors of estimate and the regression lines may be regarded as being the same for the populations from which the different groups are drawn. For this case, the present article considers tests for three hypotheses regarding the populations from which the different groups are drawn: (a)H _A, the hypothesis that all standard errors of estimate are equal; (b)H _B, the hypothesis that all regression lines are parallel, (assumingH _A); and (c)H _C, the hypothesis that the regression lines are identical, (assumingH _B). Test criteria for these three hypotheses and their sampling theory for large samples are presented. The results are extended to the case of several independent variables. An illustrative problem is presented for two groups, two independent and one dependent variable.     

Multilevel Modeling with Correlated Effects

When there exist omitted effects, measurement error, and/or simultaneity in multilevel models, explanatory variables may be correlated with random components, and standard estimation methods do not provide consistent estimates of model parameters. This paper introduces estimators that are consistent under such conditions. By employing generalized method of moments (GMM) estimation techniques in multilevel modeling, the authors present a series of estimators along a robust to efficient continuum. This continuum depends on the assumptions that the analyst makes regarding the extent of the correlated effects. It is shown that the GMM approach provides an overarching framework that encompasses well-known estimators such as fixed and random effects estimators and also provides more options. These GMM estimators can be expressed as instrumental variable (IV) estimators which enhances their interpretability. Moreover, by exploiting the hierarchical structure of the data, the current technique does not require additional variables unlike traditional IV methods. Further, statistical tests are developed to compare the different estimators. A simulation study examines the finite sample properties of the estimators and tests and confirms the theoretical order of the estimators with respect to their robustness and efficiency. It further shows that not only are regression coefficients biased, but variance components may be severely underestimated in the presence of correlated effects. Empirical standard errors are employed as they are less sensitive to correlated effects when compared to model-based standard errors. An example using student achievement data shows that GMM estimators can be effectively used in a search for the most efficient among unbiased estimators. This research was supported by the National Academy of Education/Spencer Foundation and the National Science Foundation, grant number SES-0436274. We thank the editor, associate editor, and referees for detailed feedback that helped improve the paper.     

Bayesian modeling of measurement error in predictor variables using item response theory

It is shown that measurement error in predictor variables can be modeled using item response theory (IRT). The predictor variables, that may be defined at any level of an hierarchical regression model, are treated as latent variables. The normal ogive model is used to describe the relation between the latent variables and dichotomous observed variables, which may be responses to tests or questionnaires. It will be shown that the multilevel model with measurement error in the observed predictor variables can be estimated in a Bayesian framework using Gibbs sampling. In this article, handling measurement error via the normal ogive model is compared with alternative approaches using the classical true score model. Examples using real data are given.This paper is part of the dissertation by Fox (2001) that won the 2002 Psychometric Society Dissertation Award.     

Estimation of maximal reliability: A note on a covariance structure modelling approach

A one‐step covariance structure analysis procedure for estimation of maximal reliability of linear composites with congeneric measures is outlined. The approach is readily employed within a single modelling session using popular covariance structure analysis software, and permits simultaneous estimation of the optimal measure weights with standard errors. The method is illustrated by a numerical example.     

Regression analyses of repeated measures data in cognitive research

Repeated measures designs involving nonorthogonal variables are being used with increasing frequency in cognitive psychology. Researchers usually analyze the data from such designs inappropriately, probably because the designs are not discussed in standard textbooks on regression. Two commonly used approaches to analyzing repeated measures designs are considered in this article. It is argued that both approaches use inappropriate error terms for testing the effects of independent variables. A more appropriate analysis is presented, and two alternative computational procedures for the analysis are illustrated.     

纳入式分类分析法在潜在剖面模型的后续多元回归中的应用

纳入式分类分析法能克服传统的分类分析法对后续一元回归模型参数的低估,发挥潜在类别模型的后续分析简化变量间交互作用的功能。本文进一步将纳入式分类分析法拓展至潜在剖面模型后续的多元统计分析中。通过蒙特卡洛模拟实验,比较各种纳入变量的方法思路与后续分析模型在四种常见的多元回归模型中参数估计的表现。结果发现,纳入式分类分析法所需纳入的变量取决于后续分析中与因变量、潜类别变量的关系,且后续分析使用含交互作用的模型更为稳健。     

Analysis of Interactions and Nonlinear Effects with Missing Data: A Factored Regression Modeling Approach Using Maximum Likelihood Estimation

Abstract

When estimating multiple regression models with incomplete predictor variables, it is necessary to specify a joint distribution for the predictor variables. A convenient assumption is that this distribution is a multivariate normal distribution, which is also the default in many statistical software packages. This distribution will in general be misspecified if predictors with missing data have nonlinear effects (e.g., x²) or are included in interaction terms (e.g., x·z). In the present article, we introduce a factored regression modeling approach for estimating regression models with missing data that is based on maximum likelihood estimation. In this approach, the model likelihood is factorized into a part that is due to the model of interest and a part that is due to the model for the incomplete predictors. In three simulation studies, we showed that the factored regression modeling approach produced valid estimates of interaction and nonlinear effects in regression models with missing values on categorical or continuous predictor variables under a broad range of conditions. We developed the R package mdmb, which facilitates a user-friendly application of the factored regression modeling approach, and present a real-data example that illustrates the flexibility of the software.     

Moderation Analysis Using a Two-Level Regression Model

Moderation analysis is widely used in social and behavioral research. The most commonly used model for moderation analysis is moderated multiple regression (MMR) in which the explanatory variables of the regression model include product terms, and the model is typically estimated by least squares (LS). This paper argues for a two-level regression model in which the regression coefficients of a criterion variable on predictors are further regressed on moderator variables. An algorithm for estimating the parameters of the two-level model by normal-distribution-based maximum likelihood (NML) is developed. Formulas for the standard errors (SEs) of the parameter estimates are provided and studied. Results indicate that, when heteroscedasticity exists, NML with the two-level model gives more efficient and more accurate parameter estimates than the LS analysis of the MMR model. When error variances are homoscedastic, NML with the two-level model leads to essentially the same results as LS with the MMR model. Most importantly, the two-level regression model permits estimating the percentage of variance of each regression coefficient that is due to moderator variables. When applied to data from General Social Surveys 1991, NML with the two-level model identified a significant moderation effect of race on the regression of job prestige on years of education while LS with the MMR model did not. An R package is also developed and documented to facilitate the application of the two-level model.     

Abstract: Genome-Wide Complex Trait Analysis of Shared Genetic Effects Between Brain Electrophysiology Measures and Alcohol Dependence

Multilevel analysis is an appropriate tool for the analysis of hierarchically structured data. There may, however, be reasons to ignore one of the levels of nesting in the data analysis. In this article a three level model with one predictor variable is used as a reference model and the top or intermediate level is ignored in the data analysis. Analytical results show that this has an effect on the estimated variance components and that standard errors of regression coefficients estimators may be overestimated, leading to a lower power of the test of the effect of the predictor variable. The magnitude of these results depends on the ignored level and the level at which the predictor variable varies, and on the values of the variance components and the sample sizes.     

Regression analyses as a tool for studying reading processes: Comment on Just and Carpenters eye fixation theory

Just and Carpenter (1980) presented a theory of reading based on eye fixations wherein their “psycholinguistic” variables accounted for 72% of the variance in word gaze durations. This comment raises some statistical and theoretical problems with their use of simultaneous regression analysis of gaze duration measures and with the resulting theory of reading. A major problem was the confounding of perceptual with psycholinguistic factors. New eye fixation data are presented to support these criticisms. Analysis of fixations within words revealed that most gaze duration variance was contributed by number of fixations rather than by fixation duration.     

Isolating and Examining Sources of Suppression and Multicollinearity in Multiple Linear Regression

The presence of suppression (and multicollinearity) in multiple regression analysis complicates interpretation of predictor-criterion relationships. The mathematical conditions that produce suppression in regression analysis have received considerable attention in the methodological literature but until now nothing in the way of an analytic strategy to isolate, examine, and remove suppression effects has been offered. In this article such an approach, rooted in confirmatory factor analysis theory and employing matrix algebra, is developed. Suppression is viewed as the result of criterion-irrelevant variance operating among predictors. Decomposition of predictor variables into criterion-relevant and criterion-irrelevant components using structural equation modeling permits derivation of regression weights with the effects of criterion-irrelevant variance omitted. Three examples with data from applied research are used to illustrate the approach: the first assesses child and parent characteristics to explain why some parents of children with obsessive-compulsive disorder accommodate their child's compulsions more so than do others, the second examines various dimensions of personal health to explain individual differences in global quality of life among patients following heart surgery, and the third deals with quantifying the relative importance of various aptitudes for explaining academic performance in a sample of nursing students. The approach is offered as an analytic tool for investigators interested in understanding predictor-criterion relationships when complex patterns of intercorrelation among predictors are present and is shown to augment dominance analysis.     

Two-Part Predictors in Regression Models

Researchers often build regression models to relate a response to a set of predictor variables. In some cases, there are predictors that apply to some participants, or to some measurement occasions, but not others. For example, a romantic partner's substance use may be a key predictor of one's own substance use. However, not all participants have a partner, and in a longitudinal study, participants may have a partner during only some occasions. This could be viewed as missing data, but of a very distinctive type: the values are not just unknown but also undefined. In this paper, we present a simple method to accommodate this situation, along with a motivating example, the algebraic justification, a simulation study, and examples on how to carry out the technique.     

Relative Importance Analysis: A Useful Supplement to Regression Analysis

This article advocates for the wider use of relative importance indices as a supplement to multiple regression analyses. The goal of such analyses is to partition explained variance among multiple predictors to better understand the role played by each predictor in a regression equation. Unfortunately, when predictors are correlated, typically relied upon metrics are flawed indicators of variable importance. To that end, we highlight the key benefits of two relative importance analyses, dominance analysis and relative weight analysis, over estimates produced by multiple regression analysis. We also describe numerous situations where relative importance weights should be used, while simultaneously cautioning readers about the limitations and misconceptions regarding the use of these weights. Finally, we present step-by-step recommendations for researchers interested in incorporating these analyses in their own work and point them to available web resources to assist them in producing these weights.     

SEX ROLE ORIENTATION, COGNITIVE FLEXIBILITY, AND LIFE SATISFACTION AMONG OLDER ADULTS

Bem's Sex Role Inventory (Bem, 1974) was employed to categorize 101 older adults into masculine, feminine, androgynous, and undifferentiated sex role orientations. Relationships among these sex role orientations and cognitive flexibility and life satisfaction were explored. Additionally, hierarchical multiple regression analyses were performed to assess the contributions of masculinity, femininity, and the interaction term (masculinity x femininity) in the prediction of cognitive flexibility and life satisfaction. These older adults did not vary significantly in either their cognitive flexibility or their life satisfaction as a function of their sex role categorization, nor was an appreciable percentage of variance accounted for by the predictor variables in the regression analyses. Issues are raised regarding the validity of the typological (median-split based) approaches used by researchers to assess expectations of the differentiation of masculine and feminine components in older adults. An alternative structural developmental approach, based on a factor-analytic methodology, is proposed.     

Conditional mixed models with crossed random effects

The analysis of continuous hierarchical data such as repeated measures or data from meta‐analyses can be carried out by means of the linear mixed‐effects model. However, in some situations this model, in its standard form, does pose computational problems. For example, when dealing with crossed random‐effects models, the estimation of the variance components becomes a non‐trivial task if only one observation is available for each cross‐classified level. Pseudolikelihood ideas have been used in the context of binary data with standard generalized linear multilevel models. However, even in this case the problem of the estimation of the variance remains non‐trivial. In this paper, we first propose a method to fit a crossed random‐effects model with two levels and continuous outcomes, borrowing ideas from conditional linear mixed‐effects model theory. We also propose a crossed random‐effects model for binary data combining ideas of conditional logistic regression with pseudolikelihood estimation. We apply this method to a case study with data coming from the field of psychometrics and study a series of items (responses) crossed with participants. A simulation study assesses the operational characteristics of the method.