Heteroscedasticity as a Basis of Direction Dependence in Reversible Linear Regression Models

Heteroscedasticity is a well-known issue in linear regression modeling. When heteroscedasticity is observed, researchers are advised to remedy possible model misspecification of the explanatory part of the model (e.g., considering alternative functional forms and/or omitted variables). The present contribution discusses another source of heteroscedasticity in observational data: Directional model misspecifications in the case of nonnormal variables. Directional misspecification refers to situations where alternative models are equally likely to explain the data-generating process (e.g., x → y versus y → x). It is shown that the homoscedasticity assumption is likely to be violated in models that erroneously treat true nonnormal predictors as response variables. Recently, Direction Dependence Analysis (DDA) has been proposed as a framework to empirically evaluate the direction of effects in linear models. The present study links the phenomenon of heteroscedasticity with DDA and describes visual diagnostics and nine homoscedasticity tests that can be used to make decisions concerning the direction of effects in linear models. Results of a Monte Carlo simulation that demonstrate the adequacy of the approach are presented. An empirical example is provided, and applicability of the methodology in cases of violated assumptions is discussed.     

基于多元回归的调节效应分析

在心理学和其他社科研究领域,大量实证研究建立调节模型,以分析自变量对因变量关系的影响机制,但在基于多元回归的调节效应分析实践中仍存在不足。我们回顾了均值中心化在基于多元回归的调节效应分析中的作用,均值中心化不影响乘积项(即调节效应)的检验,仅对一阶项(即主效应)的检验有影响。讨论了简单斜率的检验方法,建议在调节变量为连续变量时,使用Johnson-Neyman法进行简单斜率检验;在调节变量为类别变量或研究者对某个调节变量值感兴趣时,使用选点法。并用一个实际例子演示如何进行调节效应分析。随后展望了调节效应检验的拓展方向。     

Direction of dependence in measurement error models

Methods to determine the direction of a regression line, that is, to determine the direction of dependence in reversible linear regression models (e.g., x→y vs. y→x), have experienced rapid development within the last decade. However, previous research largely rested on the assumption that the true predictor is measured without measurement error. The present paper extends the direction dependence principle to measurement error models. First, we discuss asymmetric representations of the reliability coefficient in terms of higher moments of variables and the attenuation of skewness and excess kurtosis due to measurement error. Second, we identify conditions where direction dependence decisions are biased due to measurement error and suggest method of moments (MOM) estimation as a remedy. Third, we address data situations in which the true outcome exhibits both regression and measurement error, and propose a sensitivity analysis approach to determining the robustness of direction dependence decisions against unreliably measured outcomes. Monte Carlo simulations were performed to assess the performance of MOM-based direction dependence measures and their robustness to violated measurement error assumptions (i.e., non-independence and non-normality). An empirical example from subjective well-being research is presented. The plausibility of model assumptions and links to modern causal inference methods for observational data are discussed.     

Conditional Direction Dependence Analysis: Evaluating the Causal Direction of Effects in Linear Models with Interaction Terms

Abstract

Direction dependence analysis (DDA) makes use of higher than second moment information of variables (x and y) to detect potential confounding and to probe the causal direction of linear variable relations (i.e., whether x?→?y or y?→?x better approximates the underlying causal mechanism). The “true” predictor is assumed to be a continuous nonnormal exogenous variable. Existing methods compatible with DDA, however, are of limited use when the relation of a focal predictor and an outcome is affected by a moderator. This study presents a conditional direction dependence analysis (CDDA) framework which enables researchers to evaluate the causal direction of conditional regression effects. Monte–Carlo simulations were used to evaluate two different moderation scenarios: Study 1 evaluates the performance of CDDA tests when a moderator affects the strength of the causal effect x?→?y. Study 2 evaluates cases in which the causal direction itself (x?→?y vs y?→?x) depends on moderator values. Study 3 evaluates the robustness of DDA tests in the presence of functional model misspecifications. Results suggest that significance tests compatible with CDDA are suitable in both moderation scenarios, i.e., CDDA allows one to discern regions of a moderator in which the causal direction is uniquely identifiable. An empirical example is provided to illustrate the approach.     

Linear relations amongk sets of variables

A family of solutions for linear relations amongk sets of variables is proposed. It is shown how these solutions apply fork=2, and how they can be generalized from there tok3.The family of solutions depends on three independent choices: (i) to what extent a solution may be influenced by differences in variances of components within each set; (ii) to what extent the sets may be differentially weighted with respect to their contribution to the solution—including orthogonality constraints; (iii) whether or not individual sets of variables may be replaced by an orthogonal and unit normalized basis.Solutions are compared with respect to their optimality properties. For each solution the appropriate stationary equations are given. For one example it is shown how the determinantal equation of the stationary equations can be interpreted.     

Tests of Simple Slopes in Multiple Regression Models with an Interaction: Comparison of Four Approaches

In multiple regression researchers often follow up significant tests of the interaction between continuous predictors X and Z with tests of the simple slope of Y on X at different sample-estimated values of the moderator Z (e.g., ±1 SD from the mean of Z). We show analytically that when X and Z are randomly sampled from the population, the variance expression of the simple slope at sample-estimated values of Z differs from the traditional variance expression obtained when the values of X and Z are fixed. A simulation study using randomly sampled predictors compared four approaches: (a) the Aiken and West (1991 Aiken, L. S., &; West, S. G. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage.[Crossref] , [Google Scholar]) test of simple slopes at fixed population values of Z, (b) the Aiken and West test at sample-estimated values of Z, (c) a 95% percentile bootstrap confidence interval approach, and (d) a fully Bayesian approach with diffuse priors. The results showed that approach (b) led to inflated Type 1 error rates and 95% confidence intervals with inadequate coverage rates, whereas other approaches maintained acceptable Type 1 error rates and adequate coverage of confidence intervals. Approach (c) had asymmetric rejection rates at small sample sizes. We used an empirical data set to illustrate these approaches.     

Fungible Weights in Multiple Regression

Every set of alternate weights (i.e., nonleast squares weights) in a multiple regression analysis with three or more predictors is associated with an infinite class of weights. All members of a given class can be deemed fungiblebecause they yield identical SSE (sum of squared errors) and R ² values. Equations for generating fungible weights are reviewed and an example is given that illustrates how fungible weights can be profitably used to evaluate parameter sensitivity in multiple regression. The author wishes to thank Drs. Robyn Dawes, William Grove, Markus Keel, Leslie Yonce, Joe Rausch, the editor, and three anonymous reviewers for helpful comments on earlier versions of this article.     

A Comparative Investigation of Confidence Intervals for IndependentVariables in Linear Regression

In linear regression, the most appropriate standardized effect size for individual independent variables having an arbitrary metric remains open to debate, despite researchers typically reporting a standardized regression coefficient. Alternative standardized measures include the semipartial correlation, the improvement in the squared multiple correlation, and the squared partial correlation. No arguments based on either theoretical or statistical grounds for preferring one of these standardized measures have been mounted in the literature. Using a Monte Carlo simulation, the performance of interval estimators for these effect-size measures was compared in a 5-way factorial design. Formal statistical design methods assessed both the accuracy and robustness of the four interval estimators. The coverage probability of a large-sample confidence interval for the semipartial correlation coefficient derived from Aloe and Becker was highly accurate and robust in 98% of instances. It was better in small samples than the Yuan-Chan large-sample confidence interval for a standardized regression coefficient. It was also consistently better than both a bootstrap confidence interval for the improvement in the squared multiple correlation and a noncentral interval for the squared partial correlation.     

Significance tests to determine the direction of effects in linear regression models

Previous studies have discussed asymmetric interpretations of the Pearson correlation coefficient and have shown that higher moments can be used to decide on the direction of dependence in the bivariate linear regression setting. The current study extends this approach by illustrating that the third moment of regression residuals may also be used to derive conclusions concerning the direction of effects. Assuming non‐normally distributed variables, it is shown that the distribution of residuals of the correctly specified regression model (e.g., Y is regressed on X) is more symmetric than the distribution of residuals of the competing model (i.e., X is regressed on Y). Based on this result, 4 one‐sample tests are discussed which can be used to decide which variable is more likely to be the response and which one is more likely to be the explanatory variable. A fifth significance test is proposed based on the differences of skewness estimates, which leads to a more direct test of a hypothesis that is compatible with direction of dependence. A Monte Carlo simulation study was performed to examine the behaviour of the procedures under various degrees of associations, sample sizes, and distributional properties of the underlying population. An empirical example is given which illustrates the application of the tests in practice.     

Isolating and Examining Sources of Suppression and Multicollinearity in Multiple Linear Regression

The presence of suppression (and multicollinearity) in multiple regression analysis complicates interpretation of predictor-criterion relationships. The mathematical conditions that produce suppression in regression analysis have received considerable attention in the methodological literature but until now nothing in the way of an analytic strategy to isolate, examine, and remove suppression effects has been offered. In this article such an approach, rooted in confirmatory factor analysis theory and employing matrix algebra, is developed. Suppression is viewed as the result of criterion-irrelevant variance operating among predictors. Decomposition of predictor variables into criterion-relevant and criterion-irrelevant components using structural equation modeling permits derivation of regression weights with the effects of criterion-irrelevant variance omitted. Three examples with data from applied research are used to illustrate the approach: the first assesses child and parent characteristics to explain why some parents of children with obsessive-compulsive disorder accommodate their child's compulsions more so than do others, the second examines various dimensions of personal health to explain individual differences in global quality of life among patients following heart surgery, and the third deals with quantifying the relative importance of various aptitudes for explaining academic performance in a sample of nursing students. The approach is offered as an analytic tool for investigators interested in understanding predictor-criterion relationships when complex patterns of intercorrelation among predictors are present and is shown to augment dominance analysis.     

Order Algebras as Models of Linear Logic

The starting point of the present study is the interpretation of intuitionistic linear logic in Petri nets proposed by U. Engberg and G. Winskel. We show that several categories of order algebras provide equivalent interpretations of this logic, and identify the category of the so called strongly coherent quantales arising in these interpretations. The equivalence of the interpretations is intimately related to the categorical facts that the aforementioned categories are connected with each other via adjunctions, and the compositions of the connecting functors with co-domain the category of strongly coherent quantales are dense. In particular, each quantale canonically induces a Petri net, and this association gives rise to an adjunction between the category of quantales and a category whose objects are all Petri nets.     

Assessment of Multiple Membership Multilevel Models: An Application to Interviewer Effects on Nonresponse

Multilevel multiple membership models account for situations where lower level units are nested within multiple higher level units from the same classification. Not accounting correctly for such multiple membership structures leads to biased results. The use of a multiple membership model requires selection of weights reflecting the hypothesized contribution of each level two unit and their relationship to the level one outcome. The Deviance Information Criterion (DIC) has been proposed to identify such weights. For the case of logistic regression, this study assesses, through simulation, the model identification rates of the DIC to detect the correct multiple membership weights, and the properties of model variance estimators for different weight specifications across a range of scenarios. The study is motivated by analyzing interviewer effects across waves in a longitudinal study. Interviewers can substantially influence the behavior of sample survey respondents, including their decision to participate in the survey. In the case of a longitudinal survey several interviewers may contact sample members to participate across different waves. Multilevel multiple membership models are suitable to account for the inclusion of higher-level random effects for interviewers at various waves, and to assess, for example, the relative importance of previous and current wave interviewers on current wave nonresponse. To illustrate the application, multiple membership models are applied to the UK Family and Children Survey to identify interviewer effects in a longitudinal study. The paper takes a critical view on the substantive interpretation of the model weights and provides practical guidance to statistical modelers. The main recommendation is that it is best to specify the weights in a multiple membership model by exploring different weight specifications based on the DIC, rather than prespecifying the weights.     

Properties of the maximum likelihood solution in factor analysis regression

Algebraic properties of the normal theory maximum likelihood solution in factor analysis regression are investigated. Two commonly employed measures of the within sample predictive accuracy of the factor analysis regression function are considered: the variance of the regression residuals and the squared correlation coefficient between the criterion variable and the regression function. It is shown that this within sample residual variance and within sample squared correlation may be obtained directly from the factor loading and unique variance estimates, without use of the original observations or the sample covariance matrix.     

纳入式分类分析法在潜在剖面模型的后续多元回归中的应用

纳入式分类分析法能克服传统的分类分析法对后续一元回归模型参数的低估,发挥潜在类别模型的后续分析简化变量间交互作用的功能。本文进一步将纳入式分类分析法拓展至潜在剖面模型后续的多元统计分析中。通过蒙特卡洛模拟实验,比较各种纳入变量的方法思路与后续分析模型在四种常见的多元回归模型中参数估计的表现。结果发现,纳入式分类分析法所需纳入的变量取决于后续分析中与因变量、潜类别变量的关系,且后续分析使用含交互作用的模型更为稳健。     

Optimal Scaling of Interaction Effects in Generalized Linear Models

This study involved two phases: first, when classification was based on the calibration sample; and second, in a cross-validation setting. Computer generated data were used. Results obtained from rules based on probabilities of group membership were compared for accuracy when classifying in the discriminant space and in the predictor variable spaces. In the first phase accuracy was greater in the predictor variable spaces, while the reverse was true in the second phase. In general, rules based on probabilities of group membership were approximately equally accurate and more accurate than a rule related to a multiple regression analysis. Other findings are also discussed.     

Dealing with Multiple Solutions in Structural Vector Autoregressive Models

Structural vector autoregressive models (VARs) hold great potential for psychological science, particularly for time series data analysis. They capture the magnitude, direction of influence, and temporal (lagged and contemporaneous) nature of relations among variables. Unified structural equation modeling (uSEM) is an optimal structural VAR instantiation, according to large-scale simulation studies, and it is implemented within an SEM framework. However, little is known about the uniqueness of uSEM results. Thus, the goal of this study was to investigate whether multiple solutions result from uSEM analysis and, if so, to demonstrate ways to select an optimal solution. This was accomplished with two simulated data sets, an empirical data set concerning children's dyadic play, and modifications to the group iterative multiple model estimation (GIMME) program, which implements uSEMs with group- and individual-level relations in a data-driven manner. Results revealed multiple solutions when there were large contemporaneous relations among variables. Results also verified several ways to select the correct solution when the complete solution set was generated, such as the use of cross-validation, maximum standardized residuals, and information criteria. This work has immediate and direct implications for the analysis of time series data and for the inferences drawn from those data concerning human behavior.     

Neuropsychological Aspects of Multiple Sclerosis

Since S. Rao's [Neuropsychology of Multiple Sclerosis: A Critical Review, A Journal of Clinical and Experimental Neuropsychology, Vol. 85, pp. 503–542] (1986) seminal review, considerable research has been undertaken on the neuropsychological consequences of multiple sclerosis. This review incorporates the research literature of the last decade in presenting an overview of the current state of our knowledge concerning the etiology, course, symptoms, assessment, consequences, and treatment of multiple sclerosis (MS). The concept of subcortical dementia is revisited in light of the most recent literature documenting the neuropsychological deficits in patients with MS. The view that cognitively heterogeneous patient groups may disguise more specific patterns of focal neuropsychological impairment is considered. A critical review of the recent literature is also presented, detailing the degree to which recent research has addressed the areas of research need identified by Rao in 1986. Given recent advances in our knowledge, the need for more attention to be directed toward the evaluation of rehabilitation and psychological intervention is highlighted.     

Testing the significance of the successive components in redundancy analysis

In this paper we study the interrelationships between two sets of data measured on the same subjects via redundancy analysis. We consider redundancy analysis from an inferential point of view. Under the hypothesis of multinormality, tests of significance are obtained for each successive redundancy component so that only the significant factors are retained for prediction purposes. An example illustrates the method. The authors would like to thank the Editor and the referees for their helpful comments. This research has been partly financed by NSERC (Canada).