Introduction to the Special Issue on Mediation Analyses: What If Planetary Scientists Used Mediation Analysis to Infer Causation?

Mediation analyses have become extremely popular in the social sciences generally, and in social psychology specifically, because they seem to grant the researcher the ability to draw causal conclusions from correlational data. But this is illusory. Correlation provides poor quality evidence of causation not only in the case of zero-order correlations but also in the case of more complex mediation analyses. The articles in this special issue provide compelling arguments as to why mediation analyses make a weak case for actual mediation. My goal is to cognitively prime the reader for these articles by presenting analyses pertaining to the mean orbital momentum, kinetic energy, and conversion energy of the eight planets of our solar system. Our solar system provides a dramatic case in point where mediation analyses provide blatantly wrong conclusions.     

Neuroanatomical substrates of age-related cognitive decline

There are many reports of relations between age and cognitive variables and of relations between age and variables representing different aspects of brain structure and a few reports of relations between brain structure variables and cognitive variables. These findings have sometimes led to inferences that the age-related brain changes cause the age-related cognitive changes. Although this conclusion may well be true, it is widely recognized that simple correlations are not sufficient to warrant causal conclusions, and other types of correlational information, such as mediation and correlations between longitudinal brain changes and longitudinal cognitive changes, also have limitations with respect to causal inferences. These issues are discussed, and the existing results on relations of regional volume, white matter hyperintensities, and diffusion tensor imaging measures of white matter integrity to age and to measures of cognitive functioning are reviewed. It is concluded that at the current time the evidence that these aspects of brain structure are neuroanatomical substrates of age-related cognitive decline is weak. The final section contains several suggestions concerning measurement and methodology that may lead to stronger conclusions in the future.     

Causal Inference from Noise

Correlation is not causation is one of the mantras of the sciences—a cautionary warning especially to fields like epidemiology and pharmacology where the seduction of compelling correlations naturally leads to causal hypotheses. The standard view from the epistemology of causation is that to tell whether one correlated variable is causing the other, one needs to intervene on the system—the best sort of intervention being a trial that is both randomized and controlled. In this paper, we argue that some purely correlational data contains information that allows us to draw causal inferences: statistical noise. Methods for extracting causal knowledge from noise provide us with an alternative to randomized controlled trials that allows us to reach causal conclusions from purely correlational data.     

Correlational Data,Causal Hypotheses,and Validity

A shared problem across the sciences is to make sense of correlational data coming from observations and/or from experiments. Arguably, this means establishing when correlations are causal and when they are not. This is an old problem in philosophy. This paper, narrowing down the scope to quantitative causal analysis in social science, reformulates the problem in terms of the validity of statistical models. Two strategies to make sense of correlational data are presented: first, a ‘structural strategy’, the goal of which is to model and test causal structures that explain correlational data; second, a ‘manipulationist or interventionist strategy’, that hinges upon the notion of invariance under intervention. It is argued that while the former can offer a solution the latter cannot.     

Mood and personality: A search for the causal relationship1

Previous findings of correlations between mood and personality variables were noted. It was suggested that the establishment of a causal relationship was crucially important, not merely to the understanding of how mood is related to personality but to the conceptualization of mood as a source of error in personality measurement as well. A prior attempt to establish a causal role for mood failed, but may have done so for methodological reasons. The present study investigated the influence of a mood manipulation on personality measures. There was also an opportunity to replicate prior correlational findings independent of the effect of the mood manipulation. College students were asked to think of happy, sad, or affectively neutral events and then filled out several personality measures. A manipulation check confirmed that different moods were induced by this procedure. The prior correlations between mood and personality variables were replicated, but the mood manipulation did not affect any of the personality measures. The implications of these results for personality measurement and for the relation between mood and behavior were discussed.     

Does Smoking by Pregnant Women Influence IQ, Birth Weight, and Developmental Disabilities in Their Infants? A Methodological Review and Multivariate Analysis

Neuropsychologists are asked frequently to address the issue of the cause of a variety of central nervous system problems that may affect higher cortical function. One such issue is the relationship of maternal smoking to adverse reproductive outcomes involving neocortical insult including mental retardation, learning disabilities, attention-deficit hyperactivity disorder, and other insults that may be related to prolonged hypoxic states in utero. The instant paper develops the issue of causation as a scientific inquiry, reviews several traditional, applicable models, and critiques these models. An additional model of motility is proposed and discussed. The issue of the relationship of maternal smoking to adverse reproductive outcomes is then addressed from a review perspective along with new empirical analyses, the latter demonstrating that researchers tend to draw causal conclusions independent of whether the respective design of their studies would support conclusions about the causation of an event. Causal conclusions in the absence of causal designs have often lead to incomplete and incorrect conclusions. It is necessary to match conclusions not only to the outcomes of a research project but also to its design and accompanying limitations.     

The person in political emotion

Recent social psychological theory and research on political issues has returned to once-popular concepts such as political emotion and ideology. Strikingly, however, this work tends to avoid the notion of personality and explicit reference to individual differences. For example, the numerous studies that examine correlations between political beliefs, feelings, and preferences rarely acknowledge that such associations show an ideological coherence in individuals. Instead, correlations between abstract constructs are interpreted as suggesting causal processes. Individuals, and their responses, are aggregated to generate such correlations but remain for the most part unexamined and unmentioned. I discuss 5 practices in research and reporting that make it difficult to find the person in correlational models of political emotion. I use my own research to illustrate these practices and to show how attention to macrolevel forces such as group membership, status, and structure may be integrated with attention to the individual person and meaningful aggregates.     

Toward a process-focused model of test score validity: improving psychological assessment in science and practice

Although definitions of validity have evolved considerably since L. J. Cronbach and P. E. Meehl's classic (1955) review, contemporary validity research continues to emphasize correlational analyses assessing predictor-criterion relationships, with most outcome criteria being self-reports. The present article describes an alternative way of operationalizing validity--the process-focused (PF) model. The PF model conceptualizes validity as the degree to which respondents can be shown to engage in a predictable set of psychological processes during testing, with those processes dictated a priori by the nature of the instrument(s) used and the context in which testing takes place. In contrast to the traditional approach wherein correlational methods are used to quantify the relationship between test score and criterion, the PF model uses experimental methods to manipulate variables that moderate test score-criterion relationships, enabling researchers to draw more definitive conclusions regarding the impact of underlying psychological processes on test scores. By complementing outcome-based validity assessment with a process-driven approach, researchers will not only improve psychology's assessment procedures but also enhance their understanding of test bias and test score misuse by illuminating the intra- and interpersonal factors that lead to differential performance (and differential prediction) in different groups.     

Multiple causation,indirect measurement and generalizability in the social sciences

The fact that causal laws in the social sciences are most realistically expressed as both multivariate and stochastic has a number of very important implications for indirect measurement and generalizability. It becomes difficult to link theoretical definitions of general constructs in a one-to-one relationship to research operations, with the result that there is conceptual slippage in both experimental and nonexperimental research. It is argued that problems of this nature can be approached by developing specific multivariate causal models that incorporate sources of measurement bias, along with the theoretical variables of interest. Many general concepts are defined in such a way that causal assumptions are built into the definitions themselves. Additionally, in any given piece of research it is necessary to omit many variables from consideration, and this is often done without careful consideration of the assumptions required to justify such omissions. Finally, generalization to more inclusive populations or a diversity of settings ordinarily requires one to replace constants by variables. It is concluded that the criteria of parsimony, generalizability, and precision are incompatible, given the multivariate nature of social causation, and the author expresses his own preference for sacrificing parsimony in favor of the objectives of achieving increased precision and generalizability of social science laws.     

THE ASSUMPTIONS OF CAUSAL ANALYSES FOR INCOMPLETE CAUSAL SETS OF TWO MULTILEVEL VARIABLES

aThree causal analyses of correlations: the two-stage partial correlation analysis of Pelz and Andrews (1964); the cross-lagged panel correlation analysis of Campbell and Stanley (1963), Campbell (1963), and Rozelle and Campbell (1969); and a new three-stage analysis described in this paper are evaluated in terms of causal models representing any two variables which constitute an incomplete causal set. It is found t h a t the first two analyses make assumptions which are not generally true and which are not presently subject to practical tests. The third i s found to be useful as a way of finding out whether the correlation between two variables i s due entirely to third variable causation.     

Controlling Correlational Bias via Confirmatory Factor Analysis of MTMM Data

Confirmatory factor analysis of multitrait-multimethod (MTMM) data has proven to be a useful tool for assessing convergent and discriminant validity. However, researchers have not made full use of the results of MTMM analyses in examining the relationship between MTMM factors and variables outside the MTMM. Often, researchers simply average the various measures of each trait. Alternatively, they estimate LISREL MTMM models, but estimate only relationships between MTMM traits and the outside variables. In the present article, we show that these two approaches to analyzing data outside the MTMM produce equally highly biased parameter estimates when the actual correlations between MTMM method factors and the outside variables are substantial. An algebraic explanation and a simulated data illustration are given for the bias due to misspecification. Also, the problem is illustrated with a brief empirical example. Implications for applied research are discussed.     

Testing hypotheses on specific environmental causal effects on behavior

There have been strong critiques of the notion that environmental influences can have an important effect on psychological functioning. The substance of these criticisms is considered in order to infer the methodological challenges that have to be met. Concepts of cause and of the testing of causal effects are discussed with a particular focus on the need to consider sample selection and the value (and limitations) of longitudinal data. The designs that may be used to test hypotheses on specific environmental risk mechanisms for psychopathology are discussed in relation to a range of adoption strategies, twin designs, various types of "natural experiments," migration designs, the study of secular change, and intervention designs. In each case, consideration is given to the need for samples that "pull-apart" variables that ordinarily go together, specific hypotheses on possible causal processes, and the specification and testing of key assumptions. It is concluded that environmental risk hypotheses can be (and have been) put to the test but that it is usually necessary to use a combination of research strategies.     

Assumptions in research in sport and exercise psychology

ObjectivesThe aim of this article is to outline how certain key assumptions affect the quality and interpretation of research in quantitative sport and exercise psychology.MethodsA review of three common assumptions made in the sport and exercise psychology literature was conducted. The review focused on three assumptions relating to research validity and the treatment and interpretation of observations. A central theme to this discussion is the assumption that research observations reflect true effects in a population.ResultsAssumptions often made in sport and exercise psychology research were identified in three key areas: (1) validity, (2) inferences of causality, and (3) effect size and the “practical significance” of research findings. Findings indicated that many studies made assumptions about the validity of the self-report psychological measures adopted and few provided a comprehensive evaluation of the validity of these measures. Researchers adopting correlational designs in sport and exercise psychology often infer causality despite such conclusions being based on theory or speculation rather than empirical evidence. Research reports still do not include effect size statistics as standard and confine the discussion of findings to statistical significance alone rather than commenting on “practical significance”.ConclusionResearch quality can only be evaluated with due consideration of the common assumptions that limits empirical investigation in sport and exercise psychology. We offer some practical advice for researchers, reviewers, and journal editors to minimise the impact of these assumptions and enhance the quality of research findings in sport and exercise psychology.     

Correlation,partial correlation,and causation

Philosophers and scientists have maintained that causation, correlation, and “partial correlation” are essentially related. These views give rise to various rules of causal inference. This essay considers the claims of several philosophers and social scientists for causal systems with dichotomous variables. In section 2 important commonalities and differences are explicated among four major conceptions of correlation. In section 3 it is argued that whether correlation can serve as a measure of A's causal influence on B depends upon the conception of causation being used and upon certain background assumptions. In section 4 five major kinds of “partial correlation” are explicated, and some of the important relations are established among two conceptions of “partial correlation”, the conception of “screening off”, the conception of “partitioning”, and the measures of causal influence which have been suggested by advocates of path analysis or structural equation methods. In section 5 it is argued that whether any of these five conceptions of “partial correlation” can serve as a measure of causal influence depends upon the conception of causation being used and upon certain background assumptions. The important conclusion is that each of the approaches (considered here) to causal inference for causal systems with dichotomous variables stands in need of important qualifications and revisions if they are to be justified.     

Plausible assumptions, questionable assumptions and post hoc rationalizations: Will the real IAT, please stand up?

In a recent article, we described psychometric limitations to the Implicit Association Test (IAT). These limitations restrict the utility of this measure and render it problematic for testing many psychological theories that posit a causal role for implicit attitudes. Past failures to recognize this may have promoted faulty conclusions in the literature. In a critique of our article, Nosek and Sriram rejected our entire analysis. They asserted that our original article was based on faulty assumptions and argued that the IAT performs nicely when these assumptions are replaced by other, more plausible assumptions. We show that these plausible assumptions have all the hallmarks of post hoc rationalizations. They make little theoretical sense, are buttressed by deceptive statistical practices, contradict statements these same researchers have made in the past and do little to advance research and theory on implicit attitudes. We close by considering the vigor with which IAT researchers have dismissed meaningful criticisms of their measure.     

OUTLIER DETECTION AND TREATMENT IN I/O PSYCHOLOGY: A SURVEY OF RESEARCHER BELIEFS AND AN EMPIRICAL ILLUSTRATION

Extreme data points, or outliers, can have a disproportionate influence on the conclusions drawn from a set of bivariate correlational data. This paper addresses two aspects of outlier detection. The results of a survey regarding how published researchers prefer to deal with outliers are presented, and a set of 183 test validity studies is examined to document the effects of different approaches to the detection and exclusion of outliers on effect size measures. The study indicates that: (a) there is disagreement among researchers as to the appropriateness of deleting data points from a study; (b) researchers report greater use of visual examination of data than of numeric diagnostic techniques for detecting outliers; and (c) while outlier removal influenced effect size measures in individual studies, outlying data points were not found to be a substantial source of variance in a large test validity data set.     

From a single-level analysis to a multilevel analysis of single-case experimental designs

Multilevel modeling provides one approach to synthesizing single-case experimental design data. In this study, we present the multilevel model (the two-level and the three-level models) for summarizing single-case results over cases, over studies, or both. In addition to the basic multilevel models, we elaborate on several plausible alternative models. We apply the proposed models to real datasets and investigate to what extent the estimated treatment effect is dependent on the modeling specifications and the underlying assumptions. By considering a range of plausible models and assumptions, researchers can determine the degree to which the effect estimates and conclusions are sensitive to the specific assumptions made. If the same conclusions are reached across a range of plausible assumptions, confidence in the conclusions can be enhanced. We advise researchers not to focus on one model but conduct multiple plausible multilevel analyses and investigate whether the results depend on the modeling options.     

Teenage parenting and child development: a literature review

Research on teenage parenting together with medical and behavioral research related to child development is reviewed in an effort to determine causal factors related to reported developmental deficits among children of teenage parents. 4 general conclusions are suggested: 1) several researchers agree that children of teenage parents show poor social and intellectual competence when compared with children on non-teen parents. However, the amount of sound empirical data to support this view is minimal. For example, there are few published studies of the longterm effects of teenage parenting or of actual behavioral interactions of teenage mothers and their children. 2) It is unlikely that research along the lines of the "continuum of reproductive casualty" will lead to identification of causal factors sufficient to account for developmental deficits in children of teenage mothers. 3) Research along the lines of "the continuum of caretaking casualty" suggest numerous behavioral and environmental variables that may be related to the development of children of teenage mothers. 4) Research designs applied to the study of teenage parenting must shift from linear models to complex multivariate models that permit simultaneous analysis of organismic, environmental, and behavioral determinants of development. Finally, mental health specialists, government agencies, and researchers alike, must be willing to entertain the hypothesis that much of our knowledge of the childrearing skills of teenage mothers is based on myth rather than empirical fact. 1 such myth may be that below 19 years of age, maternal age in and of itself is an important determinant of infant development and parent-infant interaction. Poor social-economic status, family support systems, marital stability, nutrition and prenatal care may be far more important determinants of development for these children than the age of their mothers.     

A general approach to causal mediation analysis

Traditionally in the social sciences, causal mediation analysis has been formulated, understood, and implemented within the framework of linear structural equation models. We argue and demonstrate that this is problematic for 3 reasons: the lack of a general definition of causal mediation effects independent of a particular statistical model, the inability to specify the key identification assumption, and the difficulty of extending the framework to nonlinear models. In this article, we propose an alternative approach that overcomes these limitations. Our approach is general because it offers the definition, identification, estimation, and sensitivity analysis of causal mediation effects without reference to any specific statistical model. Further, our approach explicitly links these 4 elements closely together within a single framework. As a result, the proposed framework can accommodate linear and nonlinear relationships, parametric and nonparametric models, continuous and discrete mediators, and various types of outcome variables. The general definition and identification result also allow us to develop sensitivity analysis in the context of commonly used models, which enables applied researchers to formally assess the robustness of their empirical conclusions to violations of the key assumption. We illustrate our approach by applying it to the Job Search Intervention Study. We also offer easy-to-use software that implements all our proposed methods.