The evaluation of multiple and partial correlation coefficients from the factorial matrix

This paper shows how to compute multiple correlation coefficients, partial correlation coefficients, and regression coefficients from the factorial matrix. Special emphasis is given to computation technique and to approximation formulas. The method is extremely flexible in application since it may be applied to any subset of the original set of observed variables. It is also extremely useful when many of these coefficients are desired.     

Accounting for Missing Correlation Coefficients in Fixed-Effects MASEM

Meta-analytic structural equation modeling (MASEM) is increasingly applied to advance theories by synthesizing existing findings. MASEM essentially consists of two stages. In Stage 1, a pooled correlation matrix is estimated based on the reported correlation coefficients in the individual studies. In Stage 2, a structural model (such as a path model) is fitted to explain the pooled correlations. Frequently, the individual studies do not provide all the correlation coefficients between the research variables. In this study, we modify the currently optimal MASEM-method to deal with missing correlation coefficients, and compare its performance with existing methods. This study is the first to evaluate the performance of fixed-effects MASEM methods under different levels of missing correlation coefficients. We found that the often used univariate methods performed very poorly, while the multivariate methods performed well overall.     

An investigation of the accuracy of standardized path coefficients

Given the popular and ever-increasing use of path analytic research paradigms in the social sciences, it is desirable to conduct an investigation into the accuracy of the standardized path coefficients that are often the end-product of these paradigms. In pursuit of this goal, population parameters were preset concerning the correlations between all of the variables and their reliability coefficients. Based on these parameters, thousands of experiments were generated with varying numbers of cases (n). For each experiment, at each level of n, standard path analyses were conducted, and standardized path coefficients were obtained. These standardized path coefficients were then compared against the population path coefficients on which the simulations were based to determine their accuracy. The findings indicate mixed evidence for the accuracy of path analysis research paradigms.     

Can low Behavioral Activation System predict depressive mood?: An application of non‐normal structural equation modeling

Previous studies have reported a negative correlation between the Behavioral Activation System (BAS) and the depressive mood. However, which is the cause and which is the effect cannot be determined simply by evaluating the correlation coefficients. In a series of this study, we used non‐normal structural equation modeling, which could provide a statistically plausible answer to the question on the path direction. We tested for the path direction between the variables of interest by statistically comparing all possible models. The models assuming a unidirectional path from the BAS to the depressive mood were consistently supported. This suggested that low BAS could be a predisposition to developing the depressive mood. These findings could enhance our understanding of the mechanism underlying the development of the depressive mood.     

Common language effect size for correlations

The Pearson correlation coefficient can be translated to a common language effect size, which shows the probability of obtaining a certain value on one variable, given the value on the other variable. This common language effect size makes the size of a correlation coefficient understandable to laypeople. Three examples are provided to demonstrate the application of the common language effect size in interpreting Pearson correlation coefficients and multiple correlation coefficients.     

Intraclass correlations: uses in assessing rater reliability

Reliability coefficients often take the form of intraclass correlation coefficients. In this article, guidelines are given for choosing among six different forms of the intraclass correlation for reliability studies in which n target are rated by k judges. Relevant to the choice of the coefficient are the appropriate statistical model for the reliability and the application to be made of the reliability results. Confidence intervals for each of the forms are reviewed.     

Method Variance in Organizational Behavior and Human Resources Research: Effects on Correlations,Path Coefficients,and Hypothesis Testing

This research simulates the effects of method variance on correlations, standardized regression (path) coefficients, and squared multiple correlation coefficients. The results show that method variance can have extreme effects on these measures of association, depending on assumptions made about the nature of the method factors. The analysis also indicated that method variance can have strong effects on the probability of obtaining significant findings in the absence of true relationships. The link between findings of the present study and current developments on method variance in organizational behavior and human resources management research is discussed.     

Can scientifically useful hypotheses be tested with correlations?

Historically, interesting psychological theories have been phrased in terms of correlation coefficients, which are standardized covariances, and various statistics derived from them. Methodological practice over the last 40 years, however, has suggested it is necessary to transform such theories into hypotheses on covariances and statistics derived from them. This complication turns out to be unnecessary, because the methodology now exists to test hypotheses on latent structures of correlations directly. Two examples are given. Limitations of correlation structures are also noted.     

Applications of covariance structure modeling in psychology: cause for concern?

Methods of covariance structure modeling are frequently applied in psychological research. These methods merge the logic of confirmatory factor analysis, multiple regression, and path analysis within a single data analytic framework. Among the many applications are estimation of disattenuated correlation and regression coefficients, evaluation of multitrait-multimethod matrices, and assessment of hypothesized causal structures. Shortcomings of these methods are commonly acknowledged in the mathematical literature and in textbooks. Nevertheless, serious flaws remain in many published applications. For example, it is rarely noted that the fit of a favored model is identical for a potentially large number of equivalent models. A review of the personality and social psychology literature illustrates the nature of this and other problems in reported applications of covariance structure models.     

Tests of homogeneity of independent correlation coefficients

It is demonstrated that tests of homogeneity of independent correlation coefficients based on the simple forms of the normal andt approximations to the distribution of the correlation coefficients are comparable in terms of robustness, size and power.     

Interpreting canonical correlation analysis through biplots of structure correlations and weights

This paper extends the biplot technique to canonical correlation analysis and redundancy analysis. The plot of structure correlations is shown to the optimal for displaying the pairwise correlations between the variables of the one set and those of the second. The link between multivariate regression and canonical correlation analysis/redundancy analysis is exploited for producing an optimal biplot that displays a matrix of regression coefficients. This plot can be made from the canonical weights of the predictors and the structure correlations of the criterion variables. An example is used to show how the proposed biplots may be interpreted.     

The significance of rank difference coefficients of correlation

The coefficients of rank difference correlation that are barely significant at six different levels of significance are given forN's of 2 to 30. Most of the values were obtained by translation of Olds' tables of probabilities for various values of d ². Comparison of these data with those obtained by four other methods indicates that one method yields values more appropriate than those obtained from Olds' data for coefficients significant at the .01 level forN's from 11 to 25. This method also provides a convenient means of obtaining approximate values of coefficients significant at the .01 level forN's above 30. Need for caution in evaluating the significance of coefficients obtained from data involving tie rankings is indicated. The article concludes with recommendations as to choice of methods of determining the significance of rank difference coefficients.     

Hubert's multi-rater kappa revisited

There is a frequent need to measure the degree of agreement among R observers who independently classify n subjects within K nominal or ordinal categories. The most popular methods are usually kappa-type measurements. When R = 2, Cohen's kappa coefficient (weighted or not) is well known. When defined in the ordinal case while assuming quadratic weights, Cohen's kappa has the advantage of coinciding with the intraclass and concordance correlation coefficients. When R > 2, there are more discrepancies because the definition of the kappa coefficient depends on how the phrase ‘an agreement has occurred’ is interpreted. In this paper, Hubert's interpretation, that ‘an agreement occurs if and only if all raters agree on the categorization of an object’, is used, which leads to Hubert's (nominal) and Schuster and Smith's (ordinal) kappa coefficients. Formulae for the large-sample variances for the estimators of all these coefficients are given, allowing the latter to illustrate the different ways of carrying out inference and, with the use of simulation, to select the optimal procedure. In addition, it is shown that Schuster and Smith's kappa coefficient coincides with the intraclass and concordance correlation coefficients if the first coefficient is also defined assuming quadratic weights.     

Multilevel Dynamic Generalized Structured Component Analysis for Brain Connectivity Analysis in Functional Neuroimaging Data

We extend dynamic generalized structured component analysis (GSCA) to enhance its data-analytic capability in structural equation modeling of multi-subject time series data. Time series data of multiple subjects are typically hierarchically structured, where time points are nested within subjects who are in turn nested within a group. The proposed approach, named multilevel dynamic GSCA, accommodates the nested structure in time series data. Explicitly taking the nested structure into account, the proposed method allows investigating subject-wise variability of the loadings and path coefficients by looking at the variance estimates of the corresponding random effects, as well as fixed loadings between observed and latent variables and fixed path coefficients between latent variables. We demonstrate the effectiveness of the proposed approach by applying the method to the multi-subject functional neuroimaging data for brain connectivity analysis, where time series data-level measurements are nested within subjects.     

On the adoption of partial least squares in psychological research: Caveat emptor

The partial least squares technique (PLS) has been touted as a viable alternative to latent variable structural equation modeling (SEM) for evaluating theoretical models in the differential psychology domain. We bring some balance to the discussion by reviewing the broader methodological literature to highlight: (1) the misleading characterization of PLS as an SEM method; (2) limitations of PLS for global model testing; (3) problems in testing the significance of path coefficients; (4) extremely high false positive rates when using empirical confidence intervals in conjunction with a new “sign-change correction” for path coefficients; (5) misconceptions surrounding the supposedly superior ability of PLS to handle small sample sizes and non-normality; and (6) conceptual and statistical problems with formative measurement and the application of PLS to such models. Additionally, we also reanalyze the dataset provided by Willaby et al. (2015; http://dx.doi.org/10.1016/j.paid.2014.09.008) to highlight the limitations of PLS. Our broader review and analysis of the available evidence makes it clear that PLS is not useful for statistical estimation and testing.     

Commentary on “Mediation analysis and categorical variables: The final frontier” by Dawn Iacobucci

The commentary discusses several topics in Dr. Iacobucci's article on mediation analysis with categorical variables including the lack of equivalency of mediated effect measures, ab, and c–c′, that are algebraically equivalent in ordinary least squares regression models. The commentary raises the issue of correlated mediation coefficients and presents formulas to include the correlation in the test of significance. References to new developments in causal mediation analysis for categorical variables are given and the importance of programs of research to establish a mediation theory is emphasized.     

Standard errors of two-level scalability coefficients

For the construction of tests and questionnaires that require multiple raters (e.g., a child behaviour checklist completed by both parents) a novel ordinal scaling technique is currently being further developed, called two-level Mokken scale analysis. The technique uses within-rater and between-rater coefficients to assess the scalability of the test. These coefficients are generalizations of Mokken's scalability coefficients. In this paper we derived standard errors for the two-level coefficients and for their ratios. The coefficients, the estimates, the estimated standard errors and the software implementation are discussed and illustrated using a real-data example, and a small-scale simulation study demonstrates the accuracy of the estimates.