Fixed- and random-effects meta-analytic structural equation modeling: Examples and analyses in R

Meta-analytic structural equation modeling (MASEM) combines the ideas of meta-analysis and structural equation modeling for the purpose of synthesizing correlation or covariance matrices and fitting structural equation models on the pooled correlation or covariance matrix. Cheung and Chan (Psychological Methods 10:40–64, 2005b, Structural Equation Modeling 16:28–53, 2009) proposed a two-stage structural equation modeling (TSSEM) approach to conducting MASEM that was based on a fixed-effects model by assuming that all studies have the same population correlation or covariance matrices. The main objective of this article is to extend the TSSEM approach to a random-effects model by the inclusion of study-specific random effects. Another objective is to demonstrate the procedures with two examples using the metaSEM package implemented in the R statistical environment. Issues related to and future directions for MASEM are discussed.     

接纳承诺疗法的作用机制——基于元分析结构方程模型

接纳承诺疗法(Acceptance and Commitment Therapy, ACT)被认为是行为治疗“第三浪潮”的重要代表。本研究使用元分析结构方程模型, 考察ACT的作用机制。通过数据库检索与筛选, 最终纳入文献50篇。结果发现: ACT所假设的心理灵活性、接纳、此时此刻、价值的中介作用都达到统计显著, 认知解离这一中介变量并不显著; 中介机制在网络化干预中仍然得到检验; 相较之传统CBT, ACT在所假设的机制上有其区别于CBT的优势。后续临床研究应更全面地测量6大核心机制, 关注对美好生活提升的影响, 采用多点瞬时评价法, 并尽可能使用更高级、更先进的统计方法检验其作用机制。     

Meta-analytic structural equation modeling: a two-stage approach

To synthesize studies that use structural equation modeling (SEM), researchers usually use Pearson correlations (univariate r), Fisher z scores (univariate z), or generalized least squares (GLS) to combine the correlation matrices. The pooled correlation matrix is then analyzed by the use of SEM. Questionable inferences may occur for these ad hoc procedures. A 2-stage structural equation modeling (TSSEM) method is proposed to incorporate meta-analytic techniques and SEM into a unified framework. Simulation results reveal that the univariate-r, univariate-z, and TSSEM methods perform well in testing the homogeneity of correlation matrices and estimating the pooled correlation matrix. When fitting SEM, only TSSEM works well. The GLS method performed poorly in small to medium samples.     

认知诊断缺失数据处理方法的比较：零替换、多重插补与极大似然估计法

数据缺失在测验中经常发生, 认知诊断评估也不例外, 数据缺失会导致诊断结果的偏差。首先, 通过模拟研究在多种实验条件下比较了常用的缺失数据处理方法。结果表明：(1)缺失数据导致估计精确性下降, 随着人数与题目数量减少、缺失率增大、题目质量降低, 所有方法的PCCR均下降, Bias绝对值和RMSE均上升。(2)估计题目参数时, EM法表现最好, 其次是MI, FIML和ZR法表现不稳定。(3)估计被试知识状态时, EM和FIML表现最好, MI和ZR表现不稳定。其次, 在PISA2015实证数据中进一步探索了不同方法的表现。综合模拟和实证研究结果, 推荐选用EM或FIML法进行缺失数据处理。     

Fisher transformation based confidence intervals of correlations in fixed- and random-effects meta-analysis

Meta-analyses of correlation coefficients are an important technique to integrate results from many cross-sectional and longitudinal research designs. Uncertainty in pooled estimates is typically assessed with the help of confidence intervals, which can double as hypothesis tests for two-sided hypotheses about the underlying correlation. A standard approach to construct confidence intervals for the main effect is the Hedges-Olkin-Vevea Fisher-z (HOVz) approach, which is based on the Fisher-z transformation. Results from previous studies (Field, 2005, Psychol. Meth., 10, 444; Hafdahl and Williams, 2009, Psychol. Meth., 14, 24), however, indicate that in random-effects models the performance of the HOVz confidence interval can be unsatisfactory. To this end, we propose improvements of the HOVz approach, which are based on enhanced variance estimators for the main effect estimate. In order to study the coverage of the new confidence intervals in both fixed- and random-effects meta-analysis models, we perform an extensive simulation study, comparing them to established approaches. Data were generated via a truncated normal and beta distribution model. The results show that our newly proposed confidence intervals based on a Knapp-Hartung-type variance estimator or robust heteroscedasticity consistent sandwich estimators in combination with the integral z-to-r transformation (Hafdahl, 2009, Br. J. Math. Stat. Psychol., 62, 233) provide more accurate coverage than existing approaches in most scenarios, especially in the more appropriate beta distribution simulation model.     

Evaluation of Student Performance through a Multidimensional Finite Mixture IRT Model

In the Italian academic system, a student can enroll for an exam immediately after the end of the teaching period or can postpone it; in this second case the exam result is missing. We propose an approach for the evaluation of a student performance throughout the course of study, accounting also for nonattempted exams. The approach is based on an item response theory model that includes two discrete latent variables representing student performance and priority in selecting the exams to take. We explicitly account for nonignorable missing observations as the indicators of attempted exams also contribute to measure the performance (within-item multidimensionality). The model also allows for individual covariates in its structural part.     

家庭支持型主管行为对员工的影响与作用机制：基于元分析的证据

为厘清家庭支持型主管行为对员工的独特影响,并比较不同的作用机制,本研究对包含204个独立样本、340个效应值及91145名员工的164篇文献进行了元分析,结果发现：(1)与一般主管支持行为相比,家庭支持型主管行为对员工的任务绩效、创新行为和生活满意度有更强的积极影响。(2)工作对家庭冲突(资源视角)、领导-成员交换(交换视角)和情感承诺(情感视角)均能解释家庭支持型主管行为对员工的作用机制,并互为补充。具体而言,三者均能中介家庭支持型主管行为对任务绩效的影响;领导-成员交换和情感承诺在家庭支持型主管行为与创新行为间起中介作用;工作对家庭冲突和领导-成员交换则在家庭支持型主管行为影响生活满意度中发挥中介效应。研究结果为家庭支持型主管行为的影响效果提供了可靠结论,也有助于深入理解其作用机制。     

Generalized canonical correlation analysis of matrices with missing rows: a simulation study

A method is presented for generalized canonical correlation analysis of two or more matrices with missing rows. The method is a combination of Carroll’s (1968) method and the missing data approach of the OVERALS technique (Van der Burg, 1988). In a simulation study we assess the performance of the method and compare it to an existing procedure called GENCOM, proposed by Green and Carroll (1988). We find that the proposed method outperforms the GENCOM algorithm both with respect to model fit and recovery of the true structure. The research of Michel van de Velden was partly funded through EU Grant HPMF-CT-2000-00664. The authors would like to thank the associate editor and three anonymous referees for their constructive comments and suggestions that led to a considerable improvement of the paper.     

Analysis of Variance of Multiply Imputed Data

As a procedure for handling missing data, Multiple imputation consists of estimating the missing data multiple times to create several complete versions of an incomplete data set. All these data sets are analyzed by the same statistical procedure, and the results are pooled for interpretation. So far, no explicit rules for pooling F tests of (repeated-measures) analysis of variance have been defined. In this article we outline the appropriate procedure for the results of analysis of variance (ANOVA) for multiply imputed data sets. It involves both reformulation of the ANOVA model as a regression model using effect coding of the predictors and applying already existing combination rules for regression models. The proposed procedure is illustrated using 3 example data sets. The pooled results of these 3 examples provide plausible F and p values.     

2PLM下缺失数据处理方法及其比较

项目反应理论(IRT)是用于客观测量的现代教育与心理测量理论之一,广泛用于缺失数据十分常见的大尺度测验分析。IRT中两参数逻辑斯蒂克模型(2PLM)下仅有完全随机缺失机制下缺失反应和缺失能力处理的EM算法。本研究推导2PLM下缺失反应忽略的EM 算法,并提出随机缺失机制下缺失反应和缺失能力处理的EM算法和考虑能力估计和作答反应不确定性的多重借补法。研究显示：在各种缺失机制、缺失比例和测验设计下,缺失反应忽略的EM算法和多重借补法表现理想。     

Full Information Maximum Likelihood Estimation for Latent Variable Interactions With Incomplete Indicators

Researchers have developed missing data handling techniques for estimating interaction effects in multiple regression. Extending to latent variable interactions, we investigated full information maximum likelihood (FIML) estimation to handle incompletely observed indicators for product indicator (PI) and latent moderated structural equations (LMS) methods. Drawing on the analytic work on missing data handling techniques in multiple regression with interaction effects, we compared the performance of FIML for PI and LMS analytically. We performed a simulation study to compare FIML for PI and LMS. We recommend using FIML for LMS when the indicators are missing completely at random (MCAR) or missing at random (MAR) and when they are normally distributed. FIML for LMS produces unbiased parameter estimates with small variances, correct Type I error rates, and high statistical power of interaction effects. We illustrated the use of these methods by analyzing the interaction effect between advanced cancer patients’ depression and change of inner peace well-being on future hopelessness levels.     

The Importance of the Default Mode Network in Creativity—A Structural MRI Study

Anecdotal reports as well as behavioral studies have suggested that creative performance benefits from unconscious processes. So far, however, little is known about how creative ideas arise from the brain. In the current study, we aimed to investigate the neural correlates of creativity by means of structural MRI research. Given that unconscious and less controlled processes are important in creative thinking, structural brain research may find a positive correlation between well‐established creativity measures and cortical thickness in brain structures of the default mode network (i.e., the counterpart of the cognitive control network). Individuals performed the Alternative Uses task by which an individual's cognitive flexibility and the average uniqueness and average creativity of a participant's ideas were assessed. We computed optimized voxel‐based‐morphometry (VBM) to explore the correlation between inter‐individual differences in creativity and inter‐individual differences in gray matter volume. For all creativity measures, a positive correlation was found between creative performance and gray matter volume of the default mode network. These findings support the idea that the default mode network is important in creativity, and provide neurostructural support for the idea that unconscious forms of information processing are important in creativity. Theoretical and practical implications are discussed.     

More efficient parameter estimates for factor analysis of ordinal variables by ridge generalized least squares

Data in psychology are often collected using Likert‐type scales, and it has been shown that factor analysis of Likert‐type data is better performed on the polychoric correlation matrix than on the product‐moment covariance matrix, especially when the distributions of the observed variables are skewed. In theory, factor analysis of the polychoric correlation matrix is best conducted using generalized least squares with an asymptotically correct weight matrix (AGLS). However, simulation studies showed that both least squares (LS) and diagonally weighted least squares (DWLS) perform better than AGLS, and thus LS or DWLS is routinely used in practice. In either LS or DWLS, the associations among the polychoric correlation coefficients are completely ignored. To mend such a gap between statistical theory and empirical work, this paper proposes new methods, called ridge GLS, for factor analysis of ordinal data. Monte Carlo results show that, for a wide range of sample sizes, ridge GLS methods yield uniformly more accurate parameter estimates than existing methods (LS, DWLS, AGLS). A real‐data example indicates that estimates by ridge GLS are 9–20% more efficient than those by existing methods. Rescaled and adjusted test statistics as well as sandwich‐type standard errors following the ridge GLS methods also perform reasonably well.     

Postmodeling Sensitivity Analysis to Detect the Effect of Missing Data Mechanisms

Incomplete or missing data is a common problem in almost all areas of empirical research. It is well known that simple and ad hoc methods such as complete case analysis or mean imputation can lead to biased and/or inefficient estimates. The method of maximum likelihood works well; however, when the missing data mechanism is not one of missing completely at random (MCAR) or missing at random (MAR), it too can result in incorrect inference. Statistical tests for MCAR have been proposed, but these are restricted to a certain class of problems. The idea of sensitivity analysis as a means to detect the missing data mechanism has been proposed in the statistics literature in conjunction with selection models where conjointly the data and missing data mechanism are modeled. Our approach is different here in that we do not model the missing data mechanism but use the data at hand to examine the sensitivity of a given model to the missing data mechanism. Our methodology is meant to raise a flag for researchers when the assumptions of MCAR (or MAR) do not hold. To our knowledge, no specific proposal for sensitivity analysis has been set forth in the area of structural equation models (SEM). This article gives a specific method for performing postmodeling sensitivity analysis using a statistical test and graphs. A simulation study is performed to assess the methodology in the context of structural equation models. This study shows success of the method, especially when the sample size is 300 or more and the percentage of missing data is 20% or more. The method is also used to study a set of real data measuring physical and social self-concepts in 463 Nigerian adolescents using a factor analysis model.     

Incomplete Information, Inferences, and Individual Differences: The Case of Environmental Judgments

A model for inference of missing information is explicated and tested in two studies (Ns = 74, 76) of judgments about two environmental issues (endangered species reintroduction and the siting of a waste processing facility). Participants made judgments of scenarios in which information relevant to the judgment was varied orthogonally and, in some cases, relevant information was missing. The results showed individual differences-as well as intraindividual differences-in the assumptions participants made about missing information and in the tendency to infer missing cues. Reported assumptions about missing information predicted some aspects of the judgments. The data for only a small minority (15%) of the participants were consistent with the inferred values model. Participants may use different methods for dealing with missing information at different times or may not generally follow either an inference or averaging model in such contexts. Less favorable judgments were given for scenarios with incomplete information (the "penalty" effect), and this effect showed individual and intraindividual variation that was related to reported assumptions about missing information. We discuss the implications of these results for societal conflicts over controversial issues and for understanding the sources of individual differences in judgments. Copyright 2000 Academic Press.     

Modeling Concordance Correlation Coefficient for Longitudinal Study Data

Measures of agreement are used in a wide range of behavioral, biomedical, psychosocial, and health-care related research to assess reliability of diagnostic test, psychometric properties of instrument, fidelity of psychosocial intervention, and accuracy of proxy outcome. The concordance correlation coefficient (CCC) is a popular measure of agreement for continuous outcomes. In modern-day applications, data are often clustered, making inference difficult to perform using existing methods. In addition, as longitudinal study designs become increasingly popular, missing data have become a serious issue, and the lack of methods to systematically address this problem has hampered the progress of research in the aforementioned fields. In this paper, we develop a novel approach to tackle the complexities involved in addressing missing data and other related issues for performing CCC analysis within a longitudinal data setting. The approach is illustrated with both real and simulated data.     

追踪研究方法在国内心理研究中的应用述评

以1982~2012年中国期刊网收录的88例追踪研究为对象,从应用现状、设计特征、数据处理三方面分析和评估追踪研究方法在国内心理研究的应用情况及存在的问题。结果显示,2005年之前追踪研究方法应用增长缓慢,2005年开始呈显著增长趋势,研究对象以未成年及成年早期群体为主。主要采用固定样本追踪设计,大部分研究测量2-3次、样本量在10~300之间、持续时间在3年内。61例有缺失的研究中,38例用删除法处理缺失;主要运用 HLM、方差分析、t检验和SEM分析追踪数据。相当部分研究存在测量次数少、样本量较小、持续时间短、被试缺失严重及数据处理方法相对陈旧问题。追踪研究方法的应用应注意,根据理论模型和研究有效性要求确定设计类型和设计特征,根据数据特征选择缺失处理方法和追踪数据分析方法。     

On the use of beta coefficients in meta-analysis

This research reports an investigation of the use of standardized regression (beta) coefficients in meta-analyses that use correlation coefficients as the effect-size metric. The investigation consisted of analyzing more than 1,700 corresponding beta coefficients and correlation coefficients harvested from published studies. Results indicate that, under certain conditions, using knowledge of corresponding beta coefficients to input missing correlations (effect sizes) generally produces relatively accurate and precise population effect-size estimates. Potential benefits from applying this knowledge include smaller sampling errors because of increased numbers of effect sizes and smaller non-sampling errors because of the inclusion of a broader array of research designs.     

Modeling storage and retrieval processes with clinical populations with applications examining alcohol-induced amnesia and Korsakoff amnesia

Multinomial processing tree models can provide for measures of underlying cognitive processes. In this paper, the Chechile [Chechile, R. A. (2004). New multinomial models for the Chechile-Meyer task. Journal of Mathematical Psychology, 48, 364-384] 6P model is described and applied to several applications involving clinical populations. The model provides for separate measures of storage and retrieval. Monte Carlo studies were conducted to examine the relative accuracy of two methods for obtaining an overall condition estimate for the 6P model, i.e., averaging estimates found for individuals versus pooling the multinomial frequency data before estimating the model parameters. The sampling studies showed that the pooling of frequencies resulted in more accurate parameter estimates. However, psychological assessment in clinical psychology requires precise measurement on an individual basis. In order to recover information about individuals from pooled frequency information, a modified jackknife method was advanced. The jackknife method is based on a contrast between the overall pooled frequency information and the pool frequency without the observations from a single individual. Another series of Monte Carlo simulations demonstrate that the new jackknife method resulted in better recovery of the correct individual parameter values relative to estimates based on only the data from the individual. Finally, the 6P model was used to examine the data from two previously reported studies with clinical populations. One application addressed the effect of alcohol-induced amnesia, and the other application dealt with Korsakoff amnesia. In both cases the pattern of storage and retrieval measurements resulted in a clarification of the underlying storage and retrieval differences between the clinical group and the control group.