Postselection Inference in Structural Equation Modeling

Abstract

Most statistical inference methods were established under the assumption that the fitted model is known in advance. In practice, however, researchers often obtain their final model by some data-driven selection process. The selection process makes the finally fitted model random, and it also influences the sampling distribution of the estimator. Therefore, implementing naive inference methods may result in wrong conclusions—which is probably a prime source of the reproducibility crisis in psychological science. The present study accommodates three valid state-of-the-art postselection inference methods for structural equation modeling (SEM) from the statistical literature: data splitting (DS), postselection inference (PoSI), and the polyhedral (PH) method. A simulation is conducted to compare the three methods with the commonly used naive procedure under selection events made by L₁-penalized SEM. The results show that the naive method often yields incorrect inference, and that the valid methods control the coverage rate in most cases with their own pros and cons. Real world data examples show the practical use of the valid inference methods.     

Quantifying Parsimony in Structural Equation Modeling

Fitting propensity (FP) is defined as a model's average ability to fit diverse data patterns, all else being equal. The relevance of FP to model selection is examined in the context of structural equation modeling (SEM). In SEM it is well known that the number of free model parameters influences FP, but other facets of FP are routinely excluded from consideration. It is shown that models possessing the same number of free parameters but different structures may exhibit different FPs. The consequences of this fact are demonstrated using illustrative examples and models culled from published research. The case is made that further attention should be given to quantifying FP in SEM and considering it in model selection. Practical approaches are suggested.     

结构方程建模中的题目打包策略

结构方程建模中题目打包法的优缺点包括：指标数据质量变好、模型拟合程度提高; 估计偏差不大, 可校正; 估计稳定, 但降低了敏感性与可证伪性。打包法的前提条件是单维、同质, 适合结构模型分析, 不适合测量模型分析。对于单维测验, 给出了一个打包流程。对于通常的多个子量表(多维结构)测验, 推荐在子量表内打包, 每个子量表打包成1个指标或者3个指标, 用于结构方程建模。     

项目组合在结构方程模型中的应用

项目组合（itemparceling）是对同一量表中的若干项目进行整合并形成新的观测指标的过程。虽然一直以来它都是一个有争议的议题,但随着其在结构方程模型中的应用日益广泛,它越来越受到研究者重视。文章从项目组合的基本逻辑、优缺点以及具体方法等方面对项目组合的研究进行了概括,并在此基础上提出了使用的具体建议：（1）根据研究的目的与具体情境选择是否需要组合;（2）组合之前必须首先确定概念的维度性;（3）项目组合最好建立在一定的理论基础上等等。未来的研究可以深入探讨各种组合方法对模型拟合以及参数估计的影响以及项目组合在一些SEM高阶应用中的效果,并进一步与项目反应理论等测量理论相结合     

不同适应水平小学生有意义事件报告比较——小学生自我调节机制的一项研究

本研究选取发展良好与适应不良的三——五年级小学生为被试,以问卷访谈形式调查他们心目中的正性和负性意义事件。结果表明,两组被试在事件意义体验的人际亲和性、内在一外在性和丰富性方面有明显不同的反映。感受的丰富性表现在正负性意义事件两方面,某种事件负性意义感受的出现是良好适应的表现。     

基于结构方程模型的有调节的中介效应分析

有调节的中介模型是中介过程受到调节变量影响的模型。指出了目前有调节的中介效应分析普遍存在的问题：当前有调节的中介效应检验大多使用多元线性回归分析,忽略了测量误差;而基于结构方程模型(SEM)的有调节的中介效应分析需要产生乘积指标,又会面临乘积指标生成和乘积项非正态分布的问题。在简介潜调节结构方程(LMS)方法后,建议使用LMS方法得到偏差校正的bootstrap置信区间来进行基于SEM的有调节的中介效应分析。总结出一个有调节的中介SEM分析流程,并有示例和相应的Mplus程序。文末展望了LMS和有调节的中介模型的发展方向。     

Evaluating Food Policy Councils Using Structural Equation Modeling

At least 282 Food Policy Councils (FPCs) are currently working to improve access to healthy foods in their communities by connecting food system sectors, gathering community input, and advising food policy. Empirical research on FPCs is limited. This study empirically evaluates FPCs to better understand the relationships between Organizational Capacity, Social Capital, and Council Effectiveness by testing a FPC Framework adapted from Allen and colleagues (2012). Members of all FPCs in the U.S., Canada, and Native American Tribes and First Nations were invited to complete the Food Policy Council Self‐Assessment Tool (FPC‐SAT). Structural equation modeling was used to test the FPC Framework. Three hundred and fifty‐four FPC members from 95 councils completed the FPC‐SAT. After slight modification, a revised FPC Framework was a good fit with the data (χ² = 40.085, df = 24, p‐value = .021, comparative fit index = 0.988, Tucker Lewis index = 0.982, root mean squared error of approximation = 0.044, p‐close = .650). A moderation analysis revealed that community context influences the relationship between Social Capital and Council Effectiveness within the FPC Framework. The FPC Framework can guide capacity building interventions and FPC evaluations. The empirically tested framework can help FPCs efficiently work toward achieving their missions and improving their local food system.     

管理胜任力特征分析:结构方程模型检验

管理胜任力特征分析是人事选拔与评价的重要内容之一。本研究在运用基于胜任力的职位分析并总结国内外有关文献的基础上,编制了管理综合素质评价量表,并运用此量表调查了220名中高层管理者,采用因素分析和结构方程模型检验企业高级管理者胜任力特征的结构。结果表明,管理胜任力特征结构由管理素质和管理技能等两个维度构成,但在维度要素及其关键度上,职位层次间存在显著差异。本研究为管理职位的测评选拔提供了新的理论依据。     

On Nonequivalence of Several Procedures of Structural Equation Modeling

The normal theory based maximum likelihood procedure is widely used in structural equation modeling. Three alternatives are: the normal theory based generalized least squares, the normal theory based iteratively reweighted least squares, and the asymptotically distribution-free procedure. When data are normally distributed and the model structure is correctly specified, the four procedures are asymptotically equivalent. However, this equivalence is often used when models are not correctly specified. This short paper clarifies conditions under which these procedures are not asymptotically equivalent. Analytical results indicate that, when a model is not correct, two factors contribute to the nonequivalence of the different procedures. One is that the estimated covariance matrices by different procedures are different, the other is that they use different scales to measure the distance between the sample covariance matrix and the estimated covariance matrix. The results are illustrated using real as well as simulated data. Implication of the results to model fit indices is also discussed using the comparative fit index as an example. The work described in this paper was supported by a grant from the Research Grants Council of Hong Kong Special Administrative Region (Project No. CUHK 4170/99M) and by NSF grant DMS04-37167.     

A Sandwich-Type Standard Error Estimator of SEM Models with Multivariate Time Series

Structural equation models are increasingly used as a modeling tool for multivariate time series data in the social and behavioral sciences. Standard error estimators of SEM models, originally developed for independent data, require modifications to accommodate the fact that time series data are inherently dependent. In this article, we extend a sandwich-type standard error estimator of independent data to multivariate time series data. One required element of this estimator is the asymptotic covariance matrix of concurrent and lagged correlations among manifest variables, whose closed-form expression has not been presented in the literature. The performance of the adapted sandwich-type standard error estimator is evaluated using a simulation study and further illustrated using an empirical example.     

Structural Equation Modeling Approaches for Analyzing Partially Nested Data

Study designs involving clustering in some study arms, but not all study arms, are common in clinical treatment-outcome and educational settings. For instance, in a treatment arm, persons may be nested in therapy groups, whereas in a control arm there are no groups. Methodological approaches for handling such partially nested designs have recently been developed in a multilevel modeling framework (MLM-PN) and have proved very useful. We introduce two alternative structural equation modeling (SEM) approaches for analyzing partially nested data: a multivariate single-level SEM (SSEM-PN) and a multiple-arm multilevel SEM (MSEM-PN). We show how SSEM-PN and MSEM-PN can produce results equivalent to existing MLM-PNs and can be extended to flexibly accommodate several modeling features that are difficult or impossible to handle in MLM-PNs. For instance, using an SSEM-PN or MSEM-PN, it is possible to specify complex structural models involving cluster-level outcomes, obtain absolute model fit, decompose person-level predictor effects in the treatment arm using latent cluster means, and include traditional factors as predictors/outcomes. Importantly, implementation of such features for partially nested designs differs from that for fully nested designs. An empirical example involving a partially nested depression intervention combines several of these features in an analysis of interest for treatment-outcome studies.     

Structural Equation Modeling with Small Samples: Test Statistics

Structural equation modeling is a well-known technique for studying relationships among multivariate data. In practice, high dimensional nonnormal data with small to medium sample sizes are very common, and large sample theory, on which almost all modeling statistics are based, cannot be invoked for model evaluation with test statistics. The most natural method for nonnormal data, the asymptotically distribution free procedure, is not defined when the sample size is less than the number of nonduplicated elements in the sample covariance. Since normal theory maximum likelihood estimation remains defined for intermediate to small sample size, it may be invoked but with the probable consequence of distorted performance in model evaluation. This article studies the small sample behavior of several test statistics that are based on maximum likelihood estimator, but are designed to perform better with nonnormal data. We aim to identify statistics that work reasonably well for a range of small sample sizes and distribution conditions. Monte Carlo results indicate that Yuan and Bentler's recently proposed F-statistic performs satisfactorily.     

Alternative Multiple Imputation Inference for Categorical Structural Equation Modeling

The use of item responses from questionnaire data is ubiquitous in social science research. One side effect of using such data is that researchers must often account for item level missingness. Multiple imputation is one of the most widely used missing data handling techniques. The traditional multiple imputation approach in structural equation modeling has a number of limitations. Motivated by Lee and Cai’s approach, we propose an alternative method for conducting statistical inference from multiple imputation in categorical structural equation modeling. We examine the performance of our proposed method via a simulation study and illustrate it with one empirical data set.     

OpenMx 2.0: Extended Structural Equation and Statistical Modeling

The new software package OpenMx 2.0 for structural equation and other statistical modeling is introduced and its features are described. OpenMx is evolving in a modular direction and now allows a mix-and-match computational approach that separates model expectations from fit functions and optimizers. Major backend architectural improvements include a move to swappable open-source optimizers such as the newly written CSOLNP. Entire new methodologies such as item factor analysis and state space modeling have been implemented. New model expectation functions including support for the expression of models in LISREL syntax and a simplified multigroup expectation function are available. Ease-of-use improvements include helper functions to standardize model parameters and compute their Jacobian-based standard errors, access to model components through standard R $ mechanisms, and improved tab completion from within the R Graphical User Interface.     

Moment Testing for Interaction Terms in Structural Equation Modeling

Starting with Kenny and Judd (Psychol. Bull. 96:201–210, 1984) several methods have been introduced for analyzing models with interaction terms. In all these methods more information from the data than just means and covariances is required. In this paper we also use more than just first- and second-order moments; however, we are aiming to adding just a selection of the third-order moments. The key issue in this paper is to develop theoretical results that will allow practitioners to evaluate the strength of different third-order moments in assessing interaction terms of the model. To select the third-order moments, we propose to be guided by the power of the goodness-of-fit test of a model with no interactions, which varies with each selection of third-order moments. A theorem is presented that relates the power of the usual goodness-of-fit test of the model with the power of a moment test for the significance of third-order moments; the latter has the advantage that it can be computed without fitting a model. The main conclusion is that the selection of third-order moments can be based on the power of a moment test, thus assessing the relevance in the analysis of different sets of third-order moments can be computationally simple. The paper gives an illustration of the method and argues for the need of refraining from adding into the analysis an excess of higher-order moments.     

SEM Based CARMA Time Series Modeling for Arbitrary N

This article explains in detail the state space specification and estimation of first and higher-order autoregressive moving-average models in continuous time (CARMA) in an extended structural equation modeling (SEM) context for N = 1 as well as N > 1. To illustrate the approach, simulations will be presented in which a single panel model (T = 41 time points) is estimated for a sample of N = 1,000 individuals as well as for samples of N = 100 and N = 50 individuals, followed by estimating 100 separate models for each of the one-hundred N = 1 cases in the N = 100 sample. Furthermore, we will demonstrate how to test the difference between the full panel model and each N = 1 model by means of a subject-group-reproducibility test. Finally, the proposed analyses will be applied in an empirical example, in which the relationships between mood at work and mood at home are studied in a sample of N = 55 women. All analyses are carried out by ctsem, an R-package for continuous time modeling, interfacing to OpenMx.     

基于多层结构方程模型的情境效应分析—— 兼与多层线性模型比较

多层(嵌套)数据的变量关系研究, 必须借助多层模型来实现。两层模型中, 层一自变量Xij按组均值中心化, 并将组均值 置于层2截距方程式中, 可将Xij对因变量Yij的效应分解为组间和组内部分, 二者之差被称为情境效应, 称为情境变量。多层结构方程模型(MSEM)将多层线性模型(MLM)和结构方程模型(SEM)相结合, 通过设置潜变量和多指标的方法校正了MLM在情境效应分析中出现的抽样误差和测量误差, 同时解决了数据的多层(嵌套)结构和潜变量的估计问题。除了分析原理的说明, 还以班级平均竞争氛围对学生竞争表现的情境效应为例进行分析方法的示范, 并比较MSEM和MLM的异同, 随后展望了MSEM情境效应模型、情境效应无偏估计方法和情境变量研究的拓展方向。