Sensitivity of Fit Indices to Model Misspecification and Model Types

The search for cut-off criteria of fit indices for model fit evaluation (e.g., Hu &; Bentler, 1999 Hu, L. and Bentler, P. M. 1999. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling., 6: 1–55. [Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]) assumes that these fit indices are sensitive to model misspecification, but not to different types of models. If fit indices were sensitive to different types of models that are misspecified to the same degree, it would be very difficult to establish cut-off criteria that would be generally useful. The issue about SEM fit indices being sensitive to different types of models has not received sufficient attention, although there is some research suggesting that this might be the case (e.g., Kenny &; McCoach, 2003 Kenny, D. A. and McCoach, D. B. 2003. Effect of the number of variables on measures of fit in structural equation modeling. Structural Equation Modeling., 10: 333–351. [Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]). This study examines if fit indices are sensitive to different types of models while controlling for the severity of model misspecification. The findings show that most fit indices, including some very popular ones (e.g., RMSEA), may be sensitive to different types of models that have the same degree of specification error. The findings suggest that, for most fit indices, it would be difficult to establish cut-off criteria that would be generally useful in SEM applications.     

认知诊断模型下整体和项目拟合指标

认知诊断模型能否拟合测验数据,直接决定诊断结果的准确性。目前国内鲜有研究涉及认知诊断测验下的模型-资料拟合检验。文章将模型整体拟合指标及基于PPMC的项目拟合指标应用于认知诊断模型-资料拟合检验。模拟研究基于DINA,R-DINA和R-RUM三个诊断模型检验各拟合指标的表现。结果显示整体和项目拟合指标在识别数据产生模型时皆有较高准确率。采用整体和项目拟合指标比较了三个竞争模型与Tatsuoka带分数减法数据的拟合情况,显示R-RUM拟合最好。     

Sensitivity of Fit Indices to Fake Perturbation of Ordinal Data: A Sample by Replacement Approach

In many psychological questionnaires the need to analyze empirical data raises the fundamental problem of possible fake or fraudulent observations in the data. This aspect is particularly relevant for researchers working on sensitive topics such as, for example, risky sexual behaviors and drug addictions. Our contribution presents a new probabilistic approach, called Sample Generation by Replacement (SGR), to address the problem of evaluating the sensitivity of 8 commonly used SEM-based fit indices (Goodness of Fit Index, GFI; Adjusted Goodness of Fit Index, AGFI; Expected Cross Validation Index, ECVI; Standardized Root-Mean-Square Residual Index, SRMR; Root-Mean-Square Error of Approximation, RMSEA; Comparative Fit Index, CFI; Nonnormed Fit Index, NNFI; and Normed Fit Index, NFI) to fake-good ordinal data. We used SGR to perform a simulation study involving 3 different SEM models, 2 sample size conditions, and 2 estimation methods: maximum likelihood (ML) and weighted least squares (WLS). Our results show that the incremental fit indices (CFI, NNFI, and NFI) are clearly more sensitive to fake perturbation than the absolute fit indices (GFI, AGFI, and ECVI). Overall, NFI turned out to be the best and most reliable fit index. We also applied SGR to real behavioral data on (non)compliance in liver transplant patients.     

The Multigroup Multilevel Categorical Latent Growth Curve Models

Longitudinal data describe developmental patterns and enable predictions of individual changes beyond sampled time points. Major methodological issues in longitudinal data include modeling random effects, subject effects, growth curve parameters, and autoregressive residuals. This study embedded the longitudinal model within a multigroup multilevel framework and allowed for autoregressive residuals. The parameter in the new model can be estimated using the computer program WinBUGS, which adopts Markov Chain Monte Carlo algorithms. Two simulation studies were conducted. An empirical example was raised and established based on models generated by the results of empirical data, which have been fitted and compared.     

On Insensitivity of the Chi-Square Model Test to Nonlinear Misspecification in Structural Equation Models

In this paper, we show that for some structural equation models (SEM), the classical chi-square goodness-of-fit test is unable to detect the presence of nonlinear terms in the model. As an example, we consider a regression model with latent variables and interactions terms. Not only the model test has zero power against that type of misspecifications, but even the theoretical (chi-square) distribution of the test is not distorted when severe interaction term misspecification is present in the postulated model. We explain this phenomenon by exploiting results on asymptotic robustness in structural equation models. The importance of this paper is to warn against the conclusion that if a proposed linear model fits the data well according to the chi-quare goodness-of-fit test, then the underlying model is linear indeed; it will be shown that the underlying model may, in fact, be severely nonlinear. In addition, the present paper shows that such insensitivity to nonlinear terms is only a particular instance of a more general problem, namely, the incapacity of the classical chi-square goodness-of-fit test to detect deviations from zero correlation among exogenous regressors (either being them observable, or latent) when the structural part of the model is just saturated.     

Sample Sizes for Two-Group Second-Order Latent Growth Curve Models

Second-order latent growth curve models (S. C. Duncan &; Duncan, 1996 Duncan, S. C. and Duncan, T. E. 1996. A multivariate growth curve analysis of adolescent substance use.. Structural Equation Modeling, 3: 323–347. [Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]; McArdle, 1988 McArdle, J. J. 1988. “Dynamic but structural equation modeling of repeated measures data.”. In Handbook of multivariate experimental psychology, , 2nd ed. Edited by: Cattell, R. B. and Nesselroade, J. 564–614. New York: Plenum.. [Crossref] , [Google Scholar]) can be used to study group differences in change in latent constructs. We give exact formulas for the covariance matrix of the parameter estimates and an algebraic expression for the estimation of slope differences. Formulas for calculations of the required sample size are presented, illustrated, and discussed. They are checked by Monte Carlo simulations in Mplus and also by Satorra and Saris's (1985) Satorra, A. and Saris, W. E. 1985. The power of the likelihood ratio test in covariance structure analysis.. Psychometrika, 50: 83–90. [Crossref], [Web of Science ®] , [Google Scholar] power approximation techniques for small and medium effect sizes (Cohen, 1988 Cohen, J. 1988. Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum..  [Google Scholar]). Results are similar across methods. Not surprisingly, sample sizes decrease with effect sizes, indicator reliabilities, number of indicators, frequency of observation, and duration of study. The relative importance of these factors is also discussed, alone and in combination. The use of the sample size formula is illustrated using a modification of empirical results from Stoel, Peetsma, and Roeleveld (2003) Stoel, R. D., Peetsma, T. T. and Roeleveld, J. 2003. Relations between the development of school investment, self-confidence, and language achievement in elementary education: A multivariate latent growth curve approach.. Learning and Individual Differences, 13: 313–333. [Crossref], [Web of Science ®] , [Google Scholar].     

Evaluation of Model Fit in Cognitive Diagnosis Models

Cognitive diagnosis models (CDMs) estimate student ability profiles using latent attributes. Model fit to the data needs to be ascertained in order to determine whether inferences from CDMs are valid. This study investigated the usefulness of some popular model fit statistics to detect CDM fit including relative fit indices (AIC, BIC, and CAIC), and absolute fit indices (RMSEA2, ABS(fcor) and MAX(χ²_jj′)). These fit indices were assessed under different CDM settings with respect to Q-matrix misspecification and CDM misspecification. Results showed that relative fit indices selected the correct DINA model most of the times and selected the correct G-DINA model well across most conditions. Absolute fit indices rejected the true DINA model if the Q-matrix was misspecified in any way. Absolute fit indices rejected the true G-DINA model whenever the Q-matrix was under-specified. RMSEA2 could be artificially low when the Q-matrix was over-specified.     

A Latent Growth Curve Modeling Approach to Pooled Interrupted Time Series Analyses

This paper presents a latent variable approach for the estimation of treatment effects within a pooled interrupted time series (ITS) design. Although considered quasi-experimental, the ITS design has been noted as representing one of the strongest alternatives to the randomized experiment, making it highly appropriate for use in documenting the presence of effects that might warrant further evaluation in a large-scale randomized study. Results suggest that the latent variable growth modeling (LGM) is capable of detecting simultaneous differences in both level and slope, and provides tests of significance for these two necessary indicators of an ITS intervention effect. As shown in the analyses, the LGM framework provides a comprehensive and flexible approach to research design and data analysis, making available to a wide audience of researchers an analytical framework for a variety of analyses of growth and developmental processes.     

Sensitivity Analysis in Structural Equation Models: Cases and Their Influence

The detection of outliers and influential observations is routine practice in linear regression. Despite ongoing extensions and development of case diagnostics in structural equation models (SEM), their application has received limited attention and understanding in practice. The use of case diagnostics informs analysts of the uncertainty of model estimates under different subsets of the data and highlights unusual and important characteristics of certain cases. We present several measures of case influence applicable in SEM and illustrate their implementation, presentation, and interpretation with two empirical examples: (a) a common factor model on verbal and visual ability (Holzinger &; Swineford, 1939 Holzinger, K. and Swineford, F. 1939. A study in factor analysis: The stability of a bi-factor solution. Chicago, IL: University of Chicago..  [Google Scholar]) and (b) a general structural equation model assessing the effect of industrialization on democracy in a mediating model using country-level data (Bollen, 1989 Bollen, K. A. 1989. Structural equation models with latent variables New York, NY: Wiley.. [Crossref] , [Google Scholar]; Bollen &; Arminger, 1991 Bollen, K. A. and Arminger, G. 1991. Observational residuals in factor analysis and structural equation models. Sociological Methodology, 21: 235–262. [Crossref], [Web of Science ®] , [Google Scholar]). Throughout these examples, three issues are emphasized. First, cases may impact different aspects of results as identified by different measures of influence. Second, the important distinction between outliers and influential cases is highlighted. Third, the concept of good and bad cases is introduced—these are influential cases that improve/worsen overall model fit based on their presence in the sample. We conclude with a discussion on the utility of detecting influential cases in SEM and present recommendations for the use of measures of case influence.     

Nonlinear and Quasi-Simplex Patterns in Latent Growth Models

In the SEM literature, simplex and latent growth models have always been considered competing approaches for the analysis of longitudinal data, even if they are strongly connected and both of specific importance. General dynamic models, which simultaneously estimate autoregressive structures and latent curves, have been recently proposed in the literature. We discuss the properties of Autoregressive Latent Trajectories (ALT) with the aim of deriving their relationship with nonlinear growth models. We show how the quasi-simplex part of the ALT admits an equivalent nonlinear growth representation. A simulation study is performed to examine how the relationship holds in the presence of polynomial and bounded growths over time, whereas an empirical application on student achievement highlights the usefulness of the equivalence. The evaluation of the formative process in the European University system has been assuming an ever increasing importance since the beginning of the Bologna process. In this context, the analysis of student performances and capabilities using different approaches plays a fundamental role.     

Incorporating Student Mobility in Achievement Growth Modeling: A Cross-Classified Multiple Membership Growth Curve Model

Multiple membership random effects models (MMREMs) have been developed for use in situations where individuals are members of multiple higher level organizational units. Despite their availability and the frequency with which multiple membership structures are encountered, no studies have extended the MMREM approach to hierarchical growth curve modeling (GCM). This study introduces a cross-classified multiple membership growth curve model (CCMM-GCM) for modeling, for example, academic achievement trajectories in the presence of student mobility. Real data are used to demonstrate and compare growth curve model estimates using the CCMM-GCM and a conventional GCM that ignores student mobility. Results indicate that the CCMM-GCM represents a promising option for modeling growth for multiple membership data structures.     

Diagnostics of Robust Growth Curve Modeling Using Student's t Distribution

Growth curve models with different types of distributions of random effects and of intraindividual measurement errors for robust analysis are compared. After demonstrating the influence of distribution specification on parameter estimation, 3 methods for diagnosing the distributions for both random effects and intraindividual measurement errors are proposed and evaluated. The methods include (a) distribution checking based on individual growth curve analysis; (b) distribution comparison based on Deviance Information Criterion, and (c) post hoc checking of degrees of freedom estimates for t distributions. The performance of the methods is compared through simulation studies. When the sample size is reasonably large, the method of post hoc checking of degrees of freedom estimates works best. A web interface is developed to ease the use of the 3 methods. Application of the 3 methods is illustrated through growth curve analysis of mathematical ability development using data on the Peabody Individual Achievement Test Mathematics assessment from the National Longitudinal Survey of Youth 1997 Cohort (Bureau of Labor Statistics, U.S. Department of Labor, 2005).     

Nonlinear Structured Growth Mixture Models in Mplus and OpenMx

Growth mixture models (GMMs; B. O. Muthén & Muthén, 2000; B. O. Muthén & Shedden, 1999) are a combination of latent curve models (LCMs) and finite mixture models to examine the existence of latent classes that follow distinct developmental patterns. GMMs are often fit with linear, latent basis, multiphase, or polynomial change models because of their common use, flexibility in modeling many types of change patterns, the availability of statistical programs to fit such models, and the ease of programming. In this article, we present additional ways of modeling nonlinear change patterns with GMMs. Specifically, we show how LCMs that follow specific nonlinear functions can be extended to examine the presence of multiple latent classes using the Mplus and OpenMx computer programs. These models are fit to longitudinal reading data from the Early Childhood Longitudinal Study–Kindergarten Cohort to illustrate their use.     

Detecting Multiple Random Changepoints in Bayesian Piecewise Growth Mixture Models

Piecewise growth mixture models are a flexible and useful class of methods for analyzing segmented trends in individual growth trajectory over time, where the individuals come from a mixture of two or more latent classes. These models allow each segment of the overall developmental process within each class to have a different functional form; examples include two linear phases of growth, or a quadratic phase followed by a linear phase. The changepoint (knot) is the time of transition from one developmental phase (segment) to another. Inferring the location of the changepoint(s) is often of practical interest, along with inference for other model parameters. A random changepoint allows for individual differences in the transition time within each class. The primary objectives of our study are as follows: (1) to develop a PGMM using a Bayesian inference approach that allows the estimation of multiple random changepoints within each class; (2) to develop a procedure to empirically detect the number of random changepoints within each class; and (3) to empirically investigate the bias and precision of the estimation of the model parameters, including the random changepoints, via a simulation study. We have developed the user-friendly package BayesianPGMM for R to facilitate the adoption of this methodology in practice, which is available at https://github.com/lockEF/BayesianPGMM. We describe an application to mouse-tracking data for a visual recognition task.     

Observations on the Use of Growth Mixture Models in Psychological Research

Psychologists are applying growth mixture models at an increasing rate. This article argues that most of these applications are unlikely to reproduce the underlying taxonic structure of the population. At a more fundamental level, in many cases there is probably no taxonic structure to be found. Latent growth classes then categorically approximate the true continuum of individual differences in change. This approximation, although in some cases potentially useful, can also be problematic. The utility of growth mixture models for psychological science thus remains in doubt. Some ways in which these models might be more profitably used are suggested.     

The ABC’s of LGM: An Introductory Guide to Latent Variable Growth Curve Modeling

In recent years, we have witnessed an increase in the complexity of theoretical models that attempt to explain behavior from both contextual and developmental perspectives. This increase in the complexity of our theoretical propositions regarding behavior parallels recent methodological advances for the analysis of change. These new analysis techniques have fundamentally altered how we conceptualize and study change. Researchers have begun to identify larger frameworks to integrate our knowledge regarding the analysis of change. One such framework is latent growth modeling, perhaps the most important and influential statistical revolution to have recently occurred in the social and behavioral sciences. This study presents a basic introduction to a latent growth modeling approach for analyzing repeated measures data. Included is the specification and interpretation of the growth factors, primary extensions such as the analysis of growth in multiple populations, and structural models including both precursors of growth, and subsequent outcomes hypothesized to be influenced by the growth functions.     

Sensitivity to syntax in visual cortex

One of the most intriguing findings on language comprehension is that violations of syntactic predictions can affect event-related potentials as early as 120 ms, in the same time-window as early sensory processing. This effect, the so-called early left-anterior negativity (ELAN), has been argued to reflect word category access and initial syntactic structure building (Friederici, 2002). In two experiments, we used magnetoencephalography to investigate whether (a) rapid word category identification relies on overt category-marking closed-class morphemes and (b) whether violations of word category predictions affect modality-specific sensory responses. Participants read sentences containing violations of word category predictions. Unexpected items varied in whether or not their word category was marked by an overt function morpheme. In Experiment 1, the amplitude of the visual evoked M100 component was increased for unexpected items, but only when word category was overtly marked by a function morpheme. Dipole modeling localized the generator of this effect to the occipital cortex. Experiment 2 replicated the main results of Experiment 1 and eliminated two non-morphology-related explanations of the M100 contrast we observed between targets containing overt category-marking and targets that lacked such morphology. Our results show that during reading, syntactically relevant cues in the input can affect activity in occipital regions at around 125 ms, a finding that may shed new light on the remarkable rapidity of language processing.     

Cognitive Development of Lead Exposed Children from Ages 6 to 15 Years: An Application of Growth Curve Analysis

The effect of lead exposure on cognitive growth patterns was assessed in a longitudinal study of 196 children. Performances on tests of verbal comprehension and perceptual organization (Vocabulary & Block Design, Wechsler Intelligence Scales for Children) were measured at ages 6.5, 11 and 15 years. Growth curve analyses revealed that the quadratic model best described the relationship between test scores and age. Children with higher lead levels, as measured at age 15 years, demonstrated lower verbal comprehension scores over time and greater decline in their rate of Vocabulary development at age 15 years, as compared to children with lower lead levels. Lead exposure was not significantly associated with growth in perceptual organization test scores. Socioeconomic status and maternal intelligence were statistically significantly associated with growth patterns for both test scores, independent of the effects of lead. The findings suggest that lead negatively impacts the developmental progression of specific cognitive skills from childhood through adolescence.     

Measuring Patterns of Change in Personality Assessments: An Annotated Application of Latent Growth Curve Modeling

Latent growth curve (LGC) modeling within the framework of structural equation modeling (SEM) is now highly regarded as one of the most powerful and informative approaches to the analysis of longitudinal data (see, e.g., Curran &; Hussong, 2003 Curran, P. J. and Hussong, A. M. 2003. The use of latent trajectory models in psychopathology research. Journal of Abnormal Psychology, 112: 526–544. [Crossref], [PubMed], [Web of Science ®] , [Google Scholar]). Whereas LGC modeling enables researchers to test for differences in developmental trajectories across time, conventional repeated measures analyses do not provide this opportunity. Nonetheless, a review of studies reported in most psychology journals reveals scant application of this methodological approach. One possible explanation for this limited use of LGC modeling is a lack of knowledge related to its application. The intent of this article, then, is to address this deficiency by presenting an annotated application of LGC modeling to health psychology data. Based on a sample of 405 Hong Kong Chinese women who recently underwent breast cancer surgery, we walk the readers through SEM modeling procedures that test for differences in both the initial status and rate of change in Psychological Morbidity and Social Adjustment at 1, 4, and 8 months postsurgery. We interpret findings from both a methodological and a substantive perspective.