Missing data techniques for structural equation modeling

As with other statistical methods, missing data often create major problems for the estimation of structural equation models (SEMs). Conventional methods such as listwise or pairwise deletion generally do a poor job of using all the available information. However, structural equation modelers are fortunate that many programs for estimating SEMs now have maximum likelihood methods for handling missing data in an optimal fashion. In addition to maximum likelihood, this article also discusses multiple imputation. This method has statistical properties that are almost as good as those for maximum likelihood and can be applied to a much wider array of models and estimation methods.     

The effect of multiple indicators on the power to detect inter‐individual differences in change

Hertzog et al. evaluated the statistical power of linear latent growth curve models (LGCMs) to detect individual differences in change, i.e., variances of latent slopes, as a function of sample size, number of longitudinal measurement occasions, and growth curve reliability. We extend this work by investigating the effect of the number of indicators per measurement occasion on power. We analytically demonstrate that the positive effect of multiple indicators on statistical power is inversely related to the relative magnitude of occasion‐specific latent residual variance and is independent of the specific model that constitutes the observed variables, in particular of other parameters in the LGCM. When designing a study, researchers have to consider trade‐offs of costs and benefits of different design features. We demonstrate how knowledge about power equivalent transformations between indicator measurement designs allows researchers to identify the most cost‐efficient research design for detecting parameters of interest. Finally, we integrate different formal results to exhibit the trade‐off between the number of measurement occasions and number of indicators per occasion for constant power in LGCMs.     

Applying mixed‐effects modeling to single‐subject designs: An introduction

Behavior analysis and statistical inference have shared a conflicted relationship for over fifty years. However, a significant portion of this conflict is directed toward statistical tests (e.g., t‐tests, ANOVA) that aggregate group and/or temporal variability into means and standard deviations and as a result remove much of the data important to behavior analysts. Mixed‐effects modeling, a more recently developed statistical test, addresses many of the limitations of more basic tests by incorporating random effects. Random effects quantify individual subject variability without eliminating it from the model, hence producing a model that can predict both group and individual behavior. We present the results of a generalized linear mixed‐effects model applied to single‐subject data taken from Ackerlund Brandt, Dozier, Juanico, Laudont, & Mick, 2015, in which children chose from one of three reinforcers for completing a task. Results of the mixed‐effects modeling are consistent with visual analyses and importantly provide a statistical framework to predict individual behavior without requiring aggregation. We conclude by discussing the implications of these results and provide recommendations for further integration of mixed‐effects models in the analyses of single‐subject designs.     

Non‐parametric three‐way mixed ANOVA with aligned rank tests

Research problems that require a non‐parametric analysis of multifactor designs with repeated measures arise in the behavioural sciences. There is, however, a lack of available procedures in commonly used statistical packages. In the present study, a generalization of the aligned rank test for the two‐way interaction is proposed for the analysis of the typical sources of variation in a three‐way analysis of variance (ANOVA) with repeated measures. It can be implemented in the usual statistical packages. Its statistical properties are tested by using simulation methods with two sample sizes (n = 30 and n = 10) and three distributions (normal, exponential and double exponential). Results indicate substantial increases in power for non‐normal distributions in comparison with the usual parametric tests. Similar levels of Type I error for both parametric and aligned rank ANOVA were obtained with non‐normal distributions and large sample sizes. Degrees‐of‐freedom adjustments for Type I error control in small samples are proposed. The procedure is applied to a case study with 30 participants per group where it detects gender differences in linguistic abilities in blind children not shown previously by other methods.     

Properties of bootstrap tests for N‐of‐1 studies

N‐of‐1 study designs involve the collection and analysis of repeated measures data from an individual not using an intervention and using an intervention. This study explores the use of semi‐parametric and parametric bootstrap tests in the analysis of N‐of‐1 studies under a single time series framework in the presence of autocorrelation. When the Type I error rates of bootstrap tests are compared to Wald tests, our results show that the bootstrap tests have more desirable properties. We compare the results for normally distributed errors with those for contaminated normally distributed errors and find that, except when there is relatively large autocorrelation, there is little difference between the power of the parametric and semi‐parametric bootstrap tests. We also experiment with two intervention designs: ABAB and AB, and show the ABAB design has more power. The results provide guidelines for designing N‐of‐1 studies, in the sense of how many observations and how many intervention changes are needed to achieve a certain level of power and which test should be performed.     

Experimental Power for Indirect Effects in Group-randomized Studies with Group-level Mediators

Mediation analyses have provided a critical platform to assess the validity of theories of action across a wide range of disciplines. Despite widespread interest and development in these analyses, literature guiding the design of mediation studies has been largely unavailable. Like studies focused on the detection of a total or main effect, an important design consideration is the statistical power to detect indirect effects if they exist. Understanding the sensitivity to detect indirect effects is exceptionally important because it directly influences the scale of data collection and ultimately governs the types of evidence group-randomized studies can bring to bear on theories of action. However, unlike studies concerned with the detection of total effects, literature has not established power formulas for detecting multilevel indirect effects in group-randomized designs. In this study, we develop closed-form expressions to estimate the variance of and the power to detect indirect effects in group-randomized studies with a group-level mediator using two-level linear models (i.e., 2-2-1 mediation). The results suggest that when carefully planned, group-randomized designs may frequently be well positioned to detect mediation effects with typical sample sizes. The resulting power formulas are implemented in the R package PowerUpR and the PowerUp!-Mediator software (causalevaluation.org).     

Bayesian analysis of mixtures in structural equation models with non‐ignorable missing data

Structural equation models (SEMs) have become widely used to determine the interrelationships between latent and observed variables in social, psychological, and behavioural sciences. As heterogeneous data are very common in practical research in these fields, the analysis of mixture models has received a lot of attention in the literature. An important issue in the analysis of mixture SEMs is the presence of missing data, in particular of data missing with a non‐ignorable mechanism. However, only a limited amount of work has been done in analysing mixture SEMs with non‐ignorable missing data. The main objective of this paper is to develop a Bayesian approach for analysing mixture SEMs with an unknown number of components and non‐ignorable missing data. A simulation study shows that Bayesian estimates obtained by the proposed Markov chain Monte Carlo methods are accurate and the Bayes factor computed via a path sampling procedure is useful for identifying the correct number of components, selecting an appropriate missingness mechanism, and investigating various effects of latent variables in the mixture SEMs. A real data set on a study of job satisfaction is used to demonstrate the methodology.     

A stronger latent‐variable methodology to actual–ideal discrepancy

We introduce a latent actual–ideal discrepancy (LAID) approach based on structural equation models (SEMs) with multiple indicators and empirically weighted variables. In Study 1, we demonstrate with simulated data, the superiority of a weighted approach to discrepancy in comparison to a classic unweighted one. In Study 2, we evaluate the effects of actual and ideal appearance on physical self‐concept and self‐esteem. Actual appearance contributes positively to physical self‐concept and self‐esteem, whereas ideal appearance contributes negatively. In support of multidimensional perspective, actual‐ and ideal‐appearance effects on self‐esteem are substantially—but not completely—mediated by physical self‐concept. Whereas this pattern of results generalises across gender and age, multiple‐group invariance tests show that the effect of actual appearance on physical self‐concept is larger for women than for men. Copyright © 2008 John Wiley & Sons, Ltd.     

Minimax d-optimal designs for item response theory models

Various different item response theory (IRT) models can be used in educational and psychological measurement to analyze test data. One of the major drawbacks of these models is that efficient parameter estimation can only be achieved with very large data sets. Therefore, it is often worthwhile to search for designs of the test data that in some way will optimize the parameter estimates. The results from the statistical theory on optimal design can be applied for efficient estimation of the parameters.A major problem in finding an optimal design for IRT models is that the designs are only optimal for a given set of parameters, that is, they are locally optimal. Locally optimal designs can be constructed with a sequential design procedure. In this paper minimax designs are proposed for IRT models to overcome the problem of local optimality. Minimax designs are compared to sequentially constructed designs for the two parameter logistic model and the results show that minimax design can be nearly as efficient as sequentially constructed designs.     

Contextualized Personality Questionnaires: A Case for Copulas in Structural Equation Models for Categorical Data

For structural equation models (SEMs) with categorical data, correlated measurement residuals are not easily implemented. The problem lies mainly in the absence of a categorical analogue to the multivariate normal distribution and the absence of closed form formulas in SEMs for categorical data. We present a novel technique to handle measurement residuals that keeps the attractive SEM mainframe intact yet adds flexibility in dependence modeling without excessive computational burden. The technique is based upon the concept of copula functions and is introduced with a data set of ordinal responses originating from a contextualized personality study on aggression. Focus is on models arising in a multitrait-multimethod context, where the flexibility in dependence structures allows for method effects that can vary across the latent trait dimension. The empirical application illustrates that ignoring design-implied correlated measurement residuals can potentially influence study results and conclusions in both a quantitative as well as a qualitative way. Model parameter estimates can be biased, but more important, model inferences can be heavily distorted.     

Testing for negligible interaction: A coherent and robust approach

Researchers often want to demonstrate a lack of interaction between two categorical predictors on an outcome. To justify a lack of interaction, researchers typically accept the null hypothesis of no interaction from a conventional analysis of variance (ANOVA). This method is inappropriate as failure to reject the null hypothesis does not provide statistical evidence to support a lack of interaction. This study proposes a bootstrap‐based intersection–union test for negligible interaction that provides coherent decisions between the omnibus test and post hoc interaction contrast tests and is robust to violations of the normality and variance homogeneity assumptions. Further, a multiple comparison strategy for testing interaction contrasts following a non‐significant omnibus test is proposed. Our simulation study compared the Type I error control, omnibus power and per‐contrast power of the proposed approach to the non‐centrality‐based negligible interaction test of Cheng and Shao (2007, Statistica Sinica, 17, 1441). For 2 × 2 designs, the empirical Type I error rates of the Cheng and Shao test were very close to the nominal α level when the normality and variance homogeneity assumptions were satisfied; however, only our proposed bootstrapping approach was satisfactory under non‐normality and/or variance heterogeneity. In general a × b designs, although the omnibus Cheng and Shao test, as expected, is the most powerful, it is not robust to assumption violation and results in incoherent omnibus and interaction contrast decisions that are not possible with the intersection–union approach.     

The Power to Detect and Predict Individual Differences in Intra-Individual Variability Using the Mixed-Effects Location-Scale Model

Our goal is to provide empirical scientists with practical tools and advice with which to test hypotheses related to individual differences in intra-individual variability using the mixed-effects location-scale model. To that end, we evaluate Type I error rates and power to detect and predict individual differences in intra-individual variability using this model and provide empirically-based guidelines for building scale models that include random and/or systematically-varying fixed effects. We also provide two power simulation programs that allow researchers to conduct a priori empirical power analyses. Our results aligned with statistical power theory, in that, greater power was observed for designs with more individuals, more repeated occasions, greater proportions of variance available to be explained, and larger effect sizes. In addition, our results indicated that Type I error rates were acceptable in situations when individual differences in intra-individual variability were not initially detectable as well as when the scale-model individual-level predictor explained all initially detectable individual differences in intra-individual variability. We conclude our paper by providing study design and model building advice for those interested in using the mixed-effects location-scale model in practice.     

Linking property crime using offender crime scene behaviour: A comparison of methods

This study compared the ability of seven statistical models to distinguish between linked and unlinked crimes. The seven models utilised geographical, temporal, and modus operandi information relating to residential burglaries (n = 180), commercial robberies, (n = 118), and car thefts (n = 376). Model performance was assessed using receiver operating characteristic analysis and by examining the success with which the seven models could successfully prioritise linked over unlinked crimes. The regression‐based and probabilistic models achieved comparable accuracy and were generally more accurate than the tree‐based models tested in this study. The Logistic algorithm achieved the highest area under the curve (AUC) for residential burglary (AUC = 0.903) and commercial robbery (AUC = 0.830) and the SimpleLogistic algorithm achieving the highest for car theft (AUC = 0.820). The findings also indicated that discrimination accuracy is maximised (in some situations) if behavioural domains are utilised rather than individual crime scene behaviours and that the AUC should not be used as the sole measure of accuracy in behavioural crime linkage research.     

Bayesian Semiparametric Structural Equation Models with Latent Variables

Structural equation models (SEMs) with latent variables are widely useful for sparse covariance structure modeling and for inferring relationships among latent variables. Bayesian SEMs are appealing in allowing for the incorporation of prior information and in providing exact posterior distributions of unknowns, including the latent variables. In this article, we propose a broad class of semiparametric Bayesian SEMs, which allow mixed categorical and continuous manifest variables while also allowing the latent variables to have unknown distributions. In order to include typical identifiability restrictions on the latent variable distributions, we rely on centered Dirichlet process (CDP) and CDP mixture (CDPM) models. The CDP will induce a latent class model with an unknown number of classes, while the CDPM will induce a latent trait model with unknown densities for the latent traits. A simple and efficient Markov chain Monte Carlo algorithm is developed for posterior computation, and the methods are illustrated using simulated examples, and several applications.     

How satisfied are you with your job? Estimating the reliability of scores on a single‐item job satisfaction measure

Estimating the reliability of scores on single‐item measures can be difficult because commonly used internal consistency estimates of reliability cannot be calculated. When longitudinal data is available, statistical models can be used to decompose the variability in the latent variable at each wave into trait versus state variance. Then, reliability can be estimated as a ratio of the sum of the trait variance that is captured in repeated assessments over the total variance. The current study used latent trait‐state‐error models on a nine‐year longitudinal data (N = 5,003) to estimate the test–retest reliability of scores on a single‐item measure of job satisfaction. Results showed that job satisfaction scores were somewhat unreliable (r_xx = .49–.59) and amenable to change.     

Quantified choice of root-mean-square errors of approximation for evaluation and power analysis of small differences between structural equation models

MacCallum, Browne, and Cai (2006) proposed a new framework for evaluation and power analysis of small differences between nested structural equation models (SEMs). In their framework, the null and alternative hypotheses for testing a small difference in fit and its related power analyses were defined by some chosen root-mean-square error of approximation (RMSEA) pairs. In this article, we develop a new method that quantifies those chosen RMSEA pairs and allows a quantitative comparison of them. Our method proposes the use of single RMSEA values to replace the choice of RMSEA pairs for model comparison and power analysis, thus avoiding the differential meaning of the chosen RMSEA pairs inherent in the approach of MacCallum et al. (2006). With this choice, the conventional cutoff values in model overall evaluation can directly be transferred and applied to the evaluation and power analysis of model differences.     

Power analysis for multivariate and repeated measures designs: A flexible approach using the SPSS MANOVA procedure

Although power analysis is an important component in the planning and implementation of research designs, it is often ignored. Computer programs for performing power analysis are available, but most have limitations, particularly for complex multivariate designs. An SPSS procedure is presented that can be used for calculating power for univariate, multivariate, and repeated measures models with and without time-varying and time-constant covariates. Three examples provide a framework for calculating power via this method: an ANCOVA, a MANOVA, and a repeated measures ANOVA with two or more groups. The benefits and limitations of this procedure are discussed.     

The WIG (weighted individual and group) shrinkage estimator

Regularization, or shrinkage estimation, refers to a class of statistical methods that constrain the variability of parameter estimates when fitting models to data. These constraints move parameters toward a group mean or toward a fixed point (e.g., 0). Regularization has gained popularity across many fields for its ability to increase predictive power over classical techniques. However, articles published in JEAB and other behavioral journals have yet to adopt these methods. This paper reviews some common regularization schemes and speculates as to why articles published in JEAB do not use them. In response, we propose our own shrinkage estimator that avoids some of the possible objections associated with the reviewed regularization methods. Our estimator works by mixing weighted individual and group (WIG) data rather than by constraining parameters. We test this method on a problem of model selection. Specifically, we conduct a simulation study on the selection of matching‐law‐based punishment models, comparing WIG with ordinary least squares (OLS) regression, and find that, on average, WIG outperforms OLS in this context.