Simple imputation methods versus direct likelihood analysis for missing item scores in multilevel educational data

Missing data, such as item responses in multilevel data, are ubiquitous in educational research settings. Researchers in the item response theory (IRT) context have shown that ignoring such missing data can create problems in the estimation of the IRT model parameters. Consequently, several imputation methods for dealing with missing item data have been proposed and shown to be effective when applied with traditional IRT models. Additionally, a nonimputation direct likelihood analysis has been shown to be an effective tool for handling missing observations in clustered data settings. This study investigates the performance of six simple imputation methods, which have been found to be useful in other IRT contexts, versus a direct likelihood analysis, in multilevel data from educational settings. Multilevel item response data were simulated on the basis of two empirical data sets, and some of the item scores were deleted, such that they were missing either completely at random or simply at random. An explanatory IRT model was used for modeling the complete, incomplete, and imputed data sets. We showed that direct likelihood analysis of the incomplete data sets produced unbiased parameter estimates that were comparable to those from a complete data analysis. Multiple-imputation approaches of the two-way mean and corrected item mean substitution methods displayed varying degrees of effectiveness in imputing data that in turn could produce unbiased parameter estimates. The simple random imputation, adjusted random imputation, item means substitution, and regression imputation methods seemed to be less effective in imputing missing item scores in multilevel data settings.     

Neuropsychological outcomes of pediatric demyelinating diseases: a review

Immune-mediated central nervous system (CNS) demyelinating diseases impact various areas of the brain, optic nerves, and/or spinal cord and can result in a wide range of neurologic symptoms including adverse cognitive outcomes. Neuropsychological outcomes in adult multiple sclerosis (MS) are well documented, while literature on such outcomes in pediatric cohorts is more limited. Furthermore, literature on neuropsychological outcomes in pediatric acute disseminated encephalomyelitis (ADEM), neuromyelitis optica (NMO), and transverse myelitis (TM) is even more limited. This paper is the first to review what is known about neuropsychological outcomes associated with immune-mediated CNS demyelinating diseases, with a focus on pediatric MS, ADEM, NMO, and TM. Additionally, this review illuminates the need to clarify differences in neuropsychological sequelae between conditions, characterize longitudinal cognitive outcomes, and investigate neuropsychological outcomes in relation to clinical variables (e.g., age of onset, disease duration, number of relapses) and psychosocial variables (e.g., fatigue, emotional problems, behavioral functioning) to better understand neuropsychological outcomes associated with these conditions.     

Bias in longitudinal data analysis with missing data using typical linear mixed‐effects modelling and pattern‐mixture approach: An analytical illustration

We analytically derive the fixed‐effects estimates in unconditional linear growth curve models by typical linear mixed‐effects modelling (TLME) and by a pattern‐mixture (PM) approach with random‐slope‐dependent two‐missing‐pattern missing not at random (MNAR) longitudinal data. Results showed that when the missingness mechanism is random‐slope‐dependent MNAR, TLME estimates of both the mean intercept and mean slope are biased because of incorrect weights used in the estimation. More specifically, the estimate of the mean slope is biased towards the mean slope for completers, whereas the estimate of the mean intercept is biased towards the opposite direction as compared to the estimate of the mean slope. We also discuss why the PM approach can provide unbiased fixed‐effects estimates for random‐coefficients‐dependent MNAR data but does not work well for missing at random or outcome‐dependent MNAR data. A small simulation study was conducted to illustrate the results and to compare results from TLME and PM. Results from an empirical data analysis showed that the conceptual finding can be generalized to other real conditions even when some assumptions for the analytical derivation cannot be met. Implications from the analytical and empirical results were discussed and sensitivity analysis was suggested for longitudinal data analysis with missing data.     

Missing not at random models for latent growth curve analyses

The past decade has seen a noticeable shift in missing data handling techniques that assume a missing at random (MAR) mechanism, where the propensity for missing data on an outcome is related to other analysis variables. Although MAR is often reasonable, there are situations where this assumption is unlikely to hold, leading to biased parameter estimates. One such example is a longitudinal study of substance use where participants with the highest frequency of use also have the highest likelihood of attrition, even after controlling for other correlates of missingness. There is a large body of literature on missing not at random (MNAR) analysis models for longitudinal data, particularly in the field of biostatistics. Because these methods allow for a relationship between the outcome variable and the propensity for missing data, they require a weaker assumption about the missing data mechanism. This article describes 2 classic MNAR modeling approaches for longitudinal data: the selection model and the pattern mixture model. To date, these models have been slow to migrate to the social sciences, in part because they required complicated custom computer programs. These models are now quite easy to estimate in popular structural equation modeling programs, particularly Mplus. The purpose of this article is to describe these MNAR modeling frameworks and to illustrate their application on a real data set. Despite their potential advantages, MNAR-based analyses are not without problems and also rely on untestable assumptions. This article offers practical advice for implementing and choosing among different longitudinal models.     

Abstract: Evaluation of Test Statistics for Robust Structural Equation Modeling With Nonnormal Missing Data

Traditional structural equation modeling (SEM) techniques have trouble dealing with incomplete and/or nonnormal data that are often encountered in practice. Yuan and Zhang (2011a) developed a two-stage procedure for SEM to handle nonnormal missing data and proposed four test statistics for overall model evaluation. Although these statistics have been shown to work well with complete data, their performance for incomplete data has not been investigated in the context of robust statistics.

Focusing on a linear growth curve model, a systematic simulation study is conducted to evaluate the accuracy of the parameter estimates and the performance of five test statistics including the naive statistic derived from normal distribution based maximum likelihood (ML), the Satorra-Bentler scaled chi-square statistic (RML), the mean- and variance-adjusted chi-square statistic (AML), Yuan-Bentler residual-based test statistic (CRADF), and Yuan-Bentler residual-based F statistic (RF). Data are generated and analyzed in R using the package rsem (Yuan & Zhang, 2011b).

Based on the simulation study, we can observe the following: (a) The traditional normal distribution-based method cannot yield accurate parameter estimates for nonnormal data, whereas the robust method obtains much more accurate model parameter estimates for nonnormal data and performs almost as well as the normal distribution based method for normal distributed data. (b) With the increase of sample size, or the decrease of missing rate or the number of outliers, the parameter estimates are less biased and the empirical distributions of test statistics are closer to their nominal distributions. (c) The ML test statistic does not work well for nonnormal or missing data. (d) For nonnormal complete data, CRADF and RF work relatively better than RML and AML. (e) For missing completely at random (MCAR) missing data, in almost all the cases, RML and AML work better than CRADF and RF. (f) For nonnormal missing at random (MAR) missing data, CRADF and RF work better than AML. (g) The performance of the robust method does not seem to be influenced by the symmetry of outliers.     

Neuropsychological profiles of six children with anoxic brain injury

Anoxic brain injury (ABI) often results in severe memory impairment and other cognitive and behavioral deficits, although limited information is available regarding pediatric cases. This study reported the neuropsychological outcomes in six children and adolescents who sustained ABI. Profiles were compared by mechanism of injury (ischemic vs. hypoxemic) and three cases were evaluated more than once. Severe intellectual, attention, memory, and behavioral impairments were observed in all six cases although academic achievement, internalizing behavioral problems, and visuospatial deficits were in general less severe than other cognitive and behavioral deficits. The longitudinal case studies varied but showed steady increases in memory and intellectual performance in the younger children with strongest improvement in nonverbal abilities and little change in parent-reported behavior. This study raises several possible hypotheses about specific cognitive and behavioral outcomes observed in pediatric ABI.     

Estimating age change in list recall in asset and health dynamics of the oldest-old: the effects of attrition bias and missing data treatment

Average change in list recall was evaluated as a function of missing data treatment (Study 1) and dropout status (Study 2) over ages 70 to 105 in Asset and Health Dynamics of the Oldest-Old data. In Study 1 the authors compared results of full-information maximum likelihood (FIML) and the multiple imputation (MI) missing-data treatments with and without independent predictors of missingness. Results showed declines in all treatments, but declines were larger for FIML and MI treatments when predictors were included in the treatment of missing data, indicating that attrition bias was reduced. In Study 2, models that included dropout status had better fits and reduced random variance compared with models without dropout status. The authors conclude that change estimates are most accurate when independent predictors of missingness are included in the treatment of missing data with either MI or FIML and when dropout effects are modeled.     

Asymptotic bias of normal-distribution-based maximum likelihood estimates of moderation effects with data missing at random

Moderation analysis is useful for addressing interesting research questions in social sciences and behavioural research. In practice, moderated multiple regression (MMR) models have been most widely used. However, missing data pose a challenge, mainly because the interaction term is a product of two or more variables and thus is a non-linear function of the involved variables. Normal-distribution-based maximum likelihood (NML) has been proposed and applied for estimating MMR models with incomplete data. When data are missing completely at random, moderation effect estimates are consistent. However, simulation results have found that when data in the predictor are missing at random (MAR), NML can yield inaccurate estimates of moderation effects when the moderation effects are non-null. Simulation studies are subject to the limitation of confounding systematic bias with sampling errors. Thus, the purpose of this paper is to analytically derive asymptotic bias of NML estimates of moderation effects with MAR data. Results show that when the moderation effect is zero, there is no asymptotic bias in moderation effect estimates with either normal or non-normal data. When the moderation effect is non-zero, however, asymptotic bias may exist and is determined by factors such as the moderation effect size, missing-data proportion, and type of missingness dependence. Our analytical results suggest that researchers should apply NML to MMR models with caution when missing data exist. Suggestions are given regarding moderation analysis with missing data.     

Finite Mixtures of Hidden Markov Models for Longitudinal Responses Subject to Drop out

Abstract

Drop out is a typical issue in longitudinal studies. When the missingness is non-ignorable, inference based on the observed data only may be biased. This paper is motivated by the Leiden 85+ study, a longitudinal study conducted to analyze the dynamics of cognitive functioning in the elderly. We account for dependence between longitudinal responses from the same subject using time-varying random effects associated with a heterogeneous hidden Markov chain. As several participants in the study drop out prematurely, we introduce a further random effect model to describe the missing data mechanism. The potential dependence between the random effects in the two equations (and, therefore, between the two processes) is introduced through a joint distribution specified via a latent structure approach. The application of the proposal to data from the Leiden 85+ study shows its effectiveness in modeling heterogeneous longitudinal patterns, possibly influenced by the missing data process. Results from a sensitivity analysis show the robustness of the estimates with respect to misspecification of the missing data mechanism. A simulation study provides evidence for the reliability of the inferential conclusions drawn from the analysis of the Leiden 85+ data.     

追踪研究方法在国内心理研究中的应用述评

以1982~2012年中国期刊网收录的88例追踪研究为对象,从应用现状、设计特征、数据处理三方面分析和评估追踪研究方法在国内心理研究的应用情况及存在的问题。结果显示,2005年之前追踪研究方法应用增长缓慢,2005年开始呈显著增长趋势,研究对象以未成年及成年早期群体为主。主要采用固定样本追踪设计,大部分研究测量2-3次、样本量在10~300之间、持续时间在3年内。61例有缺失的研究中,38例用删除法处理缺失;主要运用 HLM、方差分析、t检验和SEM分析追踪数据。相当部分研究存在测量次数少、样本量较小、持续时间短、被试缺失严重及数据处理方法相对陈旧问题。追踪研究方法的应用应注意,根据理论模型和研究有效性要求确定设计类型和设计特征,根据数据特征选择缺失处理方法和追踪数据分析方法。     

The Role of Neuropsychological Assessment in the Functional Outcomes of Children with ADHD

The value of evidence-based services is now recognized both within clinical communities and by the public at large. Increasingly, neuropsychologists must justify the necessity of often costly and time-consuming neuropsychological assessments in the diagnosis and treatment of common childhood disorders, such as Attention-deficit/Hyperactivity Disorder (ADHD). Published medical guidelines and prominent researchers, however, have argued against the need for formal neuropsychological assessment of ADHD. The present review examines the literature on developmental outcomes in childhood ADHD, with emphasis on the utility of formal neuropsychological assessment among children diagnosed and treated in primary care settings. The review yields three central findings: 1) adherence to published diagnostic guidelines for ADHD is poor among pediatric and primary care physicians; 2) ADHD most often co-exists with other disorders, thus diagnoses made without formal psychometric assessment can be incomplete or incorrect, ultimately increasing treatment costs; and, 3) untreated children with ADHD, and those who have untreated comorbidities, are at greater risk for poor outcomes in social, academic, vocational, and practical settings. The available literature suggests that neuropsychological assessment provides information that can potentially reduce risks for poor outcomes and improve quality of life among children with ADHD. Controlled studies directly examining the impact of neuropsychological assessments in improving outcomes among children with ADHD are needed.     

Estimating Incremental Validity Under Missing Data

A common form of missing data is caused by selection on an observed variable (e.g., Z). If the selection variable was measured and is available, the data are regarded as missing at random (MAR). Selection biases correlation, reliability, and effect size estimates when these estimates are computed on listwise deleted (LD) data sets. On the other hand, maximum likelihood (ML) estimates are generally unbiased and outperform LD in most situations, at least when the data are MAR. The exception is when we estimate the partial correlation. In this situation, LD estimates are unbiased when the cause of missingness is partialled out. In other words, there is no advantage of ML estimates over LD estimates in this situation. We demonstrate that under a MAR condition, even ML estimates may become biased, depending on how partial correlations are computed. Finally, we conclude with recommendations about how future researchers might estimate partial correlations even when the cause of missingness is unknown and, perhaps, unknowable.     

Postmodeling Sensitivity Analysis to Detect the Effect of Missing Data Mechanisms

Incomplete or missing data is a common problem in almost all areas of empirical research. It is well known that simple and ad hoc methods such as complete case analysis or mean imputation can lead to biased and/or inefficient estimates. The method of maximum likelihood works well; however, when the missing data mechanism is not one of missing completely at random (MCAR) or missing at random (MAR), it too can result in incorrect inference. Statistical tests for MCAR have been proposed, but these are restricted to a certain class of problems. The idea of sensitivity analysis as a means to detect the missing data mechanism has been proposed in the statistics literature in conjunction with selection models where conjointly the data and missing data mechanism are modeled. Our approach is different here in that we do not model the missing data mechanism but use the data at hand to examine the sensitivity of a given model to the missing data mechanism. Our methodology is meant to raise a flag for researchers when the assumptions of MCAR (or MAR) do not hold. To our knowledge, no specific proposal for sensitivity analysis has been set forth in the area of structural equation models (SEM). This article gives a specific method for performing postmodeling sensitivity analysis using a statistical test and graphs. A simulation study is performed to assess the methodology in the context of structural equation models. This study shows success of the method, especially when the sample size is 300 or more and the percentage of missing data is 20% or more. The method is also used to study a set of real data measuring physical and social self-concepts in 463 Nigerian adolescents using a factor analysis model.     

加速追踪设计的方法和应用

加速追踪设计(ALD)是一种选择相邻多个群组同时进行短期追踪研究, 获得在测量上有重叠的多个群组追踪数据, 对多个群组数据进行合并建构一条在时间跨度上较长的发展趋势或增长曲线的方法。ALD结合真追踪和横断设计的特征, 既保持真追踪设计的大部分优点, 克服真追踪研究中由于重测效应和被试缺失导致的问题, 又尝试分离年龄、群组和历史时间效应, 在发展心理研究有重要应用。已有研究探讨ALD的数据分析方法、ALD的有效性及设计特征。未来研究应关注拓展设计条件下ALD的适应性, 探索非线性假设或群组效应显著时的数据分析方法和ALD中缺失数据处理问题。     

A Two-Year Longitudinal Study of Neuropsychological and Cognitive Performance in Relation to Behavioral Problems and Competencies in Elementary School Children

Despite interest in early neuropsychological status as a possible contributor to children's behavioral development, prospective longitudinal investigations of neuropsychological measures in relation to later behavioral outcomes in childhood are few. A 2-year longitudinal study in a nonselected childhood sample is reported. The study tested the influence of early neuropsychological performance (verbal fluency, mental inhibitory control, and visual spatial ability) on later childhood behavioral problems and social competency. Regular education children (n = 235) were assessed at three time points 1 year apart. To control for autocorrelation of outcome measures, Time 1 behavior was partialed while testing the effects of Time 1 neuropsychological scores on Time 3 outcome. To control for autocorrelation of neuropsychological scores, Time 2 scores were partialed while testing the predictive effect of Time 1 scores on Time 3 outcome. Both sets of regression models suggested modest but statistically significant effects for inhibitory control and verbal fluency, but not IQ, reading, or visual spatial ability, on behavioral outcome. Study results are consistent with a modest causal effect of selected neuropsychological skills on later behavioral adjustment. The findings support theories that implicate subtle neuropsychological dysfunction in the development of behavioral problems in childhood.     

A Class of Distribution-Free Models for Longitudinal Mediation Analysis

Mediation analysis constitutes an important part of treatment study to identify the mechanisms by which an intervention achieves its effect. Structural equation model (SEM) is a popular framework for modeling such causal relationship. However, current methods impose various restrictions on the study designs and data distributions, limiting the utility of the information they provide in real study applications. In particular, in longitudinal studies missing data is commonly addressed under the assumption of missing at random (MAR), where current methods are unable to handle such missing data if parametric assumptions are violated. In this paper, we propose a new, robust approach to address the limitations of current SEM within the context of longitudinal mediation analysis by utilizing a class of functional response models (FRM). Being distribution-free, the FRM-based approach does not impose any parametric assumption on data distributions. In addition, by extending the inverse probability weighted (IPW) estimates to the current context, the FRM-based SEM provides valid inference for longitudinal mediation analysis under the two most popular missing data mechanisms; missing completely at random (MCAR) and missing at random (MAR). We illustrate the approach with both real and simulated data.     

Innovative Retention Methods in Longitudinal Research: A Case Study of the Developmental Trends Study

Minimizing participant attrition is vital to the success of longitudinal research. The Developmental Trends Study (DTS), a longitudinal study of the development of disruptive behavior disorders, has achieved a low attrition rate throughout the study. The development of early retention strategies, managing contact and scheduling history through the use of electronic databases, interviewer persistence, and the emergence of new electronic search methods have contributed to the success of our study. A literature review of retention methodology and practical solutions to maintain participant cooperation is described. A case study of the DTS is presented to inform researchers in longitudinal research on new methods used to maintain high retention rates.     

Full Information Maximum Likelihood Estimation for Latent Variable Interactions With Incomplete Indicators

Researchers have developed missing data handling techniques for estimating interaction effects in multiple regression. Extending to latent variable interactions, we investigated full information maximum likelihood (FIML) estimation to handle incompletely observed indicators for product indicator (PI) and latent moderated structural equations (LMS) methods. Drawing on the analytic work on missing data handling techniques in multiple regression with interaction effects, we compared the performance of FIML for PI and LMS analytically. We performed a simulation study to compare FIML for PI and LMS. We recommend using FIML for LMS when the indicators are missing completely at random (MCAR) or missing at random (MAR) and when they are normally distributed. FIML for LMS produces unbiased parameter estimates with small variances, correct Type I error rates, and high statistical power of interaction effects. We illustrated the use of these methods by analyzing the interaction effect between advanced cancer patients’ depression and change of inner peace well-being on future hopelessness levels.     

Growth modeling with nonignorable dropout: alternative analyses of the STAR*D antidepressant trial

This article uses a general latent variable framework to study a series of models for nonignorable missingness due to dropout. Nonignorable missing data modeling acknowledges that missingness may depend not only on covariates and observed outcomes at previous time points as with the standard missing at random assumption, but also on latent variables such as values that would have been observed (missing outcomes), developmental trends (growth factors), and qualitatively different types of development (latent trajectory classes). These alternative predictors of missing data can be explored in a general latent variable framework with the Mplus program. A flexible new model uses an extended pattern-mixture approach where missingness is a function of latent dropout classes in combination with growth mixture modeling. A new selection model not only allows an influence of the outcomes on missingness but allows this influence to vary across classes. Model selection is discussed. The missing data models are applied to longitudinal data from the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) study, the largest antidepressant clinical trial in the United States to date. Despite the importance of this trial, STAR*D growth model analyses using nonignorable missing data techniques have not been explored until now. The STAR*D data are shown to feature distinct trajectory classes, including a low class corresponding to substantial improvement in depression, a minority class with a U-shaped curve corresponding to transient improvement, and a high class corresponding to no improvement. The analyses provide a new way to assess drug efficiency in the presence of dropout.     

Neuropsychological dysfunction in asthmatic children

Neuropsychological dysfunction associated with pediatric asthma is reviewed. Significant methodological confounds associated with clinical research, including problems with the objective measure of asthma, are prevalent in many studies. Most evidence does not support the belief that asthma alone results in homogeneous neuropsychological compromise. Studies of adverse reactions to asthma medications indicate medication-specific effects including slight improvements in some aspects of neuropsychological functioning, such as attention, and deficiencies in other aspects of neuropsychological functioning, such as memory. The acute neuropsychological effects of various medication regimens appears to be reversible with cessation of the asthma medication under suspicion, although no data yet exist regarding the long-term effects of therapeutic dosages of asthma medications upon a developing nervous system. The hypothetical effects of asthma on school performance have been related to non-neuropsychological variables such as a child's socioeconomic status, though there is also evidence suggesting that poorly controlled asthma is related to learning problems. Implications for pediatric neuropsychologists are discussed.