How to handle missing data: A comparison of different approaches

Many researchers face the problem of missing data in longitudinal research. Especially, high risk samples are characterized by missing data which can complicate analyses and the interpretation of results. In the current study, our aim was to find the most optimal and best method to deal with the missing data in a specific study with many missing data on the outcome variable. Therefore, different techniques to handle missing data were evaluated, and a solution to efficiently handle substantial amounts of missing data was provided. A simulation study was conducted to determine the most optimal method to deal with the missing data. Results revealed that multiple imputation (MI) using predictive mean matching was the most optimal method with respect to lowest bias and the smallest confidence interval (CI) while maintaining power. Listwise deletion and last observation carried backward also scored acceptable with respect to bias; however, CIs were much larger and sample size almost halved using these methods. Longitudinal research in high risk samples could benefit from using MI in future research to handle missing data. The paper ends with a checklist for handling missing data.     

Exploratory Factor Analysis With Small Samples and Missing Data

Exploratory factor analysis (EFA) is an extremely popular method for determining the underlying factor structure for a set of variables. Due to its exploratory nature, EFA is notorious for being conducted with small sample sizes, and recent reviews of psychological research have reported that between 40% and 60% of applied studies have 200 or fewer observations. Recent methodological studies have addressed small size requirements for EFA models; however, these models have only considered complete data, which are the exception rather than the rule in psychology. Furthermore, the extant literature on missing data techniques with small samples is scant, and nearly all existing studies focus on topics that are not of primary interest to EFA models. Therefore, this article presents a simulation to assess the performance of various missing data techniques for EFA models with both small samples and missing data. Results show that deletion methods do not extract the proper number of factors and estimate the factor loadings with severe bias, even when data are missing completely at random. Predictive mean matching is the best method overall when considering extracting the correct number of factors and estimating factor loadings without bias, although 2-stage estimation was a close second.     

Career patterns: A twenty-year panel study

Using 20 years of employment and job mobility data from a representative German sample (N = 1259), we employ optimal matching analysis (OMA) to identify six career patterns which deviate from the traditional career path of long-term, full-time employment in one organization. Then, in further analyses, we examine which socio-demographic predictors affect whether or not individuals follow that traditional career path. Results indicate that age, gender, marital status, number of children, education, and career starts in the public sector significantly predicted whether or not individuals followed the traditional career path. The article concludes with directions for future theoretical and methodological research on career patterns.     

概化理论缺失数据方差分量估计

各种心理调查、心理实验中, 数据的缺失随处可见。由于数据缺失, 给概化理论分析非平衡数据的方差分量带来一系列问题。基于概化理论框架下, 运用Matlab 7.0软件, 自编程序模拟产生随机双面交叉设计p×i×r缺失数据, 比较和探讨公式法、REML法、拆分法和MCMC法在估计各个方差分量上的性能优劣。结果表明：(1) MCMC方法估计随机双面交叉设计p×i×r缺失数据方差分量, 较其它3种方法表现出更强的优势; (2) 题目和评分者是缺失数据方差分量估计重要的影响因素。     

Bayesian Inference for Growth Mixture Models with Latent Class Dependent Missing Data

Growth mixture models (GMMs) with nonignorable missing data have drawn increasing attention in research communities but have not been fully studied. The goal of this article is to propose and to evaluate a Bayesian method to estimate the GMMs with latent class dependent missing data. An extended GMM is first presented in which class probabilities depend on some observed explanatory variables and data missingness depends on both the explanatory variables and a latent class variable. A full Bayesian method is then proposed to estimate the model. Through the data augmentation method, conditional posterior distributions for all model parameters and missing data are obtained. A Gibbs sampling procedure is then used to generate Markov chains of model parameters for statistical inference. The application of the model and the method is first demonstrated through the analysis of mathematical ability growth data from the National Longitudinal Survey of Youth 1997 (Bureau of Labor Statistics, U.S. Department of Labor, 1997). A simulation study considering 3 main factors (the sample size, the class probability, and the missing data mechanism) is then conducted and the results show that the proposed Bayesian estimation approach performs very well under the studied conditions. Finally, some implications of this study, including the misspecified missingness mechanism, the sample size, the sensitivity of the model, the number of latent classes, the model comparison, and the future directions of the approach, are discussed.     

Women’s occupational career patterns over 27 years: Relations to family of origin, life careers, and wellness

This study aimed at identifying and describing occupational career patterns (OCPs) from age 16 to 43 by applying optimal matching techniques to sequence data obtained from a sample of Swedish women. Women’s occupational trajectories were found to be diverse. Upward mobility (3 patterns) and stable careers (4 patterns) were prevalent, but there were also women characterized by downward mobility, fluctuation and being outside the labor market (1 pattern each). Women’s OCPs were related to family of origin, but more strongly to their overall life career (i.e., multiple role constellations over the life course). The study indicates that occupational mobility patterns do matter in terms of job perceptions, work attitudes, and quality of life. The results generally confirm the popular belief of the advantage of upward mobility, followed by stable occupational careers, whereas women with downward or fluctuating careers fared worst.     

2PL模型的两种马尔可夫蒙特卡洛缺失数据处理方法比较

马尔科夫蒙特卡洛（MCMC）是项目反应理论中处理缺失数据的一种典型方法。文章通过模拟研究比较了在不同被试人数,项目数,缺失比例下两种MCMC方法（M-H within Gibbs和DA-T Gibbs）参数估计的精确性,并结合了实证研究。研究结果表明,两种方法是有差异的,项目参数估计均受被试人数影响很大,受缺失比例影响相对更小。在样本较大缺失比例较小时,M-H within Gibbs参数估计的均方误差（RMSE）相对略小,随着样本数的减少或缺失比例的增加,DA-T Gibbs方法逐渐优于M-H within Gibbs方法     

Managerial career patterns: A review of the empirical evidence

Despite the ubiquitous presence of the term “career patterns” in the discourse about careers, the existing empirical evidence on (managerial) career patterns is rather limited. From this literature review of 33 published empirical studies of managerial and similar professional career patterns found in electronic bibliographic databases, it is clear that upward mobility is still the norm, even when contrasting traditional to “new” careers. We argue that the nature and number of unique career patterns identified is strongly influenced by where and when the data were collected (i.e. empirical access), and how career patterns are measured. Our review further shows organizations clearly still act as containing social structures for the patterning of managerial careers, and that contemporary managerial careers, despite some evidence of increasing inter-organizational mobility, have new boundaries which induce linearity. We provide insights for further conceptualization of managerial career patterns and for advancing methodological approaches, including the use of optimal matching analysis and narratives. By expanding the scope of career pattern dimensions beyond time and direction, this review provides ground for further research on managerial career patterns.     

基于增长模型的非随机缺失数据处理:选择模型和极大似然方法

对含有非随机缺失数据的潜变量增长模型,为了考察基于不同假设的缺失数据处理方法:极大似然(ML)方法与DiggleKenward选择模型的优劣,通过Monte Carlo模拟研究,比较两种方法对模型中增长参数估计精度及其标准误估计的差异,并考虑样本量、非随机缺失比例和随机缺失比例的影响。结果表明,符合前提假设的Diggle-Kenward选择模型的参数估计精度普遍高于ML方法;对于标准误估计值,ML方法存在一定程度的低估,得到的置信区间覆盖比率也明显低于Diggle-Kenward选择模型。     

Response stereotypy without automaticity: Not quite involuntary attention in the pigeon

Previous research has shown that when pigeons are required to peck each of two keys four times in any order for reinforcement, stereotyped response sequences develop, though they are not required by the task. However, despite this stereotypy, sequences are not automatized. They require sufficient attention that sequence performance is disrupted by a concurrent, delayed matching-to-sample task. In the present experiments, pigeons were required to execute response sequences and delayed matching-to-sample concurrently. Experiment 1 showed that the matching task disrupts sequence performance even when the matching sample stimuli do not have to be processed concurrently with sequence execution. The presence of sample stimuli during sequence execution seems to command attention even if the sample stimuli are also available for processing when sequences are not being executed. Experiment 2 used the directed forgetting technique to show that pigeons do seem able to ignore sample stimuli if they are given information that tells them they will not be required to remember those stimuli later. Together, the experiments suggest attentional mechanisms in the pigeon that are not quite involuntary.     

Searching for patterns in random sequences.

In a probability-guessing paradigm, participants predict which of two events will occur on each trial. Participants generally frequency match even though frequency matching is nonoptimal with random sequences. The optimal strategy is to guess the most frequent event, maximizing. We hypothesize that frequency matching results from a search for patterns, even in random sequences. Using both callisotomy patients and patients with frontal brain damage, Wolford, Miller, and Gazzaniaga (2000) found frequency matching in the left hemisphere but maximizing in the right hemisphere. In this paper, we show that a secondary task that competes for left hemisphere resources moves the participants toward maximizing but that a right-hemisphere task preserves frequency matching. We also show that a misunderstanding of randomness contributes to frequency matching.     

Postmodeling Sensitivity Analysis to Detect the Effect of Missing Data Mechanisms

Incomplete or missing data is a common problem in almost all areas of empirical research. It is well known that simple and ad hoc methods such as complete case analysis or mean imputation can lead to biased and/or inefficient estimates. The method of maximum likelihood works well; however, when the missing data mechanism is not one of missing completely at random (MCAR) or missing at random (MAR), it too can result in incorrect inference. Statistical tests for MCAR have been proposed, but these are restricted to a certain class of problems. The idea of sensitivity analysis as a means to detect the missing data mechanism has been proposed in the statistics literature in conjunction with selection models where conjointly the data and missing data mechanism are modeled. Our approach is different here in that we do not model the missing data mechanism but use the data at hand to examine the sensitivity of a given model to the missing data mechanism. Our methodology is meant to raise a flag for researchers when the assumptions of MCAR (or MAR) do not hold. To our knowledge, no specific proposal for sensitivity analysis has been set forth in the area of structural equation models (SEM). This article gives a specific method for performing postmodeling sensitivity analysis using a statistical test and graphs. A simulation study is performed to assess the methodology in the context of structural equation models. This study shows success of the method, especially when the sample size is 300 or more and the percentage of missing data is 20% or more. The method is also used to study a set of real data measuring physical and social self-concepts in 463 Nigerian adolescents using a factor analysis model.     

Planned Missing Data Designs for Research in Cognitive Development

Data collection can be the most time- and cost-intensive part of developmental research. This article describes some long-proposed but little-used research designs that have the potential to maximize data quality (reliability and validity) while minimizing research cost. In planned missing data designs, missing data are used strategically to improve the validity of data collection in one of two ways. Multiform designs allow one to increase the number of measures assessed on each participant without increasing each participant's burden. Two-method measurement designs allow one to reap the benefits of a cost-intensive gold-standard measure, using a larger sample size made possible by a rougher, cheaper measure. We explain each method using examples relevant to cognitive development research. With the use of analysis methods that produce unbiased results, planned missing data designs are an efficient way to manage cost, improve data quality, and reduce participant fatigue and practice effects.     

Using Principal Components as Auxiliary Variables in Missing Data Estimation

To deal with missing data that arise due to participant nonresponse or attrition, methodologists have recommended an “inclusive” strategy where a large set of auxiliary variables are used to inform the missing data process. In practice, the set of possible auxiliary variables is often too large. We propose using principal components analysis (PCA) to reduce the number of possible auxiliary variables to a manageable number. A series of Monte Carlo simulations compared the performance of the inclusive strategy with eight auxiliary variables (inclusive approach) to the PCA strategy using just one principal component derived from the eight original variables (PCA approach). We examined the influence of four independent variables: magnitude of correlations, rate of missing data, missing data mechanism, and sample size on parameter bias, root mean squared error, and confidence interval coverage. Results indicate that the PCA approach results in unbiased parameter estimates and potentially more accuracy than the inclusive approach. We conclude that using the PCA strategy to reduce the number of auxiliary variables is an effective and practical way to reap the benefits of the inclusive strategy in the presence of many possible auxiliary variables.     

Women's career patterns: A study of Swedish women born in the 1950s

Patterns of career development have been found to be an important factor for long‐term career rewards and well‐being. However, existing career models excessively focus on men or elite women and upon paid work, typically without considering other roles. Based on a life course perspective, this study aimed to identify women's career development patterns by examining the dynamic interactions between individuals' involvement in working life and other career‐related domains of life. Career biographies, from the ages of 16 to 43, were recorded through retrospective reports from a representative sample of Swedish women (N = 549) participating in a longitudinal programme on individual development. Seven career‐related activities were coded and combined into career sequences covering the entire period. Data were analysed using optimal matching, and nine distinct career patterns ‐ disparate in terms of the timing, ordering and duration of activities ‐ were identified. There were significant differences between the career patterns in early educational aspirations and early sexual experiences, as well as in life‐role values and socio‐economic status in middle age. With respect to the consequences of career patterns for well‐being, there were significant differences in self‐rated health but not in job satisfaction or life satisfaction. The diversity of patterns is discussed from a perspective that takes account of both life course theory and the choices made by individual women in a society that provides childcare facilities, parental leave and other types of support to working parents.     

Sequences of spaced responses: Behavioral units and the role of contiguity

Sequences of temporally spaced responses were reinforced to investigate the effects of delay of reinforcement on the formation of functional behavioral units. In Experiment 1, rats' two- and three-response demarcated sequences of left and right lever presses were reinforced such that different response distributions would occur depending on whether the sequences themselves or individual responses were functional units. The matching law could thus be obeyed either by individual responses or by sequences, but not by both; intermediate results were possible. Both regular (nonretractable) and retractable levers were used; the retractable levers precluded the occurrence of insufficiently spaced responses. At a minimum interresponse time of 5 s for regular levers and 7 s for retractable ones, matching results were intermediate, with greater evidence of sequence conditionability in the two-response sequences than in the three-response sequences. In Experiment 2, the required minimum interresponse spacing for two-response retractable-lever sequences was varied in an attempt to locate the sequence matching threshold. This attempt was unsuccessful, but the sequences (instead of individual responses) more closely obeyed the matching law. In the shortest spaced condition, conditional probability data on Lag 1 sequence emission order showed marked, highly similar patterning for all rats, indicating sequential control of the sequences. Post hoc definition of the behavioral unit in these studies is ambiguous. Although reinforcement contiguity was important, aspects of the results could support both molar- and molecular-level interpretations.     

Dual imputation model for incomplete longitudinal data

Missing values are a practical issue in the analysis of longitudinal data. Multiple imputation (MI) is a well‐known likelihood‐based method that has optimal properties in terms of efficiency and consistency if the imputation model is correctly specified. Doubly robust (DR) weighing‐based methods protect against misspecification bias if one of the models, but not necessarily both, for the data or the mechanism leading to missing data is correct. We propose a new imputation method that captures the simplicity of MI and protection from the DR method. This method integrates MI and DR to protect against misspecification of the imputation model under a missing at random assumption. Our method avoids analytical complications of missing data particularly in multivariate settings, and is easy to implement in standard statistical packages. Moreover, the proposed method works very well with an intermittent pattern of missingness when other DR methods can not be used. Simulation experiments show that the proposed approach achieves improved performance when one of the models is correct. The method is applied to data from the fireworks disaster study, a randomized clinical trial comparing therapies in disaster‐exposed children. We conclude that the new method increases the robustness of imputations.     

Task analysis in curriculum design: a hierarchically sequenced introductory mathematics curriculum

A method of systematic task analysis is applied to the problem of designing a sequence of learning objectives that will provide an optimal match for the child's natural sequence of acquisition of mathematical skills and concepts. The authors begin by proposing an operational definition of the number concept in the form of a set of behaviors which, taken together, permit the inference that the child has an abstract concept of "number". These are the "objectives" of the curriculum. Each behavior in the defining set is then subjected to an analysis that identifies hypothesized components of skilled performance and prerequisites for learning these components. On the basis of these analyses, specific sequences of learning objectives are proposed. The proposed sequences are hypothesized to be those that will best facilitate learning, by maximizing transfer from earlier to later objectives. Relevant literature on early learning and cognitive development is considered in conjunction with the analyses and the resulting sequences. The paper concludes with a discussion of the ways in which the curriculum can be implemented and studied in schools. Examples of data on individual children are presented, and the use of such data for improving the curriculum itself, as well as for examining the effects of other treatment variables, is considered.     

A New Procedure to Test Mediation With Missing Data Through Nonparametric Bootstrapping and Multiple Imputation

This article proposes a new procedure to test mediation with the presence of missing data by combining nonparametric bootstrapping with multiple imputation (MI). This procedure performs MI first and then bootstrapping for each imputed data set. The proposed procedure is more computationally efficient than the procedure that performs bootstrapping first and then MI for each bootstrap sample. The validity of the procedure is evaluated using a simulation study under different sample size, missing data mechanism, missing data proportion, and shape of distribution conditions. The result suggests that the proposed procedure performs comparably to the procedure that combines bootstrapping with full information maximum likelihood under most conditions. However, caution needs to be taken when using this procedure to handle missing not-at-random or nonnormal data.     

Optimal choice

We present a classification and theoretical analysis of discrete-trial and free-operant choice procedures in which reinforcement is assigned to one alternative only, or independently to both, is either always available or conditionally available, and is either "held" or not from trial to trial. Momentary-maximizing and (globally) optimal choice sequences are defined in terms of initializing and marker events. Free-operant choice is analyzed in terms of a clock space whose axes are the times since the last A and B choices. The analysis shows that most molar matching data are derivable from momentary maximizing, and that the momentary-maximizing hypothesis has not been adequately tested in either discrete-trial or free-operant situations.