The use of item scores and response times to detect examinees who may have benefited from item preknowledge

According to Wollack and Schoenig (2018, The Sage encyclopedia of educational research, measurement, and evaluation. Thousand Oaks, CA: Sage, 260), benefiting from item preknowledge is one of the three broad types of test fraud that occur in educational assessments. We use tools from constrained statistical inference to suggest a new statistic that is based on item scores and response times and can be used to detect examinees who may have benefited from item preknowledge for the case when the set of compromised items is known. The asymptotic distribution of the new statistic under no preknowledge is proved to be a simple mixture of two χ² distributions. We perform a detailed simulation study to show that the Type I error rate of the new statistic is very close to the nominal level and that the power of the new statistic is satisfactory in comparison to that of the existing statistics for detecting item preknowledge based on both item scores and response times. We also include a real data example to demonstrate the usefulness of the suggested statistic.     

Multilevel and Latent Variable Modeling with Composite Links and Exploded Likelihoods

Composite links and exploded likelihoods are powerful yet simple tools for specifying a wide range of latent variable models. Applications considered include survival or duration models, models for rankings, small area estimation with census information, models for ordinal responses, item response models with guessing, randomized response models, unfolding models, latent class models with random effects, multilevel latent class models, models with log-normal latent variables, and zero-inflated Poisson models with random effects. Some of the ideas are illustrated by estimating an unfolding model for attitudes to female work participation. We wish to thank The Research Council of Norway for a grant supporting our collaboration.     

Doubly Robust-Type Estimation for Covariate Adjustment in Latent Variable Modeling

Due to the difficulty in achieving a random assignment, a quasi-experimental or observational study design is frequently used in the behavioral and social sciences. If a nonrandom assignment depends on the covariates, multiple group structural equation modeling, that includes the regression function of the dependent variables on the covariates that determine the assignment, can provide reasonable estimates under the condition of correct specification of the regression function. However, it is usually difficult to specify the correct regression function because the dimensions of the dependent variables and covariates are typically large. Therefore, the propensity score adjustment methods have been proposed, since they do not require the specification of the regression function and have been applied to several applied studies. However, these methods produce biased estimates if the assignment mechanism is incorrectly specified. In order to make a more robust inference, it would be more useful to develop an estimation method that integrates the regression approach with the propensity score methodology. In this study we propose a doubly robust-type estimation method for marginal multiple group structural equation modeling. This method provides a consistent estimator if either the regression function or the assignment mechanism is correctly specified. A simulation study indicates that the proposed estimation method is more robust than the existing methods. This research was partially supported by the Ministry of Education, Science, Sports and Culture, Grant-in-Aid for Young Scientists (B), 187-30406.     

Independence of sampling of motion parallax and binocular disparity in discrimination of depth1

Abstract: The sampling strategy of the visual system in binocular disparity and motion parallax to discriminate depth was investigated. Human observers were asked to discriminate between the depths of two surfaces defined by both cues. Gaussian noise was added to the depths represented by each cue, and the correlation in noise was manipulated. Human performance was compared with two types of likelihood models. The first was based on independent sampling, in which data from the two cues were gathered from independent sets of points in the display. The second was based on paired sampling, in which data from these cues were gathered from the same set of points. The former model yielded a better fit with human performance. This suggests that the visual system is more likely to adopt independent sampling.     

Automatic Extraction of Property Norm‐Like Data From Large Text Corpora

Traditional methods for deriving property‐based representations of concepts from text have focused on either extracting only a subset of possible relation types, such as hyponymy/hypernymy (e.g., car is‐a vehicle ) or meronymy/metonymy (e.g., car has wheels ), or unspecified relations (e.g., car — petrol ). We propose a system for the challenging task of automatic, large‐scale acquisition of unconstrained, human‐like property norms from large text corpora, and discuss the theoretical implications of such a system. We employ syntactic, semantic, and encyclopedic information to guide our extraction, yielding concept‐relation‐feature triples (e.g., car be fast , car require petrol , car cause pollution ), which approximate property‐based conceptual representations. Our novel method extracts candidate triples from parsed corpora (Wikipedia and the British National Corpus) using syntactically and grammatically motivated rules, then reweights triples with a linear combination of their frequency and four statistical metrics. We assess our system output in three ways: lexical comparison with norms derived from human‐generated property norm data, direct evaluation by four human judges, and a semantic distance comparison with both WordNet similarity data and human‐judged concept similarity ratings. Our system offers a viable and performant method of plausible triple extraction: Our lexical comparison shows comparable performance to the current state‐of‐the‐art, while subsequent evaluations exhibit the human‐like character of our generated properties.     

Rejoinder: Remaining Challenges in Investigating Grade-Retention Effectiveness

This rejoinder, in response to the commentaries of Steiner, Park, and Kim (this issue) and Reshetnyak, Cham, and Hughes (this issue), discusses remaining challenges in grade retention research. First, a same-age comparison assumes that the instruments used in different grades measure ability equally well. We discuss the importance of evaluating the properties of the scaling process to address whether this assumption has been met. Second, we discuss issues in the selection of covariates to be included in the weights. Third, we discuss the unconfoundedness assumption and the problem of remaining imbalance. Finally, we provide an empirical illustration showing that studying grade-retention effectiveness comes with multiple methodological decisions that are rooted in a bias–variance trade-off.     

Using Marginal Structural Modeling for Grade Retention Effects

Vandecandelaere, Vansteelandt, De Fraine, and Van Damme (this issue) described marginal structural modeling (MSM) and used it to estimate the effects of a time-varying intervention, retention (holding back) in school grades, on students' math achievement. This commentary supplements Vandecandelaere et al. (this issue) and discusses several topics in retention studies and MSM. First, we discuss the importance of equating time-varying confounders in retention studies. Second, we discuss same-grade and same-age comparisons in retention studies. Third, we discuss one important section in the authors' overview of MSM: why standard methods (e.g., ANCOVA, propensity score analysis) cannot properly adjust for time-varying confounders. Finally, using the grade retention analyses in Vandecandelaere et al. (this issue) as an example, we provide our insights on four aspects of MSM: (a) covariate selection, (b) estimation of weights, (c) evaluation of balance properties, and (d) missing data handling.     

缺失数据的结构方程建模:全息极大似然估计时辅助变量的作用

本研究通过蒙特卡洛模拟考查了采用全息极大似然估计进行缺失数据建模时辅助变量的作用。具体考查了辅助变量与研究变量的共缺机制、共缺率、相关程度、辅助变量数目与样本量等因素对参数估计结果精确性的影响。结果表明,当辅助与研究变量共缺时:(1)对于完全随机缺失的辅助变量,结果更容易出现偏差;(2)对于MAR-MAR组合机制,纳入单个辅助变量是有益的;对于MAR-MCAR或MAR-MNAR组合机制,纳入多于一个辅助变量的效果更好;(3)纳入与研究变量低相关的辅助变量对结果也是有益的。     

四参数Logistic模型潜在特质参数的Warm加权极大似然估计

本文以四参数Logistic(4-parameter Logistic,4PL)模型为研究对象,根据Warm的加权极大似然估计技巧,提出了4PL模型潜在特质参数的加权极大似然估计方法,并借助模拟研究对加权极大似然估计的性质进行验证。研究结果表明,与通常的极大似然估计和后验期望估计相比,加权极大似然估计的偏差(bias)明显减小,并且具有良好的返真性能。此外,在测试的长度较短和项目的区分度较小的情况下,加权极大似然估计依然保持了良好的统计性质,表现出更加显著的优势。     

Random effects and extended generalized partial credit models

In this paper it is shown that under the random effects generalized partial credit model for the measurement of a single latent variable by a set of polytomously scored items, the joint marginal probability distribution of the item scores has a closed-form expression in terms of item category location parameters, parameters that characterize the distribution of the latent variable in the subpopulation of examinees with a zero score on all items, and item-scaling parameters. Due to this closed-form expression, all parameters of the random effects generalized partial credit model can be estimated using marginal maximum likelihood estimation without assuming a particular distribution of the latent variable in the population of examinees and without using numerical integration. Also due to this closed-form expression, new special cases of the random effects generalized partial credit model can be identified. In addition to these new special cases, a slightly more general model than the random effects generalized partial credit model is presented. This slightly more general model is called the extended generalized partial credit model. Attention is paid to maximum likelihood estimation of the parameters of the extended generalized partial credit model and to assessing the goodness of fit of the model using generalized likelihood ratio tests. Attention is also paid to person parameter estimation under the random effects generalized partial credit model. It is shown that expected a posteriori estimates can be obtained for all possible score patterns. A simulation study is carried out to show the usefulness of the proposed models compared to the standard models that assume normality of the latent variable in the population of examinees. In an empirical example, some of the procedures proposed are demonstrated.