Response Time Based Nonparametric Kullback-Leibler Divergence Measure for Detecting Aberrant Test-Taking Behavior |
| |
Authors: | Kaiwen Man Jeffery R. Harring Yunbo Ouyang Sarah L. Thomas |
| |
Affiliation: | 1. Educational Measurement, Statistics and Evaluation, University of Maryland, USA;2. Department of Human Development and Quantitative Methodology, University of Maryland, USA;3. Department of Statistics, University of Illinois at Urbana and Champaign, USA;4. Caveon Test Security, USA |
| |
Abstract: | Many important high-stakes decisions—college admission, academic performance evaluation, and even job promotion—depend on accurate and reliable scores from valid large-scale assessments. However, examinees sometimes cheat by copying answers from other test-takers or practicing with test items ahead of time, which can undermine the effectiveness of such assessments in yielding accurate, precise information of examinees' performances. This study focuses on the utility of a new nonparametric person-fit index using examinees' response times to detect two types of cheating behaviors. The feasibility of this method was investigated vis-à-vis a Monte Carlo simulation as well as through analyzing data from a large-scale assessment. Findings indicate that the proposed index was quite successful in detecting pre-knowledge cheating and extreme one-item cheating. |
| |
Keywords: | aberrant examinee behavior response times computer-based testing person-fit index copy cheating pre-knowledge cheating |
|
|