Statistical inferences about true scores

Formulas are derived for unbiased sample estimators of any raw or central moment of the frequency distribution of true scores. A general method is developed for obtaining from each examinee's observed score a least squares estimate of his true score.This research was carried out under contract Nonr-2214(00) with the Office of Naval Research, Department of the Navy.     

Ordinal consistency and ordinal true scores

This paper argues that test data are ordinal, that latent trait scores are only determined ordinally, and that test data are used largely for ordinal purposes. Therefore it is desirable to develop a test theory based only on ordinal assumptions. A set of ordinal assumptions is presented, including an ordinal version of local independence. From these assumptions it is first shown that the gamma-correlation between two tests is the product of their gamma-correlations with the true latent order. The theory is generalized to allow for heterogeneous tests by defining a weighted average local independence. The tau-correlations between total score and the latent order can be found in both homogeneous and heterogeneous cases, and a system of differential item weighting to maximize the tau-correlation between weighted items and the latent order is provided. Thus a purely ordinal test theory seems possible.Part of this work was done while the author was a Visiting Fellow at Macquarrie University. The paper has benefitted from discussions with Professors Thomas J. Reynolds and Roderick P. McDonald and from the comments of several anonymous reviewers.     

Prediction of true test scores from observed item scores and ancillary data

In many educational tests which involve constructed responses, a traditional test score is obtained by adding together item scores obtained through holistic scoring by trained human raters. For example, this practice was used until 2008 in the case of GRE^®General Analytical Writing and until 2009 in the case of TOEFL^® iBT Writing. With use of natural language processing, it is possible to obtain additional information concerning item responses from computer programs such as e‐rater^®. In addition, available information relevant to examinee performance may include scores on related tests. We suggest application of standard results from classical test theory to the available data to obtain best linear predictors of true traditional test scores. In performing such analysis, we require estimation of variances and covariances of measurement errors, a task which can be quite difficult in the case of tests with limited numbers of items and with multiple measurements per item. As a consequence, a new estimation method is suggested based on samples of examinees who have taken an assessment more than once. Such samples are typically not random samples of the general population of examinees, so that we apply statistical adjustment methods to obtain the needed estimated variances and covariances of measurement errors. To examine practical implications of the suggested methods of analysis, applications are made to GRE General Analytical Writing and TOEFL iBT Writing. Results obtained indicate that substantial improvements are possible both in terms of reliability of scoring and in terms of assessment reliability.     

Shrinkage estimation of linear combinations of true scores

This paper is concerned with combining observed scores from sections of tests. It is demonstrated that in the presence of population information a linear combination of true scores can be estimated more efficiently than by the same linear combination of the observed scores. Three criteria for optimality are discussed, but they yield the same solution which can be described and motivated as a multivariate shrinkage estimator.Input from Eric Bradlow, Charles Lewis, and Linda Zeger is acknowledged. Research for this paper was funded by the Program Research Council (ETS). Suggestions of the Editor and of anonymous referees were instrumental in several improvements of the paper.     

Bayesian inference and the classical test theory model: Reliability and true scores

A general one-way analysis of variance components with unequal replication numbers is used to provide unbiased estimates of the true and error score variance of classical test theory. The inadequacy of the ANOVA theory is noted and the foundations for a Bayesian approach are detailed. The choice of prior distribution is discussed and a justification for the Tiao-Tan prior is found in the particular context of the “n-split” technique. The posterior distributions of reliability, error score variance, observed score variance and true score variance are presented with some extensions of the original work of Tiao and Tan. Special attention is given to simple approximations that are available in important cases and also to the problems that arise when the ANOVA estimate of true score variance is negative. Bayesian methods derived by Box and Tiao and by Lindley are studied numerically in relation to the problem of estimating true score. Each is found to be useful and the advantages and disadvantages of each are discussed and related to the classical test-theoretic methods. Finally, some general relationships between Bayesian inference and classical test theory are discussed. Supported in part by the National Institute of Child Health and Human Development under Research Grant 1 PO1 HDO1762. Reproduction, translation, use or disposal by or for the United States Government is permitted.     

Testing a linear relation between true scores of two measures

We concern ourselves with the hypothesis that two variables have a perfect disattenuated correlation, hence measure the same trait except for errors of measurement. This hypothesis is equivalent to saying, within the adopted model, that true scores of two psychological tests satisfy a perfect linear relation. Statistical tests of this hypothesis are derived when the relation is specified with the exception of the additive constant. Two approaches are presented and various assumptions concerning the error parameters are used. Then the results are reinterpreted in terms of the possible existence of an unspecified perfect linear relation between true scores of two psychological tests. A numerical example is appended by way of illustration. Research reported in this paper has been supported by grant GB-18230 from National Science Foundation.     

An interval estimate for making statistical inferences about true scores

A numerical procedure is outlined for obtaining an interval estimate of the regression of true score on observed score. Only the frequency distribution of observed scores is needed for this. The procedure assumes that the conditional distribution of observed scores for fixed true score is binomial. The procedure is applied to several sets of test data.This research was sponsored in part by the Personnel and Training Research Programs, Psychological Sciences Division, Office of Naval Research, under Contract No. N00014-69-C-0017, Contract Authority Identification Number, NR No. 150-303, and Educational Testing Service. Reproduction in whole or in part is permitted for any purpose of the United States Government.     

Platonic Personal Immortality

I argue that Plato distinguishes between personal immortality and immortality of the soul. I begin by criticizing the consensus view that Plato identifies the person and the soul. I then turn to the issue of immortality. By considering passages from Symposium and Timaeus, I make the case that Plato thinks that while the soul is immortal by nature, if a person is going to be immortal, they must become so. Finally, I argue that Plato has a psychological continuity approach to personal identity. Thus, for Plato, a person becomes immortal by avoiding reincarnation and securing for themselves psychological continuity forever.     

《巴门尼德》与柏拉图式辩证术

《巴门尼德》的主题是形相论(理念论)与辩证术,柏拉图式辩证术问题与他应对智术师运动及其背后的前苏格拉底“存在之战”直接相关。本文考察柏拉图改造修辞术为辩证术的过程和哲学意义,以此定位形相与辩证术的位置。《巴门尼德》的形相论批判和辩证术展示指向共同的目的,即呈现逻辑推理与范畴描述对于人类认识存在的作用及限度,并由此凸显辩证术对于哲学探究新路彳圣的意义,这篇对话传达了柏拉图对人类认识与存在之问的深刻反思。