Reliability for the law of comparative judgment

A variance-components analysis is presented for paired comparisons in terms of three components:s, the scale value of the stimuli;d, a deviation from the linear model specified by the law of comparative judgment; andb, a binomial error component. Estimates are given for each of the three variances, _s ² , _d ² , and _b ² . Several coefficients, analogous to reliability coefficients, based on these three variances are indicated. The techniques are illustrated in a replicated comparison of handwriting specimens.This research was jointly supported in part by Princeton University, the Office of Naval Research under contract Nonr-1858(15), and the National Science Foundation under grant NSF G-642, and in part by Educational Testing Service. Reproduction in whole or in part is permitted for any purpose of the United States Government.Thanks are due to Ledyard Tucker and Frederic Lord for valuable suggestions on the development presented here.     

A revised law of comparative judgment

In contrast to Thurstone's Law of Comparative Judgment, a model in which a comparison pair and its complement are assumed to give rise to two different distributions of differences is considered. The consequences of this revised model on scaling problems is developed.The research in this article was supported jointly by the Army, Navy, and Air Force under contract with the Massachusetts Institute of Technology.     

The rasch model,the law of comparative judgment and additive conjoint measurement

Relationships between the Rasch model and both the law of comparative judgment and additive conjoint measurement are discussed. The distance between the ability of Persona and the difficult of Itemi is, in the Rasch model, the baseline value corresponding to the probability thata will respond correctly toi, where this probability is interpreted as the area under a logistic curve (which is substantially equivalent to the normal curve) and is thus an application of the law of comparative judgment. Under certain assumptions, the Rasch model is also a special case of additive conjoint measurement and, properly reinterpreted, may be usefully applied in contexts other than individual differences.     

A solution for case iii of the law of comparative judgment

A solution for Case III of the Law of Comparative Judgment, modeled after Thurstone's solution for Case IV but eliminating the restrictive assumption of relatively equal discriminal dispersions, is developed.     

An internal consistency check for scale values determined by the method of successive intervals

The method of successive intervals is a psychological scaling procedure in which stimuli are classified into successive intervals according to the degree of some defined attribute which they are judged to possess. A psychological continuum is defined and the scale values are then taken as the medians of the distributions of judgments on the psychological continuum. It is assumed that the distributions of judgments for each stimulus are normal on the psychological continuum as defined.An internal consistency check indicates that the cumulative distributions of empirical judgments for the various stimuli can be reproduced by means of a limited number of parameters with an average error that compares favorably with that usually reported for paired comparison data. Furthermore, the scale values obtained by successive interval scaling, for the data reported, are shown to be linearly related to those obtained by the method of paired comparisons.This paper was written while the first author was a post-doctoral Research Training Fellow of the Social Science Research Council studying at the University of Chicago. It reports research undertaken in cooperation with the Quartermaster Food and Container Institute for the Armed Forces. The views or conclusions contained in this report are those of the authors. They are not to be construed as necessarily reflecting the views or endorsement of the Department of Defense.     

A least-squares solution for case IV of the law of comparative judgment

Case IV of Thurstone's Law of Comparative Judgment is displayed as a system of homogeneous linear equations for which a least-squares solution is presented, using various conditional equations which fix the origin and the unit of measurement. The computational load is, however, quite heavy.     

The validity of the successive intervals method of psychometric scaling

The degree to which scale values computed by the method of successive intervals diverge from theoretically true values is seen to be due to three types of error: error due to inequalities in variances of the distributions from which the scale values are computed, error due to non-normality of the distributions, and sampling error. The contribution of each type of error to the total error is evaluated; the latter is seen to be surprisingly small under appropriate conditions. Certain aspects of the formal methodology underlying scaling procedures are also briefly considered.This paper reports research undertaken in cooperation with the Quartermaster Food and Container Institute for the Armed Forces, and has been assigned number 475 in the series of papers approved for publication. The views or conclusions contained in this report are those of the authors. They are not to be construed as necessarily reflecting the views or indorsement of the Department of Defense.     

The estimation of the discriminal dispersion in the method of successive intervals

A new algebraic formula is derived for estimation of the discriminal dispersion in the method of successive intervals. The legitimate use of the formula requires that as many normal deviates as possible be present in the matrix. For this reason, it is recommended that deviates corresponding to the interval (0.01, 0.99) of the cumulative proportions be used, instead of those corresponding to (0.05, 0.95), the interval used by Edwards and Thurstone. Computations on data published by Edwards and Thurstone showed that when adjustment was made for variability in dispersions calculated by the formula of this paper, a reduction of fifty per cent in mean absolute discrepancy was produced. Since the formula is easy to use and avoids the disadvantages of its predecessors, it should have fairly wide applicability in psychological research.This research was supported in part by the United States Air Force under Contract No. AF 33(038)-25726 monitored by Air Force Personnel and Training Research Center. Permission is granted for reproduction, translation, publication, use and disposal in whole and in part by or for the United States Government. The writer is grateful to Dr. A. L. Edwards for a critical reading of an earlier version of this paper, and to Dr. L. H. Lanier and Dr. L. M. Stolurow for editorial advice on the present version, which was written at the University of Illinois. The editors ofPsychometrika have informed the writer that H. J. A. Rimoldi and M. Hormaeche (7) have independently derived the same formula for the discriminal dispersion from a different set of postulates using the law of comparative judgment.     

A score method of constructing asymmetric confidence intervals for the mean of a rating scale item

This article presents a generalization of the Score method of constructing confidence intervals for the population proportion (E. B. Wilson, 1927) to the case of the population mean of a rating scale item. A simulation study was conducted to assess the properties of the Score confidence interval in relation to the traditional Wald (A. Wald, 1943) confidence interval under a variety of conditions, including sample size, number of response options, extremeness of the population mean, and kurtosis of the response distribution. The results of the simulation study indicated that the Score interval usually outperformed the Wald interval, suggesting that the Score interval is a viable method of constructing confidence intervals for the population mean of a rating scale item.     

A double law of comparative judgment for the analysis of preferential choice and similarities data

By virtue of certain modifications in the Law of Comparative Judgment, equations are developed which (i) permit the construction of a joint scale of individuals and items, as in the case of attitude measurement, directly from their pair-comparison preferences, and (ii) take into account the variable of laterality which is significant for the construction of group preference scales.This work was supported by grant NSF-G5820 from the National Science Foundation.     

A modification of the method of successive intervals

A modification of the method of successive intervals is presented which yields scale values correlating .995 with those from Thurstone's method described by Saffir. Values yielded by the present method can be obtained in 25 per cent of the time required by the older method and are shown to be, on apriori grounds, more reliable as well.     

An axiomatic formulation and generalization of successive intervals scaling

A formal set of axioms is presented for the method of successive intervals, and directly testable consequences of the scaling assumptions are derived. Then by a systematic modification of basic axioms the scaling model is generalized to non-normal stimulus distributions of both specified and unspecified form.