Classical Psychophysics and the Assessment of Agreement and Accuracy In Judgments of Personality

ABSTRACT The just noticeable difference (jnd) unit of classical psycho-physics is introduced as a new way to describe accuracy and agreement in observer evaluations of personality. A formula for estimating jnd's from typically available summary statistics is derived from Thurstone's law of comparative judgment. A study examining four traits judged in 10 samples of subjects where the design permitted the calculation of jnd's by the method of paired comparisons indicated that the formula predicted the empirically derived jnd's associated with the mean judge ratings with considerable precision. Jnd's of criterion measures were also predicted, and while fit was somewhat less impressive in this case, there was still appreciable convergence between predicted and empirical values. The implications of jnd measures of agreement and accuracy are discussed. These implications include (a) possibilities for increased understanding of bias in observer judgments, (b) a new recognition that equal correlations to external criteria do not necessarily imply equal accuracy, and (c) alternative ways of describing the magnitude of effect in psychological research.