Admissible probability measurement procedures

Admissible probability measurement procedures utilize scoring systems with a very special property that guarantees that any student, at whatever level of knowledge or skill, can maximize his expected score if and only if he honestly reflects his degree-of-belief probabilities. Section 1 introduces the notion of a scoring system with the reproducing property and derives the necessary and sufficient condition for the case of a test item with just two possible answers. A method is given for generating a virtually inexhaustible number of scoring systems, both symmetric and asymmetric, with the reproducing property. A negative result concerning the existence of a certain subclass of reproducing scoring systems for the case of more than two possible answers is obtained. Whereas Section 1 is concerned with those instances in which the possible answers to a query are stated in the test itself, Section 2 is concerned with those instances in which the student himself must provide the possible answer(s). In this case, it is shown that a certain minor modification of a scoring system with the reproducing property yields the desired admissible probability measurement procedure.The research reported in this paper was, in part, performed at the Decision Sciences Laboratory in support of Project 4690, Information Processing in Command and Control and, in part, sponsored by the Air Force Systems Command Electronic Systems Division, Decision Sciences Laboratory, under Contract No. AF 19(628)-4304, with ARCON Inc. This report is identified as ESD-TR-65-567. Further reproduction is authorized to satisfy the needs of the U. S. Government.