Efficiency of multiple-choice tests as a function of spread of item difficulties

The validity of a univocal multiple-choice test is determined for varying distributions of item difficulty and varying degrees of item precision. Validity is a function of _d ² + _v ² , where _d measures item unreliability and _v measures the spread of item difficulties. When this variance is very small, validity is high for one optimum cutting score, but the test gives relatively little valid information for other cutting scores. As this variance increases, eta increases up to a certain point, and then begins to decrease. Screening validity at the optimum cutting score declines as this variance increases, but the test becomes much more flexible, maintaining the same validity for a wide range of cutting scores. For items of the type ordinarily used in psychological tests, the test with uniform item difficulty gives greater over-all validity, and superior validity for most cutting scores, compared to a test with a range of item difficulties. When a multiple-choice test is intended to reject the poorestF per cent of the men tested, items should on the average be located at or above the threshold for men whose true ability is at theFth percentile.This research was performed under contract Nop 536 with the Bureau of Naval Personnel, and received additional support from the Bureau of Research and Service, College of Education, University of Illinois.     

O regułach rachunku zdań

Streszczenie Zmienn istotn wyraenia rachunku zda nazywam zmienn, której warto przy pewnym ukadzie wartoci innych zmiennych wpywa na warto caego wyraenia.     

Теория нормативных предлозений

, : ... () ..., ... () ..., ... () ... , ... () ..., — . - .     

О правилах исчисления предложе ий

, .     

Prototetyka St. Lesniewskiego

Streszczenie Prototetyka St. Len'ewskiego jest uogólnieniem dwuwartociowego rachunku zda. Wystpuj w niej obok terminów tego rachunku funktory (zarówno stae jak i zmienne) tych wszystkich kategorii semantycznych, jakie mog by zdefiniowane, gdy punktem wyjcia jest kategor a zda.W pracy zreferowane s trzy systemy prototetyki. Terminem pierwotnym dwu z nich jest implikacja, terminem pierwotnym systemu trzeciego jest równowano. Systemy o terminie pierwotnym implikacji róni s. reguami wnioskowania. W jednym z nich obowizuje regua weryfikacji, w drugim regua ekstensjonalnoci. Pierwsza z tych regu jest uogólnieniem nastpujcej reguy rachunku zda, wzbogaconego o terminy 0 i 1:Wyraenie jest tez systemu, gdy tezami systemu s oba wyraenia powstae z przez podstawienie za pewn jego zmienn symboli 0 i 1.W myl reguy ekstensjonalnoci tezami systemu s prawa ekstensjonalnoci, sformuowane dla funktorów dowolnych kategorii semantycznych wac-wych prototetyce.W pracy podaj dowody równowanoci trzech systemów prototetyki i dowód ich zupenoci.Znaczna cz wyników Leniewskiego, zreferowanych w pracy, nie bya dotd opublikowana. Opracowujc te wyniki opieraem si na notatkach z wykadów Leniewskiego, spisanych przez jego uczniów. Oryginalne notatki Leniewskiego ulegy zniszczeniu w czasie powstania warszawskiego.     

Remarks on the method of paired comparisons: III. A test of significance for paired comparisons when equal standard deviations and equal correlations are assumed

A test of goodness of fit is developed for Thurstone's method of paired comparisons, Case V. The test involves the computation of , wheren is the number of observations per pair, and and are the angles obtained by applying the inverse sine transformation to the fitted and the observed proportions respectively. The number of degrees of freedom is (k–1) (k–2)/2.This research was performed in the Laboratory of Social Relations under a grant made available to Harvard University by the RAND Corporation under the Department of the Air Force, Project RAND.     

Estimation of parameters in a truncated trivariate normal distribution

This paper gives maximum-likelihood estimators for certain parameters in a truncated trivariate normal distribution when the values of the other parameters are known. The estimators are functions of a random sample. Approximate variances and covariances of the estimators, when the sample size is large, are also given. The type of truncation considered is merely restriction of the range of one of the variates, whose true mean and variance are assumed to be known. Two cases of such restriction are treated: (a) ( x < + ); (b) (- <x '), where and ' are arbitrary cutoff points which are assumed to be known. A precise statement of the estimation problem is given in Section 1. Section 2 contains preliminary calculations. The estimators appear in Section 3. The asymptotic variances and covariances of the estimators are given in Section 4. The estimators and their asymptotic variances and covariances can be easily specialized to be suitable for the case of a certain truncated bivariate normal distribution (Sections 3 and 4).     

An efficient punched-card method of computing epsilonX,epsilonX2, epsilonXY,and higher moments

A method of computing X, X ², XY and higher moments on IBM equipment is described. The basic method is that of successively summary punching, collating a variable number of blank cards behind these summary cards, gang-punching the data from the summary cards into the blank cards, and totalling the entries on these summary cards. The method appears to have several advantages over those previously described, especially if coded data are used.     

Variation in test validity with variation in the distribution of item difficulties,number of items,and degree of their intercorrelation

The relation between item difficulty distributions and the validity and reliability of tests is computed through use of normal correlation surfaces for varying numbers of items and varying degrees of item intercorrelations. Optimal or near optimal item difficulty distributions are thus identified for various possible item difficulty distributions. The results indicate that, if a test is of conventional length, is homogeneous as to content, and has a symmetrical distribution of item difficulties, correlation with a normally distributed perfect measure of the attribute common to the items does not vary appreciably with variation in the item difficulty distribution. Greater variation was evident in correlation with a second duplicate test (reliability). The general implications of these findings and their particular significance for evaluating techniques aimed at increasing reliability are considered.     

A note on correcting for chance success in objective tests

The conventional scoring formula to correct for guessing is derived and is compared with a regression method for scoring which has been recently proposed by Hamilton. It is shown that the usual formula,S=R–W/(n–1), yields a close approximation (correct within one point) to the maximum-likelihood estimate of an individual's true score on the test, if we assume that the individual knows or does not know the answer to each item, that guessing at unknown items is random, and that success at guessing is governed by the binomial law. It is also shown that the usual scoring formula yields an unbiased estimate of the individual's true score, when the true score is defined as the mean score over an indefinitely large number of independent attempts at the test or at equivalent (parallel) tests.     

A note on the estimation of test reliability by the Kuder-Richardson formula (20)

The Kuder-Richardson formula (20) is rewritten to be identical with the simplest formula, (21), except for the addition of a term involving the standard deviation, _p , of the itemp's. If _p can be estimated, a rapid and superior estimate of test reliability is possible in contrast to the simpler formula (21) used when the number of items and mean and standard deviation of test scores are known.Kuder, G. F. and Richardson, M. W. The theory of the estimation of test reliability.Psychometrika. 1937, 2, 151–160.     

The unit hierarchy and its properties

A correlation matrix may be expanded as the weighted sum of a series of unit hierarchies. The properties of the unit hierarchy are not only of theoretical interest for themselves, but lead to simpler modes of practical calculation. The analysis is analogous to a spectral set of projective operations in quantum-theory: and the analogy itself suggests many further problems and solutions.     

Über die Messung der psychischen Ermüdung mit Hilfe des Flimmertests

Ohne ZusammenfassungErklärung der Abkürzungen Add.-Lstg. Additionsleistung - AZ Arbeitszeit - B/sec Anzahl der Blitze je Sekundes - KP Kurzpause - Lstg. Leistung - M Mittelwert (arithm. Mittel) - TZ Teilzeit - Vfr. Verschmelzungsfrequenz = Flimmergrenze - VL Versuchsleiter - Vp. Versuchsperson - Vpn. Versuchspersonen - Mittlere Abweichung vom Mittelwert - M Mittlerer Fehler des Mittelwertes     

A technique for facilitating the rotation of factor axes,based on an equivalence between persons and tests

A technique is outlined which may facilitate the rotation of factor axes to a meaningful position. It is based on certain relationships between the results of test and person factor analysis, and consists essentially of supplementing the test factor space with tests which are thetest-equivalents of persons or groups of persons. These persons may be, for instance, well-known types in the domain being investigated, or even freaks. The ways in which these persons may be selected and used to determine the final rotated position of the factor axes is discussed.     

A technique for criterion-keying and selecting test items

For multiple-choice tests where noa priori key exists, the initial selection of a key for maximum validity may be made on the basis of the number of persons choosing each alternative and their mean criterion score. The keying formula is derived. Once the initial keying has been done, further precision in keying and item selection may use, in addition, the mean total test score for persons choosing each alternative. Item-selection formulas suggested by Horst and by Gulliksen for maximizing test validity are both in the form of a ratio, an item-validity index divided by an item-reliability index. The formula derived here is shown to be equivalent to the numerators of these formulas. The expression in the denominators uses the total test score. Although a radical appears in the denominator of Horst's formula and not in the denominator of Gulliksen's formula, both of them select the same items in practice.The author gratefully acknowledges the suggestions and criticisms of Dr. Harold Gulliksen, Research Adviser at the Educational Testing Service.     

Synthesis of variance

The distribution of a linear combination of two statistics distributed as is Chi-square is studied. The degree of approximation involved in assuming a Chi-square distribution is illustrated for several representative cases. It is concluded that the approximation is sufficiently accurate to use in many practical applications. Illustrations are given of its use in extending the Chi-square, the Student t and the Fisher z tests to a wider range of problems.     

The relation of the reliability of multiple-choice tests to the distribution of item difficulties

Under certain assumptions an expression, in terms of item difficulties and intercorrelations, is derived for the curvilinear correlation of test score on the ability underlying the test, this ability being defined as the common factor of the item tetrachoric intercorrelations corrected for guessing. It is shown that this curvilinear correlation is equal to the square root of the test reliability. Numerical values for these curvilinear correlations are presented for a number of hypothetical tests, defined in terms of their item parameters. These numerical results indicate that the reliability and the curvilinear correlation will be maximized by (1) minimizing the variability of item difficulty and (2) making the level of item difficulty somewhat easier than the halfway point between a chance percentage of correct answers and 100 per cent correct answers.     

Item synonymization: A method for determining the total meaning of pencil-paper reactions

Items have been studied heretofore for their value as elements of particular tests to the neglect of more fundamental research into the multiple potentiality of items. This article proposes a method of grouping items into synonymies comprising all of the items which correlate with a given key item. These synonymies can be used for interpretation of the total meaning of the key item: (1) by inspection of the constituent items and (2) by correlational study of obtained single scores of individual persons. The method is illustrated by four items with inter- and intra-correlations, and characteristics of an ideal background reservoir of items are pointed out.