Predicted differences and differences between predictions

WhenK tests are given toN individuals, and for each individual there are two criterion measures, then (1) the multiple regression weight to be applied to the standard score for each test to predict the criterion-difference score equals the difference of the weights for predicting each criterion separately; (2) the difference between the predicted scores equals the predicted difference (each test being assigned the appropriate multiple regression weight); (3) the square of the multiple correlation between predicted and actual criterion-difference scores equals the sum of squares of the multiple correlations of the battery with each criterion less the product of these correlations and the correlation between predicted scores all divided by twice the quantity one minus the criterion intercorrelation; and (4) the variance of errors of estimating the criterion-difference score equals the sum of the variances of errors of estimating each criterion score minus twice the criterion intercorrelation, plus twice the correlation between predicted scores multiplied by the product of the square root of one minus the variance of errors of estimating one criterion and the corresponding square root for the second criterion.The author wishes to express his appreciation for the suggestions and guidance given by Dr. Harold Gulliksen in the preparation of this article. He also wishes to acknowledge the helpful comments of Dr. Paul Horst and Dr. Ledyard Tucker on certain phases of the development.     

Multiple rectilinear prediction and the resolution into components

It is assumed that a battery ofn tests has been resolved into components in a common factor space ofr dimensions and a unique factor space of at mostn dimensions, wherer is much less thann. Simplified formulas for ordinary multiple and partial correlation of tests are derived directly in terms of the components. The best (in the sense of least squares) linear regression equations for predicting factor scores from test scores are derived also in terms of the components. Spearman's single factor prediction formulas emerge as special cases. The last part of the paper shows how the communality is an upper bound for multiple correlation. A necessary and sufficient condition is established for the square of the multiple correlation coefficient of testj on the remainingn—1 tests to approach the communality of testj as a limit asn increases indefinitely whiler remains constant. Limits are established for partial correlation and regression coefficients and for the prediction of factor scores.I am indebted to Professor Dunham Jackson for helpful criticism of most of this paper.     

Bias and error in multiple-choice tests

A formula for estimating real scores on a multiple-choice test from a knowledge of raw scores is derived. This formula does not involve the assumption of a binomial distribution of real scores as does the Calandra formula. Other important formulas derived show: the variance of real scores in terms of the variance of raw scores and the correlation between real scores and raw scores. If the variance of real scores (or of raw scores also) is binomial, the regression of real scores on raw scores is linear; but, otherwise the regression is curvilinear. Yet the linear estimating formula is a close approximation to the curvilinear relationship. Factors affecting the regression of real scores on raw scores and the correlation coefficient are: (1) the number of choices per question; (2) the number of questions answered; (3) the ratio of the average group raw score to the variance of raw scores.     

A FORMULA FOR THE USE OF SUPPRESSION VARIABLES IN CORRECTING INDIVIDUAL RATINGS

D unér , A. & M agnusson , D. A formula for the use of suppression variables in correcting individual ratings. Scand. J. Psychol , 1962, 3 , 226–232.—Ratings of aptitude, ability or performance used for selection and/or prediction or as criteria for the estimation of the validity of other methods are often influenced by irrelevant factors. Two correction formulas based on a suppression variable have been derived. The relation between the correlation formula derived and the partial correlation formula is discussed, as well as the effect of the application of the formula on the estimation of differential validity.     

Some properties of the communality in multiple factor theory

Several theorems concerning properties of the communaltiy of a test in the Thurstone multiple factor theory are established. The following theorems are applicable to a battery ofn tests which are describable in terms ofr common factors, with orthogonal reference vectors.1. The communality of a testj is equal to the square of the multiple correlation of testj with ther reference vectors.2. The communality of a testj is equal to the square of the multiple correlation of testj with ther reference vectors and then—1 remaining tests. Corollary: The square of the multiple correlation of a testj with then—1 remaining tests is equal to or less than the communality of testj. It cannot exceed the communality.3. The square of the multiple correlation of a testj with then—1 remaining tests equals the communality of testj if the group of tests containsr statistically independent ests teach with a communality of unity.4. With correlation coefficients corrected for attenuation, when the number of tests increases indefinitely while the rank of the correlational matrix remains unchanged, the communality of a testj equals the square of the multiple correlation of testj with then—1 remaining tests.5. With raw correlation coefficients, it is shown in a special case that the square of the multiple correlation of a testj with then—1 remaining tests approaches the communality of testj as a limit when the number of tests increases indefinitely while the rank of correlational matrix remains the same. This has not yet been proved for the general case.The author wishes to express his appreciation of the encouragement and assistance given him by Dr. L. L. Thurstone.     

双因子模型MCAT中多级评分项目选题策略的比较

双因子模型假设测验考察一个一般因子和多个组因子,符合很多教育和心理测验的因素结构。“维度缩减”方法将参数估计中多维积分计算化简为多个迭代二维积分,是双因子模型的重要特征。本文针对考察多级评分项目的计算机化自适应测验,首先推导双因子等级反应模型下Fisher信息量的计算,然后推导“维度缩减”方法在项目选择方法中的应用,最后在低、中、高双因子模式题库中比较D-优化方法、后验加权Fisher信息D优化方法(PDO)、后验加权Kullback-Leibler方法(PKL)、连续熵(CEM)和互信息(MI)方法在能力估计的相关、均方根误差、绝对值偏差和欧氏距离的表现。模拟研究表明：(1)双因子模式越强,即一般因子和组因子在项目上的区分度的差异越小,一般因子估计精度降低,组因子估计精度增加,整体能力的估计精度提高;(2)相同实验条件下,连续熵方法的测量精度最高,PKL方法的能力估计精度最低,其它方法的测量精度没有显著差异。     

The simultaneous prediction of any number of criteria by the use of a unique set of weights

When several criteria are available, it is ordinarily necessary (1) to select one of them asthe criterion, (2) to use several and thus arrive at several different sets of weights, or (3) to combine them into a single measure. A formula is derived for the determination of a unique set of weights. The use of these weights will produce the highest possible average coefficient of correlation between the various criteria and two (or more) weighted independent variables. If desired, the criteria may be assigned any predetermined weights. The weights then derived for the independent variables are such that the weighted average of the correlation coefficients between the various criteria and the independent variable composite will be a maximum. In the use of these formulas, no assumptions are necessary regarding the interrelationships existing among the criteria and it is not necessary to compute the intercorrelations among the criteria. A numerical example is included.     

Using a Response Time–Based Expected A Posteriori Estimator to Control for Differential Speededness in Computerized Adaptive Test

This study investigates using response times (RTs) with item responses in a computerized adaptive test (CAT) setting to enhance item selection and ability estimation and control for differential speededness. Using van der Linden’s hierarchical framework, an extended procedure for joint estimation of ability and speed parameters for use in CAT is developed following van der Linden; this is called the joint expected a posteriori estimator (J-EAP). It is shown that the J-EAP estimate of ability and speededness outperforms the standard maximum likelihood estimator (MLE) of ability and speededness in terms of correlation, root mean square error, and bias. It is further shown that under the maximum information per time unit item selection method (MICT)—a method which uses estimates for ability and speededness directly—using the J-EAP further reduces average examinee time spent and variability in test times between examinees above the resulting gains of this selection algorithm with the MLE while maintaining estimation efficiency. Simulated test results are further corroborated with test parameters derived from a real data example.     

Geometrical representation of two methods of linear least squares multiple correlation

Geometrical properties and relationships of the Doolittle and square root methods of multiple correlation, as represented in the variable subspace of an orthogonal person space, are shown. The method of representation is also useful for depicting zero-order and partial correlations, as well as for the more general problem of the combination of variables.     

Predicting Clinical Judgment for a Primary Grade Appreciation Battery

A six-card Primary Grade Apperception Battery (three cards from the CAT and three from the SAM) was administered to one group of first grade children on a 4 month test-retest schedule and to a new group of first graders 1 year later. Two clinical psychologists read the protocols and made clinical judgments regarding home and school adjustment on a four-point scale. An initial regression formula to predict clinical judgment from objective apperception score data was derived. The formula was validated on the retest data and the new sample. The results indicate the derived apperception score reliably predicted clinical judgment. In general, the results also suggest the six card apperception battery maybe a useful screening test for school and home adjustment for first and second grade children.     

The relation of the reliability of multiple-choice tests to the distribution of item difficulties

Under certain assumptions an expression, in terms of item difficulties and intercorrelations, is derived for the curvilinear correlation of test score on the ability underlying the test, this ability being defined as the common factor of the item tetrachoric intercorrelations corrected for guessing. It is shown that this curvilinear correlation is equal to the square root of the test reliability. Numerical values for these curvilinear correlations are presented for a number of hypothetical tests, defined in terms of their item parameters. These numerical results indicate that the reliability and the curvilinear correlation will be maximized by (1) minimizing the variability of item difficulty and (2) making the level of item difficulty somewhat easier than the halfway point between a chance percentage of correct answers and 100 per cent correct answers.     

Simplified formulas for item selection and construction

The formula for the Pearson correlation coefficient of a dichotomous variable with a multiple-categoried variable is simplified for computational purposes by effecting in the multiple-categoried variable two types of arbitrary distributions: (1) rectangular and (2) proportional to binomial expansion coefficients. The formulas which result are convenient for the selection of test items and are applicable to the objective estimation of the comparative merits of the alternatives in multiple-choice test items. It is shown that the authoritative answer should have a high positive criterion coefficient, while the omissions and several wrong-answer alternatives should each have low (algebraic) negative criterion coefficients.     

The validity of a persistence test

In order to raise the predictive efficiency of its college entrance test battery, the Educational Testing Service is working on the development of non-academic measures to supplement the standard aptitude and achievement examinations. A test of difficult number series problems was set up to measure persistence by tempting the students to give up early; the students were informed that some of the problems had no solution, and that full credit would be received by so marking them. This test was tried out and found to have some correlation with grades, while having no correlation with the other tests. Adding this test to the battery showed an appreciable rise in the battery's multiple correlation with grades.     

A technique for criterion-keying and selecting test items

For multiple-choice tests where noa priori key exists, the initial selection of a key for maximum validity may be made on the basis of the number of persons choosing each alternative and their mean criterion score. The keying formula is derived. Once the initial keying has been done, further precision in keying and item selection may use, in addition, the mean total test score for persons choosing each alternative. Item-selection formulas suggested by Horst and by Gulliksen for maximizing test validity are both in the form of a ratio, an item-validity index divided by an item-reliability index. The formula derived here is shown to be equivalent to the numerators of these formulas. The expression in the denominators uses the total test score. Although a radical appears in the denominator of Horst's formula and not in the denominator of Gulliksen's formula, both of them select the same items in practice.The author gratefully acknowledges the suggestions and criticisms of Dr. Harold Gulliksen, Research Adviser at the Educational Testing Service.     

A generalized expression for the reliability of measures

In certain situations it is important to obtain as many measures as possible, all presumably measuring the same function, for each of a group of persons. In general the number and source of the measures may vary from one member of the group to another. We take the mean of the measures for each person as the best estimate of the function for that person. The conventional formulas can not be used to determine the reliability of a set of means so obtained. A formula is developed which provides a unique estimate of the reliability of such a set of means. The formula is more general than some of the well-known reliability formulas, so that these formulas are shown to be special cases of the more general formula.     

Determining a simple structure when loadings for certain tests are known

A rigorous and an approximate solution are found for the problem: Given a primary trait matrix forn tests andr ₁ traits, and a matrix for the samen tests andr ₂ reference axes, to discover the transformation which will transform the second matrix into the first, or primary trait matrix. Formulas for determining the limits of the effect of using the approximate solution are presented. The method is applied to a set of twenty hypothetical tests, defined by their loadings on four orthogonal primary traits. After factoring the inter-correlations of these variables by Thurstone's centroid method, approximating the diagonals, the original hypothetical matrix is reproduced with a root mean square discrepancy of .014 by assuming as known the primary trait loadings of only the first eight tests. The method is applied to the results of factoring two batteries of 14 tests, having 8 tests in common, to give the factor loadings of the two batteries on the same reference axes. The method provides a means of comparing directly and quantitatively the results of two different factor studies, provided they have tests in common, and of testing the stability of simple structure under changes in the battery. The relations of the method here developed to certain problems in multiple correlation are shown.     

Standard errors of fit indices using residuals in structural equation modeling

The asymptotic standard errors of the correlation residuals and Bentler's standardized residuals in covariance structures are derived based on the asymptotic covariance matrix of raw covariance residuals. Using these results, approximations of the asymptotic standard errors of the root mean square residuals for unstandardized or standardized residuals are derived by the delta method. Further, in mean structures, approximations of the asymptotic standard errors of residuals, standardized residuals and their summary statistics are derived in a similar manner. Simulations are carried out, which show that the asymptotic standard errors of the various types of residuals and the root mean square residuals in covariance, correlation and mean structures are close to actual ones.The author is indebted to the reviewers for their comments and suggestions which have led to an improvement of this work.     

Maximizing predictive efficiency for a fixed total testing time

For any fixed total time of testing it is possible, through proper item-and-time allotment, to combine tests into a battery so that the multiple correlation with a pre-assigned criterion will be maximized. By holding constant the ratio of the length in number of items to the time length for each test, a set of general equations has been derived which will yield this maximum value of the multipleR and will enable one to determine, in any given case, the optimal fraction of total testing time that should be devoted to each type of test under consideration. The set of general equations is applied to a two-test-battery problem to obtain the optimal length of each type of test for one hour total testing time. If two other tests had been selected for the two-test sample problem, different subdivisions of the total time would generally occur. The manner in which the results would change when using other tests with different initial reliability, validity, and intercorrelation values is briefly presented. Some general implications of this method of battery development are also discussed.The writer is indebted to Max Woodbury for his assistance and especially to Dr. N. J. F. Van Steenberg and Dr. Anna S. Henriques, who provided valuable guidance and aid in the development of the solution to this problem. This paper is a revision of a thesis submitted in 1939 at the University of Utah in partial fulfillment of the requirements for the master's degree.     

Correction of item-total correlations in item analysis

The biserial correlation between an item and the total test of which the item is a part tends to be misleadingly high when used in item analysis, since the item is included in the total test. Two formulas with correction for this overlap are derived and compared with Zubin's and Guilford's formulas. One of the new coefficients is invariant to test length.     

Inverse Images of Box Formulas in Modal Logic

We investigate, for several modal logics but concentrating on KT, KD45, S4 and S5, the set of formulas B for which ${\square B}$ is provably equivalent to ${\square A}$ for a selected formula A (such as p, a sentence letter). In the exceptional case in which a modal logic is closed under the (‘cancellation’) rule taking us from ${\square C \leftrightarrow \square D}$ to ${C \leftrightarrow D}$ , there is only one formula B, to within equivalence, in this inverse image, as we shall call it, of ${\square A}$ (relative to the logic concerned); for logics for which the intended reading of “ ${\square}$ ” is epistemic or doxastic, failure to be closed under this rule indicates that from the proposition expressed by a knowledge- or belief-attribution, the propositional object of the attitude in question cannot be recovered: arguably, a somewhat disconcerting situation. More generally, the inverse image of ${\square A}$ may comprise a range of non-equivalent formulas, all those provably implied by one fixed formula and provably implying another—though we shall see that for several choices of logic and of the formula A, there is not even such an ‘interval characterization’ of the inverse image (of ${\square A}$ ) to be found.