A theoretical distribution for mental test scores

The negative hypergeometric distribution of raw scores on mental tests is derived from certain assumptions relating to test theory. This result is checked empirically in a number of examples. Further derivations lead to the bivariate distribution of parallel tests which is also verified with actual data. The bivariate distribution of raw score and true score is also derived from a further assumption. This distribution is used to set confidence limits for true scores for persons with a given raw score.This work was supported in part by contract Nonr-2752(00) between the Office of Naval Research and Educational Testing Service. Reproduction in whole or in part for any purpose of the United States Government is permitted.     

Some generalizations of a theoretical distribution of mental test scores

The negative hypergeometric distribution may be generalized to provide distributions involving 3, 4, or more parameters. It is shown that, in the case of the binomial error model, such distributions correspond to quadratic, cubic, etc., regressions of true scores on raw scores. Explicit formulas are given for the bivariate moments required to fit these regressions and so estimate the parameters of the generalized hypergeometric distribution. Equations are also developed for fitting the 3- and 4-parameter distributions directly, i.e., without reference to the model, and the method for generalizing these is indicated. The methods developed are applied to objective test data with satisfactory results.This research was supported in part by Office of Naval Research Contract Nonr2752(00).The author is now at the University of Queensland, St. Lucia, Queensland, Australia.     

Statistical inferences about true scores

Formulas are derived for unbiased sample estimators of any raw or central moment of the frequency distribution of true scores. A general method is developed for obtaining from each examinee's observed score a least squares estimate of his true score.This research was carried out under contract Nonr-2214(00) with the Office of Naval Research, Department of the Navy.     

The stochastic model of mental testing theory and an application

An axiomatization of mental test theory is presented. The presentation deals explicitly with the following concepts: true score, parallel forms, reliability, validity, and combined test score.This project was supported by the Office of Naval Research under Contract N6onr 270-(20) and by the National Science Foundation, Grant NSF G-642. Work now continued under Nonr 1858-(150) and NSF G-22889.     

An IBM method for computing intraserial correlations

A method for computing intraserial correlations using a 602-A Calculating Punch, an 077 Collator, a 513 Gang Punch, and a 403 Tabulator is described. An example of the use of the procedure and an estimate of the time needed with each machine are given. This procedure is compared with another method, which makes use of a more powerful IBM machine.This research was supported in part by the United States Air Force under Contract No. AF 33(038)-25726, monitored by the Air Force Personnel and Training Research Center. Permission is granted for reproduction, translation, publication, use and disposal in whole and in part by or for the United States Government.     

The mean and standard deviation of the distribution of group assembly sums

An interesting problem in linear programming is the group assembly problem which is mathematically equivalent to the general transportation problem of economics. Computer programs designed for the determination of exact and approximate optimal group assemblies have been available for some time. This paper presents formulas for the mean and squared standard deviation of the distribution of all possible group assembly sums. Computational techniques are presented and the results are related to those of the analysis of variance of ak-factor problem withn levels of each factor.This research was supported in part by the United States Air Force under Project No. 7713, Task No. 77232, and Contract No. AF 18(600)-1050, monitored by the Crew Research Laboratory of the Air Force Personnel and Training Research Center. Permission is granted for reproduction, translation, publication, use, and disposal in whole and in part by or for the United States Government.     

An empirical study of the normality and independence of errors of measurement in test scores

An empirical study of test scores shows the variance of the errors of measurement to be significantly associated with true score in each of four groups studied; it also shows the distribution of the errors of measurement to be significantly skewed in three of these four groups. The mathematical rationale underlying the statistical treatment is presented. Standard error formulas are given for making the necessary significance tests.This research was in part carried out under Contracts Nonr-2214(00) and Nonr-2752(00) with the Office of Naval Research, Department of the Navy.     

On the estimation of parameters in latent structure analysis

In this note, we describe the iterative procedure introduced earlier by Goodman to calculate the maximum likelihood estimates of the parameters in latent structure analysis, and we provide here a simple and direct proof of the fact that the parameter estimates obtained with the iterative procedure cannot lie outside the allowed interval. Formann recently stated that Goodman's algorithm can yield parameter estimates that lie outside the allowed interval, and we prove in the present note that Formann's contention is incorrect.This research was supported in part by Research Contract No. NSF SOC 76-80389 from the Division of the Social Sciences of the National Science Foundation. The author is indebted to C. C. Clogg for helpful comments and for the numerical results reported here (see, e.g., Table 1).     

Percentage points of Wilks'L mvc andL vc criteria

Likelihood ratio tests have been proposed by Wilks for testing the hypothesis of equal means, variances, and covariances (H _mvc) and the hypothesis of equal variances and covariances (H _vc) in ap-variate normal distribution. Using exact distributions of the appropriate likelihood ratio statistics, tables of the .05 and .01 points of these distributions are constructed forp = 4, 5, 6, 7 and sample sizen = 25 (5) 60 (10) 100. A correction factor is recommended for largern. Two numerical examples illustrate use of the tables. A nonparametric test is proposed forH _mvc when the multivariate parent population is known to be non-normal.This research was supported partly by the Office of Naval Research under Contract No. Nonr-855(06) and partly by the United States Air Force through the Air Force Office of Scientific Research of the Air Research and Development Command, under Contract No. 18(600)-83. Reproduction in whole or in part for any purpose of the United States Government is permitted.     

Bayesian inference and the classical test theory model: Reliability and true scores

A general one-way analysis of variance components with unequal replication numbers is used to provide unbiased estimates of the true and error score variance of classical test theory. The inadequacy of the ANOVA theory is noted and the foundations for a Bayesian approach are detailed. The choice of prior distribution is discussed and a justification for the Tiao-Tan prior is found in the particular context of the “n-split” technique. The posterior distributions of reliability, error score variance, observed score variance and true score variance are presented with some extensions of the original work of Tiao and Tan. Special attention is given to simple approximations that are available in important cases and also to the problems that arise when the ANOVA estimate of true score variance is negative. Bayesian methods derived by Box and Tiao and by Lindley are studied numerically in relation to the problem of estimating true score. Each is found to be useful and the advantages and disadvantages of each are discussed and related to the classical test-theoretic methods. Finally, some general relationships between Bayesian inference and classical test theory are discussed. Supported in part by the National Institute of Child Health and Human Development under Research Grant 1 PO1 HDO1762. Reproduction, translation, use or disposal by or for the United States Government is permitted.     

Probability-of-coverage tolerance intervals for the beta binomial

True score tolerance intervals, which are designed to cover a chosen proportion of the conditional distribution of true scores given an observed score, are suggested as alternatives to true score confidence intervals. Using large sample theory, a tolerance interval estimator for the beta binomial is derived. An example indicates that with moderate sample sizes, tolerance intervals with high probability of coverage will not be much wider than when the two beta true score parameters are known.The author acknowledges valuable comments from Richard Sawyer.Most work was completed while the author was at the American College Testing Program.     

Bayesian estimation of item response curves

Item response curves for a set of binary responses are studied from a Bayesian viewpoint of estimating the item parameters. For the two-parameter logistic model with normally distributed ability, restricted bivariate beta priors are used to illustrate the computation of the posterior mode via the EM algorithm. The procedure is illustrated by data from a mathematics test.This work was supported under Contract No. N00014-85-K-0113, NR 150-535, from Personnel and Training Research Programs, Psychological Sciences Division, Office of Naval Research. The authors wish to thank Mark D. Reckase for providing the ACT data used in the illustration and Michael J. Soltys for computational assistance. They also wish to thank the editor and four anonymous reviewers for many valuable suggestions.     

Maximum validity of a test with equivalent items

It is assumed that a scale of true scores on a function exists and that the probability of answering an item correctly is a curve of the type of the integral of the normal curve. The product moment correlation between the test score and true score is derived for a normal distribution of subjects and a test composed of equivalent items. Numerical examples demonstrate that the maximum correlation between test scores and true scores occurs for a one hundred item test when the point correlation between items is less than three tenths.     

Notes on an approximation method for fitting parabolic equations to experimental data

When a numerical transformation of raw data is used only to simplify the arithmetic of curve fitting, the transformation may lead to undesirable and even highly distorted results. This principle is illustrated with an approximation method of fitting parabolic equations to experimental data, as described recently in texts by Johnson and Lewis. Although the approximation method will never yield as good fits as the exact, least-squares method, satisfactory results are in general achieved whenever the transformed scores yield a linear plot as a function ofX. The principal difficulty with the method is that some data which fall along a parabola may not yield a linear plot of the transformed scores versusX, and so cannot be fitted satisfactorily by the approximation method.This study was done in cooperation with the Systems Division, Naval Research Laboratory, under Contract N5-ori-166, Task Order I, between the Office of Naval Research and The Johns Hopkins University. This is Report No. 166-I-156, Project Designation No. NR-507-470, under that contract. The author is indebted to Dr. Hermann von Schelling, of the Naval Medical Research Laboratory, U. S. Naval Submarine Base, New London, Connecticut, for technical advice. Miss Judith T. Parker and Mr. William T. Pollock assisted capably in the tedious computations required for this note.     

Penalized Best Linear Prediction of True Test Scores

Psychometrika - In best linear prediction (BLP), a true test score is predicted by observed item scores and by ancillary test data. If the use of BLP rather than a more direct estimate of a true...     

Validity of the NIMH Diagnostic Interview Schedule for Children: A comparison between psychiatric and pediatric referrals

The NIMH Diagnostic Interview Schedule for Children (DISC) was used to evaluate 40 psychiatric referrals aged 7–11 and 40 pediatric referrals matched for age, sex, race, and socioeconomic status. Each parent and child was interviewed separately using parallel versions of the interview, DISC-P and DISC-C, and parents completed the Child Behavior Checklist (CBCL). The psychiatric referrals had more psychiatric diagnoses and higher symptom scores than the pediatric referrals. Parent reports discriminated better between the criterion groups than child reports. In both groups, mild oppositional behavior and fears were commonly reported by parents, whereas mild separation anxiety, fears, and dysthymia were commonly reported by children. Attention deficit disorder, conduct disorder, and affective disorders were much more common among psychiatrically referred children. There was an association (x²=37.1, p < 001) between abnormally high CBCL scores and diagnoses derived from the interview with the parent, but the association between the CBCL and the child interview was not significant. Over all, the results support the validity of the DISC-P, and to a lesser extent that of the DISC-C, in discriminating psychiatric from pediatric referrals, at the level of both symptoms and severe diagnoses, but not at the mild/moderate level of diagnosis.This work was supported in part by NIMH Clinical Research Center Grant No. MH30915 and by NIMH Contract No. RFP-DB-81-0027. The second author is supported by an NIMH Research Scientist Development Award No. MH00403 and by a Faculty Scholar's Award from the William T. Grant Foundation. The authors are most grateful to Noelle Conover for her assistance in data analysis.     

Sex-role expectations,power, and psychological distress

Data collected in a community survey of psychiatric epidemiology in New Haven, Connecticut, in 1967 are reanalyzed to examine the relationship of sex-role expectations and power to rates of psychological distress. The results indicate that people who occupy powerful roles have low rates of distress, regardless of whether they conform to or deviate from role expectations. The occupation of the powerless role, however, is particularly productive of distress when the occupant of this role deviates from sex-appropriate behavior. The findings suggest that sex differences in distress may in part be deducible from more general differences in power and role expectations.The author is grateful to Jerome K. Myers for generously allowing use of his data in this article. Myers' research was supported by PGS Contract No. 43-67-743 and Research Grant No. MH 15522 from the National Institute of Mental Health, Department of Health, Education and Welfare.     

Estimating true-score distributions in psychological testing (an empirical bayes estimation problem)

The following problem is considered: Given that the frequency distribution of the errors of measurement is known, determine or estimate the distribution of true scores from the distribution of observed scores for a group of examinees. Typically this problem does not have a unique solution. However, if the true-score distribution is smooth, then any two smooth solutions to the problem will differ little from each other. Methods for finding smooth solutions are developed a) for a population and b) for a sample of examinees. The results of a number of tryouts on actual test data are summarized.The writer wishes to thank Diana Lees and Virginia Lennon, who wrote the computer programs, carried out some of the mathematical derivations, and helped with other important aspects of the work. This work was supported in part by contract Nonr-2752(00) between the Office of Naval Research and Educational Testing Service. Reproduction, translation, use and disposal in whole or in part by or for the United States Government is permitted.     

Some developments in multivariate generalizability

This article is concerned with estimation of components of maximum generalizability in multifacet experimental designs involving multiple dependent measures. Within a Type II multivariate analysis of variance framework, components of maximum generalizability are defined as those composites of the dependent measures that maximize universe score variance for persons relative to observed score variance. The coefficient of maximum generalizability, expressed as a function of variance component matrices, is shown to equal the squared canonical correlation between true and observed scores. Emphasis is placed on estimation of variance component matrices, on the distinction between generalizability- and decision-studies, and on extension to multifacet designs involving crossed and nested facets. An example of a two-facet partially nested design is provided.Appreciation is expressed to the Office of Research in Medical Education, University of Texas Medical Branch, for permitting use of their data.     

Optimal test length for maximum absolute prediction

The concepts of multiple differential prediction and multiple absolute prediction are developed in earlier papers (2, 3). The problem of determining the optimal distribution of testing time for multiple differential prediction has been previously considered (4). This paper develops an analogous procedure for multiple absolute prediction. A numerical example illustrating the procedure is presented. The mathematical rationale underlying the procedure is given.This research was carried out under Contract Nonr-477(08) between the University of Washington and the Office of Naval Research. The computations were carried out by Robert Dear and Donald Mills. Much credit is due the typist, Elizabeth Cross. Supervision of both computational and editorial activities was provided by William Clemans. To each of these able contributors we are deeply grateful.