Overestimation of subjective probabilities

Abstract.— When asked to estimate the probability of outcomes of draws from a binomial population, student subjects tend to report p values that clearly exceed the objective ones. The probability of specific binomial sequences was found to be even more overestimated, while the answers became much more conservative when the outcomes were grouped into a few categories. These findings were replicated in a second experiment, where the probability of heights in a male and a female student population was estimated. When the task was to estimate frequency of occurrence, instead of probability, the answers became more realistic. The conclusion is drawn that the direct p estimates are relatively independent of frequency judgments, the chief determinant being the properties of the particular sample to be evaluated, irrespective of the number and probabilities of other possible samples.     

An algorithm to generate discrete probability distributions: Binomial,hypergeometric, negative binomial,inverse hypergeometric,and Poisson

A recursion procedure for generating point probability values, cumulative probability values, and interval probability values of discrete distributions is described. Algorithms and associated FORTRAN functions for five representative discrete probability distributions are presented : binomial, hypergeometric, negative binomial, inverse hypergeometric, and Poisson. The joint use of recursion and a small arbitrary initial value ensures computational efficiency.     

Comparing examinees to a control

When comparing examinees to a control, the examiner usually does not know the probability of correctly classifying the examinees based on the number of items used and the number of examinees tested. Using ranking and selection techniques, a general framework is described for deriving a lower bound on this probability. We illustrate how these techniques can be applied to the binomial error model. New exact results are given for normal populations having unknown and unequal variances.The work upon which this publication is based was performed pursuant to a grant [Grant No. NIE-G-76-0083] with the National Institute of Education, Department of Health, Education and Welfare. Points of view or opinions stated do not necessarily represent official NIE position or policy.     

Exact probabilities for contingency tables using binomial coefficients

The use of binomial coefficients in place of factorials to shorten the calculation of exact probabilities for 2 × 2 and 2 ×r contingency tables is discussed. A useful set of inequalities for estimating the cumulative probabilities in the tail of the distribution from the probability of a single table is given. A table of binomial coefficients with four significant places andn through 60 is provided.     

Nonasymptotic significance tests for two measures of agreement.

The kappa agreement coefficient of Cohen from 1960 and Brennan and Prediger from 1981 are defined and compared. A FORTRAN program is described that computes Cohen's kappa and Brennan and Prediger's kappa and their associated probability values based on Monte Carlo resampling and the binomial distribution, respectively.     

Typological personality research by configural frequency analysis

After discussing diverse concepts of types or syndromes the definition of types, according to configural frequency analysis (CFA), is given. A type, in this theory, is assumed to be a configuration of categories belonging to different attributes. This configuration should occur with a probability which is higher than the conditional probability for given univariate marginal frequencies. The conditional probability is computed under the null hypothesis of independence of the attributes. Types are identified by simultaneous conditional binomial tests and interpreted by means of an interaction structure analysis in a multivariate contingency table. Two further versions of CFA are explained. By prediction CFA it is possible to predict certain configurations by other ones while by c-sample CFA it is possible to discriminate between populations by means of configurations. The procedures are illustrated by an example concerning the responses of patients to lumbar punctures.     

Scoring formulas and probability considerations

Bayes' theorem of inverse probability is made the basis of a general equation for scoring objective examinations. The equation so obtained is evaluated by assuming a binomial distribution of examinee knowledge and guessing tendency. A graphical illustration of the application of this equation to a hypothetical test situation is presented. The limitations inherent in the use of Bayes' theorem make it inadvisable to recommend the practical use of the equation unless future experimental evidence indicates an increase in scoring validity which more than compensates for the increase in scoring difficulty.     

Decision rules and signal detectability in a reinforcement-density discrimination

Two probabilistic schedules of reinforcement, one richer in reinforcement, the other leaner, were overlapping stimuli to be discriminated in a choice situation. One of two schedules was in effect for 12 seconds. Then, during a 6-second choice period, the first left-key peck was reinforced if the richer schedule had been in effect, and the first right-key peck was reinforced if the leaner schedule had been in effect. The two schedule stimuli may be viewed as two binomial distributions of the number of reinforcement opportunities. Each schedule yielded different frequencies of 16 substimuli. Each substimulus had a particular type of outcome pattern for the 12 seconds during which a schedule was in effect, and consisted of four consecutive light-cued 3-second T-cycles, each having 0 or 1 reinforced center-key pecks. Substimuli therefore contained 0 to 4 reinforcers. On any 3-second cycle, the first center-key peck darkened that key and was reinforced with probability .75 or .25 in the richer or leaner schedules, respectively. In terms of the theory of signal detection, detectability neared the maximum possible d′ for all four pigeons. Left-key peck probability increased when number of reinforcers in a substimulus increased, when these occurred closer to choice, or when pellets were larger for correct left-key pecks than for correct right-key pecks. Averaged over different temporal patterns of reinforcement in a substimulus, substimuli with the same number of reinforcers produced choice probabilities that matched relative expected payoff rather than maximized one alternative.     

Latent Class Modeling of Phases in the Development of Transitive Reasoning

Fuzzy trace theory posits that during development the use of verbatim information for solving transitive relationships shifts to the use of gist information. In cognitive developmental research that uses a cross-sectional design, the binomial mixture model is often used to identify such shifts. Because the binomial mixture model assumes equal task difficulty and uses the number of correctly solved tasks for data analysis, it may be too restrictive and the more flexible latent class model is adopted as an alternative. This model allows varying task difficulty and uses the pattern of task scores as input for data analysis. The binomial mixture model and the latent class model are compared theoretically, and applied to transitive reasoning test data obtained from a cross-sectional sample of 615 children. The latent class model is found to be more appropriate for identifying multiple phases. Three phases are distinguished which can be interpreted well by means of fuzzy trace theory. These phases do not encompass fixed age periods.     

Probability-of-coverage tolerance intervals for the beta binomial

True score tolerance intervals, which are designed to cover a chosen proportion of the conditional distribution of true scores given an observed score, are suggested as alternatives to true score confidence intervals. Using large sample theory, a tolerance interval estimator for the beta binomial is derived. An example indicates that with moderate sample sizes, tolerance intervals with high probability of coverage will not be much wider than when the two beta true score parameters are known.The author acknowledges valuable comments from Richard Sawyer.Most work was completed while the author was at the American College Testing Program.     

Carnap's inductive probabilities as a contribution to decision theory

Common probability theories only allow the deduction of probabilities by using previously known or presupposed probabilities. They do not, however, allow the derivation of probabilities from observed data alone. The question thus arises as to how probabilities in the empirical sciences, especially in medicine, may be arrived at. Carnap hoped to be able to answer this question byhis theory of inductive probabilities. In the first four sections of the present paper the above mentioned problem is discussed in general. After a short presentation of Carnap's theory it is then shown that this theory cannot claim validity for arbitrary random processes. It is suggested that the theory be only applied to binomial and multinomial experiments. By application of de Finetti's theorem Carnap's inductive probabilities are interpreted as consecutive probabilities of the Bayesian kind. Through the introduction of a new axiom the decision parameter λ can be determined even if no a priori knowledge is given. Finally, it is demonstrated that the fundamental problem of Wald's decision theory, i.e., the determination of a plausible criterion where no a priori knowledge is available, can be solved for the cases of binomial and multinomial experiments.     

Bias and error in multiple-choice tests

A formula for estimating real scores on a multiple-choice test from a knowledge of raw scores is derived. This formula does not involve the assumption of a binomial distribution of real scores as does the Calandra formula. Other important formulas derived show: the variance of real scores in terms of the variance of raw scores and the correlation between real scores and raw scores. If the variance of real scores (or of raw scores also) is binomial, the regression of real scores on raw scores is linear; but, otherwise the regression is curvilinear. Yet the linear estimating formula is a close approximation to the curvilinear relationship. Factors affecting the regression of real scores on raw scores and the correlation coefficient are: (1) the number of choices per question; (2) the number of questions answered; (3) the ratio of the average group raw score to the variance of raw scores.     

Simulation of complex survival distributions

This paper describes a method for generating sample survival distributions from a hypothetical population, as would be required for running Monte Carlo simulations. The method is based on the concept of a quincunx. Cases are entered into a life table and allowed to drop out or die during each interval with probabilities that mirror the hypothetical population. By repeating this process many times and tracking the results, the researcher is able to study the sampling distribution of effect size indices and test statistics, and can generate empirical estimates of power and precision for planned studies. Unlike other methods that are commonly used for this purpose, the model proposed here allows the researcher to define a population in which the hazard rates and/or attrition rates vary substantially from one time point to the next, as may be the case in clinical trials or studies of processing times. The method requires less than 100 lines of code and runs some 10,000 simulations per hour on a microcomputer.     

BINOMIX: A BASIC program for maximum likelihood analyses of finite and beta-binomial mixture distributions

If a discrete random variableY is binomially distributed for each observation unit in a sample, and if the binomial parameterp varies between different units, then the distribution ofY underlying a sample of observations based on different units is called a binomial mixture distribution. BINOMIX is a BASIC program to estimate parameters and to evaluate the goodness of fit for the two most important types of binomial mixtures, finite and beta-binomial.     

Variability signatures distinguish verbal from nonverbal counting for both large and small numbers

Humans appear to share with animals a nonverbal counting process. In a nonverbal counting condition, subjects pressed a key a numeral-specified number of times, while saying “the” at every press. The mean number of presses increased as a power function of the target number, with a constant coefficient of variation (c.v.), both within and beyond the proposed subitizing range (1–4 or 5), suggesting small numbers are represented on the same continuum as larger numbers and subject to the same noise process (scalar variability). By contrast, when subjects counted their presses out loud as fast as they could, the c.v. decreased as the inverse square root of the target value (binomial variability instead of scalar variability). The unexpected power-law relation between target value and mean number of presses in nonverbal counting suggests a new hypothesis about the development of the function relating number symbols to mental magnitudes.     

非参数认知诊断方法下诊断结果的概率化表征

非参数认知诊断分类方法非常适合课堂评估,其诊断结果采用0-1形式而缺乏概率化表征,不能精细地区分被试属性掌握程度的差异或变化,还缺乏可用于评价真实测验分类结果的信度和效度指标。要刻画被试属性掌握程度的差异,首要的问题是要为非参数认知诊断方法提供一种可以量化属性掌握概率的方法。针对此问题,基于二项分布和玻尔兹曼分布提出非参数认知诊断方法下诊断结果的概率化表征方法,并用于构建分类准确性和分类一致性指标。模拟研究与实测数据分析结果显示：概率化表征方法与非参数认知诊断方法的分类结果高度一致;概率化表征方法与认知诊断模型所得的属性掌握概率十分接近;概率化表征方法所得的属性（模式）掌握概率可用于计算属性（模式）分类准确性和分类一致性指标,在实际测验情景下可作为信度和效度指标,评价诊断结果的重测一致率和判准率。     

A statistical description of verbal learning

Free-recall verbal learning is analyzed in terms of a probability model. The general theory assumes that the probability of recalling a word on any trial is completely determined by the number of times the word has been recalled on previous trials. Three particular cases of this general theory are examined. In these three cases, specific restrictions are placed upon the relation between probability of recall and number of previous recalls. The application of these special cases to typical experimental data is illustrated. An interpretation of the model in terms of set theory is suggested but is not essential to the argument.     

The influence of signal probability and the number of non-signal categories on selective response times

Signal probability and the number of non-signal categories were varied independently in a selective response task. Response latencies were significantly affected by changes in signal probability but not by changes in the number of non-signal categories.