Asymptotic log-linear analysis: some cautions concerning sparse frequency tables

Traditional asymptotic probability values resulting from log-linear analyses of sparse frequency tables are often much too large. Asymptotic probability values for chi-squared and likelihood-ratio statistics are compared to nonasymptotic and exact probability values for selected log-linear models. The asymptotic probability values are all too often substantially larger than the exact probability values for the analysis of sparse frequency tables. An exact nondirectional permutation method is presented to analyze combined independent multinomial distributions. Exact nondirectional permutation methods to analyze hypergeometric distributions associated with r-way frequency tables are confined to r = 2.     

Combining probability values from independent permutation tests: a discrete analog of Fisher's classical method

Permutation tests are based on all possible arrangements of observed data sets. Consequently, such tests yield exact probability values obtained from discrete probability distributions. An exact nondirectional method to combine independent probability values that obey discrete probability distributions is introduced. The exact method is the discrete analog to Fisher's classical method for combining probability values from independent continuous probability distributions. If the combination of probability values includes even one probability value that obeys a sparse discrete probability distribution, then Fisher's classical method may be grossly inadequate.     

Some generalizations of a theoretical distribution of mental test scores

The negative hypergeometric distribution may be generalized to provide distributions involving 3, 4, or more parameters. It is shown that, in the case of the binomial error model, such distributions correspond to quadratic, cubic, etc., regressions of true scores on raw scores. Explicit formulas are given for the bivariate moments required to fit these regressions and so estimate the parameters of the generalized hypergeometric distribution. Equations are also developed for fitting the 3- and 4-parameter distributions directly, i.e., without reference to the model, and the method for generalizing these is indicated. The methods developed are applied to objective test data with satisfactory results.This research was supported in part by Office of Naval Research Contract Nonr2752(00).The author is now at the University of Queensland, St. Lucia, Queensland, Australia.     

Learning Continuous Probability Distributions with Symmetric Diffusion Networks

In this article we present symmetric diffusion networks, a family of networks that instantiate the principles of continuous, stochastic, adaptive and interactive propagation of information. Using methods of Markovion diffusion theory, we formalize the activation dynamics of these networks and then show that they can be trained to reproduce entire multivariate probability distributions on their outputs using the contrastive Hebbion learning rule (CHL). We show that CHL performs gradient descent on an error function that captures differences between desired and obtained continuous multivariate probability distributions. This allows the learning algorithm to go beyond expected values of output units and to approximate complete probability distributions on continuous multivariate activation spaces. We argue that learning continuous distributions is an important task underlying a variety of real-life situations that were beyond the scope of previous connectionist networks. Deterministic networks, like back propagation, cannot learn this task because they are limited to learning average values of independent output units. Previous stochastic connectionist networks could learn probability distributions but they were limited to discrete variables. Simulations show that symmetric diffusion networks can be trained with the CHL rule to approximate discrete and continuous probability distributions of various types.     

Overestimation of subjective probabilities

Abstract.— When asked to estimate the probability of outcomes of draws from a binomial population, student subjects tend to report p values that clearly exceed the objective ones. The probability of specific binomial sequences was found to be even more overestimated, while the answers became much more conservative when the outcomes were grouped into a few categories. These findings were replicated in a second experiment, where the probability of heights in a male and a female student population was estimated. When the task was to estimate frequency of occurrence, instead of probability, the answers became more realistic. The conclusion is drawn that the direct p estimates are relatively independent of frequency judgments, the chief determinant being the properties of the particular sample to be evaluated, irrespective of the number and probabilities of other possible samples.     

Nonasymptotic significance tests for two measures of agreement.

The kappa agreement coefficient of Cohen from 1960 and Brennan and Prediger from 1981 are defined and compared. A FORTRAN program is described that computes Cohen's kappa and Brennan and Prediger's kappa and their associated probability values based on Monte Carlo resampling and the binomial distribution, respectively.     

Simulating the binomial distribution with a software quincunx

The software quincunx is a graphics-based simulation of the binomial distribution designed as a teaching aid for statistics and probability instruction. The user can define the binomial probability, number of trials, and number of repetitions. The program graphically simulates the action of a mechanical quincunx and provides a summary of observed and expected data.     

Empirical bayes estimates of domain scores under binomial and hypergeometric distributions for test scores

We introduce two simple empirical approximate Bayes estimators (EABEs)— and —for estimating domain scores under binomial and hypergeometric distributions, respectively. Both EABEs (derived from corresponding marginal distributions of observed test scorex without relying on knowledge of prior domain score distributions) have been proven to hold -asymptotic optimality in Robbins' sense of convergence in mean. We found that, where and are the monotonized versions of and under Van Houwelingen's monotonization method, respectively, the convergence rate of the overall expected loss of Bayes risk in either or depends on test length, sample size, and ratio of test length to size of domain items. In terms of conditional Bayes risk, and outperform their maximum likelihood counterparts over the middle range of domain scales. In terms of mean-squared error, we also found that: (a) given a unimodal prior distribution of domain scores, performs better than both and a linear EBE of the beta-binomial model when domain item size is small or when test items reflect a high degree of heterogeneity; (b) performs as well as when prior distribution is bimodal and test items are homogeneous; and (c) the linear EBE is extremely robust when a large pool of homogeneous items plus a unimodal prior distribution exists.The authors are indebted to both anonymous reviewers, especially Reviewer 2, and the Editor for their invaluable comments and suggestions. Thanks are also due to Yuan-Chin Chang and Chin-Fu Hsiao for their help with our simulation and programming work.     

Peer Disagreement and Independence Preservation

It has often been recommended that the differing probability distributions of a group of experts should be reconciled in such a way as to preserve each instance of independence common to all of their distributions. When probability pooling is subject to a universal domain condition, along with state-wise aggregation, there are severe limitations on implementing this recommendation. In particular, when the individuals are epistemic peers whose probability assessments are to be accorded equal weight, universal preservation of independence is, with a few exceptions, impossible. Under more reasonable restrictions on pooling, however, there is a natural method of preserving the independence of any fixed finite family of countable partitions, and hence of any fixed finite family of discrete random variables.     

Information-theoretic latent distribution modeling: distinguishing discrete and continuous latent variable models

Distinguishing between discrete and continuous latent variable distributions has become increasingly important in numerous domains of behavioral science. Here, the authors explore an information-theoretic approach to latent distribution modeling, in which the ability of latent distribution models to represent statistical information in observed data is emphasized. The authors conclude that loss of statistical information with a decrease in the number of latent values provides an attractive basis for comparing discrete and continuous latent variable models. Theoretical considerations as well as the results of 2 Monte Carlo simulations indicate that information theory provides a sound basis for modeling latent distributions and distinguishing between discrete and continuous latent variable models in particular.     

BINOMIX: A BASIC program for maximum likelihood analyses of finite and beta-binomial mixture distributions

If a discrete random variableY is binomially distributed for each observation unit in a sample, and if the binomial parameterp varies between different units, then the distribution ofY underlying a sample of observations based on different units is called a binomial mixture distribution. BINOMIX is a BASIC program to estimate parameters and to evaluate the goodness of fit for the two most important types of binomial mixtures, finite and beta-binomial.     

Properties of reverse hazard functions

For continuous distributions reverse hazard is defined as the probability density divided by the cumulative probability, F(x); whereas the usual hazard function is the density divided by the survivor function, 1−F(x). Reverse hazard corresponds to the conditional density of an immediate failure or state change, conditioned by the fact that the state change occurred. For example, of all the items that failed, the proportion of those items that immediately failed is reverse hazard. Reverse hazard exhibits many symmetrical properties with hazard. In this paper a set of theorems are developed that explicate the properties of reverse hazard for both continuous and discrete probability distributions. Taken together, hazard and reverse hazard are a powerful set of theoretical constructs that are valuable for understanding stochastic systems.     

Prior distributions for random choice structures

We study various axioms of discrete probabilistic choice, measuring how restrictive they are, both alone and in the presence of other axioms, given a specific class of prior distributions over a complete collection of finite choice probabilities. We do this by using Monte Carlo simulation to compute, for a range of prior distributions, probabilities that various simple and compound axioms hold. For example, the probability of the triangle inequality is usually many orders of magnitude higher than the probability of random utility. While neither the triangle inequality nor weak stochastic transitivity imply the other, the conditional probability that one holds given the other holds is greater than the marginal probability, for all priors in the class we consider. The reciprocal of the prior probability that an axiom holds is an upper bound on the Bayes factor in favor of a restricted model, in which the axiom holds, against an unrestricted model. The relatively high prior probability of the triangle inequality limits the degree of support that data from a single decision maker can provide in its favor. The much lower probability of random utility implies that the Bayes factor in favor of it can be much higher, for suitable data.     

The redundancy of recursion and infinity for natural language

An influential line of thought claims that natural language and arithmetic processing require recursion, a putative hallmark of human cognitive processing (Chomsky in Evolution of human language: biolinguistic perspectives. Cambridge University Press, Cambridge, pp 45–61, 2010; Fitch et al. in Cognition 97(2):179–210, 2005; Hauser et al. in Science 298(5598):1569–1579, 2002). First, we question the need for recursion in human cognitive processing by arguing that a generally simpler and less resource demanding process—iteration—is sufficient to account for human natural language and arithmetic performance. We argue that the only motivation for recursion, the infinity in natural language and arithmetic competence, is equally approachable by iteration and recursion. Second, we submit that the infinity in natural language and arithmetic competence reduces to imagining infinite embedding or concatenation, which is completely independent from the ability to implement infinite processing, and thus, independent from both recursion and iteration. Furthermore, we claim that a property of natural language is physically uncountable finity and not discrete infinity.     

Some improved diagnostics for failure of the Rasch model

Although several goodness of fit tests have been developed for the Rasch model for dichotomous items, most of them are of a global, asymptotic, and confirmatory type. This paper, based on ideas from a recent thesis by Van den Wollenberg, offers some suggestions for local, small sample, and exploratory techniques: difficulty plots for person groups scoring right and wrong on a specific item, a slope test per item based on a binomial distribution per score group, and a unidimensionality check based on an extended hypergeometric distribution per score group. This paper owes much to the inspiring and pioneering work of Arnold Van den Wollenberg, of which only minor aspects are criticized. Thanks go to Charles Lewis for stimulating discussions and for solutions to some programming problems.     

The frequency accrual speed test (FAST): A new measure of ‘mental speed’?

This paper presents a rationale for a new measure of ‘mental speed’, based on a task in which subjects discriminate the relative frequency of flashes on one or the other of two lamps. In this ‘frequency accrual speed test’ (FAST), accuracy is interpreted as a measure of the rate at which flashes are registered. By assuming that sub-optimal performance is due to a random failure to register flashes, predictions are derived for a binomial and a hypergeometric version of the test. A procedure for estimating inspection time (IT) is given, and optimal values for the task parameters suggested. Results from eight studies are presented. These show that accuracy in the FAST procedure provides a stable, reliable, and robust measure, not obviously susceptible to different strategies or to practice. In four out of six studies, accuracy in this type of task showed a significant positive correlation with tests of intellectual performance, and in five out of five studies accuracy was negatively correlated with IT (significantly in two). These results show that accuracy in the FAST procedure may provide a useful measure of the speed with which sensory information is sampled. A programme for further testing is outlined.     

QMPE: estimating Lognormal, Wald, and Weibull RT distributions with a parameter-dependent lower bound.

We describe and test quantile maximum probability estimator (QMPE), an open-source ANSI Fortran 90 program for response time distribution estimation. QMPE enables users to estimate parameters for the ex-Gaussian and Gumbel (1958) distributions, along with three "shifted" distributions (i.e., distributions with a parameter-dependent lower bound): the Lognormal, Wald, and Weibul distributions. Estimation can be performed using either the standard continuous maximum likelihood (CML) method or quantile maximum probability (QMP; Heathcote & Brown, in press). We review the properties of each distribution and the theoretical evidence showing that CML estimates fail for some cases with shifted distributions, whereas QMP estimates do not. In cases in which CML does not fail, a Monte Carlo investigation showed that QMP estimates were usually as good, and in some cases better, than CML estimates. However, the Monte Carlo study also uncovered problems that can occur with both CML and QMP estimates, particularly when samples are small and skew is low, highlighting the difficulties of estimating distributions with parameter-dependent lower bounds.     

QMPE: Estimating Lognormal, Wald, and Weibull RT distributions with a parameter-dependent lower bound

We describe and test quantile maximum probability estimator (QMPE), an open-source ANSI Fortran 90 program for response time distribution estimation.¹ QMPE enables users to estimate parameters for the ex-Gaussian and Gumbel (1958) distributions, along with three “shifted” distributions (i.e., distributions with a parameter-dependent lower bound): the Lognormal, Wald, and Weibull distributions. Estimation can be performed using either the standard continuous maximum likelihood (CML) method or quantile maximum probability (QMP; Heathcote & Brown, in press). We review the properties of each distribution and the theoretical evidence showing that CML estimates fail for some cases with shifted distributions, whereas QMP estimates do not. In cases in which CML does not fail, a Monte Carlo investigation showed that QMP estimates were usually as good, and in some cases better, than CML estimates. However, the Monte Carlo study also uncovered problems that can occur with both CML and QMP estimates, particularly when samples are small and skew is low, highlighting the difficulties of estimating distributions with parameter-dependent lower bounds.     

Sets of probability distributions, independence, and convexity

This paper analyzes concepts of independence and assumptions of convexity in the theory of sets of probability distributions. The starting point is Kyburg and Pittarelli??s discussion of ??convex Bayesianism?? (in particular their proposals concerning E-admissibility, independence, and convexity). The paper offers an organized review of the literature on independence for sets of probability distributions; new results on graphoid properties and on the justification of ??strong independence?? (using exchangeability) are presented. Finally, the connection between Kyburg and Pittarelli??s results and recent developments on the axiomatization of non-binary preferences, and its impact on ??complete?? independence, are described.