A frequentist interpretation of probability for model-based inductive inference

The main objective of the paper is to propose a frequentist interpretation of probability in the context of model-based induction, anchored on the Strong Law of Large Numbers (SLLN) and justifiable on empirical grounds. It is argued that the prevailing views in philosophy of science concerning induction and the frequentist interpretation of probability are unduly influenced by enumerative induction, and the von Mises rendering, both of which are at odds with frequentist model-based induction that dominates current practice. The differences between the two perspectives are brought out with a view to defend the model-based frequentist interpretation of probability against certain well-known charges, including [i] the circularity of its definition, [ii] its inability to assign ‘single event’ probabilities, and [iii] its reliance on ‘random samples’. It is argued that charges [i]–[ii] stem from misidentifying the frequentist ‘long-run’ with the von Mises collective. In contrast, the defining characteristic of the long-run metaphor associated with model-based induction is neither its temporal nor its physical dimension, but its repeatability (in principle); an attribute that renders it operational in practice. It is also argued that the notion of a statistical model can easily accommodate non-IID samples, rendering charge [iii] simply misinformed.     

The use of decomposition in probability assessments of continuous variables

In this study, decomposition is used as a tool for the assessment of continuous probability distributions. The goal of using decomposition is to obtain better calibrated probability distributions by expressing a variable as a known function of several component variables. Three target quantities were used in the study. Each was assessed holistically and using two different decompositions. Thus, each subject provided three distributions for each of the three target quantities. The recomposed assessments were compared to holistic assessments. The distributions obtained using decomposition were found to be much better calibrated than those obtained holistically. Two methods of aggregating distributions from multiple subjects were also examined. One involves aggregating (averaging) distributions before recomposing while the second method involves recomposing and then averaging distributions for the target variable. The second method was found to be slightly better, although both showed better calibration than was found in the individual assessments.     

A taxonomy of latent structure assumptions for probability matrix decomposition models

A taxonomy of latent structure assumptions (LSAs) for probability matrix decomposition (PMD) models is proposed which includes the original PMD model (Maris, De Boeck, & Van Mechelen, 1996) as well as a three-way extension of the multiple classification latent class model (Maris, 1999). It is shown that PMD models involving different LSAs are actually restricted latent class models with latent variables that depend on some external variables. For parameter estimation a combined approach is proposed that uses both a mode-finding algorithm (EM) and a sampling-based approach (Gibbs sampling). A simulation study is conducted to investigate the extent to which information criteria, specific model checks, and checks for global goodness of fit may help to specify the basic assumptions of the different PMD models. Finally, an application is described with models involving different latent structure assumptions for data on hostile behavior in frustrating situations.Note: The research reported in this paper was partially supported by the Fund for Scientific Research-Flanders (Belgium) (project G.0207.97 awarded to Paul De Boeck and Iven Van Mechelen), and the Research Fund of K.U. Leuven (F/96/6 fellowship to Andrew Gelman, OT/96/10 project awarded to Iven Van Mechelen and GOA/2000/02 awarded to Paul De Boeck and Iven Van Mechelen). We thank Marcel Croon and Kristof Vansteelandt for commenting on an earlier draft of this paper.     

Analyzing the accuracy of probability judgments for multiple events: An extension of the covariance decomposition

The probability score (PS) can be used to measure the overall accuracy of probability judgments for a single event, e.g., “Rain falls,” or “This patient has cancer.” It has been shown previously how a “covariance decomposition” of the mean of PS over many occasions indexes several distinct aspects of judgment performance (J. F. Yates, Organizational Behavior and Human Performance, 30, 132–156 (1982)). There are many situations in which probability judgments are reported for sample space partitions containing more than one event and its complement, e.g., medical situations in which a patient might suffer from Disease X, Disease Y, or Disease Z, or testing situations in which the correct answer to an item might be any one of alternatives (a) through (e). The probability score for multiple events (PSM) serves as a measure of the overall accuracy of probability judgments for the events in partitions of any size. The present article describes and interprets an extension of the covariance decomposition to the mean of PSM. The decomposition is illustrated with data from two contexts, medicine and education.     

Subjective probability of disjunctive hypotheses: local-weight models for decomposition of evidential support

When the probability of a single member of a set of mutually exclusive and exhaustive possibilities is judged, its alternatives are evaluated as a composite "residual" hypothesis. Support theory (Rottenstreich & Tversky, 1997; Tversky & Koehler, 1994) implies that the process of packing alternatives together in the residual reduces the perceived evidential support for the set of alternatives and consequently inflates the judged probability of the focal hypothesis. Previous work has investigated the global weights that determine the extent to which the overall evidential support for the alternatives is discounted by this packing operation (Koehler, Brenner, & Tversky, 1997). In the present investigation, we analyze this issue in greater detail, examining the local weights that measure the specific contribution of each component hypothesis included implicitly in the residual. We describe a procedure for estimating local weights and introduce a set of plausible properties that impose systematic ordinal relationships among local weights. Results from four experiments testing these properties are reported, and a local-weight model is developed that accounts for nearly all of the variance in the probability judgments in these empirical tests. Local weights appear to be sensitive both to the individual component with which they are associated and to the residual hypothesis in which the component resides.     

Exact and resampling probability values for measures of categorical variation and consensus

When analyzing categorical data, it is often important to assess the magnitude of variation, or consensus, among observations in unordered categories. Utilizing the theory of partitions, exact solutions for five commonly used measures of categorical variation are presented. When the number of partitions is very large, resampling methods provide close approximations to exact probability values.     

Nonasymptotic significance tests for two measures of agreement.

The kappa agreement coefficient of Cohen from 1960 and Brennan and Prediger from 1981 are defined and compared. A FORTRAN program is described that computes Cohen's kappa and Brennan and Prediger's kappa and their associated probability values based on Monte Carlo resampling and the binomial distribution, respectively.     

Clarifications on the application and interpretation of the test for excess significance and its extensions

This commentary discusses challenges in the application of the test for excess significance (Ioannidis & Trikalinos, 2007) including the definition of the body of evidence, the plausible effect size for power calculations and the threshold of statistical significance. Interpretation should be cautious, given that it is not possible to separate different mechanisms of bias (classic publication bias, selective analysis, and fabrication) that lead to an excess of significance and in some fields significance-related biases may follow a complex pattern (e.g. Proteus phenomenon and occasional preference for “negative” results). Likelihood ratio estimates can be used to generate the post-test probability of bias, and correcting effect estimates for bias is possible in theory, but may not necessarily be reliable.     

Canonical analysis of longitudinal and repeated measures data with stationary weights

When measuring the same variables on different occasions, two procedures for canonical analysis with stationary compositing weights are developed. The first, SUMCOV, maximizes the sum of the covariances of the canonical variates subject to norming constraints. The second, COLLIN, maximizes the largest root of the covariances of the canonical variates subject to norming constraints. A characterization theorem establishes a model building approach. Both methods are extended to allow for Cohort Sequential Designs. Finally a numerical illustration utilizing Nesselroade and Baltes data is presented.The authors wish to thank John Nesselroade for permitting us to use the data whose analysis we present.     

Testing the significance of interobserver agreement measures in the presence of autocorrelation: The jackknife procedure

Users of interobserver agreement statistics have heretofore ignored the problem of autocorrelation in behavior sequences when testing the statistical significance of agreement measures. Due to autocorrelation traditional reliability tests based on the 2 × 2 contingency-table model (e.g., kappa, phi) are incorrect. Correct tests can be developed by using the bivariate time series as a statistical model. Seen from this perspective, testing the significance of interobserver agreement becomes formally equivalent to testing the significance of the lag-zero cross-correlation between two time series. The robust procedure known as the jackknife is suggested for this purpose.     

The significance of kierkegaard's interpretation of Don Giovanni in relation to Hegel's philosophy of art

Kant's Intuitionism: A Commentary on the Transcendental Aesthetic. Lorne Falkenstein. Toronto, University of Toronto Press, 1995. pp. xxiii + 465. £45–50. ISBN 0–8020–2973–6.     

Exact and resampling probability values for measures associated with ordered r by c contingency tables

Permutation procedures to compute exact and resampling probability values associated with measures of association for ordered r x c contingency tables are described. Comparisons with asymptotic probability values demonstrated that exact and resampling permutation procedures were preferred for sparse contingency tables.