Estimating true score in the compound binomial error model

Several procedures have been proposed in the statistical literature for estimating simultaneously the mean of each ofk binomial populations. In terms of mental test theory, however, it is not clear that these procedures should be used when an item sampling model applies since the binomial error model is usually viewed as an oversimplification of the true situation. In this study we compare empirically several of these estimation techniques. Particular attention is given to situations where observations are generated according to a two-term approximation to the compound binomial distribution.The author would like to thank Shelley Niwa for writing the computer programs used in this study.The work upon which this publication is based was performed pursuant to Grant # NIE-G-76-0083 with the National Institute of Education, Department of Health, Education and Welfare. Points of view or opinions stated do not necessarily represent official NIE position or policy.     

MCMC estimation and some model-fit analysis of multidimensional IRT models

A Bayesian procedure to estimate the three-parameter normal ogive model and a generalization of the procedure to a model with multidimensional ability parameters are presented. The procedure is a generalization of a procedure by Albert (1992) for estimating the two-parameter normal ogive model. The procedure supports analyzing data from multiple populations and incomplete designs. It is shown that restrictions can be imposed on the factor matrix for testing specific hypotheses about the ability structure. The technique is illustrated using simulated and real data. The authors would like to thank Norman Verhelst for his valuable comments and ACT, CITO group and SweSAT for the use of their data.     

Temporal Interaction of Information and Belief

The temporal updating of an agent’s beliefs in response to a flow of information is modeled in a simple modal logic that, for every date t, contains a normal belief operator B _t and a non-normal information operator I _t which is analogous to the ‘only knowing’ operator discussed in the computer science literature. Soundness and completeness of the logic are proved and the relationship between the proposed logic, the AGM theory of belief revision and the notion of plausibility is discussed. A first draft of this paper was presented at the Workshop on Belief Change in Rational Agents: Perspectives from Artificial Intelligence, Philosophy and Economics, Dagstuhl (Germany), August 2005. Special Issue Formal Epistemology II. Edited by Branden Fitelson     

Combining versus analyzing multiple causes: how domain assumptions and task context affect integration rules

In everyday life, people typically observe fragments of causal networks. From this knowledge, people infer how novel combinations of causes they may never have observed together might behave. I report on 4 experiments that address the question of how people intuitively integrate multiple causes to predict a continuously varying effect. Most theories of causal induction in psychology and statistics assume a bias toward linearity and additivity. In contrast, these experiments show that people are sensitive to cues biasing various integration rules. Causes that refer to intensive quantities (e.g., taste) or to preferences (e.g., liking) bias people toward averaging the causal influences, whereas extensive quantities (e.g., strength of a drug) lead to a tendency to add. However, the knowledge underlying these processes is fallible and unstable. Therefore, people are easily influenced by additional task-related context factors. These additional factors include the way data are presented, the difficulty of the inference task, and transfer from previous tasks. The results of the experiments provide evidence for causal model and related theories, which postulate that domain-general representations of causal knowledge are influenced by abstract domain knowledge, data-driven task factors, and processing difficulty.     

The role of causal models in multiple judgments under uncertainty

Two studies examined a novel prediction of the causal Bayes net approach to judgments under uncertainty, namely that causal knowledge affects the interpretation of statistical evidence obtained over multiple observations. Participants estimated the conditional probability of an uncertain event (breast cancer) given information about the base rate, hit rate (probability of a positive mammogram given cancer) and false positive rate (probability of a positive mammogram in the absence of cancer). Conditional probability estimates were made after observing one or two positive mammograms. Participants exhibited a causal stability effect: there was a smaller increase in estimates of the probability of cancer over multiple positive mammograms when a causal explanation of false positives was provided. This was the case when the judgments were made by different participants (Experiment 1) or by the same participants (Experiment 2). These results show that identical patterns of observed events can lead to different estimates of event probability depending on beliefs about the generative causes of the observations.     

Why Bayesian Psychologists Should Change the Way They Use the Bayes Factor

The discussion following Bem’s (2011 Bem, D. J. (2011). Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology, 100, 407–425. doi: 10.1037/a0021524[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]) psi research highlights that applications of the Bayes factor in psychological research are not without problems. The first problem is the omission to translate subjective prior knowledge into subjective prior distributions. In the words of Savage (1961 Savage, L. J. (1961). The foundations of statistical inference reconsidered. In J. Neyman, (Ed.), Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, pp. 575–586. BerkeleyCA: University of California. [Google Scholar]): “they make the Bayesian omelet without breaking the Bayesian egg.” The second problem occurs if the Bayesian egg is not broken: the omission to choose default prior distributions such that the ensuing inferences are well calibrated. The third problem is the adherence to inadequate rules for the interpretation of the size of the Bayes factor. The current paper will elaborate these problems and show how to avoid them using the basic hypotheses and statistical model used in the first experiment described in Bem (2011 Bem, D. J. (2011). Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology, 100, 407–425. doi: 10.1037/a0021524[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]). It will be argued that a thorough investigation of these problems in the context of more encompassing hypotheses and statistical models is called for if Bayesian psychologists want to add a well-founded Bayes factor to the tool kit of psychological researchers.     

Calibrated Bayes Factors Should Not Be Used: A Reply to Hoijtink,van Kooten,and Hulsker

Hoijtink, Kooten, and Hulsker (2016 Hoijtink, H., van Kooten, P., &; Hulsker, K. (2016). Why Bayesian psychologists should change the way they use the Bayes factor. Multivariate Behavioral Research, 51, 1--9. doi: 10.1080/00273171.2014.969364.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]) present a method for choosing the prior distribution for an analysis with Bayes factor that is based on controlling error rates, which they advocate as an alternative to our more subjective methods (Morey &; Rouder, 2014 Morey, R.D., &; Rouder, J.N. (2014). Bayesfactor: Computation of Bayes factors for common designs. R package version 0.9.9. Retrieved from http://CRAN.R-project.org/package=BayesFactor [Google Scholar]; Rouder, Speckman, Sun, Morey, &; Iverson, 2009 Rouder, J.N., Speckman, P.L., Sun, D., Morey, R.D., &; Iverson, G. (2009). Bayesian t-tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin and Review, 16, 225–237. doi: 10.3758/PBR.16.2.225[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]; Wagenmakers, Wetzels, Borsboom, &; van der Maas, 2011 Wagenmakers, E.-J., Wetzels, R., Borsboom, D., &; van der Maas, H. (2011). Why psychologists must change the way they analyze their data: The case of psi. A comment on Bem (2011). Journal of Personality and Social Psychology, 100, 426–432. doi: 10.1037/a0022790[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]). We show that the method they advocate amounts to a simple significance test, and that the resulting Bayes factors are not interpretable. Additionally, their method fails in common circumstances, and has the potential to yield arbitrarily high Type II error rates. After critiquing their method, we outline the position on subjectivity that underlies our advocacy of Bayes factors.     

Evidence for Mixed Processes in Normal/Mirror Discrimination of Rotated Letters: A Bayesian Model Comparison Between Single- and Mixed-Distribution Models1

When determining whether a rotated letter is normal or mirrored, an observer mentally rotates the letter to its canonical orientation. To account for patterns of response times (RTs) for the normal/mirror discrimination of rotated letters, previous research formulated a model that postulated a mixture of trials with and without mental rotation. While this model could explain the curvilinear relationship that has been found between averaged RT and letter orientations, the curved RT function is still open to alternative explanations without assuming mixed processes. To address this issue and test the mixed-process hypothesis more directly, we analyzed trial-by-trial RT data instead of averaged RTs by employing a Bayesian model comparison technique. If rotation and non-rotation trials are mixed, trial-by-trial RTs for letters in a particular orientation should not follow a single distribution but a mixed one formed from the superposition of two separate distributions, one for rotation and one for non-rotation trials. In the present study, we compared single- and mixed-distribution models. Bayes-factor analysis showed decisive support for the mixed-distribution model over the single-distribution model. In addition, using the widely applicable information criterion (WAIC), the predictive accuracy of the mixed-distribution model was found to be as high as that of the single-distribution model. These results indicated the involvement of mixed processes in normal/mirror discrimination of rotated letters. The usefulness of statistical modeling in psychological study and necessary precautions to take in the interpretation of the parameters of unconfirmed models are also discussed.     

Naïve and Robust: Class‐Conditional Independence in Human Classification Learning

Humans excel in categorization. Yet from a computational standpoint, learning a novel probabilistic classification task involves severe computational challenges. The present paper investigates one way to address these challenges: assuming class‐conditional independence of features. This feature independence assumption simplifies the inference problem, allows for informed inferences about novel feature combinations, and performs robustly across different statistical environments. We designed a new Bayesian classification learning model (the dependence‐independence structure and category learning model, DISC‐LM) that incorporates varying degrees of prior belief in class‐conditional independence, learns whether or not independence holds, and adapts its behavior accordingly. Theoretical results from two simulation studies demonstrate that classification behavior can appear to start simple, yet adapt effectively to unexpected task structures. Two experiments—designed using optimal experimental design principles—were conducted with human learners. Classification decisions of the majority of participants were best accounted for by a version of the model with very high initial prior belief in class‐conditional independence, before adapting to the true environmental structure. Class‐conditional independence may be a strong and useful default assumption in category learning tasks.     

Bayesian Analysis of Nonlinear Structural Equation Models with Nonignorable Missing Data

A Bayesian approach is developed for analyzing nonlinear structural equation models with nonignorable missing data. The nonignorable missingness mechanism is specified by a logistic regression model. A hybrid algorithm that combines the Gibbs sampler and the Metropolis–Hastings algorithm is used to produce the joint Bayesian estimates of structural parameters, latent variables, parameters in the nonignorable missing model, as well as their standard errors estimates. A goodness-of-fit statistic for assessing the plausibility of the posited nonlinear structural equation model is introduced, and a procedure for computing the Bayes factor for model comparison is developed via path sampling. Results obtained with respect to different missing data models, and different prior inputs are compared via simulation studies. In particular, it is shown that in the presence of nonignorable missing data, results obtained by the proposed method with a nonignorable missing data model are significantly better than those that are obtained under the missing at random assumption. A real example is presented to illustrate the newly developed Bayesian methodologies. This research is fully supported by a grant (CUHK 4243/03H) from the Research Grant Council of the Hong Kong Special Administration Region. The authors are thankful to the editor and reviewers for valuable comments for improving the paper, and also to ICPSR and the relevant funding agency for allowing the use of the data. Requests for reprints should be sent to Professor S.Y. Lee, Department of Statistics, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong.