Marginal distributions for the estimation of proportions inm groups

A Bayesian Model II approach to the estimation of proportions inm groups (discussed by Novick, Lewis, and Jackson) is extended to obtain posterior marginal distributions for the proportions. It is anticipated that these will be useful in applications (such as Individually Prescribed Instruction) where decisions are to be made separately for each proportion, rather than jointly for the set of proportions. In addition, the approach is extended to allow greater use of prior information than previously and the specification of this prior information is discussed.We are grateful to a reviewer for suggestions that made possible a more concise and complete presentation of our work.     

The estimation of proportions inm groups

In many applications, it is desirable to estimate binomial proportions inm groups where it is anticipated that these proportions are similar but not identical. Following a general approach due to Lindley, a Bayesian Model II aposteriori modal estimate is derived that estimates the inverse sine transform of each proportion by a weighted average of the inverse sine transform of the observed proportion in the individual group and the average of the estimated values. Comparison with a classical method due to Jackson spotlights some desirable features of Model II analyses. The simplicity of the present formulation makes it possible to study the behavior of the Bayesian Model II approach more closely than in more complex formulations. Also, it is possible to estimate the amount of gain afforded by the Model II analyses.     

Understanding bias in proportion production

The Stevens exponent (beta) can be obtained from proportion estimation judgments using the power model. In this article, the authors extend that model to proportion production, in which the relative magnitudes of 2 stimuli are adjusted to correspond to a numeric proportion (e.g., 1/4 or .25). The model predicts that when beta < 1, small proportions are underproduced, and large proportions are overproduced, but it predicts the reverse when beta > 1, which is the opposite of the predicted patterns for estimation. Eight participants estimated and produced magnitudes and proportions with spatial volume (beta < 1; Experiment 1) and color saturation (beta > 1; Experiment 2). The model's predictions were generally supported. An extension of the model using reference points can account for multicycle patterns shown by some participants.     

Posterior predictive checks can and should be Bayesian: Comment on Gelman and Shalizi, ‘Philosophy and the practice of Bayesian statistics’

Bayesian inference is conditional on the space of models assumed by the analyst. The posterior distribution indicates only which of the available parameter values are less bad than the others, without indicating whether the best available parameter values really fit the data well. A posterior predictive check is important to assess whether the posterior predictions of the least bad parameters are discrepant from the actual data in systematic ways. Gelman and Shalizi (2012a) assert that the posterior predictive check, whether done qualitatively or quantitatively, is non‐Bayesian. I suggest that the qualitative posterior predictive check might be Bayesian, and the quantitative posterior predictive check should be Bayesian. In particular, I show that the ‘Bayesian p‐value’, from which an analyst attempts to reject a model without recourse to an alternative model, is ambiguous and inconclusive. Instead, the posterior predictive check, whether qualitative or quantitative, should be consummated with Bayesian estimation of an expanded model. The conclusion agrees with Gelman and Shalizi regarding the importance of the posterior predictive check for breaking out of an initially assumed space of models. Philosophically, the conclusion allows the liberation to be completely Bayesian instead of relying on a non‐Bayesian deus ex machina. Practically, the conclusion cautions against use of the Bayesian p‐value in favour of direct model expansion and Bayesian evaluation.     

Fitting the psychometric function

A constrained generalized maximum likelihood routine for fitting psychometric functions is proposed, which determines optimum values for the complete parameter set--that is, threshold and slope--as well as for guessing and lapsing probability. The constraints are realized by Bayesian prior distributions for each of these parameters. The fit itself results from maximizing the posterior distribution of the parameter values by a multidimensional simplex method. We present results from extensive Monte Carlo simulations by which we can approximate bias and variability of the estimated parameters of simulated psychometric functions. Furthermore, we have tested the routine with data gathered in real sessions of psychophysical experimenting.     

The effects on the predictive variance of a new subject’s score for a new test

Abstract: It is often required to predict the scores or their variations under interest. Ishii and Watanabe (2001) investigated, in the context of psychological measurement, the Bayesian predictive distribution of a new subject’s scores for tests and subjects’ scores for a new test. In this paper, the Bayesian posterior predictive distribution of a new subject’s scores for a new parallel test were considered. And the effects of the number of subjects, the number of the tests, and the test reliability were investigated. Then, it was found that, under assumptions that (co)variance parameters are known, the predictive variance of a new subject’s score for a new test was equal to the predictive variances of the new subject’s scores for the existent tests. It was also found that the effect of the number of subjects was relatively large and the effect of the number of tests was relatively small, when a new subject’s scores for existent tests were not observed.     

Bias in proportion judgments: the cyclical power model

When participants make part-whole proportion judgments, systematic bias is commonly observed. In some studies, small proportions are overestimated and large proportions underestimated; in other studies, the reverse pattern occurs. Sometimes the bias pattern repeats cyclically with a higher frequency (e.g., overestimation of proportions less than .25 and between .5 and .75; underestimation otherwise). To account for the various bias patterns, a cyclical power model was derived from Stevens' power law. The model proposes that the amplitude of the bias pattern is determined by the Stevens exponent, beta (i.e., the stimulus continuum being judged), and that the frequency of the pattern is determined by a choice of intermediate reference points in the stimulus. When beta < 1, an over-then-under pattern is predicted; when beta > 1, the under-then-over pattern is predicted. Two experiments confirming the model's assumptions are described. A mixed-cycle version of the model is also proposed that predicts observed asymmetries in bias patterns when the set of reference points varies across trials.     

Extending exploratory diagnostic classification models: Inferring the effect of covariates

Diagnostic models provide a statistical framework for designing formative assessments by classifying student knowledge profiles according to a collection of fine-grained attributes. The context and ecosystem in which students learn may play an important role in skill mastery, and it is therefore important to develop methods for incorporating student covariates into diagnostic models. Including covariates may provide researchers and practitioners with the ability to evaluate novel interventions or understand the role of background knowledge in attribute mastery. Existing research is designed to include covariates in confirmatory diagnostic models, which are also known as restricted latent class models. We propose new methods for including covariates in exploratory RLCMs that jointly infer the latent structure and evaluate the role of covariates on performance and skill mastery. We present a novel Bayesian formulation and report a Markov chain Monte Carlo algorithm using a Metropolis-within-Gibbs algorithm for approximating the model parameter posterior distribution. We report Monte Carlo simulation evidence regarding the accuracy of our new methods and present results from an application that examines the role of student background knowledge on the mastery of a probability data set.     

A Bayesian Power Analysis Procedure Considering Uncertainty in Effect Size Estimates from a Meta-analysis

In conventional frequentist power analysis, one often uses an effect size estimate, treats it as if it were the true value, and ignores uncertainty in the effect size estimate for the analysis. The resulting sample sizes can vary dramatically depending on the chosen effect size value. To resolve the problem, we propose a hybrid Bayesian power analysis procedure that models uncertainty in the effect size estimates from a meta-analysis. We use observed effect sizes and prior distributions to obtain the posterior distribution of the effect size and model parameters. Then, we simulate effect sizes from the obtained posterior distribution. For each simulated effect size, we obtain a power value. With an estimated power distribution for a given sample size, we can estimate the probability of reaching a power level or higher and the expected power. With a range of planned sample sizes, we can generate a power assurance curve. Both the conventional frequentist and our Bayesian procedures were applied to conduct prospective power analyses for two meta-analysis examples (testing standardized mean differences in example 1 and Pearson's correlations in example 2). The advantages of our proposed procedure are demonstrated and discussed.     

Estimating correlations with missing data,a bayesian approach

The posterior distribution of the bivariate correlation is analytically derived given a data set wherex is completely observed buty is missing at random for a portion of the sample. Interval estimates of the correlation are then constructed from the posterior distribution in terms of highest density regions (HDRs). Various choices for the form of the prior distribution are explored. For each of these priors, the resulting Bayesian HDRs are compared with each other and with intervals derived from maximum likelihood theory.     

A bayesian alternative to least squares and equal weighting coefficients in regression

This paper details a Bayesian alternative to the use of least squares and equal weighting coefficients in regression. An equal weight prior distribution for the linear regression parameters is described with regard to the conditional normal regression model, and resulting posterior distributions for these parameters are detailed. Some interesting connections between this Bayesian procedure and several other methods for estimating optimal weighting coefficients are discussed. In addition, results are presented of a Monte Carlo investigation which compared the effectiveness of the Bayesian procedure relative to least squares, equal weight, ridge, and Bayesian exchangeability estimations.     

贝叶斯题组随机效应模型的必要性及影响因素

题组模型可以解决传统IRT模型由于题目间局部独立性假设违背时所导致的参数估计偏差。为探讨题组随机效应模型的适用范围, 采用Monte Carlo模拟研究, 分别使用2-PL贝叶斯题组随机效应模型(BTRM)和2-PL贝叶斯模型(BM)对数据进行拟合, 考虑了题组效应、题组长度、题目数量和局部独立题目比例的影响。结果显示：(1) BTRM不受题组效应和题组长度影响, BM对参数估计的误差随题组效应和题组长度增加而增加。(2) BTRM具有一定的普遍性, 且当题组效应大, 题组长, 题目数量大时使用该模型能减少估计误差, 但是当题目数量较小时, 两个模型得到的能力估计误差都较大。(3)当局部独立题目的比例较大时, 两种模型得到的参数估计差异不大。     

The meaning of malingering data: further applications of Bayes' theorem

A previous Behavioral Sciences and the Law article (Mossman & Hart, 1996) asserted that information from malingering tests is best conceptualized using Bayes' theorem, and that courts therefore deserve Bayesian interpretations when mental health professionals present evidence about malingering. Mossman and Hart gave several examples of estimated Bayesian posterior probabilities, but they did not systematically address how one constructs confidence intervals for these estimates. This article explains how the usually imperfect nature of humanly created diagnostic tests mandates Bayesian interpretations of test results, and describes methods for generating confidence intervals for posterior probabilities. Sample calculations show that Bayesian reasoning is quite feasible and would not require investigators to expend unusual efforts when constructing and validating malingering instruments. Bayesian interpretations most accurately capture what malingering tests do: provide information that alters one's beliefs about the likelihood of malingering.     

Bayesian estimation and model selection in ordered latent class models for polytomous items

In a latent class IRT model in which the latent classes are ordered on one dimension, the class specific response probabilities are subject to inequality constraints. The number of these inequality constraints increase dramatically with the number of response categories per item, if assumptions like monotonicity or double monotonicity of the cumulative category response functions are postulated. A Markov chain Monte Carlo method, the Gibbs sampler, can sample from the multivariate posterior distribution of the parameters under the constraints. Bayesian model selection can be done by posterior predictive checks and Bayes factors. A simulation study is done to evaluate results of the application of these methods to ordered latent class models in three realistic situations. Also, an example of the presented methods is given for existing data with polytomous items. It can be concluded that the Bayesian estimation procedure can handle the inequality constraints on the parameters very well. However, the application of Bayesian model selection methods requires more research.     

Simple use of BIC to Assess Model Selection Uncertainty: An Illustration using Mediation and Moderation Models

Abstract

The Bayesian information criterion (BIC) has been used sometimes in SEM, even adopting a frequentist approach. Using simple mediation and moderation models as examples, we form posterior probability distribution via using BIC, which we call the BIC posterior, to assess model selection uncertainty of a finite number of models. This is simple but rarely used. The posterior probability distribution can be used to form a credibility set of models and to incorporate prior probabilities for model comparisons and selections. This was validated by a large scale simulation and results showed that the approximation via the BIC posterior is very good except when both the sample sizes and magnitude of parameters are small. We applied the BIC posterior to a real data set, and it has the advantages of flexibility in incorporating prior, addressing overfitting problems, and giving a full picture of posterior distribution to assess model selection uncertainty.     

Confirmation and reduction: a Bayesian account

Various scientific theories stand in a reductive relation to each other. In a recent article, we have argued that a generalized version of the Nagel-Schaffner model (GNS) is the right account of this relation. In this article, we present a Bayesian analysis of how GNS impacts on confirmation. We formalize the relation between the reducing and the reduced theory before and after the reduction using Bayesian networks, and thereby show that, post-reduction, the two theories are confirmatory of each other. We then ask when a purported reduction should be accepted on epistemic grounds. To do so, we compare the prior and posterior probabilities of the conjunction of both theories before and after the reduction and ask how well each is confirmed by the available evidence.     

Bayesian model comparison of nonlinear structural equation models with missing continuous and ordinal categorical data

Missing data are very common in behavioural and psychological research. In this paper, we develop a Bayesian approach in the context of a general nonlinear structural equation model with missing continuous and ordinal categorical data. In the development, the missing data are treated as latent quantities, and provision for the incompleteness of the data is made by a hybrid algorithm that combines the Gibbs sampler and the Metropolis‐Hastings algorithm. We show by means of a simulation study that the Bayesian estimates are accurate. A Bayesian model comparison procedure based on the Bayes factor and path sampling is proposed. The required observations from the posterior distribution for computing the Bayes factor are simulated by the hybrid algorithm in Bayesian estimation. Our simulation results indicate that the correct model is selected more frequently when the incomplete records are used in the analysis than when they are ignored. The methodology is further illustrated with a real data set from a study concerned with an AIDS preventative intervention for Filipina sex workers.     

Bayesian item fit analysis for unidimensional item response theory models

Assessing item fit for unidimensional item response theory models for dichotomous items has always been an issue of enormous interest, but there exists no unanimously agreed item fit diagnostic for these models, and hence there is room for further investigation of the area. This paper employs the posterior predictive model‐checking method, a popular Bayesian model‐checking tool, to examine item fit for the above‐mentioned models. An item fit plot, comparing the observed and predicted proportion‐correct scores of examinees with different raw scores, is suggested. This paper also suggests how to obtain posterior predictive p‐values (which are natural Bayesian p‐values) for the item fit statistics of Orlando and Thissen that summarize numerically the information in the above‐mentioned item fit plots. A number of simulation studies and a real data application demonstrate the effectiveness of the suggested item fit diagnostics. The suggested techniques seem to have adequate power and reasonable Type I error rate, and psychometricians will find them promising.