Calibrated Bayes Factors Should Not Be Used: A Reply to Hoijtink,van Kooten,and Hulsker

Hoijtink, Kooten, and Hulsker (2016 Hoijtink, H., van Kooten, P., &; Hulsker, K. (2016). Why Bayesian psychologists should change the way they use the Bayes factor. Multivariate Behavioral Research, 51, 1--9. doi: 10.1080/00273171.2014.969364.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]) present a method for choosing the prior distribution for an analysis with Bayes factor that is based on controlling error rates, which they advocate as an alternative to our more subjective methods (Morey &; Rouder, 2014 Morey, R.D., &; Rouder, J.N. (2014). Bayesfactor: Computation of Bayes factors for common designs. R package version 0.9.9. Retrieved from http://CRAN.R-project.org/package=BayesFactor [Google Scholar]; Rouder, Speckman, Sun, Morey, &; Iverson, 2009 Rouder, J.N., Speckman, P.L., Sun, D., Morey, R.D., &; Iverson, G. (2009). Bayesian t-tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin and Review, 16, 225–237. doi: 10.3758/PBR.16.2.225[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]; Wagenmakers, Wetzels, Borsboom, &; van der Maas, 2011 Wagenmakers, E.-J., Wetzels, R., Borsboom, D., &; van der Maas, H. (2011). Why psychologists must change the way they analyze their data: The case of psi. A comment on Bem (2011). Journal of Personality and Social Psychology, 100, 426–432. doi: 10.1037/a0022790[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]). We show that the method they advocate amounts to a simple significance test, and that the resulting Bayes factors are not interpretable. Additionally, their method fails in common circumstances, and has the potential to yield arbitrarily high Type II error rates. After critiquing their method, we outline the position on subjectivity that underlies our advocacy of Bayes factors.     

Exploring Factor Model Parameters across Continuous Variables with Local Structural Equation Models

Using an empirical data set, we investigated variation in factor model parameters across a continuous moderator variable and demonstrated three modeling approaches: multiple-group mean and covariance structure (MGMCS) analyses, local structural equation modeling (LSEM), and moderated factor analysis (MFA). We focused on how to study variation in factor model parameters as a function of continuous variables such as age, socioeconomic status, ability levels, acculturation, and so forth. Specifically, we formalized the LSEM approach in detail as compared with previous work and investigated its statistical properties with an analytical derivation and a simulation study. We also provide code for the easy implementation of LSEM. The illustration of methods was based on cross-sectional cognitive ability data from individuals ranging in age from 4 to 23 years. Variations in factor loadings across age were examined with regard to the age differentiation hypothesis. LSEM and MFA converged with respect to the conclusions. When there was a broad age range within groups and varying relations between the indicator variables and the common factor across age, MGMCS produced distorted parameter estimates. We discuss the pros of LSEM compared with MFA and recommend using the two tools as complementary approaches for investigating moderation in factor model parameters.     

Using Marginal Structural Modeling for Grade Retention Effects

Vandecandelaere, Vansteelandt, De Fraine, and Van Damme (this issue) described marginal structural modeling (MSM) and used it to estimate the effects of a time-varying intervention, retention (holding back) in school grades, on students' math achievement. This commentary supplements Vandecandelaere et al. (this issue) and discusses several topics in retention studies and MSM. First, we discuss the importance of equating time-varying confounders in retention studies. Second, we discuss same-grade and same-age comparisons in retention studies. Third, we discuss one important section in the authors' overview of MSM: why standard methods (e.g., ANCOVA, propensity score analysis) cannot properly adjust for time-varying confounders. Finally, using the grade retention analyses in Vandecandelaere et al. (this issue) as an example, we provide our insights on four aspects of MSM: (a) covariate selection, (b) estimation of weights, (c) evaluation of balance properties, and (d) missing data handling.     

Gaussian variational estimation for multidimensional item response theory

Multidimensional item response theory (MIRT) is widely used in assessment and evaluation of educational and psychological tests. It models the individual response patterns by specifying a functional relationship between individuals' multiple latent traits and their responses to test items. One major challenge in parameter estimation in MIRT is that the likelihood involves intractable multidimensional integrals due to the latent variable structure. Various methods have been proposed that involve either direct numerical approximations to the integrals or Monte Carlo simulations. However, these methods are known to be computationally demanding in high dimensions and rely on sampling data points from a posterior distribution. We propose a new Gaussian variational expectation--maximization (GVEM) algorithm which adopts variational inference to approximate the intractable marginal likelihood by a computationally feasible lower bound. In addition, the proposed algorithm can be applied to assess the dimensionality of the latent traits in an exploratory analysis. Simulation studies are conducted to demonstrate the computational efficiency and estimation precision of the new GVEM algorithm compared to the popular alternative Metropolis–Hastings Robbins–Monro algorithm. In addition, theoretical results are presented to establish the consistency of the estimator from the new GVEM algorithm.     

Not Simply in the Eye of the Beholder: Authenticity as a Product of Candidate Preference and Unfettered Speech

Authenticity has emerged over recent decades as a prominent theme in both the press and in political research—and peaked in the 2016 presidential contest that pitted Donald Trump against Hillary Clinton, Bernie Sanders, Marco Rubio, and Ted Cruz. In this context, we attempted to answer the question: How do voters judge a presidential candidate's authenticity? Here we use motivated reasoning and correspondent inference theory as theoretical frameworks to examine how partisan preference combines with perceptions of unfettered speech and strategic impression management to influence voter judgments of a candidate's authenticity. An online survey of 525 respondents demonstrated that individuals' partisan preferences influenced both judgments of a candidate's authenticity and their perceptions of behaviors signifying authenticity (use of unfettered speech versus strategic impression management). These behavioral signals partially mediated the relation between candidate preferences and authenticity judgments. Moreover, voters, given their partisan preferences, differentially weighted candidates' use of unfettered speech and strategic impression management tactics in their judgments of authenticity. Finally, unfiltered/politically incorrect speech was found to have both positive and negative effects on authenticity judgments. Findings further elucidate the nature of authenticity as perceived in others and identify intermediary variables and boundary conditions that influence those perceptions.     

Early motor development and cognitive abilities among Mexican preschoolers

Psychomotricity plays a very important role in children’s development, especially for learning involving reading–writing and mathematical calculations. Evaluate motor development in children 3 years old and its relationship with their cognitive abilities at the age of 5 years. Based on a cohort study, we analyzed the information about motor performance evaluated at 3 years old by Peabody Motor Scale and cognitive abilities at 5 years old. The association was estimated using linear regression models adjusted by mother’s intelligence quotient, sex, Bayley mental development index at 18 months, and quality of the environment at home (HOME scale). 148 children whose motor performance was determined at age 3 and was evaluated later at age 5 to determine their cognitive abilities. Cognitive abilities (verbal, quantitative, and memory) measured by McCarthy Scales. Significant positive associations were observed between stationary balance at age 3 with verbal abilities (β = 0.67, p = .04) and memory (β = 0.81, p = .02) at 5 years. Grasping and visual-motor integration were significant and positively associated with quantitative abilities (β = 0.74, p = .005; β = 0.61, p = .01) and memory (β = 2.11, p = .001; β = 1.74, p = .004). The results suggest that early motor performance contributes to the establishment of cognitive abilities at 5 years. Evaluation and early motor stimulation before the child is faced with formal learning likely helps to create neuronal networks that facilitate the acquisition of academic knowledge.     

Naïve and Robust: Class‐Conditional Independence in Human Classification Learning

Humans excel in categorization. Yet from a computational standpoint, learning a novel probabilistic classification task involves severe computational challenges. The present paper investigates one way to address these challenges: assuming class‐conditional independence of features. This feature independence assumption simplifies the inference problem, allows for informed inferences about novel feature combinations, and performs robustly across different statistical environments. We designed a new Bayesian classification learning model (the dependence‐independence structure and category learning model, DISC‐LM) that incorporates varying degrees of prior belief in class‐conditional independence, learns whether or not independence holds, and adapts its behavior accordingly. Theoretical results from two simulation studies demonstrate that classification behavior can appear to start simple, yet adapt effectively to unexpected task structures. Two experiments—designed using optimal experimental design principles—were conducted with human learners. Classification decisions of the majority of participants were best accounted for by a version of the model with very high initial prior belief in class‐conditional independence, before adapting to the true environmental structure. Class‐conditional independence may be a strong and useful default assumption in category learning tasks.     

Of BOLD Claims and Excessive Fears: A Call for Caution and Patience Regarding Political Neuroscience

Published and informal assessments of the prospects for neuroimaging in political science have tended to range from overexuberant to reflexively dismissive. We seek to present a cautious but fair middle ground in considering this new methodology, primarily from an epistemological perspective. Our examination centers on the relationship between two levels of analysis, focusing on the potential for connection between behavior‐based theories of political psychology and cognition and the neural processes and systems involved in generating behaviors and states of mind. We explore the place of each level of analysis on its own, as well as the potential for the fruitful interaction of the two. This analysis brings together opinions and ideas presented by others in various forums and across multiple disciplines, offers a discussion of the the promises and perils of neuroimaging in its application to social science, as well as some practical thoughts regarding its early‐stage incorporation into political psychology. We argue in favor of proceeding with more substantial incorporation of neuroimaging into political psychology's methodological arsenal, but note that this will initially require both (1) greater acceptance of work more focused on presenting empirical results than on providing dispositive evidence in broader theoretical debates and (2) a commitment on the part of those conducting this research to refrain from overstating the definitiveness of its theoretical implications.     

Cheapened altruism: Discounting personally affected prosocial actors

Are charitable donors always perceived as charitable? Three studies suggest that although having a personal connection to a cause motivates much charitable giving, donors who have been personally affected by the target cause are given less “credit” for their donations, i.e., are perceived as less intrinsically charitable. These donors are perceived as having selfish motivations even when they have nothing economic or social to gain from the donation. More specifically, personally-affected donors are perceived as driven by emotional selfishness, or a desire to improve their own hedonic state rather a desire to improve the welfare of others, which lessens the charitable credit that they receive. In addition, although donors who have been personally affected by the target cause are seen as less charitable, they are perceived more favorably in other ways (e.g., more loyal).