Calibrated Bayes Factors Should Not Be Used: A Reply to Hoijtink,van Kooten,and Hulsker

Hoijtink, Kooten, and Hulsker (2016 Hoijtink, H., van Kooten, P., &; Hulsker, K. (2016). Why Bayesian psychologists should change the way they use the Bayes factor. Multivariate Behavioral Research, 51, 1--9. doi: 10.1080/00273171.2014.969364.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]) present a method for choosing the prior distribution for an analysis with Bayes factor that is based on controlling error rates, which they advocate as an alternative to our more subjective methods (Morey &; Rouder, 2014 Morey, R.D., &; Rouder, J.N. (2014). Bayesfactor: Computation of Bayes factors for common designs. R package version 0.9.9. Retrieved from http://CRAN.R-project.org/package=BayesFactor [Google Scholar]; Rouder, Speckman, Sun, Morey, &; Iverson, 2009 Rouder, J.N., Speckman, P.L., Sun, D., Morey, R.D., &; Iverson, G. (2009). Bayesian t-tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin and Review, 16, 225–237. doi: 10.3758/PBR.16.2.225[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]; Wagenmakers, Wetzels, Borsboom, &; van der Maas, 2011 Wagenmakers, E.-J., Wetzels, R., Borsboom, D., &; van der Maas, H. (2011). Why psychologists must change the way they analyze their data: The case of psi. A comment on Bem (2011). Journal of Personality and Social Psychology, 100, 426–432. doi: 10.1037/a0022790[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]). We show that the method they advocate amounts to a simple significance test, and that the resulting Bayes factors are not interpretable. Additionally, their method fails in common circumstances, and has the potential to yield arbitrarily high Type II error rates. After critiquing their method, we outline the position on subjectivity that underlies our advocacy of Bayes factors.     

Higher-Order Contingentism,Part 1: Closure and Generation

This paper is a study of higher-order contingentism – the view, roughly, that it is contingent what properties and propositions there are. We explore the motivations for this view and various ways in which it might be developed, synthesizing and expanding on work by Kit Fine, Robert Stalnaker, and Timothy Williamson. Special attention is paid to the question of whether the view makes sense by its own lights, or whether articulating the view requires drawing distinctions among possibilities that, according to the view itself, do not exist to be drawn. The paper begins with a non-technical exposition of the main ideas and technical results, which can be read on its own. This exposition is followed by a formal investigation of higher-order contingentism, in which the tools of variable-domain intensional model theory are used to articulate various versions of the view, understood as theories formulated in a higher-order modal language. Our overall assessment is mixed: higher-order contingentism can be fleshed out into an elegant systematic theory, but perhaps only at the cost of abandoning some of its original motivations.     

Mechanisms of Behavioral and Affective Treatment Outcomes in a Cognitive Behavioral Intervention for Boys

Evidence for effective treatment for behavioral problems continues to grow, yet evidence about the effective mechanisms underlying those interventions has lagged behind. The Stop Now and Plan (SNAP) program is a multicomponent intervention for boys between 6 and 11. This study tested putative treatment mechanisms using data from 252 boys in a randomized controlled trial of SNAP versus treatment as usual. SNAP includes a 3 month group treatment period followed by individualized intervention, which persisted through the 15 month study period. Measures were administered in four waves: at baseline and at 3, 9 and 15 months after baseline. A hierarchical linear modeling strategy was used. SNAP was associated with improved problem-solving skills, prosocial behavior, emotion regulation skills, and reduced parental stress. Prosocial behavior, emotion regulation skills and reduced parental stress partially mediated improvements in child aggression. Improved emotion regulation skills partially mediated treatment-related child anxious-depressed outcomes. Improvements in parenting behaviors did not differ between treatment conditions. The results suggest that independent processes may drive affective and behavioral outcomes, with some specificity regarding the mechanisms related to differing treatment outcomes.     

Effects of treatment‐integrity failures on a response‐cost procedure

Effects of incorrect or partial implementation (poor treatment integrity) on response cost are largely unknown. We evaluated reduced treatment integrity during response cost on rates of 2 concurrently available responses. College students earned points by clicking on either a black circle or a red circle on a computer screen. Experiment 1 compared 2 types of treatment‐integrity failures (omission and commission errors) across 2 levels of integrity (20% and 50%). Compared to 100% integrity conditions, omission errors did not suppress responding to the same extent, and commission errors reduced target responding but also decreased rates of alternative behavior. Experiment 2 compared the effects of 20% and 50% omission errors within subjects. Implementation at 50% integrity adequately suppressed responding, but treatment effects were lost at 20% integrity. There may be a critical level at which response cost must be implemented to suppress responding, which has important implications for application.     

Using a Computational Model to Understand Possible Sources of Skews in Distributions of Job Performance

The typical assumption that performance is distributed normally has come under question in recent years (e.g., O'Boyle & Aguinis, 2012). This paper uses a dynamic, computational model of performance‐as‐results to examine possible sources of such distributions. That is, building off the classic model of job performance (Campbell & Pritchard, 1976), components of a dynamic model are examined in 4 separate experiments using Monte Carlo simulations. The experiments indicate that positively skewed distributions can arise from pure luck, multiplicative combinations of factors where 1 of those factors has a zero origin, Matthew effects associated with learning, and feedback effects of performance on resource allocation policies by external agents. The results are discussed in terms of explanations for positively skewed performance distributions and the use and expansion of the computational model for examining dynamic performance more generally.     

Does Providing Prompts During Retrieval Practice Improve Learning?

The purpose of this investigation was to identify ways to prompt retrieval practice to make recall even more effective at producing meaningful learning. In two experiments, subjects read educational texts and practiced retrieval across two periods. During prompted retrieval, subjects were cued to explain and describe concepts from the text, whereas during free recall, subjects recalled as much of the material from the text as they could. A reading control condition was also included. Learning was assessed using both verbatim and higher‐order questions 1 week later. Practicing retrieval improved learning relative to the control on both types of questions; however, whether subjects practiced free or prompted retrieval did not matter for learning. Subjects rated prompted retrieval as less enjoyable and interesting than the other retrieval conditions. Results demonstrate practicing retrieval promoted meaningful learning, and that subjects' initial retrieval success was highest when they used their own retrieval strategies during recall. Copyright © 2016 John Wiley & Sons, Ltd.     

Adult Attachment,Multidimensional Perfectionism,and the Alliances Among Counselor Supervisees

This study examined the moderating role that 2 dimensions of perfectionism (standards [adaptive] and discrepancy [maladaptive]) had on 2 dimensions of adult attachment (anxious and avoidant) and both client and supervisory alliances among 170 counselor supervisees. Anxious attachment and discrepancy were significant negative predictors of the client working alliance. A significant 3‐way interaction was found between anxious attachment, standards, and discrepancy on the supervisory working alliance. Implications from these findings are discussed.