Calibrated Bayes Factors Should Not Be Used: A Reply to Hoijtink,van Kooten,and Hulsker

Hoijtink, Kooten, and Hulsker (2016 Hoijtink, H., van Kooten, P., &; Hulsker, K. (2016). Why Bayesian psychologists should change the way they use the Bayes factor. Multivariate Behavioral Research, 51, 1--9. doi: 10.1080/00273171.2014.969364.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]) present a method for choosing the prior distribution for an analysis with Bayes factor that is based on controlling error rates, which they advocate as an alternative to our more subjective methods (Morey &; Rouder, 2014 Morey, R.D., &; Rouder, J.N. (2014). Bayesfactor: Computation of Bayes factors for common designs. R package version 0.9.9. Retrieved from http://CRAN.R-project.org/package=BayesFactor [Google Scholar]; Rouder, Speckman, Sun, Morey, &; Iverson, 2009 Rouder, J.N., Speckman, P.L., Sun, D., Morey, R.D., &; Iverson, G. (2009). Bayesian t-tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin and Review, 16, 225–237. doi: 10.3758/PBR.16.2.225[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]; Wagenmakers, Wetzels, Borsboom, &; van der Maas, 2011 Wagenmakers, E.-J., Wetzels, R., Borsboom, D., &; van der Maas, H. (2011). Why psychologists must change the way they analyze their data: The case of psi. A comment on Bem (2011). Journal of Personality and Social Psychology, 100, 426–432. doi: 10.1037/a0022790[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]). We show that the method they advocate amounts to a simple significance test, and that the resulting Bayes factors are not interpretable. Additionally, their method fails in common circumstances, and has the potential to yield arbitrarily high Type II error rates. After critiquing their method, we outline the position on subjectivity that underlies our advocacy of Bayes factors.     

Mechanisms of Behavioral and Affective Treatment Outcomes in a Cognitive Behavioral Intervention for Boys

Evidence for effective treatment for behavioral problems continues to grow, yet evidence about the effective mechanisms underlying those interventions has lagged behind. The Stop Now and Plan (SNAP) program is a multicomponent intervention for boys between 6 and 11. This study tested putative treatment mechanisms using data from 252 boys in a randomized controlled trial of SNAP versus treatment as usual. SNAP includes a 3 month group treatment period followed by individualized intervention, which persisted through the 15 month study period. Measures were administered in four waves: at baseline and at 3, 9 and 15 months after baseline. A hierarchical linear modeling strategy was used. SNAP was associated with improved problem-solving skills, prosocial behavior, emotion regulation skills, and reduced parental stress. Prosocial behavior, emotion regulation skills and reduced parental stress partially mediated improvements in child aggression. Improved emotion regulation skills partially mediated treatment-related child anxious-depressed outcomes. Improvements in parenting behaviors did not differ between treatment conditions. The results suggest that independent processes may drive affective and behavioral outcomes, with some specificity regarding the mechanisms related to differing treatment outcomes.     

Effects of treatment‐integrity failures on a response‐cost procedure

Effects of incorrect or partial implementation (poor treatment integrity) on response cost are largely unknown. We evaluated reduced treatment integrity during response cost on rates of 2 concurrently available responses. College students earned points by clicking on either a black circle or a red circle on a computer screen. Experiment 1 compared 2 types of treatment‐integrity failures (omission and commission errors) across 2 levels of integrity (20% and 50%). Compared to 100% integrity conditions, omission errors did not suppress responding to the same extent, and commission errors reduced target responding but also decreased rates of alternative behavior. Experiment 2 compared the effects of 20% and 50% omission errors within subjects. Implementation at 50% integrity adequately suppressed responding, but treatment effects were lost at 20% integrity. There may be a critical level at which response cost must be implemented to suppress responding, which has important implications for application.     

Does Providing Prompts During Retrieval Practice Improve Learning?

The purpose of this investigation was to identify ways to prompt retrieval practice to make recall even more effective at producing meaningful learning. In two experiments, subjects read educational texts and practiced retrieval across two periods. During prompted retrieval, subjects were cued to explain and describe concepts from the text, whereas during free recall, subjects recalled as much of the material from the text as they could. A reading control condition was also included. Learning was assessed using both verbatim and higher‐order questions 1 week later. Practicing retrieval improved learning relative to the control on both types of questions; however, whether subjects practiced free or prompted retrieval did not matter for learning. Subjects rated prompted retrieval as less enjoyable and interesting than the other retrieval conditions. Results demonstrate practicing retrieval promoted meaningful learning, and that subjects' initial retrieval success was highest when they used their own retrieval strategies during recall. Copyright © 2016 John Wiley & Sons, Ltd.     

Effect of motivation on academic fluency performance in survivors of pediatric medulloblastoma

It has been proposed previously that extrinsic motivation may enable survivors of childhood medulloblastoma to significantly improve aspects of neurocognitive performance. In healthy populations, enhanced motivation has been shown to promote academic fluency, a domain likely more relevant to the educational outcomes of pediatric medulloblastoma survivors than academic skill development. The present study investigates the effect of enhanced extrinsic motivation on fluent (i.e., accurate and efficient) academic performance in pediatric medulloblastoma survivors. Participants were 36 children, ages 7–18, who had completed treatment for medulloblastoma. Participants completed a neuropsychological battery that included administration of equivalent tasks on Forms A and B of the Woodcock-Johnson III Tests of Achievement. Half were randomly assigned to an incentive condition prior to the administration of Form B. Provision of a performance-based incentive resulted in statistically significant improvement, but not normalization of function, in performance on measures of academic fluency. No demographic, treatment-related, academic, neuropsychological, or self-perception variables predicted response to incentive. Findings suggest that academic performance of survivors may significantly improve under highly motivating conditions. In addition to implications for educational services, this finding raises the novel possibility that decreased motivation represents an inherent neuropsychological deficit in this population and provides a rationale for further investigation of factors affecting individual differences in motivational processes. Further, by examining effort in a context where effort is not inherently suspect, present findings also significantly contribute to the debate regarding the effects of effort and motivation on neuropsychological performance.     

Adult Attachment,Multidimensional Perfectionism,and the Alliances Among Counselor Supervisees

This study examined the moderating role that 2 dimensions of perfectionism (standards [adaptive] and discrepancy [maladaptive]) had on 2 dimensions of adult attachment (anxious and avoidant) and both client and supervisory alliances among 170 counselor supervisees. Anxious attachment and discrepancy were significant negative predictors of the client working alliance. A significant 3‐way interaction was found between anxious attachment, standards, and discrepancy on the supervisory working alliance. Implications from these findings are discussed.     

Effects of g‐Loading and Time Lag on Retesting in Job Selection

Understanding influence on and effects of retesting is important to the selection practitioner. This article examines retesting effects on a series of selection measures for mechanically related positions to extend research that has been conducted in more controlled environments. While validity was not significantly different on retesting, time between test attempts and score increase on Spatial Reasoning were positively related, indicating the possibility of learning effects. Lower score increases were found for highly g‐loaded measures, and individuals who showed an increase scored more highly on average on their first attempt. Men tended to increase their scores more than women. We close with a discussion of the practical implications of our findings and how to build on them with future research.