Bayesian statistical inference in psychology: comment on Trafimow (2003)

D. Trafimow (2003) presented an analysis of null hypothesis significance testing (NHST) using Bayes's theorem. Among other points, he concluded that NHST is logically invalid, but that logically valid Bayesian analyses are often not possible. The latter conclusion reflects a fundamental misunderstanding of the nature of Bayesian inference. This view needs correction, because Bayesian methods have an important role to play in many psychological problems where standard techniques are inadequate. This comment, with the help of a simple example, explains the usefulness of Bayesian inference for psychology.     

The High Cost of Complexity in Experimental Design and Data Analysis: Type I and Type II Error Rates in Multiway ANOVA

The availability of statistical software packages has led to a sharp increase in use of complex research designs and complex statistical analyses in communication research. An informal examination of studies from 2 leading communication journals suggests that the analysis of variance (ANOVA) is often the statistic of choice, and a substantial proportion of published research reports using ANOVA employ complex (k ≥ 3) factorial designs, often involving multiple dependent variables. This article reports a series of Monte Carlo simulations which demonstrate that this complexity may come at a heavier cost than many communication researchers realize. As frequently used, complex factorial ANOVA yield Type I and Type II error rates that many communication scholars would likely consider unacceptable. Consequently, quality of statistical inference in many studies is highly suspect. Communication researchers are warned about problems associated with design and statistical complexity and solutions are suggested.     

Visual aesthetic complexity, attractiveness and diversive exploration.

Ratings by 60 Ss showed a significant inverted U-shaped relationship between attractiveness and complexity of 15 paintings of human figures while rated interest tended to increase with complexity. Ss most often selected and least often rejected paintings rated highly attractive and moderately complex for further viewing.     

Can cognitive processes be inferred from neuroimaging data?

There is much interest currently in using functional neuroimaging techniques to understand better the nature of cognition. One particular practice that has become common is 'reverse inference', by which the engagement of a particular cognitive process is inferred from the activation of a particular brain region. Such inferences are not deductively valid, but can still provide some information. Using a Bayesian analysis of the BrainMap neuroimaging database, I characterize the amount of additional evidence in favor of the engagement of a cognitive process that can be offered by a reverse inference. Its usefulness is particularly limited by the selectivity of activation in the region of interest. I argue that cognitive neuroscientists should be circumspect in the use of reverse inference, particularly when selectivity of the region in question cannot be established or is known to be weak.     

Working memory contributions to reading comprehension components in middle childhood children

This study examined working memory contributions to reading comprehension subskills in Greek children (mean age 9 years, 1 month). The phonological loop of the Baddeley and Hitch working memory model was assessed with 3 recall tasks (words, nonwords, and digits) and a word list matching task. The central executive (CE) was assessed with 3 tasks (listening, counting, and backward digit recall). Participants were also given a receptive vocabulary task, a reading fluency task, and written stories accompanied by comprehension questions. Canonical correlation analyses showed that the comprehension variables were related to the CE rather than the phonological loop measures. CE functions were more strongly associated with elaborative inference generation (involving significant offline processing) and comprehension control (involving metacognitive monitoring). Smaller yet significant associations were observed between the CE and the necessary inference and literal comprehension measures, whereas a moderate relationship was found in the case of the simile comprehension variable. Among the CE variables, listening recall demonstrated the highest loading on the canonical function, followed by moderate yet significant counting and backward digit recall loadings. Vocabulary was found to fully mediate several associations between working memory and comprehension measures; however, the relationship between listening recall and elaborative inferences was partly mediated. Reading fluency and, on several occasions, Greek vocabulary knowledge did not mediate the relationships between CE measures and comprehension skills assessed. This study demonstrates the usefulness of CE measures for identifying young children's possible difficulties in carrying out specific reading comprehension processes.     

Parental Narratives About Genetic Testing for Hearing Loss: A One Year Follow Up Study

Few studies examine whether and how parental attitudes towards genetic testing change over time. In this study we interviewed parents of 14 children with newly identified hearing loss at two time points: after referral to genetics and 1 year later. Qualitative analyses of parental narratives indicate that parental attitudes did not change significantly over this time. Parents who perceived genetic testing to be useful continued to value it after testing, while parents who did not perceive it as being useful for their child’s future held the same view a year later. The only parents who changed their views regarding the usefulness of genetic testing for hearing loss were those who reported that their children underwent significant changes in their hearing loss or were faced with other life threatening conditions. Parents were also often unaware of the role of the genetic counselor and how genetic counseling could help address many of their lingering questions and concerns. These emergent themes indicate the need for geneticists and genetic counselors to be aware of and sensitized to the questions and attitudes that bring parents to a genetic evaluation, as well as the reasons why parents may not follow up with genetic testing for hearing loss when recommended.     

Utilizing occupational complexity levels in vocational interest assessments: Assessing interests for STEM areas

With an aim to improve vocational interest assessments geared toward the Science, Technology, Engineering, and Mathematics (STEM) areas, we developed a new assessment by incorporating occupational complexity levels. Occupations which correspond to Holland's realistic and investigative themes were identified together with their complexity levels using the information in the Dictionary of Holland Occupational Codes. How the required skills, abilities, and work activities differ in occupations with different complexity levels was identified using the Occupational Information Network. Two studies (N = 691 total) were conducted to assess reliability and validity of a measure of STEM Interest Complexity assessing the level of interests for STEM, which was developed based on the information obtained from these occupation databases. Construct validity was demonstrated with factor analyses and based on associations with traditional interest assessments, cognitive abilities, and theoretically-related constructs. Concurrent criterion-related validity was demonstrated based on associations with attachment to STEM, intentions to persist in and further pursue a STEM field, intentions to choose a complex occupation, college major satisfaction, and STEM-related GPA. Dominance analyses indicated that the relative contribution of the STEM Interest Complexity measure exceeded that of traditional interest assessments in the prediction of criteria. The usefulness of integrating a dimension of level and consulting occupation information databases in developing vocational interest assessments are discussed.     

Psychodynamic Psychotherapy Process Research: A quick ride through what you should know about process (research)

This paper gives an overview of psychoanalytic process research, which brings to light the complexity of psychotherapy sessions. This complexity is so rich that many instruments intended to make the process measurable failed in the past because they initially used a strategy of complexity reduction. This method however did not help to further our understanding of the complexity involved. Three former presidents of the Society for Psychotherapy Research (Stiles, Hill, Elliot 2015) decided to solve the following equivalence paradox: many therapies work successfully though they all follow different theories, produce a heterogeneity of processes and often enough a misfit between what theories maintain to be a good process and their realization in the treatment room. As theories are often compared to maps, a driver would wonder which route to take. But therapists‐as‐drivers go undaunted ‐ and nevertheless achieve their goals together with their patients. Norcross and Wampold (2018) found that good therapists invent new therapies with every new patient. They hypothesize that it is because of conversation. This is in accord with the 3 former presidents’ proposal to return to detailed single case analyses including the micro‐analytic power of conversation analysis of which a few insights are outlined in this article.     

The complexity of developmental predictions from dual process models

Drawing developmental predictions from dual-process theories is more complex than is commonly realized. Overly simplified predictions drawn from such models may lead to premature rejection of the dual process approach as one of many tools for understanding cognitive development. Misleading predictions can be avoided by paying attention to several cautions about the complexity of developmental extrapolations. The complexity of developmental predictions follows from the fact that overall normative responding at a given age derives from several different mental characteristics: (1) the developmental course of Type 1 processing, (2) the developmental course of Type 2 processing, (3) the acquisition of mindware usable by Type 1 processing, (4) the acquisition of mindware usable by Type 2 processing, and (5) the practicing of the mindware available to Type 2 processing to the extent that it is available to be processed in an autonomous manner. The complexity of all these interacting processes and sources of information can sometimes result in U-shaped developmental functions on some heuristics and biases tasks, making younger children look like they are responding more optimally than older children. This is particularly true when the youngest groups are ill-equipped to even understand the task and thus respond randomly. A final caution concerns terminology: The terms normative or rational should be reserved for responses and not attributed to subpersonal processes.     

Are developmental disorders like cases of adult brain damage? Implications from connectionist modelling

It is often assumed that similar domain-specific behavioural impairments found in cases of adult brain damage and developmental disorders correspond to similar underlying causes, and can serve as convergent evidence for the modular structure of the normal adult cognitive system. We argue that this correspondence is contingent on an unsupported assumption that atypical development can produce selective deficits while the rest of the system develops normally (Residual Normality), and that this assumption tends to bias data collection in the field. Based on a review of connectionist models of acquired and developmental disorders in the domains of reading and past tense, as well as on new simulations, we explore the computational viability of Residual Normality and the potential role of development in producing behavioural deficits. Simulations demonstrate that damage to a developmental model can produce very different effects depending on whether it occurs prior to or following the training process. Because developmental disorders typically involve damage prior to learning, we conclude that the developmental process is a key component of the explanation of endstate impairments in such disorders. Further simulations demonstrate that in simple connectionist learning systems, the assumption of Residual Normality is undermined by processes of compensation or alteration elsewhere in the system. We outline the precise computational conditions required for Residual Normality to hold in development, and suggest that in many cases it is an unlikely hypothesis. We conclude that in developmental disorders, inferences from behavioural deficits to underlying structure crucially depend on developmental conditions, and that the process of ontogenetic development cannot be ignored in constructing models of developmental disorders.     

Is the past prologue for some more than others? The hobo syndrome and job complexity

The current study examines the relationship between an individual's history of changing jobs and future turnover (the so-called “hobo syndrome”). Relying on self-consistency theory, it was hypothesized that the relationship between job mobility history and turnover is moderated by job complexity. Using a sample of 393 employees from two healthcare organizations, multiple methods were used to assess the variables of interest. Job mobility history was assessed with a biodata questionnaire collected before employees were hired. Job complexity was measured objectively by a job complexity index calculated from O*NET data. Turnover was assessed with actual turnover data collected over an 18-month post-hire period. Consistent with our hypothesis, results using event history analyses revealed that previous job changes were positively related to turnover likelihood. Additionally, job complexity moderated the relationship between previous job changes and turnover likelihood, such that previous job changes were more positively related to turnover in complex jobs. Implications for future research and practice are discussed.