Dealing with the Conflicting Results of Psycholinguistic Experiments: How to Resolve Them with the Help of Statistical Meta-analysis

This paper proposes the use of the tools of statistical meta-analysis as a method of conflict resolution with respect to experiments in cognitive linguistics. With the help of statistical meta-analysis, the effect size of similar experiments can be compared, a well-founded and robust synthesis of the experimental data can be achieved, and possible causes of any divergence(s) in the outcomes can be revealed. This application of statistical meta-analysis offers a novel method of how diverging evidence can be dealt with. The workability of this idea is exemplified by a case study dealing with a series of experiments conducted as non-exact replications of Thibodeau and Boroditsky (PLoS ONE 6(2):e16782, 2011. https://doi.org/10.1371/journal.pone.0016782).     

Introduction to Bayesian Inference for Psychology

We introduce the fundamental tenets of Bayesian inference, which derive from two basic laws of probability theory. We cover the interpretation of probabilities, discrete and continuous versions of Bayes’ rule, parameter estimation, and model comparison. Using seven worked examples, we illustrate these principles and set up some of the technical background for the rest of this special issue of Psychonomic Bulletin & Review. Supplemental material is available via https://osf.io/wskex/.     

Bayesian inference for psychology,part III: Parameter estimation in nonstandard models

We demonstrate the use of three popular Bayesian software packages that enable researchers to estimate parameters in a broad class of models that are commonly used in psychological research. We focus on WinBUGS, JAGS, and Stan, and show how they can be interfaced from R and MATLAB. We illustrate the use of the packages through two fully worked examples; the examples involve a simple univariate linear regression and fitting a multinomial processing tree model to data from a classic false-memory experiment. We conclude with a comparison of the strengths and weaknesses of the packages. Our example code, data, and this text are available via https://osf.io/ucmaz/.     

The Regression Trap and Other Pitfalls of Replication Science—Illustrated by the Report of the Open Science Collaboration

The Open Science Collaboration’s 2015 Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349, aac4716. doi:10.1126/science.aac4716[Crossref], [PubMed], [Web of Science ®] , [Google Scholar] report suggests that replication effect sizes in psychology are modest. However, closer inspection reveals serious problems. When plotting replication effects are against original effects, the regression trap is lurking: Expecting replication effects to be equally strong as original effects is logically unwarranted; they are inevitably subject to regressive shrinkage. To control for regression, the reliability of original and replication studies must be taken into account. Further problems arise from missing manipulation checks and sampling biases. Our critical comment highlights the need for replication science to live up to the same methodological scrutiny as other research.     

Dutch taboo norms

This article provides norms for general taboo, personal taboo, insult, valence, and arousal for 672 Dutch words, including 202 taboo words. Norms were collected using a 7-point Likert scale and based on ratings by psychology students from the Erasmus University Rotterdam in The Netherlands. The sample consisted of 87 psychology students (58 females, 29 males). We obtained high reliability based on split-half analyses. Our norms show high correlations with arousal and valence ratings collected by another Dutch word-norms study (Moors et al.,, Behavior Research Methods, 45, 169–177, 2013). Our results show that the previously found quadratic relation (i.e., U-shaped pattern) between valence and arousal also holds when only taboo words are considered. Additionally, words rated high on taboo tended to be rated low on valence, but some words related to sex rated high on both taboo and valence. Words that rated high on taboo rated high on insult, again with the exception of words related to sex many of which rated low on insult. Finally, words rated high on taboo and insult rated high on arousal. The Dutch Taboo Norms (DTN) database is a useful tool for researchers interested in the effects of taboo words on cognitive processing. The data associated with this paper can be accessed via the Open Science Framework (https://osf.io/vk782/).     

Double Decomposition of Level-1 Variables in Multilevel Models: An Analysis of the Flynn Effect in the NSLY Data

This paper introduces an extension of cluster mean centering (also called group mean centering) for multilevel models, which we call “double decomposition (DD).” This centering method separates between-level variance, as in cluster mean centering, but also decomposes within-level variance of the same variable. This process retains the benefits of cluster mean centering but allows for context variables derived from lower level variables, other than the cluster mean, to be incorporated into the model. A brief simulation study is presented, demonstrating the potential advantage (or even necessity) for DD in certain circumstances. Several applications to multilevel analysis are discussed. Finally, an empirical demonstration examining the Flynn effect (Flynn, 1987 Flynn, J. R. (1987). Massive IQ gains in 14 nations: What IQ tests really measure. Psychological Bulletin, 101(2), 171. https://doi.org/10.1037/h0090408.[Crossref], [Web of Science ®] , [Google Scholar]), our motivating example, is presented. The use of DD in the analysis provides a novel method to narrow the field of plausible causal hypotheses regarding the Flynn effect, in line with suggestions by a number of researchers (Mingroni, 2014 Mingroni, M. A. (2014). Future efforts in Flynn effect research: Balancing reductionism with holism. Journal of Intelligence, 2(4), 122. https://doi.org/10.3390/jintelligence2040122.[Crossref] , [Google Scholar]; Rodgers, 2015 Rodgers, J. L. (2015). Methodological issues associated with studying the Flynn effect: Exploratory and confirmatory efforts in the past, present, and future. Journal of Intelligence, 3(4), 111. https://doi.org/10.3390/jintelligence3040111. [Crossref] , [Google Scholar]).     

Extending a Multilab Preregistered Replication of the Ego-Depletion Effect to a Research Program

We reanalyze the recent multilab preregistered study on ego-depletion by Hagger and Chatzisarantis (2016 Hagger, M. S., &; Chatzisarantis, N. L. D. (2016). A multilab preregistered replication of the ego-depletion effect. Perspectives on Psychological Science, 11, 546–573. doi:10.1177/1745691616652873[Crossref], [Web of Science ®] , [Google Scholar]) as if their data were obtained under the research program-strategy (Witte &; Zenker, 2016a Witte, E. H., &; Zenker, F. (2016a). Beyond schools—Reply to Marsman, Ly, and Wagenmakers. Basic and Applied Social Psychology, 38, 313–317. doi:10.1080/01973533.2016.1227710[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar], 2016b Witte, E. H., &; Zenker, F. (2016b). Reconstructing recent work on macro-social stress as a research program. Basic and Applied Social Psychology, 38, 301–307. doi:10.1080/01973533.2016.1207077[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]). This strengthens Hagger and Chatzisarantis’s (2016 Hagger, M. S., &; Chatzisarantis, N. L. D. (2016). A multilab preregistered replication of the ego-depletion effect. Perspectives on Psychological Science, 11, 546–573. doi:10.1177/1745691616652873[Crossref], [Web of Science ®] , [Google Scholar]) main conclusion, because our reanalysis more directly corroborates the absence of a medium-sized, or a small-sized, ego-depletion effect (d?=?.50 under α?=?β?=?.05; d?=?.20 under α?=?β?=?.01). We explain how a smaller ego-depletion effect of d?=?.04 can be tested under similar conditions, having determined this value by maximum likelihood estimation, and compare the research program-strategy to a standard meta-analytic integration.     

A visual familiarity account of evidence for orthographic processing in pigeons <Emphasis Type="Italic">(Columbia livia)</Emphasis>: a reply to Scarf,Corballis, Güntürkün,and Colombo (2017)

Scarf et al. (Proc Natl Acad Sci 113(40):11272–11276, 2016) demonstrated that pigeons, as with baboons (Grainger et al. in Science 336(6078):245–248, 2012; Ziegler in Psychol Sci.  https://doi.org/10.1177/0956797612474322, 2013), can be trained to display several behavioural hallmarks of human orthographic processing. But, Vokey and Jamieson (Psychol Sci 25(4):991–996, 2014) demonstrated that a standard, autoassociative neural network model of memory applied to pixel maps of the words and nonwords reproduces all of those results. In a subsequent report, Scarf et al. (Anim Cognit 20(5):999–1002, 2017) demonstrated that pigeons can reproduce one more marker of human orthographic processing: the ability to discriminate visually presented four-letter words from their mirror-reversed counterparts (e.g. “LEFT” vs. “ Open image in new window ”). The current report shows that the model of Vokey and Jamieson (2014) reproduces the results of Scarf et al. (2017) and reinforces the original argument: the recent results thought to support a conclusion of orthographic processing in pigeons and baboons are consistent with but do not force that conclusion.     

Interpolation in 16-Valued Trilattice Logics

In a recent paper we have defined an analytic tableau calculus \({{\mathbf {\mathsf{{PL}}}}}_{\mathbf {16}}\) for a functionally complete extension of Shramko and Wansing’s logic based on the trilattice \({SIXTEEN}_3\). This calculus makes it possible to define syntactic entailment relations that capture central semantic relations of the logic—such as the relations Open image in new window , Open image in new window , and Open image in new window that each correspond to a lattice order in \({SIXTEEN}_3\); and Open image in new window , the intersection of Open image in new window and Open image in new window . It turns out that our method of characterising these semantic relations—as intersections of auxiliary relations that can be captured with the help of a single calculus—lends itself well to proving interpolation. All entailment relations just mentioned have the interpolation property, not only when they are defined with respect to a functionally complete language, but also in a range of cases where less expressive languages are considered. For example, we will show that Open image in new window , when restricted to \(\mathcal {L}_{tf}\), the language originally considered by Shramko and Wansing, enjoys interpolation. This answers a question that was recently posed by M. Takano.     

A Review of Best Practice Recommendations for Text Analysis in R (and a User-Friendly App)

In recent decades, the amount of text available for organizational science research has grown tremendously. Despite the availability of text and advances in text analysis methods, many of these techniques remain largely segmented by discipline. Moreover, there is an increasing number of open-source tools (R, Python) for text analysis, yet these tools are not easily taken advantage of by social science researchers who likely have limited programming knowledge and exposure to computational methods. In this article, we compare quantitative and qualitative text analysis methods used across social sciences. We describe basic terminology and the overlooked, but critically important, steps in pre-processing raw text (e.g., selection of stop words; stemming). Next, we provide an exploratory analysis of open-ended responses from a prototypical survey dataset using topic modeling with R. We provide a list of best practice recommendations for text analysis focused on (1) hypothesis and question formation, (2) design and data collection, (3) data pre-processing, and (4) topic modeling. We also discuss the creation of scale scores for more traditional correlation and regression analyses. All the data are available in an online repository for the interested reader to practice with, along with a reference list for additional reading, an R markdown file, and an open source interactive topic model tool (topicApp; see https://github.com/wesslen/topicApp, https://github.com/wesslen/text-analysis-org-science, https://dataverse.unc.edu/dataset.xhtml?persistentId=doi:10.15139/S3/R4W7ZS).     

Digging Deeper: Why Metaphysics is More Than a Toolbox

Steven French (J Gen Philos Sci,  https://doi.org/10.1007/s10838-018-9401-8, 2018) proposes a vindication of “scientifically disinterested” metaphysics that leaves little room to its original ambitions. He claims that (1) as a discipline that looks to find out truths about the world, it is untenable; and that (2) rather, its vindication lies in its use as a “toolbox” of concepts for a philosophical discipline that does have a claim at getting us closer to truth—the philosophy of science, and more specifically of physics. I respond to both his main claims. The first claim, I argue, neglects what I call (with Ralf Busse) “archaeological” metaphysics, which tries to impose some order on phenomena by “digging deeper” from the less to the more fundamental, or from the less to the more abstract questions. The second claim imposes a hierarchical relation between metaphysics and philosophy of science which should, I argue, be replaced with a more egalitarian picture of philosophy.     

An Introduction to Kristof’s Theorem for Solving Least-Square Optimization Problems Without Calculus

Kristof’s Theorem (Kristof, 1970 Kristof, W. (1970). A theorem on the trace of certain matrix products and some applications. Journal of Mathematical Psychology, 7(3), 515–530. https://doi.org/10.1016/0022-2496(70)90037-4[Crossref], [Web of Science ®] , [Google Scholar]) describes a matrix trace inequality that can be used to solve a wide-class of least-square optimization problems without calculus. Considering its generality, it is surprising that Kristof’s Theorem is rarely used in statistics and psychometric applications. The underutilization of this method likely stems, in part, from the mathematical complexity of Kristof’s (1964 Kristof, W. (1964). Die beste orthogonale Transformation zur gegenseitigen Uberfiihrung zweier Faktorenmatrizen. Diagnostica, 10, 87–90. [Google Scholar], 1970 Kristof, W. (1970). A theorem on the trace of certain matrix products and some applications. Journal of Mathematical Psychology, 7(3), 515–530. https://doi.org/10.1016/0022-2496(70)90037-4[Crossref], [Web of Science ®] , [Google Scholar]) writings. In this article, I describe the underlying logic of Kristof’s Theorem in simple terms by reviewing four key mathematical ideas that are used in the theorem’s proof. I then show how Kristof’s Theorem can be used to provide novel derivations to two cognate models from statistics and psychometrics. This tutorial includes a glossary of technical terms and an online supplement with R (R Core Team, 2017 R Core Team (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/ [Google Scholar]) code to perform the calculations described in the text.     

Reconstructing Recent Work on Macrosocial Stress as a Research Program

We reconstruct recent work on macrosocial stress (Chou, Parmar, &; Galinsky, 2016 Chou, E. Y., Parmar, B. L., &; Galinsky, A. D. (2016). Economic insecurity increases physical pain. Psychological Science, 27, 434–454. doi:10.1177/0956797615625640[Crossref], [Web of Science ®] , [Google Scholar]) as if it were an instance of a research strategy that tests point-alternative hypotheses within a full-fledged research program. Because this strategy is free of various deficits that beset dominant strategies (e.g., meta-analysis, Bayes factor analysis), our article demonstrates one way in which the confidence crisis may be overcome.     

A Comparison of Methods for Creating Multiple Imputations of Nominal Variables

Many variables that are analyzed by social scientists are nominal in nature. When missing data occur on these variables, optimal recovery of the analysis model's parameters is a challenging endeavor. One of the most popular methods to deal with missing nominal data is multiple imputation (MI). This study evaluated the capabilities of five MI methods that can be used to treat incomplete nominal variables: multiple imputation with chained equations (MICE) using polytomous regression as the elementary imputation method; MICE based on classification and regression trees (CART); MICE based on nested logistic regressions; the ranking procedure described by Allison (2002 Allison, P. D. (2002). Missing data. Thousand Oaks, CA: Sage Publications. https://doi.org/10.4135/9780857020994.n4[Crossref] , [Google Scholar]); and a joint modeling approach based on the general location model. We first motivate our inquiry with an applied example and then present the results of a Monte Carlo simulation study that compared the performance of the five imputation methods under conditions of varying sample size, percentage of missing data, and number of nominal response categories. We found that MICE with polytomous regression was the strongest performer while the Allison (2002 Allison, P. D. (2002). Missing data. Thousand Oaks, CA: Sage Publications. https://doi.org/10.4135/9780857020994.n4[Crossref] , [Google Scholar]) ranking procedure and MICE with CART performed poorly in most conditions.     

Against Hamri’s argument for the ultimate ground of being

In this paper, I reply to Hamri's (Int J Philos Relig.  https://doi.org/10.1007/s11153-017-9625-2, 2017) new kind of cosmological argument for the ultimate ground of being by blocking the argument in more than one place.     

Occupational Therapy Students' Development of Therapeutic Communication Skills During a Service-Learning Experience

This qualitative study examined the effect community-based service-learning experiences had upon occupational therapy students' therapeutic communication skills and their application of theoretical concepts. After their semester-long course was completed, six students de-identified and voluntarily submitted their weekly reflective journals for retrospective analysis in accordance with Giorgi's strategies (1985 Giorgi , A. ( 1985 ). Phenomenology and psychological research . Pittsburgh , PA : Duquesne University Press . [Google Scholar], 1997 Giorgi , A. ( 1997 ). The theory, practice, and evaluation of the phenomenological method as a qualitative research procedure . Journal of Phenomenological Psychology , 28 , 235 – 261 .[Crossref] , [Google Scholar]). Two analysts identified eight themes: developing trust, establishing boundaries, communication, client-centered care, breakdown of preconceptions, increased self-awareness, power of pride, and power of occupation. Educators are urged to use service-learning experiences to facilitate student interpersonal skill development and integration of theoretical knowledge.     

Interactions between spontaneous instantiations to the basic level and post-event suggestions

Extensive research shows that post-event suggestions can distort the memory for a target event. In this study we examined the effect of such suggestions as they interact with the products of a spontaneous memory process: instantiation of abstract information to an intermediate level of abstractness, the basic level (Pansky & Koriat, 2004 Pansky, A. and Koriat, A. 2004. The basic-level convergence effect in memory distortions. Psychological Science, 15: 52–59. [Crossref] , [Google Scholar]). Participants read a narrative containing items presented at the superordinate level (e.g., FRUIT), were exposed to suggestions that referred to these items at the basic level (e.g., APPLE), and were finally asked to recall the original items. We found that the tendency to instantiate spontaneously in the control (non-misleading) condition, particularly over time, increased following exposure to suggestions that were likely to coincide with those instantiations. Exposure to such suggestions, either immediately or following a 24-hour delay, reduced subsequent correct recall of the original items only if the suggested information coincided with the information one tends to instantiate spontaneously in a given context. Suggestibility, in this case, was particularly pronounced and phenomenologically compelling in terms of remember/know judgements. The findings are taken to imply that effects of post-event suggestions can be understood in terms of the constructive processes that set the stage for their occurrence.     

Therapist Perceptions of Intimate Partner Violence: A Replication of Harway and Hansen's Study after More than a Decade

ABSTRACT

More than a decade ago, Hansen, Harway, and Cervantes (1991) Hansen, M., Harway, M. and Cervantes, N. 1991. Therapists' perceptions of severity in cases of family violence. Violence and Victims, 6: 225–235. [PubMed] , [Google Scholar] and Harway and Hansen (1993) Harway, M. and Hansen, M. 1993. “Therapist perceptions of family violence”. In Battering and family therapy: A feminist perspective, Edited by: Hansen, M. and Hansen, M. 42–53. Newbury Park, CA: Sage.  [Google Scholar] examined mental health providers' ability to accurately perceive violence within couples presenting for therapy and to intervene in a manner that reduces risk. Alarmingly, 40% of therapists sampled failed to identify the presence of intimate partner violence (IPV) and none predicted lethality. Over a decade later, the present replication of Harway and colleagues' study was completed with the expectation that today's therapists are better prepared to identify IPV within a clinical vignette. Results show that therapists have indeed improved their ability to identify IPV. However, only one therapist accurately predicted lethality in the present study. Implications concerning IPV training for therapists are discussed.     

Job insecurity climate's influence on employees' job attitudes: Evidence from two European countries

An important amount of literature about job insecurity and its consequences has been developed during the past few decades (Sverke, Hellgren, & Näswall, 2002 Sverke, M., Hellgren, J. and Näswall, K. 2002. No security: A meta-analysis and review of job insecurity and its consequences. Journal of Occupational Health Psychology, 7(3): 242–264. [Crossref], [PubMed] , [Google Scholar]). Most of this research has focused on an individual-analysis perspective, without taking into account social context. Although job insecurity climate has not been empirically examined, several authors have implicitly assumed that job insecurity contexts exist in some organizations where layoffs have occurred. Therefore, they examined layoff survivors' reactions. From this perspective, the aim of this study was to validate the job insecurity climate concept and examine its influence on employees' job attitudes. In order to provide additional support, this objective was examined in two samples: 428 employees belonging to 20 Spanish organizations and 550 employees in 18 Belgian organizations. The results showed that job insecurity climate emerged within the organizations in both samples, and that it influenced employees' job satisfaction and organizational commitment above and beyond individual perceptions of job insecurity in the Spanish sample. The implications and limitations of this study are discussed.     

Sexual harassment in the context of double male dominance

It has been suggested that numerically male-dominated workplaces propagate cultural norms that support sexual bravado, sexual posturing, and the denigration of feminine behaviour (Sbraga & O'Donohue, 2000 Sbraga, T. P. and O'Donohue, W. 2000. Sexual harassment. Annual Review of Sex Research, 11: 258–285. [Taylor & Francis Online] , [Google Scholar]). These cultural norms are features of normative male dominance, which have been shown to increase the risk of sexual harassment. This implies that the effect of numerical male dominance on sexual harassment may be mediated by the level of normative male dominance in the work environment. The aim of this study was to test this assumption. Our sample consisted of 1295 police women, who filled out an Internet questionnaire. The results suggest that normative male dominance indeed mediates the relation between numerical male dominance and sexual harassment. We add to the sexual harassment literature by building on Gruber's concept of double dominance. Our study helps to make clear why women are at greater risk of sexual harassment in work situations where men outnumber women than in more gender-balanced workplaces.