Methods for addressing publication bias in school psychology journals: A descriptive review of meta-analyses from 1980 to 2019

Although meta-analyses are often used to inform practitioners and researchers, the resulting effect sizes can be artificially inflated due to publication bias. There are a number of methods to protect against, detect, and correct for publication bias. Currently, it is unknown to what extent scholars publishing meta-analyses within school psychology journals use these methods to address publication bias and whether more recently published meta-analyses more frequently utilize these methods. A historical review of every meta-analysis published to date within the most prominent school psychology journals (N = 10) revealed that 88 meta-analyses were published from 1980 to early 2019. Exactly half of them included grey literature, and 60% utilized methods to detect and correct for publication bias. The most common methods were visual analysis of a funnel plot, Orwin's failsafe N, Egger's regression, and the trim and fill procedure. None of these methods were used in more than 20% of the studies. About half of the studies incorporated one method, 20% incorporated two methods, 7% incorporated three methods, and none incorporated all four methods. These methods were most evident in studies published recently. Similar to other fields, the true estimates of effects from meta-analyses published in school psychology journals may not be available, and practitioners may be utilizing interventions that are, in fact, not as strong as believed. Practitioners, researchers employing meta-analysis techniques, education programs, and editors and peer reviewers in school psychology should continue to guard against publication bias using these methods.     

Publication patterns in developmental psychology: Trends and social networks

Interest in publication patterns has been steady. Journals have instituted policies in an effort to curb bias and provide globally representative research. This study aimed to examine if publication patterns were present in two developmental psychology journals. It also explored the social networks of prominent authors and the prevalence of informal author–editor relationships, searching for any potential power groups. Data were taken from empirical articles published between 2005 and 2014 in Child Development (CD) and The International Journal of Early Childhood (IJEC) data points were geographical authorship affiliation, informal author relationships as established by co‐publishing, and connections to journal editors via identical affiliation. Results confirmed the previously established North American dominance in published research. In CD a strongly interlinked social network was identified between authors over the 10 years, with 15 chief influentialists binding groups of authors together. Results suggest that patterns are still present in published research in the realm of developmental psychology. To conclude, the potential implications of these patterns within developmental psychology are presented.     

Recovering Bartlett's social psychology of cultural dynamics

Despite the current treatment of Frederic C. Bartlett as a cognitive psychologist, his psychology was fundamentally socio‐cultural. More than half a century after the publication of Psychology and Primitive Culture (1923) and Remembering: A Study in Experimental and Social Psychology (1932), his theoretical contribution has contemporary implications for a social psychology that takes culture seriously. This paper seeks to recover and appraise Bartlett's social psychology in light of the recent literature on culture and psychology, and discusses his relevance for a social psychology of cultural dynamics. Copyright © 2000 John Wiley & Sons, Ltd.     

Latinos and latinas in community psychology: A review of the literature

Latinos and Latinas are one of the largest minority groups in the United States yet they remain a silent group. The article begins by considering the social, historical, and economic backgrounds of Latino and Latina groups in the U.S. as a context for understanding diversity and as a resource for explaining the participation of this minority group in community psychology. This article then examines the extent to which Latinos and Latinas are present and participate in community psychology throughout its publications and explores the nature of their participation through a thematic analysis of the published literature. A content analysis was conducted on 1,851 articles published from 1973 through 1992 in theAmerican Journal of Community Psychology and theJournal of Community Psychology to evaluate the presence of Latinos and Latinas in the community psychology literature. The quantitative analysis revealed that 3.7% (n=69) of all the articles reviewed focused on Latinos and Latinas or had samples with at least 15% of the participants coming from this ethnic group. Mexican Americans/Chicanos were the focus of the largest number of articles about specific Latino and Latina subgroups, followed by Cubans and Puerto Ricans. The results indicated less attention to Latino and Latina populations in the community psychology literature than would be expected from the field's goals. However, a qualitative review of the articles was more encouraging since many articles by and about Latinos and Latinas reflected values consonant with the field such as cultural pluralism or employed conceptual tools of the discipline such as empowerment or ecological approaches. Nevertheless, some of the articles reviewed also lacked a concern for the field's values in relation to Latinos and Latinas as a group. Finally, suggestions and strategies to empower this group are offered.     

出版视阈中的民国时期中国心理学发展史考察——基于民国时期心理学图书的计量分析

中国科学心理学是在20世纪初复杂的社会背景下通过传入西方心理学而建立和发展起来的.在这一过程中,心理学图书的出版和传播起到了重要作用.文章以《民国时期总书目》中收录的心理学图书为研究对象,通过对心理学图书的年出版量、编著与译著的比较、出版数量的地域差异、分支学科的出版特征等四个方面的出版状况进行计量分析,探讨其形成原因和历史背景,试图从出版视角对民国时期科学心理学在中国的发展做微观的历史考察.     

Publication bias in psychological science: prevalence, methods for identifying and controlling, and implications for the use of meta-analyses

The issue of publication bias in psychological science is one that has remained difficult to address despite decades of discussion and debate. The current article examines a sample of 91 recent meta-analyses published in American Psychological Association and Association for Psychological Science journals and the methods used in these analyses to identify and control for publication bias. Of the 91 studies analyzed, 64 (70%) made some effort to analyze publication bias, and 26 (41%) reported finding evidence of bias. Approaches to controlling publication bias were heterogeneous among studies. Of these studies, 57 (63%) attempted to find unpublished studies to control for publication bias. Nonetheless, those studies that included unpublished studies were just as likely to find evidence for publication bias as those that did not. Furthermore, authors of meta-analyses themselves were overrepresented in unpublished studies acquired, as compared with published studies, suggesting that searches for unpublished studies may increase rather than decrease some sources of bias. A subset of 48 meta-analyses for which study sample sizes and effect sizes were available was further analyzed with a conservative and newly developed tandem procedure of assessing publication bias. Results indicated that publication bias was worrisome in about 25% of meta-analyses. Meta-analyses that included unpublished studies were more likely to show bias than those that did not, likely due to selection bias in unpublished literature searches. Sources of publication bias and implications for the use of meta-analysis are discussed.     

Publication trends in school psychology: 1963–1973

Articles appearing in three school psychology journals from 1963–1973 were classified and organized into 16 topic areas. Trends of published interests over the years 1963–1966, 1967–1969, and 1970–1973 were identified. Articles concerning Professional Identity were most prominent in the early years of the decade covered; Instrument Development and Evaluation, an area which was consistently popular over the decade, ranked highest during the last period evaluated. The most noteworthy trend in publication has been the growing diversification of published interests in the field of school psychology.     

In search of cultural diversity: recent literature in cross-cultural and ethnic minority psychology

The purpose of this study was to identify where the most work on cross-cultural and ethnic minority psychology is being published and who the most productive authors are. The journals that published the most articles on cross-cultural and ethnic minority issues from 1993 to 1999 and the most prolific authors on these issues were identified by PsycINFO. Cross-cultural research is cross-national, whereas ethnic minority research involves groups of color within the United States. The citation impact of these journals and authors was determined from the 1997 Social Sciences Citation Index. The results suggest that there is very limited overlap between the literatures in cross-cultural and ethnic minority psychology. Most of the research in these areas is published in specialty journals, and there is a paucity of this research in prestigious journals. Perceived or actual barriers to publication in prestigious journals may cause some to seek specialty journals as outlets for research on cultural diversity. The top scholars in cross-cultural psychology are primarily men of European ancestry, whereas most of the top scholars in ethnic minority psychology are ethnic minority men and women. Strategies to increase the prominence of cultural diversity in the psychology literature include combining cross-cultural and ethnic minority psychology, increasing the number of editorial board members of prestigious journals having expertise in cultural diversity, and increasing the quality of specialty journals. Psychology will remain ill-equipped to face the challenges of the new millennium without increased attention to cultural diversity.     

Evaluation of publication bias in response interruption and redirection: A meta-analysis

Publication bias is the disproportionate representation of studies with large effects and statistically significant findings in the published research literature. If publication bias occurs in single-case research design studies on applied behavior-analytic (ABA) interventions, it can result in inflated estimates of ABA intervention effects. We conducted an empirical evaluation of publication bias on an evidence-based ABA intervention for children diagnosed with autism spectrum disorder, response interruption and redirection (RIRD). We determined effect size estimates for published and unpublished studies using 3 metrics, percentage of nonoverlapping data (PND), Hedges' g, and log response ratios (LRR). Omnibus effect size estimates across all 3 metrics were positive, supporting that RIRD is an effective treatment for reducing problem behavior maintained by nonsocial consequences. We observed larger PND for published compared to unpublished studies, small and nonsignificant differences in LRR for published compared to unpublished studies, and significant differences in Hedges' g for published compared to unpublished studies, with published studies showing slightly larger effect. We found little, if any, difference in methodological quality between published and unpublished studies. While RIRD appears to be an effective intervention for challenging behavior maintained by nonsocial consequences, our results reflect some degree of publication bias present in the RIRD research literature.     

The Prevalence of Well-Being on Teachers in Afri Twin Schools as Well as Non Afri Twin Schools

This study investigated the self-perceived preparedness of students from a historically disadvantaged rural South African setting for psychology studies at a university. The study sample comprised 399 beginning students with a historically Black university (age M = 22.8 years, SD = 5.1; 72.2% women; 65.7% rural background) students. The students responded to a questionnaire investigating, inter alia, prior awareness of the subject, access to psychology literature, awareness of psychologists in their communities, perceived relevance to or clashes with their culture, and parental awareness of the subject. Of the sample, 48.1% first heard about psychology upon entering a university, 75.7% had no access to books on the subject, and only 44.6% received information from their school teachers about psychology as a career. Significantly more urban (65.2%) than rural (34.8%) students had received information about psychology before coming to university.     

Population models and simulation methods: The case of the Spearman rank correlation

The purpose of this paper is to highlight the importance of a population model in guiding the design and interpretation of simulation studies used to investigate the Spearman rank correlation. The Spearman rank correlation has been known for over a hundred years to applied researchers and methodologists alike and is one of the most widely used non‐parametric statistics. Still, certain misconceptions can be found, either explicitly or implicitly, in the published literature because a population definition for this statistic is rarely discussed within the social and behavioural sciences. By relying on copula distribution theory, a population model is presented for the Spearman rank correlation, and its properties are explored both theoretically and in a simulation study. Through the use of the Iman–Conover algorithm (which allows the user to specify the rank correlation as a population parameter), simulation studies from previously published articles are explored, and it is found that many of the conclusions purported in them regarding the nature of the Spearman correlation would change if the data‐generation mechanism better matched the simulation design. More specifically, issues such as small sample bias and lack of power of the t‐test and r‐to‐z Fisher transformation disappear when the rank correlation is calculated from data sampled where the rank correlation is the population parameter. A proof for the consistency of the sample estimate of the rank correlation is shown as well as the flexibility of the copula model to encompass results previously published in the mathematical literature.     

A Bayesian approach to mitigation of publication bias

The reliability of published research findings in psychology has been a topic of rising concern. Publication bias, or treating positive findings differently from negative findings, is a contributing factor to this “crisis of confidence,” in that it likely inflates the number of false-positive effects in the literature. We demonstrate a Bayesian model averaging approach that takes into account the possibility of publication bias and allows for a better estimate of true underlying effect size. Accounting for the possibility of bias leads to a more conservative interpretation of published studies as well as meta-analyses. We provide mathematical details of the method and examples.     

Reproducible research in sport and exercise psychology: The role of sample sizes

ObjectivesWe aim to introduce the discussion on the crisis of confidence to sport and exercise psychology. We focus on an important aspect of this debate, the impact of sample sizes, by assessing sample sizes within sport and exercise psychology. Researchers have argued that publications in psychological research contain numerous false-positive findings and inflated effect sizes due to small sample sizes.MethodWe analyse the four leading journals in sport and exercise psychology regarding sample sizes of all quantitative studies published in these journals between 2009 and 2013. Subsequently, we conduct power analyses.ResultsA substantial proportion of published studies does not have sufficient power to detect effect sizes typical for psychological research. Sample sizes and power vary between research designs. Although many correlational studies have adequate sample sizes, experimental studies are often underpowered to detect small-to-medium effects.ConclusionsAs sample sizes are small, research in sport and exercise psychology may suffer from false-positive results and inflated effect sizes, while at the same time failing to detect meaningful small effects. Larger sample sizes are warranted, particularly in experimental studies.     

The project of an International Congress of Psychology by J. Ochorowicz (1881)

Since the second part of the 19th century, there has been a great increase in the number of international scientific congresses, and they appear a necessary step in the maturation of knowledge. The first person to explicitly suggest the necessity of an international congress of psychology was Julian Ochorowicz (1850–1917), who was considered the founder of Polish psychology. In 1881 he sent an article to Théodule Ribot, editor of Revue Philosophique de la France et de l'Étranger, entitled “Project of an International Congress of Psychology,” which was published in Ribot's journal. In it he described the dispersed state of psychology in 1881 and the previous 50 years, including 12 subdisciplines ranging from psychophysics to the psychology of art, and how it should aim for unification. He suggested collective efforts to progress rapidly, with widespread collaboration and the continuous exchange of observations, information, and experiments. Having international congresses would constitute a forum for such exchanges, and in his article he outlined the steps that should be taken for putting together the first international congress of psychology. The proposal was quite extensive and fanciful, making Ribot somewhat doubt its realizability. Nevertheless, the first international psychology congress took place in Paris in 1889, 8 years after the publication of Ochorowicz's article. This quick development could be attributed to the creation of the psychology societies, one of the first being La Société de Psychologie Physiologique, which was created in France in 1885. The creation of this society also contributed to psychology being recognized as a scientific domain. In line with this, the final programme of the congress was less extensive than that proposed in Ochorowicz's article, focusing on physiological aspects and ignoring the philosophical ones. In the present paper, the historical context of Ochorowicz's article is outlined, followed by a translation of his article.     

The Ecological Revolution: The Senses Considered as Perceptual Systems, 50 Years Later - Part 2

ABSTRACT

This editorial presents the second of a 2-part special issue honoring the publication of James J. Gibson's 1966 book, The Senses Considered as Perceptual Systems. As was pointed out by several reviews of this book written just after its publication (i.e., between 1967 and 1969), Gibson's 1966 book represents a revolution not only for the psychological science but also for domains beyond that of psychology. This second part of the special issue is composed of 5 theoretical contributions that represent, in addition to the 4 pieces previously published in the first part, the far-reaching influence of Gibson's ecological revolution.     

Textbooks and Authors in Social Psychology: The First Seven Decades

Abstract

A catalog of 105 social psychology textbooks published over the 70-year period from 1908 to 1977 was compiled. It included title of book, year of publication, publisher, author or authors, age of first author at publication, institutional affiliation of first author, presence or absence of a glossary, and number of references cited. A summary and analysis of the catalog revealed some characteristics of social psychology textbooks and of their authors and changes in such characteristics over time.     

Tracing the size,reach, impact,and breadth of positive psychology

This study quantitatively assessed the size, disciplinary reach, impact, and topic breadth of positive psychology (PP), in order to provide a comprehensive evaluation of the growth and significance of the field. Over 1.7?million documents in 700 PsycINFO ^® journals covering the fields of psychology, psychiatry, neuroscience, management, business, public health, and sport science were analyzed using semantic and bibliographic methods. Results indicate that PP covers many different research topics from a diverse range of disciplines, and that PP literature has been growing rapidly in significance. Over 18,000 documents were identified as belonging to PP, with 2300 published in 2011. These documents represent over 4% of PsycINFO ^® documents within the data-set published that year, and make PP close to the median size of all disciplines listed in the Thomson Reuters Journal Citation Reports ^®, Social Sciences Edition. The aggregate impact factor for PP in 2011 was 2.64.     

Guidelines Versus Practices in Cross-Lingual Assessment: A Disconcerting Disconnect

The International Test Commission's Guidelines for Translating and Adapting Tests (2010) provide important guidance on developing and evaluating tests for use across languages. These guidelines are widely applauded, but the degree to which they are followed in practice is unknown. The objective of this study was to perform a systematic review of published articles to determine whether the methodologies related to test adaptation and evaluation of cross-lingual measurement bias have improved since the initial publication of the Guidelines. Results demonstrated that although reporting reliability estimates and statistically analyzing construct equivalence improved since publication of the Guidelines, the majority of test development and validation practices in the published literature have not embraced the recommendations put forth by the International Test Commission. Overall, these findings call into question many of the inferences derived from cross-linguistic comparisons and clearly signal a need for better dissemination mechanisms of the Guidelines.     

Reform of statistical inference in psychology: The case of<Emphasis Type="Italic">Memory &; Cognition</Emphasis>

Geoffrey Loftus, Editor ofMemory & Cognition from 1994 to 1997, strongly encouraged presentation of figures with error bars and avoidance of null hypothesis significance testing (NHST). The authors examined 696Memory & Cognition articles published before, during, and after the Loftus editorship. Use of figures with bars increased to 47% under Loftus’s editorship and then declined. Bars were rarely used for interpretation, and NHST remained almost universal. Analysis of 309 articles in other psychology journals confirmed that Loftus’s influence was most evident in the articles he accepted for publication, but was otherwise limited. An e-mail survey of authors of papers accepted by Loftus revealed some support for his policy, but allegiance to traditional practices as well. Reform of psychologists’ statistical practices would require more than editorial encouragement.