Perpetuation of Retracted Publications Using the Example of the Scott S. Reuben Case: Incidences,Reasons and Possible Improvements

In 2009, Scott S. Reuben was convicted of fabricating data, which lead to 25 of his publications being retracted. Although it is clear that the perpetuation of retracted articles negatively effects the appraisal of evidence, the extent to which retracted literature is cited had not previously been investigated. In this study, to better understand the perpetuation of discredited research, we examine the number of citations of Reuben’s articles within 5 years of their retraction. Citations of Reuben’s retracted articles were assessed using the Web of Science Core Collection (Thomson Reuters, NY). All citing articles were screened to discriminate between articles in which Reuben’s work was quoted as retracted, and articles in which his data was wrongly cited without any note of the retraction status. Twenty of Reuben’s publications had been cited 274 times between 2009 and 1024. In 2014, 45 % of the retracted articles had been cited at least once. In only 25.8 % of citing articles was it clearly stated that Reuben’s work had been retracted. Annual citations decreased from 108 in 2009 to 18 in 2014; however, the percentage of publications correctly indicating the retraction status also declined. The percentage of citations in top-25 %-journals, as well as the percentage of citations in journals from Reuben’s research area, declined sharply after 2009. Our data show that even 5 years after their retraction, nearly half of Reuben’s articles are still being quoted and the retraction status is correctly mentioned in only one quarter of the citations.     

Correction and use of biomedical literature affected by scientific misconduct

The purpose of this study was to identify and describe published research articles that were named in official findings of scientific misconduct and to investigate compliance with the administrative actions contained in these reports for corrections and retractions, as represented in PubMed. Between 1993 and 2001, 102 articles were named in either the NIH Guide for Grants and Contracts ("Findings of Scientific Misconduct") or the U.S. Office of Research Integrity annual reports as needing retraction or correction. In 2002, 98 of the 102 articles were indexed in PubMed. Eighty-five of these 98 articles had indexed corrections: 47 were retracted; 26 had an erratum; 12 had a correction described in the "comment" field. Thirteen had no correction, but 10 were linked to the NIH Guide "Findings of Scientific Misconduct", leaving only 3 articles with no indication of any sort of problem. As of May 2005, there were 5,393 citations to the 102 articles, with a median of 26 citations per article (range 0-592). Researchers should be alert to "Comments" linked to the NIH Guide as these are open access, and the "Findings of Scientific Misconduct' reports are often more informative than the statements about the retraction or correction found in the journals.     

Classic articles in Psychology in the Science Citation Index Expanded: A bibliometric analysis

Who today are the most highly cited psychologists listed in the Web of Science? This paper reports answers to this question by using the Science Citation Index Expanded to find out. This index covers over 280,350 documents in the Psychology category of the Web of Science from 1900 to 2013 and lists the most highly cited papers published between 1927 and 2012. For example, in 2013, an article published by Jacob Cohen in 1992 obtained (1) the highest ranking with 1,068 citations, (2) the highest for total citations per year, and (3) was ranked 3rd for the total number of citations since publication by 2013. New technology thus provides a seismic shift in the ways that we can obtain and analyse data like these. Indeed, the results for the top 10 articles indicate that practical and methodological papers now receive more citations than the more theoretical papers usually found in earlier surveys.     

Predictors of Article Impact in Suicidology: The Bereavement Literature,A Research Note

Citation analysis has been neglected in suicidology. The present note applies a mixed‐methods approach to both test and suggest hypotheses for the variation in article impact in the bereavement literature. One hundred three articles from three core suicidology journals met the criteria for inclusion in the investigation. Citations to the articles were obtained from the Web of Science. Predictor variables included structural characteristics of the author (e.g., gender) and the article itself (e.g., years since publication). A multivariate regression analysis determined that, controlling for the other variables, the most important predictor of citations was the review article (β = .461), followed by year of publication (β = ?.414), the multiauthored article (β = .302), publication in Suicide and Life‐Threatening Behavior (SLTB) (β = .161), and male gender (β = .156). The 12 most cited articles were published between 1979 and 2004 in SLTB. The majority of these papers was written by males, were U.S. authors, and had more than one author. Four of the most cited articles were reviews. The study concludes that structural characteristics of articles and authors explained 41% of the variance in citations. The qualitative analysis determined that review papers, and papers on characteristics of suicide bereavement and psychological autopsies have been most frequently cited. Replication studies are needed for other subfields of suicidology.     

An ethnic bias in scientific citations

Recent experimental findings of subtle forms of prejudice prompted this search for a similar phenomenon outside the laboratory. In Study 1, with a sample of more than 12 000 citations by North American social scientists, names of both citing and cited authors were classified as Jewish, non Jewish, or other Author's name category was associated with 41 per cent greater odds of citing an author from the same name category Study 2 included over 17 000 citations from a much narrower research domain (prejudice research), and found a similar (40 per cent) surplus in odds of citing an author of the author's own ethnic name category. Further analyses failed to support two hypotheses — differential assortment of researchers by ethnicity to research topics, and selective citation of acquaintances' works — that were plausible alternatives to the hypothesis that the observed citation discrimination revealed implicit (unconsciously operating) prejudicial attitudes. Given the sociopolitically liberal reputation of social scientists (and of prejudice researchers especially), it seems unlikely that the observed bias in citations reflected conscious prejudicial attitudes.     

Citation Networks of Communication Journals, 1977–1985 Cliques and Positions,Citations Made and Citations Received

This article analyzes the pattern of citations among all communication journals covered by the first nine years of the Journal Citation Reports volume of the Social Sciences Citation Index (Garfield, 1977–1985). It approaches these analyses from two perspectives: (1) bibliometric analysis of citation flows, ratios, and impact factors, and (2) network analysis of both cohesion-based and position-based citation patterns. The field of communication exhibits clear clustering and inbreeding, consisting of cliques of interpersonal journals, mass media journals, and residual isolate journals. Citation patterns within and across these cliques have been very stable from 1977 through 1985. The cohesion analyses show that Human Communication Research and Communication Education provide weak ties from mass media journals to interpersonal communication journals. The Journal of Communication is cited the most strongly by the most journals, and provides, by means of citations, bidirectional channels for the flow of scientific information between mass media journals and interpersonal journals. The positional analyses show that the set of journals that has similar citing patterns is somewhat different from the set of journals that is cited similarly. In particular, the set of sources for the research reported in articles in these journals is not similar to the set of journals that refer to the articles in these journals.     

On being a whistleblower: the Needleman case

We believe that members of the scientific community have a primary obligation to promote integrity in research and that this obligation includes a duty to report observations that suggest misconduct to agencies that are empowered to examine and evaluate such evidence. Consonant with this responsibility, we became whistleblowers in the case of Herbert Needleman. His 1979 study (Needleman et al., 1979), on the effects of low-level lead exposure on children, is widely cited and highly influential in the formulation of public policy on lead. The opportunity we had to examine subject selection and data analyses from this study was prematurely halted by efforts to prevent disclosure of our observations. Nevertheless, what we saw left us with serious concerns. We hope that the events here summarized will contribute to revisions of process by which allegations of scientific misconduct are handled and that such revisions will result in less damage to scientists who speak out.     

On Being a Whistleblower: The Needleman Case

We believe that members of the scientific community have a primary obligation to promote integrity in research and that this obligation includes a duty to report observations that suggest misconduct to agencies that are empowered to examine and evaluate such evidence. Consonant with this responsibility, we became whistleblowers in the case of Herbert Needleman. His 1979 study (Needleman et al., 1979), on the effects of low-level lead exposure on children, is widely cited and highly influential in the formulation of public policy on lead. The opportunity we had to examine subject selection and data analyses from this study was prematurely halted by efforts to prevent disclosure of our observations. Nevertheless, what we saw left us with serious concerns. We hope that the events here summarized will contribute to revisions of process by which allegations of scientific misconduct are handled and that such revisions will result in less damage to scientists who speak out.     

Citation Classics in Suicide and Life Threatening Behavior: A Research Note

The number of citations a scholarly work receives is a common measure of its impact on the scientific literature; “citation classics” are the most highly cited works. The content of Suicide and Life‐Threatening Behavior (SLTB) citation classics is described here. The impact of SLTB citation classics is compared to their counterparts in journals having published the most suicide papers. All data are from the ISI electronic venue on the Web of Science and refer to the number of citations the top 1% of works received in each of ten journals from 1975 through August 10, 2011. Among all ten journals, SLTB ranked first in the number of works on suicide. The principle theme of half of SLTB suicide classics was literature review. The median number of citations for SLTB citation classics (top 1%) was 121.5, with a range between 96 and 279 citations, but classics from generalized psychiatric journals received more citations as anticipated. Journal impact factors explained 73% of the variance in classic’s citation counts across journals. On average, suicide classics received 30% more citations than all classics. Among a second group of five specialized suicide journals, however, SLTB ranked first in average 5‐year impact. Although SLTB produced the highest number of suicide articles of any journal, SLTB’s citation classics received fewer citations than suicide classics in high‐impact/prestige, general journals. Future work is needed to assess what predicts which SLTB articles ultimately become citation classics.     

Research Productivity and Performance of Journals in the Creativity Sciences: A Bibliometric Analysis

A bibliometric approach was employed to analyze the research productivity and performance of creativity studies between 1965 and 2012. A dataset was constructed using all publications and citations retrieved from four key journals that publish creativity research: Journal of Creative Behavior (JCB), Gifted Child Quarterly (GCQ), Creativity Research Journal (CRJ), and Psychology of Aesthetics, Creativity, and the Arts (PACA). Major findings in this study include: (a) During the study period, the four journals have published 1,891 articles on creativity and they have been cited 11,709 times; (b) the impact factors of the four journals increased from lower than .50 in 2002 to over 1.0 in 2012; in 2012 PACA had the highest impact factor, followed by CRJ; (c) JCB published the most creativity papers and CRJ had the most citations; (d) about a third of the articles published in the four journals have never been cited. Implications for the field of creativity are discussed.     

Citation rates for experimental psychology articles published between 1950 and 2004: Top-cited articles in behavioral cognitive psychology

From citation rates for over 85,000 articles published between 1950 and 2004 in 56 psychology journals, we identified a total of 500 behavioral cognitive psychology articles that ranked in the top 0.6?% in each half-decade, in terms of their mean citations per year using the Web of Science. Thirty nine of these articles were produced by 78 authors who authored three or more of them, and more than half were published by only five journals. The mean number of cites per year and the total number of citations necessary for an article to achieve various percentile rankings are reported for each journal. The mean number of citations necessary for an article published within each half-decade to rank at any given percentile has steadily increased from 1950 to 2004. Of the articles that we surveyed, 11?% had zero total citations, and 35?% received fewer than four total citations. Citations for post-1994 articles ranking in the 50th-75th and 90th-95th percentiles have generally continued to grow across each of their 3-year postpublication bins. For pre-1995 articles ranking in the 50th-75th and 90th-95th percentiles, citations peaked in the 4- to 6- or 7- to 9-year postpublication bins and decreased linearly thereafter, until asymptoting. In contrast, for the top-500 articles, (a) for pre-1980 articles, citations grew and peaked 10-18-year postpublication bins, and after a slight decrease began to linearly increase again; (b) for post-1979 articles, citations have continually increased across years in a nearly linear fashion. We also report changes in topics covered by the top-cited articles over the decades.     

Effectiveness of research guidelines in prevention of scientific misconduct

In response to a series of allegations of scientific misconduct in the 1980’s, a number of scientific societies, national agencies, and academic institutions, including Harvard Medical School, devised guidelines to increase awareness of optimal scientific practices and to attempt to prevent as many episodes of misconduct as possible. The chief argument for adopting guidelines is to promote good science. There is no evidence that well-crafted guidelines have had any detrimental effect on creativity since they focus on design of research studies, documentation of research findings, assignment of credit through authorship, data management and supervision of trainees, not on the origin and evolution of ideas. This paper addresses a spectrum of causes of scientific misconduct or unacceptable scientific behavior and couples these with estimates of the potential for prevention if guidelines for scientific investigation are adopted. The conclusion is that clear and understandable guidelines should help to reduce the chance that flawed research will escape from our institutions. However, they cannot be relied upon alone to prevent all instances of scientific misconduct and should be regarded rather as one means of bolstering the integrity of the entire scientific enterprise.     

The Evaluative Use of Citation Analysis for Communication Journals

This study investigated the impact of the omission of scholarly communication journals from Social Sciences Citation Index (SSCI) and Arts and Humanities Citation Index (AHCI) on citation-based appraisals of communication literature. Citation data were collected from reference lists in articles appearing in 27 communication journals published during 1990, including 14 not indexed by SSCI or AHCI. The 14 journals not included in SSCI or AHCI contributed 1,953, or 26%, of the 7,640 citations to authors during 1990, and 1,156, or 25%, of the 4,587 citations to journals in 1990. Twenty-seven of the 50 most frequently cited authors received 25% or more of their citations from journals omitted by SSCI and AHCI. This study found that an accurate evaluation of a scholar's articles based on citations received will not be possible until SSCI and/or AHCI choose to include the omitted journals, and that computer applications could be used to provide a convenient and inexpensive on-line citation index for communication literature. The Journal Impact Rating, a measure for use in comparing journals’impact on the basis of citations received, was introduced.     

Describing citation structures in sport burnout literature: A citation network analysis

ObjectivesTo evaluate sport burnout research using citation network analysis.DesignA citation network analysis.MethodsWe began by searching the term “sport and burnout” in SportDiscus, Physical Education Index, and PsycINFO. From the returned search, we then selected and analyzed all peer-reviewed English articles that were published before 2012. This allowed us to create a network of inter-citations.ResultsOur search yielded 102 articles, of which there were 11 reviews (11%), 13 qualitative articles (13%), 76 quantitative articles (75%), and two mixed methods (qualitative and quantitative) articles (2%). The 11 most cited articles were published between 1984 and 2006, appearing in 5 journals (7 of them in the same journal). Merely 2 of the most cited articles investigated coach burnout. Top-cited articles typically conducted original research using male and female individual sport athletes and were published by North American authors, with two recent exceptions. The most cited article, and also most influential burnout model, was Smith's (1986) cognitive-affective stress model.ConclusionsThe findings show that following an initial focus on coach burnout, more recent studies examine athletes. In this paper we have identified some strengths amongst the most cited papers, including a wide range of publication dates and a focus on both male and female athletes; however, there were also limitations such as a scarcity of articles investigating team sport athletes.     

Lack of Improvement in Scientific Integrity: An Analysis of WoS Retractions by Chinese Researchers (1997–2016)

This study investigated the status quo of article retractions by Chinese researchers. The bibliometric information of 834 retractions from the Web of Science SCI-expanded database were downloaded and analysed. The results showed that the number of retractions increased in the past two decades, and misconduct such as plagiarism, fraud, and faked peer review explained approximately three quarters of the retractions. Meanwhile, a large proportion of the retractions seemed typical of deliberate fraud, which might be evidenced by retractions authored by repeat offenders of data fraud and those due to faked peer review. In addition, a majority of Chinese fraudulent authors seemed to aim their articles which contained a possible misconduct at low-impact journals, regardless of the types of misconduct. The system of scientific evaluation, the “publish or perish” pressure Chinese researchers are facing, and the relatively low costs of scientific integrity may be responsible for the scientific integrity. We suggested more integrity education and severe sanctions for the policy-makers, as well as change in the peer review system and transparent retraction notices for journal administrators.     

Self- and cross-citations in the Journal of Applied Behavior Analysis and the Journal of the Experimental Analysis of Behavior: 1993-2003

Self- and cross-citations in JABA and JEAB from 1993 through 2003 were examined. Yearly mean levels of self-citation for JABA and JEAB were 34.9% and 33.2%, respectively. Overall, 7.8% of JABA citations were JEAB articles, and 0.6% of JEAB citations were JABA articles. The former value, but not the latter, is substantially higher than the cross-citation level reported for earlier years. The two JEAB articles most often cited in JABA were published over 20 years ago and are concerned with establishing operations and the matching law.     

Evolving research misconduct policies and their significance for physical scientists

Scientific misconduct includes the fabrication, falsification, and plagiarism (FFP) of concepts, data or ideas; some institutions in the United States have expanded this concept to include “other serious deviations (OSD) from accepted research practice.” It is the absence of this OSD clause that distinguishes scientific misconduct policies of the past from the “research misconduct” policies that should be the basis of future federal policy in this area. This paper introduces a standard for judging whether an action should be considered research misconduct as distinguished from scientific misconduct: by this standard, research misconduct must involve activities unique to the practice of science and must have the potential to negatively affect the scientific record. Although the number of cases of scientific misconduct is uncertain (only the NIH and the NSF keep formal records), the costs are high in terms of the integrity of the scientific record, diversions from research to investigate allegations, ruined careers of those eventually exonerated, and erosion of public confidence in science. Existing scientific misconduct policies vary from institution to institution and from government agency to government agency; some have highly developed guidelines that include OSD, others have no guidelines at all. One result has been that the federal False Claims Act has been used to pursue allegations of scientific misconduct. As a consequence, such allegations have been adjudicated in federal courts, rather than judged by scientific peers. The federal government is now establishing a first-ever research misconduct policy that would apply to all research funded by the federal government regardless of which agency funded the research or whether the research was carried out in a government, industrial or university laboratory. Physical scientists, who up to now have only infrequently been the subject of scientific misconduct allegations, must nonetheless become active in the debate over research misconduct policies and how they are implemented since they will now be explicitly covered by this new federal wide policy. Disclaimer: The authors are grateful for the support for conduct of this research provided by the United States Department of Energy (DOE). The views expressed in this paper are solely those of the authors and were formed and expressed without reference to positions taken by DOE or the Pacific Northwest National Laboratory (PNNL). The views of the authors are not intended either to reflect or imply positions of DOE or PNNL.     

On the impact of human operant research: Asymmetrical patterns of cross-citation between human and nonhuman research

Reactions to published accounts of research with human subjects, as well as research with nonhuman subjects, were assessed by examining citations in several samples of empirical articles in the Journal of the Experimental Analysis of Behavior. A stable, asymmetrical pattern emerged: Nonhuman research was cited in both human and nonhuman articles, but human research was cited primarily in human articles. Thus, human operant research appears to have had little influence on the nonhuman research which constitutes the bulk of the experimental analysis of behavior. Interpretation of this lack of impact depends on the functions one envisions for human research, several of which are discussed.     

Research misconduct among clinical trial staff

Between 1993 and 2002, 39 clinical trial staff were investigated for scientific misconduct by the Office of Research Integrity (ORI). Analysis of ORI case records reveals practices regarding workload, training and supervision that enable misconduct. Considering the potential effects on human subjects protection, quality and reliability of data, and the trustworthiness of the clinical research enterprise, regulations or guidance on use of clinical trial staff ought to be available. Current ORI regulations do not hold investigators or institutions responsible for supervision and training of clinical trial staff. Given the important issues at stake, the definition of research misconduct should encompass the intentional or negligent mismanagement of scientific projects. Individual institutions and professional associations not only can but should adopt stricter standards of conduct than those reflected in federal regulations.