The Problem of Peer Review is the Most Important Philosophical Problem

As philosophers we should have as one of our aims to produce as much philosophical knowledge as possible. A lot of potential philosophical knowledge is lost because of the flaws of the peer review system, and so a lot of philosophical knowledge would be gained were the system improved. Accordingly, as authors we should write papers about how to fix peer review, and as editors we should accept such papers if they are good. This paper presents some familiar problems with peer review, elaborates on and motivates the argument just given, and replies to some objections to it, making the case that fixing peer review is both a philosophical problem and one that admits of a solution.     

Peer review for journals: Evidence on quality control, fairness, and innovation

This paper reviews the published empirical evidence concerning journal peer review consisting of 68 papers, all but three published since 1975. Peer review improves quality, but its use to screen papers has met with limited success. Current procedures to assure quality and fairness seem to discourage scientific advancement, especially important innovations, because findings that conflict with current beliefs are often judged to have defects. Editors can use procedures to encourage the publication of papers with innovative findings such as invited papers, early-acceptance procedures, author nominations of reviewers, structured rating sheets, open peer review, results-blind review, and, in particular, electronic publication. Some journals are currently using these procedures. The basic principle behind the proposals is to change the decision from whether to publish a paper to how to publish it. The author, a professor of marketing at the Wharton School since 1968, was a founder editor of the Journal of Forecasting and the International Journal of Forecasting. This paper is based on a presentation at a workshop, “Advances in Peer Review Research”, American Association for the Advancement of Science Meeting, Baltimore, MD, February 9, 1996.     

Why Not Open the Black Box of Journal Editing in Philosophy? Make Peer Reviews of Published Papers Available

Despite general agreement within philosophy that peer review is indispensable, its fairness and reliability is often questioned. This article suggests that such worries can to a large extent be met by adopting the practice that reviews as well as earlier versions of papers are made publicly available when the final version of a paper is published. This suggestion combines the advantages of transparency with the merits of anonymity of reviewers. While there are obstacles to this suggestion, the article argues that it would be worthwhile to implement it because it can help map patterns of conduct and secure confidence in the fairness and reliability of review procedures and journal editing within philosophy.     

The promise and potential of studying the “invisible hand” of teacher influence on peer relations and student outcomes: A commentary

Peers influence children's social-emotional development and school engagement in important and unique ways. Recent research on peer social networks documents that children are affected by the nature of the school-based peer ecology, as well as by their personal peer experiences. Yet, little is known about how teachers affect the peer ecology, nor how teachers can promote positive peer influences in the school context. The four studies in this special section examine this issue. Together, they document the promise and the potential of studying the “invisible hand” of teacher influence on peer relations, and illustrate the need for further research in this area. This commentary considers the contributions of these papers for conceptualizing processes of transactional teacher and peer influence on student outcomes, and implications for interventions designed to reduce problem behaviors or increase school engagement. In addition, it considers the limitations of current knowledge and directions for future research.     

Peer review and innovation

Two important aspects of the relationship between peer review and innovation includes the acceptance of articles for publication in journals and the assessment of applications for grants for the funding of research work. While there are well-known examples of the rejection by journals of first choice of many papers that have radically changed the way we think about the world outside ourselves, such papers do get published eventually, however tortuous the process required. With grant applications the situation differs in that the refusal of a grant necessarily curtails the possible research that may be attempted. Here there are many reasons for conservatism and reservation as to the ability of a grant allocation process based on peer review to deliver truly innovative investigations. Other methods are needed; although such methods need not be applied across the board, they should constitute the methods whereby some 10–20% of the grant monies are assigned. The nomination of prizes for specific accomplishments is one way of achieving innovation although this presumes that investigators or institution already have available the money necessary to effect the innovations; otherwise it is a question of the selection and funding of particular individuals or institutions and requiring them to solve particular problems that are set in the broadest of terms.     

Childhood peer relationships: social acceptance, friendships, and peer networks

This review addresses several areas of contemporary research in children's peer relationships during the elementary and middle school years, with primary foci on children's peer acceptance, the ability to make and maintain friendships, and their participation in larger peer networks. Particular attention is given to research examining the major developments and individual differences in each of these components of children's peer relations, how these different aspects of peer functioning relate to one another, and how they contribute to development more generally, including school adjustment and achievement. Finally, it is argued that children's psychosocial development may be best informed by an integration of these somewhat independent research traditions.     

Meta-perception for pathological personality traits: do we know when others think that we are difficult?

The self allows us to reflect on our own behavior and to imagine what others think of us. Clinical experience suggests that these abilities may be impaired in people with personality disorders. They do not recognize the impact that their behavior has on others, and they have difficulty understanding how they are seen by others. We collected information regarding pathological personality traits--using both self and peer report measures--from groups of people who knew each other well (at the end of basic military training). In previous papers, we have reported that agreement between self-report and peer-report is only modest. In this paper, we address the question: Do people know that others disagree with their own perceptions of themselves? We found that expected peer scores predicted variability in peer report over and above self-report for all 10 diagnostic traits. People do have some incremental knowledge of how they are viewed by others, but they do not tell you about it unless you ask them to do so; the knowledge is not reflected in ordinary self-report data. Among participants who expect their peers to describe them as narcissistic, those who agree with this assessment are viewed as being less narcissistic by their peers than those who deny being narcissistic. It therefore appears that insight into how one is viewed by others can moderate negative impressions fostered by PD traits.     

Peer review of teaching (PRoT) in higher education – a practitioner’s reflection

Peer review of teaching (PRoT) is recommended to both develop and assure the quality of teaching practices in Higher Education. An institutional implementation of a peer review process can be viewed as a genuine desire to improve teaching quality or as an instrument of accountability and performativity. There are many approaches to the peer review of teaching operating. This article documents the impact, advantages and disadvantages of direct participation in three peer review of teaching processes.     

用公评审稿促进中国科技期刊的快速发展

学术论文评价目前普遍采用的同行评审系统,存在严重缺陷,在我国具体国情下,不易保证审稿过程的公正性,难以有效提高科研群体的审稿水平。作者提出一个“公评审稿”方案,包括实名的公开推荐和匿名的公开评审两个过程,使学术成果的发表和评价实现了清晰的功能分离,解决了发表注重快速及时、而评价注重慎重仔细这一根本矛盾。公评系统融合了中国传统的举贤制度和互联网新近发展的微博技术,其公开和共享理念,在保证学术公正的同时,能大幅度促进科研知识和经验在科研群体内的传播,迅速、有效地提高我国科技期刊的办刊质量。     

Social Biases and Solutions for Procedural Objectivity

An empirically sensitive formulation of the norms of transformative criticism must recognize that even public and shared standards of evaluation can be implemented in ways that unintentionally perpetuate and reproduce forms of social bias that are epistemically detrimental. Helen Longino's theory can explain and redress such social bias by treating peer evaluations as hypotheses based on data and by requiring a kind of perspectival diversity that bears, not on the content of the community's knowledge claims, but on the beliefs and norms of the culture of the knowledge community itself. To illustrate how socializing cognition can bias evaluations, we focus on peer‐review practices, with some discussion of peer‐review practices in philosophy. Data include responses to surveys by editors from general philosophy journals, as well as analyses of reviews and editorial decisions for the 2007 Cognitive Science Society Conference.     

When reviews attack: ethics, free speech, and the peer review process.

The peer review process, whether formally applied in publication and grant review, or informally, such as exchange of ideas in scientific and professional newsgroups, has sparked controversy. Writers in this area agree that scholarly reviews that are inappropriate in tone are not uncommon. Indeed, commentators have suggested rules and guidelines that can be used to improve the review process and to make reviewers more accountable. In this paper, we examine the relevance and impact of ethical codes on the conduct of peer review. It is our contention that the peer review process can be improved, not by a new set of rules but through closer attention to the ethical principles to which we, as psychologists, already subscribe.     

The wagon-wheel illusion in continuous light

The fact that a perceptual experience akin to the familiar wagon-wheel illusion in movies and on TV can occur in the absence of stroboscopic presentation is intriguing because of its relevance to visuo-temporal parsing. The wagon-wheel effect in continuous light has also been the source of considerable misunderstanding and dispute, as is apparent in a series of recent papers. Here we review this potentially confusing evidence and suggest how it should be interpreted.     

Introduction to Michotte's heritage in perception and cognition research

Several decades after Michotte's work was published, it continues to inspire current research in perception, cognition, and beyond. In this special issue we pay tribute to this heritage with a collection of empirical and theoretical papers on amodal completion and the perception of causality, two areas of research within which Michotte's work and ideas have had a lasting influence. As a background to better understand the remaining papers, we briefly sketch Michotte's life and work and the scope (in breadth and in depth) of his impact. We then review Michotte's seminal contributions to the areas covered in this special issue, some of the major research discoveries and themes in the intervening decades, and the major open questions and challenges we are still facing. We also include a sneak preview of the papers in this special issue, noting how they relate to Michotte's work and to each other. This review shows both how much influence Michotte has had on contemporary perception and cognition research, and how much important work remains to be done. We hope that the papers in this special issue will serve both to celebrate Michotte's heritage in this respect, and to inspire other investigators to continue the projects he began.     

Social dynamics in the preschool

In this paper, we consider how concepts from dynamic systems (such as attractors, repellors, and self-organization) can be applied to the study of young children’s peer relationships. We also consider how these concepts can be used to explore basic issues involving early peer processes. We use the dynamical systems approach called state space grid (SSG) analysis and consider how it can be expanded beyond the study of dyads to the study of larger social groups and networks. In particular, we explore the role of homophily—that is, behavioral and sex similarity—as factors in the self-organization of young children’s social groups. A dynamic systems approach allows for consideration of peer processes difficult to assess using more traditional approaches.