Peer review for journals: Evidence on quality control, fairness, and innovation

This paper reviews the published empirical evidence concerning journal peer review consisting of 68 papers, all but three published since 1975. Peer review improves quality, but its use to screen papers has met with limited success. Current procedures to assure quality and fairness seem to discourage scientific advancement, especially important innovations, because findings that conflict with current beliefs are often judged to have defects. Editors can use procedures to encourage the publication of papers with innovative findings such as invited papers, early-acceptance procedures, author nominations of reviewers, structured rating sheets, open peer review, results-blind review, and, in particular, electronic publication. Some journals are currently using these procedures. The basic principle behind the proposals is to change the decision from whether to publish a paper to how to publish it. The author, a professor of marketing at the Wharton School since 1968, was a founder editor of the Journal of Forecasting and the International Journal of Forecasting. This paper is based on a presentation at a workshop, “Advances in Peer Review Research”, American Association for the Advancement of Science Meeting, Baltimore, MD, February 9, 1996.     

Assessment of Interpersonal Influences on Adolescents: The Parent and Peer Influence Scale

Peer influence is a common topic of interest for parents, clinicians, and researchers, but results from research on the relative importance of parents’ versus peers’ influence on adolescents and young adults has been contradictory. For example, some research suggests that peer attitudes about school influences future academic aspirations, other research suggests that parents and peers differ on their level of influence based on topic (e.g., parents’ are the primary source of influence about moral issues), and other research suggests that peer-oriented children are a product of parental disregard. A reliable and valid measure of the relative strength of parent versus peer influence could enhance research in this area. As a result, the Parent and Peer Inventory Scale (PPI) was developed, which demonstrates strong reliability as well as face and concurrent validity. The PPI can be used both by researchers to address contradictions in the literature, as well as clinicians seeking to evaluate level of peer influence in a family.     

NSF programs in the behavioral and neural sciences

The National Science Foundation’s new Division of Behavioral and Neural Sciences supports basic research and related activities in psychology, ethology, anthropology, linguistics, and the neurosciences. Research equipment and facilities can be provided either through a grant supporting a research project or through a separate grant. The scientific importance and soundness of the research to be conducted, as evaluated by reviewers from the research community, are the main criteria for support. Preparation of proposals is briefly discussed, and sources of further information are noted.     

THE EFFECTS OF PERSONALITY SIMILARITY ON PEER RATINGS OF CONTEXTUAL WORK BEHAVIORS

The present field study investigates whether rater-ratee similarity in the Big Five personality factors influences peer ratings of contextual work behaviors. It overcomes problems that previous studies have had by using a polynomial regression analysis and by correcting the potential biases from nonindependence. Using more than 500 peer dyads, we found that rater-ratee similarity in Conscientiousness, but not in other dimensions, was positively associated with peer ratings even after controlling for interpersonal affect. These results suggest that the observed effect of personality similarity may reflect actual behavioral differences rather than biases due to interpersonal affect. Implications of the findings are discussed along with recommendations for future research.     

A detection theory approach to the evaluation of assessors in assessment centres

The ratings given to job applicants in an assessment centre (AC) will be influenced both by an assessor's sensitivity to the evidence of suitability provided by the applicants, and by whether the assessor has a liberal or a conservative rating tendency. In this study we explore the usefulness of signal detection methodology for evaluating and comparing both aspects of performance. A group of eight managerial assessors in a real‐world AC rated 195 applicants for retail sales positions. The sensitivity and response bias of assessors was evaluated using receiver operating characteristic (ROC) analyses, and their performance was evaluated by means of critical operating characteristic (COC) analyses. RscorePlus software ( Harvey, 2002 ) was used for these purposes. We conclude that, in this and similar organizational contexts, such analyses are potentially useful for estimating and comparing the performances of assessors, thereby highlighting the need for, and subsequently evaluating the effectiveness of, any remedial intervention.     

Developing consciousness in organizations: The transcendental meditation program in business

This paper reviews retrospective, prospective, and case research on workplace applications of Maharishi's Transcendental Meditation technique for developing consciousness and human potential. The distinctive psychophysiological state of restful alertness produced by the Transcendental Meditation technique appears to improve employee health, well-being, job satisfaction, efficiency and productivity, in turn influencing organizational climate, absenteeism, and financial performance.This research has been supported by a grant from the Schweisfurth Foundation of Munich, Germany. The authors also wish to thank the two anonymous reviewers for their comments and Sheila Swanson for editorial assistance.     

Assessment Center Procedures: Cognitive Load During the Observation Phase

This study explores the traditional procedure of observing assessment center exercises while taking notes vs. an alternative procedure where assessors merely observe and postpone note‐taking until immediately after the exercise. The first procedure is considered to be cognitively demanding due to the requirement of simultaneous note‐taking and observing. Also, dual task processing (concurrent observing and note‐taking) is considered to be especially demanding for assessors without rating experience. The procedures are evaluated using a 2 × 2 design (with note‐taking/without note‐taking × experienced/inexperienced). Some 121 experienced and inexperienced assessors rated videotaped candidates, observing either with or without taking notes. Results showed that experienced assessors yield significantly higher differential accuracy than inexperienced assessors. We did not find an effect of observation procedure on accuracy, interrater reliability or halo. Implications for future research are described.     

气质和儿童同伴交往类型关系的研究

该文采用问卷法和社会测量法相结合的方法,以班级为单位对484名3~9岁儿童的气质和同伴关系类型的关系进行研究。结果表明:3~9岁儿童同伴关系类型已形成分化,其中被忽视儿童和被拒绝儿童所占总体样本的比例分别为16.9%和9.6%,比例较高值得重视。儿童气质对同伴交往类型的形成影响很大,四种同伴关系交往类型在气质各维度上方差检验都达到了差异显著,有着明显不同的气质特点。     

同伴关系的发展研究

同伴关系因其在个体心理社会适应中的重要作用而受到研究者的广泛重视。本文介绍了作者及其团队在同伴关系领域基于13年的"中美儿童同伴关系数据库"开展的长期研究工作,主要包括儿童同伴关系的影响因素及影响后效、同伴关系的追踪研究、同伴关系的中美比较和网络同伴关系等主题,较为全面地阐述了儿童同伴关系领域的现有研究基础,概括了儿童同伴关系的童年期特点、发展趋势及其文化差异,总结了对儿童同伴关系及其发展的主要指标研究得到的结论,并从网络交往、同伴关系的社会网络分析和个体中心的分析等视角指出了同伴关系领域后续研究的可行方向。     

Using technology to explore social networks and mechanisms underlying peer effects in classrooms

Peer interactions among children have long interested social scientists. Identifying causal peer effects is difficult, and a number of studies have used random assignment to produce evidence that peers affect each other's outcomes. This focus by sociologists and economists on whether peers affect each other has not been matched by direct evidence on how these effects operate. The authors argue that one reason for the small number of studies in sociology and economics on the mechanisms underlying peer effects is the difficulty of collecting data on microinteractions. They argue technology reduces data collection costs relative to direct observation and allows for realistic school activities with randomly assigned peers. The authors describe a novel strategy for collecting data on peer interactions and discuss how this approach might shed light on mechanisms underlying peer influence. The centerpiece of this strategy is the use of handheld computers by middle and high school students as part of interactive math and science lessons called the Discussion Game. The handhelds collect data on interactions between students and track how students' answers evolve as they interact with different peers.     

Frequency and Type of Conflicts of Interest in the Peer Review of Basic Biomedical Research Funding Applications: Self-Reporting Versus Manual Detection

Despite the presumed frequency of conflicts of interest in scientific peer review, there is a paucity of data in the literature reporting on the frequency and type of conflicts that occur, particularly with regard to the peer review of basic science applications. To address this gap, the American Institute of Biological Sciences (AIBS) conducted a retrospective analysis of conflict of interest data from the peer review of 282 biomedical research applications via several onsite review panels. The overall conflicted-ness of these panels was significantly lower than that reported for regulatory review. In addition, the majority of identified conflicts were institutional or collaborative in nature. No direct financial conflicts were identified, although this is likely due to the relatively basic science nature of the research. It was also found that 65 % of identified conflicts were manually detected by AIBS staff searching reviewer CVs and application documents, with the remaining 35 % resulting from self-reporting. The lack of self-reporting may be in part attributed to a lack of perceived risk of the conflict. This result indicates that many potential conflicts go unreported in peer review, underscoring the importance of improving detection methods and standardizing the reporting of reviewer and applicant conflict of interest information.     

The perils of centralized research funding systems

The distribution of research funds by key funding agencies in Canada and the U.S. is based on the idea of selectivity, i.e., the policy of NOT funding a significant fraction of the applicants. Despite the fact that this policy is superficially justified by the peer review-enforced goal of "excellence," in practice it tends to favor research along well-established lines and discourages novel approaches, innovation, and risk taking. Furthermore, the secretive nature of the funding system efficiently turns it into a self-serving network operating on the principle of an "old boys’ club." This even further undercuts the veneer of objectivity in the peer review assessment because funding panels can arbitrarily interpret reviews at their whim without oversight. Although the system calls itself "competition,", the idea of real interactive competition is actually betrayed. What we have, in fact, is a centralized and authoritarian quasi-socialist grant distribution system. The article disputes the common belief that the major problem of a granting system is "government under-funding." On the contrary, it argues that much of a trivial research is actually OVER-funded. The main problem is the distribution system, not the overall level of funding. An alternative funding model is proposed. In a nutshell, it is a "fund researchers, not proposals" strategy. Drastic simplification of the funding process is advocated. The proposed measures are largely at odds with the present paradigm of selectivity and fierce competition as catalyzers of "excellence." The latter notion is dismissed as misleading. In order to see more innovation, we need a more uniform funding system (a sliding funding scale), even it means lower average funding levels. He immigrated to Canada in 1978 and in 1980 joined the Engineering faculty of McMaster University, Hamilton, Ontario, where he is presently professor of Engineering Physics. He is an author or co-author of over 120 peer-reviewed papers in semiconductor and plasma physics, electronics, environmental engineering, biophysics, philosophical problems of physics, and social aspects of scientific research.     

Does being attractive always help? Positive and negative effects of attractiveness on social decision making

Previous studies of organizational decision making demonstrate an abundance of positive biases directed toward highly attractive individuals. The current research, in contrast, suggests that when the person being evaluated is of the same sex as the evaluator, attractiveness hurts, rather than helps. Three experiments assessing evaluations of potential job candidates (Studies 1 and 3) and university applicants (Study 2) demonstrated positive biases toward highly attractive other-sex targets but negative biases toward highly attractive same-sex targets. This pattern was mediated by variability in participants' desire to interact with versus avoid the target individual (Studies 1 and 2) and was moderated by participants' level of self-esteem (Study 3); the derogation of attractive same-sex targets was not observed among people with high self-esteem. Findings demonstrate an important exception to the positive effects of attractiveness in organizational settings and suggest that negative responses to attractive same-sex targets stem from perceptions of self-threat.     

Peer tutoring to prevent firearm play acquisition, generalization, and long-term maintenance of safety skills

Hundreds of accidental injuries and deaths to children occur annually in the United States as a result of firearm play. Behavioral skills training (BST) and in situ training have been found to be effective in teaching children the skills to use if they find a firearm, but training requires substantial time and effort. The current study examined the use of peers as tutors as a potential way to decrease the time and resources needed to teach these safety skills to youngsters. Peer trainers conducted BST and in situ training with other children. Children taught by the peer trainers acquired the safety skills and demonstrated them in naturalistic situations in which the skills were needed. Furthermore, all of the peer trainers acquired and maintained the skills. These results support the use of peer tutoring for teaching safety skills to other children.     

La motivation et son ancrage utilitaire/désirable dans l’évaluation

Motivation has an important place among the personal characteristics that professional assessors take in account. A first study shows that different professionals do not choose the same criteria to assess motivation and also shows that normative factors affect their judgement. Then a series of studies focuses on the valorization of intrinsic motivation and on the anchoring in utility / desirability of this attribution of value. It appears that, compared with extrinsically motivated applicants, intrinsically motivated applicants are rather described by judges with desirable traits than with utility traits. This result is found as well with professionals and non professional respondents. So it seems that the valorization of intrinsic motivation would be more anchored in desirability than in utility, which is a difference with the norm of internality that is clearly anchored in utility.     

青少年早期同伴拒绝、同伴侵害与抑郁的关系：交叉滞后分析

采用间隔1年的追踪设计,使用结构方程建模对1068名青少年早期个体的同伴拒绝、同伴侵害与抑郁间的关系进行交叉滞后分析。结果发现,（1）在不同时间点上,同伴拒绝、同伴侵害和青少年抑郁之间存在显著正相关;（2）先前的抑郁水平对随后个体遭受的身体侵害和关系侵害均有显著预测作用,而先前的同伴侵害不能显著预测随后青少年的抑郁水平。抑郁对同伴拒绝、以及同伴拒绝对抑郁的预测作用均不显著。（3）同伴拒绝、同伴侵害与抑郁间的关系具有跨性别的一致性。这表明,青少年早期同伴侵害和抑郁间存在单向预测关系,抑郁导致同伴侵害。