Use and misuse of metrics in research evaluation

This paper addresses some critical issues in the applicability of quantitative performance measures (including bibliometric, economic, and co-occurrence measures) to the assessment of basic research. The strengths and weaknesses of metrics applied as research performance measures are examined. It is concluded that metrics have a useful role to play in the evaluation of research. Each metric employed, whether bibliometric, economic, co-occurrence, or others, brings a new dimension of potential insight to the complex problem of research assessment. However, when used in a stand-alone mode, metrics can be easily misinterpreted and result in misleading conclusions. Metrics should be an integral part of a more comprehensive approach to research evaluation, in which the leading role is assumed by expert peer review. Under this structure, the reviewers must be broadly constituted, so that the technical correctness of the research approaches being taken (doing the job right) can be evaluated by technical experts and the correctness of the research target (doing the right job) can be evaluated as well. This comprehensive approach insures quality in the implementation of existing research paradigms and allows the introduction of revolutionary new research paradigms as well. The views contained in this paper are solely those of the author and do not represent the views of the Department of the Navy. Ronald Kostoff received a Ph.D. from Princeton University, performed technical and economic research at Bell Laboratories, managed energy programs at U.S. Department of Energy, was Director of Technical Assessment at the Office of Naval Research for many years and presently manages the Navy’s In-House Laboratory Independent Research Program.     

The Journal of Abnormal Child Psychology at 25

This issue, redesigned in a new format, marks the beginning of Volume 25, the beginning of the silver anniversary year of the Journal of Abnormal Child Psychology. Data were compiled from the first 24 volumes to study changes over time, and Pearson correlations were calculated to determine linear trends in the journal's development The number of articles per volume has doubled, from about 25 to 50. In recent years the first authors of articles were more likely to be located outside the U.S. (up from about 5% to about 25%). Also, in recent years the first authors of articles were less likely to be affiliated with nonmedical school university departments and more likely to be affiliated with medical schools or research institutes. An introduction to this special 25th anniversary issue is provided.     

Internal Objects - A Theoretical Analysis of Jungian and Kleinian Models

In this paper I discuss the ways in which experimental and objective research from cognitive science and developmental psychology can help analysts evaluate the theoretical models of mental objects which we use; I indicate the ways in which such evidence tends to support models of internal objects as mental representations or developmental capacities rather than as wish-fulfilling expressions of instinctual drives. This land of empirical evidence is not just of academic interest but also has direct clinical relevance, particularly with borderline patients; such patients' sense of identity is totally dependent on the analyst's understanding of their internal world and for this to be misunderstood by the analyst can be catastrophic. An accurate theoretical model of mental objects can therefore help analysts to contain their patients more effectively.     

Self- and cross-citations in the Journal of Applied Behavior Analysis and the Journal of the Experimental Analysis of Behavior: 1983-1992

We examined self- and cross-citation practices in JABA and JEAB from 1983 through 1992. Mean levels of self-citation for JABA and for JEAB were 22.6% and 36.1%, respectively. Overall, 2.4% of JABA citations were JEAB articles, and 0.6% of JEAB citations were JABA articles, which suggests limited integration of basic and applied research.     

Jungian Psychology, the Body, and the Future

Impressive reaffirmation of Jung's archetypal hypothesis has come from developments in behavioural biology (Tinbergen 1951; Cosmides 1985), psycholinguistics (Chomsky 1965), structural anthropology (Lévi-Strauss 1967), developmental psychology (Bowlby 1969), dream research (Jouvet 1975). neuroscience (MacLean 1976), sociobiology (Wilson 1978), and evolutionary psychiatry (Gardner 1988; Gilbert 1989). A close correspondence exists between Jungian theories of dreaming in human beings and modern biological theories of dreaming in animals. A paradigm shift is under way in the direction of a growing cross-disciplinary awareness that all human sciences are about archetypal manifestations and that these apply as much to the body as to the mind. Jungian psychology must keep abreast of these developments if it is not to be sidelined and superseded by less humane therapeutic philosophies.     

A general linear model for estimating effect size in the presence of publication bias

When the process of publication favors studies with smallp-values, and hence large effect estimates, combined estimates from many studies may be biased. This paper describes a model for estimation of effect size when there is selection based on one-tailedp-values. The model employs the method of maximum likelihood in the context of a mixed (fixed and random) effects general linear model for effect sizes. It offers a test for the presence of publication bias, and corrected estimates of the parameters of the linear model for effect magnitude. The model is illustrated using a well-known data set on the benefits of psychotherapy.Authors' note: The contributions of the authors are considered equal, and the order of authorship was chosen to be reverse-alphabetical.     

Implications of college-age narcissism for psychosocial functioning at midlife: Findings from a longitudinal study of women

Positive and negative implications of two types of college-age narcissism on psychosocial functioning at midlife were studied in a longitudinal sample of women. Both types were scored with self-report measures when the women were, on the average, age 21. Throughout the first half of their adult life, high scorers on covert narcissism presented themselves as lacking in confidence and having low morale; high scorers on overt narcissism described themselves as wanting admiration, being independent, and being forceful. Positive outcomes associated with covert narcissism included the development of insight and of a differentiated sense of the self and, in the case of overt narcissism, entailed an involvement in high-status and creative work careers. At midlife, high scorers on covert narcissism experienced negative affect and feelings of being neglected, whereas high scorers on overt narcissism experienced family conflict and problems with impulse control. The personality characteristics associated with both types of narcissism remained stable from the early 20s to early 50s.     

Ethics education in science and engineering: The case of animal research

Conclusion The past one hundred fifty years of debate over the use of animals in research and testing has been characterized mainly byad hominem attacks and on uncritical rejection of the other sides’ arguments. In the classroom, it is important to avoid repeating exercises in public relations and to demand sound scholarship. This paper is a modification of material originally included in the handbook which accompanied the American Association for the Advancement of Science (AAAS) Seminar “Teaching Ethics in Science and Engineering”, 10–11 February 1993.     

The percentage bend correlation coefficient

A well-known result is that the usual correlation coefficient,, is highly nonrobust: very slight changes in only one of the marginal distributions can alter by a substantial amount. There are a variety of methods for correcting this problem. This paper identifies one particular method which is useful in psychometrics and provides a simple test for independence. It is not recommended that the new test replace the usual test ofH ₀: = 0, but the new test has important advantages over the usual test in terms of both Type I errors and power.