Use and misuse of metrics in research evaluation

This paper addresses some critical issues in the applicability of quantitative performance measures (including bibliometric, economic, and co-occurrence measures) to the assessment of basic research. The strengths and weaknesses of metrics applied as research performance measures are examined. It is concluded that metrics have a useful role to play in the evaluation of research. Each metric employed, whether bibliometric, economic, co-occurrence, or others, brings a new dimension of potential insight to the complex problem of research assessment. However, when used in a stand-alone mode, metrics can be easily misinterpreted and result in misleading conclusions. Metrics should be an integral part of a more comprehensive approach to research evaluation, in which the leading role is assumed by expert peer review. Under this structure, the reviewers must be broadly constituted, so that the technical correctness of the research approaches being taken (doing the job right) can be evaluated by technical experts and the correctness of the research target (doing the right job) can be evaluated as well. This comprehensive approach insures quality in the implementation of existing research paradigms and allows the introduction of revolutionary new research paradigms as well. The views contained in this paper are solely those of the author and do not represent the views of the Department of the Navy. Ronald Kostoff received a Ph.D. from Princeton University, performed technical and economic research at Bell Laboratories, managed energy programs at U.S. Department of Energy, was Director of Technical Assessment at the Office of Naval Research for many years and presently manages the Navy’s In-House Laboratory Independent Research Program.     

The Journal of Abnormal Child Psychology at 25

This issue, redesigned in a new format, marks the beginning of Volume 25, the beginning of the silver anniversary year of the Journal of Abnormal Child Psychology. Data were compiled from the first 24 volumes to study changes over time, and Pearson correlations were calculated to determine linear trends in the journal's development The number of articles per volume has doubled, from about 25 to 50. In recent years the first authors of articles were more likely to be located outside the U.S. (up from about 5% to about 25%). Also, in recent years the first authors of articles were less likely to be affiliated with nonmedical school university departments and more likely to be affiliated with medical schools or research institutes. An introduction to this special 25th anniversary issue is provided.     

DIE MARXISMUS – DEBATTE IM “SILBERNEN ZEITALTER”

Im Anfang des 20.Jahrhunderts unterzog eine Reihe von jungen, später berühmt gewordenen Philosophen den Marxismus einer tiefen Kritik. Ihnen ist es gelungen, wichtige Defekte der revolutionären Doktrinen aufzudecken und ihre verhängnivsollen Konsequenzen vorwegzunehmen. Diese Kritik ist auch für die gegenwärtige geistig-politische Situation aktuell.     

Internal Objects - A Theoretical Analysis of Jungian and Kleinian Models

In this paper I discuss the ways in which experimental and objective research from cognitive science and developmental psychology can help analysts evaluate the theoretical models of mental objects which we use; I indicate the ways in which such evidence tends to support models of internal objects as mental representations or developmental capacities rather than as wish-fulfilling expressions of instinctual drives. This land of empirical evidence is not just of academic interest but also has direct clinical relevance, particularly with borderline patients; such patients' sense of identity is totally dependent on the analyst's understanding of their internal world and for this to be misunderstood by the analyst can be catastrophic. An accurate theoretical model of mental objects can therefore help analysts to contain their patients more effectively.     

Self- and cross-citations in the Journal of Applied Behavior Analysis and the Journal of the Experimental Analysis of Behavior: 1983-1992

We examined self- and cross-citation practices in JABA and JEAB from 1983 through 1992. Mean levels of self-citation for JABA and for JEAB were 22.6% and 36.1%, respectively. Overall, 2.4% of JABA citations were JEAB articles, and 0.6% of JEAB citations were JABA articles, which suggests limited integration of basic and applied research.     

TRACING A TRAJECTORY

Abstract. Theology and ethics intersect with sciences at different points depending upon whether the scholars involved are interested in, for example, general epistemological issues or practical moral judgments. The intersection affects theology and ethics in different ways, depending upon various commitments or resistances on the part of theologians. The author surveys his own writings to show how openness to the sciences has had an impact on various phases of his work and what issues remain somewhat unresolved.     

Jungian Psychology, the Body, and the Future

Impressive reaffirmation of Jung's archetypal hypothesis has come from developments in behavioural biology (Tinbergen 1951; Cosmides 1985), psycholinguistics (Chomsky 1965), structural anthropology (Lévi-Strauss 1967), developmental psychology (Bowlby 1969), dream research (Jouvet 1975). neuroscience (MacLean 1976), sociobiology (Wilson 1978), and evolutionary psychiatry (Gardner 1988; Gilbert 1989). A close correspondence exists between Jungian theories of dreaming in human beings and modern biological theories of dreaming in animals. A paradigm shift is under way in the direction of a growing cross-disciplinary awareness that all human sciences are about archetypal manifestations and that these apply as much to the body as to the mind. Jungian psychology must keep abreast of these developments if it is not to be sidelined and superseded by less humane therapeutic philosophies.     

A general linear model for estimating effect size in the presence of publication bias

When the process of publication favors studies with smallp-values, and hence large effect estimates, combined estimates from many studies may be biased. This paper describes a model for estimation of effect size when there is selection based on one-tailedp-values. The model employs the method of maximum likelihood in the context of a mixed (fixed and random) effects general linear model for effect sizes. It offers a test for the presence of publication bias, and corrected estimates of the parameters of the linear model for effect magnitude. The model is illustrated using a well-known data set on the benefits of psychotherapy.Authors' note: The contributions of the authors are considered equal, and the order of authorship was chosen to be reverse-alphabetical.     

Fallacies and moral dilemmas

The continuing debate between utilitarians and deontologists often takes the form of disagreement over how particular moral dilemmas are to be resolved, but protagonists on both sides tend to overlook the possibility of resolving a dilemma with remainder, such as regret. The importance of remainder is also overlooked by critics of some absolutist ways of resolving or slipping between the horns of certain moral dilemmas. Moreover, deontologists, if not utilitarians, can be criticised for overlooking the possibility that, according to their theory, some dilemmas may be irresolvable. Virtue ethics, with its concentration on the agent, readily accommodates both mention of remainder and irresolvable dilemmas, and yields a specification of tragic dilemmas which the other two theories might like to take on.     

Why are there no objective values?

Using the mathematical frameworks of economic preference ranking, subjective probability, and rational learning through empirical evidence, the epistemological implications of teleological ethical intuitionism are pointed out to the extent to which the latter is based on cognitivist and objectivist concepts of value. The notions of objective value and objective norm are critically analysed with reference to epistemological criteria of intersubjectively shared valuative experience. It is concluded that one cannot meaningfully postulate general material theories of morality that could be tested, confirmed or refuted by intersubjective empirical evidence of preferences and values, however loosely the empirical evidence of values may be interpreted. This situation is explained with reference to the ways in which preceived values become systematically influenced by the concomitants of individual valuative experience, but which have nothing to do with contingent subjective interests.