Use and misuse of metrics in research evaluation

This paper addresses some critical issues in the applicability of quantitative performance measures (including bibliometric, economic, and co-occurrence measures) to the assessment of basic research. The strengths and weaknesses of metrics applied as research performance measures are examined. It is concluded that metrics have a useful role to play in the evaluation of research. Each metric employed, whether bibliometric, economic, co-occurrence, or others, brings a new dimension of potential insight to the complex problem of research assessment. However, when used in a stand-alone mode, metrics can be easily misinterpreted and result in misleading conclusions. Metrics should be an integral part of a more comprehensive approach to research evaluation, in which the leading role is assumed by expert peer review. Under this structure, the reviewers must be broadly constituted, so that the technical correctness of the research approaches being taken (doing the job right) can be evaluated by technical experts and the correctness of the research target (doing the right job) can be evaluated as well. This comprehensive approach insures quality in the implementation of existing research paradigms and allows the introduction of revolutionary new research paradigms as well. The views contained in this paper are solely those of the author and do not represent the views of the Department of the Navy. Ronald Kostoff received a Ph.D. from Princeton University, performed technical and economic research at Bell Laboratories, managed energy programs at U.S. Department of Energy, was Director of Technical Assessment at the Office of Naval Research for many years and presently manages the Navy’s In-House Laboratory Independent Research Program.     

The effect of spatial frequency on global precedence and hemispheric differences

There are many conditions in which identification proceeds faster for the global form of a hierarchical pattern than for its local parts. Since the global form usually contains more lower spatial frequencies than do the local forms, it has frequently been suggested that the higher transmission rate of low spatial frequencies is responsible for the global advantage. There are also functional hemispheric differences. While the right hemisphere is better at processing global information, the left hemisphere has an advantage with respect to local information. In accordance with the spatial-frequency hypothesis, it has been speculated that this difference is due to a differential capacity of the hemispheres for processing low and high spatial frequencies. To test whether low spatial frequencies were responsible for the global advantage and/or for the observed hemispheric differences, two experiments were carried out with unfiltered and highpass-filtered compound-letter stimuli presented at the left, right, or center visual field. The first experiment, in which the target level was randomized in each trial block, revealed that low spatial frequencies were not necessary for either global advantage or for hemispheric differences. Highpass filtering merely increased the response times. In the second experiment, the target level was held constant in each block. This generally increased the speed of responding and produced interactions between filtering and global-local processing. It was concluded that both sensory and attentional or control mechanisms were responsible for global precedence and that the hemispheres differed with respect to the latter.     

Personality Factors and Inhibited Career Development: Testing the Unique Contribution of Shyness

This study evaluated the hypothesis that shyness is uniquely associated with delayed development of personal resources for dealing with career tasks during the college years. In addition to shyness, the set of relevant personality factors included academic self-esteem and dysphoria. Canonical correlation and semipartial analysis were used to assess the mutual and unique variance associated with these three personality factors and a set of career development variables pertinent to college students in late adolescence. Results revealed that shyness was uniquely related with vocational self-concept crystallization and aspects of vocational maturity (e.g., attitudes toward planning), but unrelated with indexes of self and environmental exploration. Academic self-esteem was only related to vocational self-concept, while dysphoria evidenced no unique association with any career variables. Implications of the expected and unexpected findings are discussed.     

Acquisition of blood, injury, and needle fears and phobias

The origins of fear and phobia of blood, injury, and injections were investigated in a sample of 128 fearful university students. Based on Mutilation Questionnaire scores, subjects were designated as common fear, high fear, or phobic. Ss reports of their onset experiences obtained from structured interviews were categorized into one or more acquisition pathways of conditioning, vicarious observation, and information. Of the 73% of Ss who recalled one or more onset experiences, 76% reported conditioning-like events as the primary pathway with the majority reporting fear-related UCSs. Vicarious experiences were reported as primary by 20% and 3% reported information as being primary in their fear onset. Severity of fear was unrelated to the pathway by which it was acquired, to whether the onset was recalled, and if recalled, whether it was due to a single or to multiple traumatic events. Results are discussed in terms of methodological problems of memory issues and means by which data are collected.     

Book Reviews

G rünbaum , A dolf . Validation in the Clinical Theory of Psychoanalysis: A Study in the Philosophy of Psychoanalysis
H ollis , J ames . The Middle Passage: From Misery to Meaning in Midlife
B ertoia , J udi . Drawings from a Dying Child: Insights into Death from a Jungian Perspective
H olt , D avid . The Psychology of Carl Jung: Essays in Application and Deconstruction     

A DEVELOPMENTAL DEFICIT IN LOCALIZING OBJECTS FROM VISION

Abstract— We describe a college student, A. H., with a developmental deficit in determining the location of objects from vision. The deficit is selective in that (a) localization from auditory or tactile information is intact, (b) A H reports the identity of mislocalized objects accurately, (c) visual localization errors preserve certain parameters of the target location, and (d) visual localization is severely impaired under certain stimulus conditions, but nearly intact under other conditions. These results bear on the representation and processing of location information in the visual system, and also have implications for understanding developmental dyslexia.     

Demonstrative induction,old and new evidence and the accuracy of the electrostatic inverse square law

Maxwell claimed that the electrostatic inverse square law could be deduced from Cavendish's spherical condenser experiment. This is true only if the accuracy claims made by Cavendish and Maxwell are ignored, for both used the inverse square law as a premise in their analyses of experimental accuracy. By so doing, they assumed the very law the accuracy of which the Cavendish experiment was supposed to test. This paper attempts to make rational sense of this apparently circular procedure and to relate it to some variants of traditional problems concerning old and new evidence.My interest in demonstrative induction developed as the result of being given the opportunity by Joseph Pitt to comment on an earlier version of John Norton's Science and Certainty (1994) at a conference held at the Virginia Polytechnic Institute and State University. I want to thank Joe and John for much stimulating discussion and John specifically for sharing his work on demonstrative induction with me. My gratitude also goes to the National Science Foundation (DIR-8920699) and Ohio State University for providing funding for a larger project of which this paper forms a part.