Use and misuse of metrics in research evaluation

This paper addresses some critical issues in the applicability of quantitative performance measures (including bibliometric, economic, and co-occurrence measures) to the assessment of basic research. The strengths and weaknesses of metrics applied as research performance measures are examined. It is concluded that metrics have a useful role to play in the evaluation of research. Each metric employed, whether bibliometric, economic, co-occurrence, or others, brings a new dimension of potential insight to the complex problem of research assessment. However, when used in a stand-alone mode, metrics can be easily misinterpreted and result in misleading conclusions. Metrics should be an integral part of a more comprehensive approach to research evaluation, in which the leading role is assumed by expert peer review. Under this structure, the reviewers must be broadly constituted, so that the technical correctness of the research approaches being taken (doing the job right) can be evaluated by technical experts and the correctness of the research target (doing the right job) can be evaluated as well. This comprehensive approach insures quality in the implementation of existing research paradigms and allows the introduction of revolutionary new research paradigms as well. The views contained in this paper are solely those of the author and do not represent the views of the Department of the Navy. Ronald Kostoff received a Ph.D. from Princeton University, performed technical and economic research at Bell Laboratories, managed energy programs at U.S. Department of Energy, was Director of Technical Assessment at the Office of Naval Research for many years and presently manages the Navy’s In-House Laboratory Independent Research Program.     

Using the Web to facilitate international academic exchange

Use of the World-Wide Web among academics has grown explosively during the last few years. To date, most applications of the Web have focused on the storage and dissemination of relatively static information, in part because of the cost associated with manual collating and updating of data. This paper presents an example of a dynamic worldwide database devoted to maintaining inherently transient information: The database accepts international travel itineraries submitted by individual academic psychologists, and makes them publicly available to potential hosts on the basis of various search keys. The objective of the database (http://www.psy.uwa.edu.au/wave/) is to facilitate international exchange among academics.     

Asymmetries and errors in perception of depth from disparity suggest a multicomponent model of disparity processing

In three experiments, asymmetries between the processing of crossed and uncrossed disparities were investigated. The target was a luminance-defined circle concentric to a fixation mark, viewed stereoscopically on a computer monitor for 105 msec. Fifteen disparities were presented according to the method of constant stimuli. Observers indicated the apparent direction of target depth relative to fixation. All experiments measured both the accuracy and latency of this response. Experiment 1 showed fewer errors and shorter reaction times for identifying crossed disparities. Experiments 2 and 3 replicated Experiment 1 and also showed that observers may often perceive a target in the direction opposite that prescribed by the disparity information. We propose that the asymmetries and reversals result from differences in computation of sign, not of magnitude. This notion is consistent with a scheme of continuous disparity tuning and accounts for such asymmetries and errors without positing disparity pooling mechanisms.     

Tic douloureux, Parkinson’s disease and the herpes connection

Several cases encountered in psychiatric practice are described in which herpetic involvement of the trigeminal nerve appeared to be crucial to the pathophysiology of the patients’s illness. A brief history of herpes is discussed in terms of the anatomy and physiology of the trigeminal nerve. Particular attention is given to the effects of intraneuronal herpes on gasserian ganglion and spinal nuclei discharge threshoolds, as well as viral pathways directly impacting the substantia nigra. Evidence characterizing Parkinson’s disease as one of several important manifestations of CNS herpes is covered. This article will be presented, by invitation, at the Neuroscience Seminar, Department of Medicine, University of Otago, Dunedin, New Zealand, on 19 August 1997.     

Evidence Supporting the Importance of Peripheral Visual Information for the Directional Control of Aiming Movement

The focus of the present study was on determining whether the high level of directional accuracy found in aiming studies in which the subjects can see their hand in the visual periphery supports the existence of a kinetic visual channel or, rather, the advantage of binocular over monocular vision for movement directional control. The limits of this kinetic visual channel were also explored. The results of the 1st experiment indicated that seeing one's hand in the visual periphery is sufficient to ensure optimal directional aiming accuracy. Further, no differences in aiming accuracy were noted between monocular and binocular vision. These results supported the existence of a visual kinetic channel. In the 2nd experiment, whether this kinetic visual channel would operate with movements slower (55 degrees /s) than those usually used in studies that had proved its existence (over 110 degrees /s) was delineated. The results indicated that this visual kinetic channel was operative even at relatively slow movement velocities. Central vision of the hand seemed to be used for on-line directional control of relatively slow movements.     

Influences of Destructive Personality Information on Decision Making1

This study examined the effects of information about a landlord's personality on juror judgments for a landlord-tenant civil trial. The personality information manipulated was specifically relevant to destructive acts. Alternative versions of witness testimony were used to describe a landlord either high or low on need for power. treating people as objects, and negative life themes. This information strongly influenced aspects of the schema for this case constructed by the individual mock jurors. Destructive personality information caused dislike of the landlord and lowered the credibility of his story. If the landlord treated people as objects, the credibility of the tenant's story and positive evaluations of the tenant increased. Juror judgments about relative fault were strongly shifted by destructive personality information. An empirical model for juror decisions indicated a dynamic interplay of story components and fault judgments.     

The Ramsey Test and Conditional Semantics

Proponents of the projection strategy take an epistemic rule for the evaluation of English conditionals, the Ramsey test, as clue to the truth-conditional semantics of conditionals. They also construe English conditionals as stronger than the material conditional. Given plausible assumptions, however, the Ramsey test induces the semantics of the material conditional. The alleged link between Ramsey test and truth conditions stronger than those of the material conditional can be saved by construing conditionals as ternary, rather than binary, propositional functions with a hidden contextual parameter. But such a ternary construal raises problems of its own.